Conducting Research Evaluation

In many ways, research assessment is similar to research analysis except it happens after program implementation and aims to articulate the outcomes of a policy or program. It is mainly concerned with the measuring, describing and understanding the outcomes (positive and negative) of a policy action.

By applying good research design principles, research evaluation aims to understand if a policy or program was effective in reducing a problem or burden. If a postive benefit was not realised or not realised to the expected extent, a researcher should seek to understand what caused the benefit failure.

If benefits were realised, a researcher might explore whether or not the beneftis reached all the intended taget groups. This may be an important research focus, because often policy actions benefit the powerful, well connected or wealthier members of a society. If inequity in benefit distribution is found, the researcher should seek to understand the cause of the inequity.

Lastly, research assessments often focus on questions about cost-efficiency and sustainability. Post implementation data on the benefits realised from policy programs, combined with actual policy cost data, can be used to calcuate a social return on investment or cost benefit ratio. This kind of research can be difficult, however, because controlling for other confounding factors that may have played a role in the accrual of benefits is very difficult in the real world environments into which most policy actions are implemented.

Both qualitative and quantitative methods can be employed in research evaluation and there are many theories for use when evaluating policy implementation. Researchers should match the data collection approach to the specific aim of the research question. Details on the strengths and limitations of each data collection approach were presented earlier in this module.

Please read a short summary on research methods in the following.