Research quality and research translation: Never the twain shall meet?

Today's blogger is Martin Bortz, who is a research fellow at the Melbourne School of Government. Martin works in our research translation team, and is completing a PhD in the role of consultants in public policy. He also convenes the knowledge translation network at Melbourne University.

The focus of today’s entry is the ways in which knowledge translation can contribute to higher quality research. There are two fundamental questions embedded in this topic. First, what is quality research? And, second, what is the relationship between quality research and knowledge translation (if any)?

What is quality research?

The way in which quality research is understood varies, depending on whether it is quantitative or qualitative. Good quantitative research is assessed as meeting several hallmarks. First, it is objective. This means it takes advantage of generally accepted western scientific tools and approaches both to data collection as well as analysis. Second, it is internally valid so that research results logically follow the modes of analysis and data collection. Third, results are externally valid so can be generalised beyond the study in question. Finally, it means that the study could be replicated and produce similar results.

On the other hand, good qualitative research tends to conform to a different set of characteristics.  Some of the more common criteria include appropriate use of theory; linking of methods to research questions; systematic analysis; and use of negative cases.

Whether qualitative or quantitative, the sets of criteria tend to refer to both the process and presentation of research. That is, to generate quality research, the criteria either require that specific steps be taken during the research process, or that the researcher conduct their analysis and write-up in a specific way. In this way, we can consider the criteria largely process-oriented.

The knowledge translation process can also be seen in this light. That is, effective knowledge translation is a process through which researchers create knowledge, disseminate that knowledge, and seek to integrate it into policy and practice.

So how can we link research translation to research quality?

The diagram below [from Graham et al. (2006)] is a helpful starting point. This diagram embeds knowledge creation within a broader knowledge-to-action (or knowledge translation) process. We can see the multiple stages at which knowledge translation can (and should) occur. If we include policymakers and practitioners at each step of this process, it becomes much clearer as to how knowledge exchange can improve the overall quality of research. For instance, engaging policymakers in the problem identification stage may produce a more refined definition of the problem. Likewise, seeking to monitor how knowledge is used may result in a refinement of the kinds of claims that are made in the research itself.

More to the point, we can also apply the criteria of quality research (whether qualitative or quantitative) to the steps involved in research translation. The criteria for external validity can be enhanced through subjecting findings to the scrutiny of key stakeholders. These stakeholders will be in a position to tell you if findings align with their understanding of both the problem under investigation, as well as any proposed solutions. Likewise, researchers in the qualitative paradigm may benefit from research users’ informing them of any potential negative cases to investigate – either once the findings have been developed, or else when the research questions are determined.

However, not all criteria for quality research are likely to benefit. For instance, assessing whether the methods are suitable to address the kinds of research questions being asked requires research specific skill and experience, one which policymakers and/or practitioners may not have. Likewise, assessing the extent to which a study is replicable may also require research-specific skills, either in making the assessment, or in running the study at a later stage.

Research translation has the potential to enhance the quality of research, insofar as it will subject components of the research process to further scrutiny and refinement. This requires bringing external stakeholders and research users into the research process early, and allowing those stakeholders to influence the way in which the research is conducted, as well as what can sensibly be said about a given phenomenon.