Measuring research impact at CIMVHR

The Canadian Institute for Military and Veteran Health Research (CIMVHR) has two main priorities:

• Knowledge translation (KT) of research needs and results related to military and Veterans’ health
• Partnerships that will advance the development and uptake of research on military and veteran's health  

We want to “move knowledge to action” to improve the health of those who serve in the military, Veterans and their families.  The scope of our work also pertains to first responder groups such as police, fire-fighters and paramedics, who work in similarly stressful occupations, and humanitarians who work in similarly unstable world locales.

Ultimately, the impact of the work developed through the CIMVHR networks applies to all Canadians as current medical advances lead to changes in policies, programs and practices in Canadian mental health organizations, emergency departments and rehabilitation hospitals, among others.

Our challenge is measuring the impact of the knowledge translation (KT) and Partnership/Collaboration efforts we are now engaged in and are developing based on identified needs from stakeholders. I hope that this discussion of CIMVHR's impact measurement framework is able to assist with the Federation’s ongoing Impacts of Humanities and Social Science Research project.

Can you describe the impact measurement framework used by CIMVHR?

We are working closely with Health Canada in developing a clear understanding of breadth of the intended impacts, e.g. jurisdictional vs. pan-Canadian impacts, and KT. It is important that we build a common understanding of expectations for KT and ensure buy-in from our stakeholders.

In an ideal world, all who access any of CIMVHR’s KT products will eagerly report increased awareness and access, i.e. that they’ve learned something and then used the knowledge.  It is impossible and unwieldy to track all users through the different levels of outcomes. Instead we must determine the intended audiences for significant products/mechanisms and those whom CIMVHR wants to ensure use these to help address our priorities and theirs. The current Performance Measurement plan (2015-2019) is concentrating on measuring the impacts on our university partners, researchers, policy-makers and on health-care providers.  We need to keep in mind the burden of tracking several user groups through these outcomes.

We know that not all KT is equal and not everything needs to be or should be measured. Decisions of what is significant enough to measure are currently being considered and will be based on the following criteria: amount of financial investment, level of effort, strategic importance, degree of impact, or other considerations

We’ve identified numerous indicators for our KT and Partnership/Collaboration outputs and are confident that we will demonstrate increased awareness and understanding through pre/post learning event surveys of stakeholders, and/or through reported increased use and benefit of knowledge products. Over the next 4 years we intend to conduct interviews with our partners and collaborators to capture impacts on health care systems. In due course we want to be able to demonstrate impacts on healthcare/well-being outcomes in clinical practice settings as this aligns most closely with our key beneficiaries.

Active and conscious participation of researchers and research users will be paramount to successfully measuring the impacts. 

What different kinds of impact exist and can be measured?

The list of potential impacts is lengthy. For example, in the area of Research & Knowledge, consider capacity:  

• Capacity to perform high quality research
• Advancing methods and data sets to provide capacity for more complex problem solving
• Increasing the capacity for researchers to use existing research
• Developing and progressing researchers through their careers
• Building the reputations of CIMVHR and its affiliated institutions
• Increasing research revenues for military and veteran’s health research
• Improving the relevance of research (to improve to capacity for action from research)

Consider also the impacts of using research on policy and healthcare systems[1] to:

• Identify areas where CIMVHR research has been used already to inform health policies.
• Working with our stakeholders, examine areas of policy to identify knowledge gaps and problems and to determine if research is required to address these problems;
• Evaluate and integrate research findings to directly influence the policy decision;
• Inform alternative perspectives to deal with the policy issue;
• Persuade targeted stakeholders to support a predetermined decision.

Further, CIMVHR aims to include supporting high-quality research evidence to inform new practices and programs for military, Veterans and their families.  Measuring the outcomes and impact of these practices and programs is also critical.

What further tools do you need to continue to measure research impacts?

• Tools to help health services and policy researchers to describe the impact of their work.

Can we develop a tool, useable by individual researchers without specific training in research impact assessment, to think through and describe the possible impacts of their work? We are exploring models such as the Research Impact Framework developed at the London School of Hygiene and Tropical Medicine.  

• Tools to measure research use in health policy

Recently, Australian researchers published a comprehensive, multi-modal comprehensive measurement and scoring tool that assesses four domains of research use (i.e. instrumental, conceptual, tactical, and imposed). It is one example of an existing tool that we are investigating to assist us.