In September 2016, the Canadian Institutes of Health Research (CIHR) launched an International Peer Review Expert Panel under the Chairmanship of Sir Peter Gluckman, Chief Scientific Advisor to the Prime Minister of New Zealand, to assess the design and implementation of CIHR’s new grants adjudication processes. Nominated by the Federation for the Humanities and Social Sciences, Matthew Herder presented his testimony to the International Peer Review Expert Panel on January 17, 2017. You can read Herder’s full testimony here as well as his blog below, originally posted on Impact Ethics blog.
There are many problems with the current peer review system managed by the Canadian Institutes of Health Research (CIHR) – Canada’s federal funding agency for health-related research. Three discrete problems, however, merit special attention given their relevance to proposed reforms to CIHR’s investigator-initiated research programs.
Failure to address these problems will exacerbate the challenges that researchers in the social sciences and humanities already face when applying for funding from CIHR. It will also mean that CIHR continues to spend (at least some of its) health research dollars in a less than efficient and effective manner. Consider the following scenarios:
Scenario 1: A team of anthropologists applies for CIHR funding to study the development of vaccines using ‘actor network theory.’ The grant is rejected because the reviewers are baffled as to why the researchers need ‘actors’ to carry out their research.
Actor network theory is a legitimate approach to social theory and research, yet the reviewers believed the applicants proposed to hire thespians as part of their grant. While the evidence is anecdotal, scholars in the social sciences and humanities frequently report receiving rejection letters from CIHR that unequivocally betray the peer reviewers’ lack of understanding of the applicants’ fields of research.
Part of the problem in providing appropriate peer review at CIHR is that the pool of potential reviewers with training in the social sciences and humanities as applied to the field of health is fairly small. Compounding this problem is the fact that many researchers in that pool have become so disillusioned with CIHR that they refuse to participate in CIHR’s peer review process.
Scenario 2: A team of neuroscientists, ethicists, epidemiologists and criminologists submit an interdisciplinary grant on the biological, moral, structural, and social underpinnings of family violence. Peer reviewers from each of the disciplines are assigned for virtual review. Each reviewer judges those aspects of the grant that are within her discipline to be strong, but nevertheless concludes that the overall application is weak. The grant is given a modest overall score. The grant is rejected without face-to-face discussion amongst reviewers.
The ending of this second scenario points to a structural problem in CIHR’s peer review process that unfairly disadvantages interdisciplinary researchers’ applications. In an effort to better shepherd financial and human resources, CIHR has elected to proceed with virtual peer review for the initial stage of the grant review process. The problem with virtual peer review of interdisciplinary research, as this example illustrates, is that such review can be less than the sum of its parts.
Researchers in biomedicine and in the humanities and social sciences occupy entirely different ‘epistemic cultures.’ The research methods and contributions to knowledge are produced and measured in fundamentally different ways. Lacking an opportunity to address this divide in a face-to-face conversation, online reviewers would seem ill-equipped to assess interdisciplinary research proposals.
Scenario 3: A team of health law and bioethics researchers are writing a grant on conscientious objection as it applies to physicians in the context of assisted dying, but they don’t want to apply to CIHR for funding because of past experiences with poor quality peer review. They consider refocusing their research on conscientious objection and war, because then they can apply to the Social Sciences and Humanities Research Council (SSHRC) – Canada’s federal funding agency for researchers trained in the social sciences and humanities.
This last scenario illustrates the choice that researchers in the social sciences and humanities who work on health sometimes face. They can either take their chances with CIHR and hope that peer reviewers will have relevant expertise, or they can reorient their research away from health and apply to SSHRC where they can be assured of appropriate peer review —meaning, peer review by researchers with relevant disciplinary training. In the scenario given, health researchers working in the social sciences and humanities who would opt to stick with health research are all the more ‘boxed in’ by CIHR since 2009 when SSHRC—the federal funding agency best positioned to ensure appropriate peer review—’boxed out’ all health-related research as ineligible for funding.
This current state of affairs increases both the risk that important health-related research by social sciences and humanities researchers will be left ‘undone’ and the risk that when such research is funded by CIHR it will not have had the benefit of rigorous peer view and thus be of lower quality.
CIHR needs to reform its funding programs to better support health-related research in the humanities and social sciences. It needs to renew its outreach efforts to ensure qualified, rigorous peer review for health-related research in the social sciences and humanities. And, interdisciplinary research involving researchers from the humanities and social sciences along with researchers from the biomedical sciences should not be rejected in the absence of face-to-face peer review by researchers spanning those disciplines.