Peer review: An insight into MSFHR’s principles and process

24 May 2017

Rigorous peer review is the cornerstone of MSFHR funding programs. It ensures the Foundation supports research that meets the highest scientific standards, invests in BC’s best and brightest researchers, and builds our province’s capacity for world-class health research.

At MSFHR we adhere to six principles shared by all members of the National Alliance of Provincial Health Research Organizations (NAPHRO): integrity, accountability, transparency, balance, confidentiality and impartiality.

In line with these principles we follow these steps for each review cycle:

  • Applications by theme. Depending on the program, applications are categorized by theme with a review panel for each (e.g. biomedical, clinical, health services and population health research). The Chairs and Scientific Officers of all review panels cross check these groupings to ensure the research proposed within each application aligns with the panel it is assigned to.
  • Match reviewers to applications. Once the application deadline has passed, we recruit reviewers who are subject matter experts in areas that directly align with the applications we’ve received. We’ve been matching reviewers to applications based on expertise for 15 years, drawing on ‘external readers’, often from the international community, for highly specialized applications.
  • Hold in-person review meetings. Review panels for our major funding awards are held in person. Each reviewer is encouraged to share unbiased feedback and discuss scoring for each application. We find that through this frank and open discussion, reviewers challenge and hold each other accountable for adjudication decisions.
    The Convening and Collaborating and Reach Awards which provide smaller funding amounts are not reviewed in-person.
  • Have three rounds of scoring. Two lead reviewers (chosen for their subject matter expertise) share their initial scores. Following panel discussions the lead reviewers have the opportunity to revise their scores, creating the consensus score which forms the basis for the voting range. Finally, all panel members (excluding those with conflicts) score the application anonymously. The average of these scores is the final score for the application.
  • Debrief with all reviewers. Our reviewers are seasoned researchers and many sit on panels for a number of funding organizations. At the end of each review meeting we invite them to share feedback on the program, the resources provided to applicants and the review process itself. This ensures we continue to refine our programs and processes to ensure they are of high quality and we continue to meet the highest scientific standards.

MSFHR’s structured peer review process is tailored to meet the needs of each individual funding program. Below is a visual overview of the process for our Scholar program, which has been running since 2001.

“Having served on review panels for numerous agencies over many years I’d say MSFHR has an excellent process in place. The advance planning in disseminating information to the panel members, distributing the grants to be reviewed through an online site, and then making use of a very efficient reviewing site makes the process easy and smooth. By incorporating the online scoring system that works in real time at the panel meetings, it also makes for a very efficient experience at the face-to-face meetings that results in scoring that directly compares all applications using well defined criteria. Overall, a robust and well run peer review process,” Dr. Vincent Duronio, Professor, Department of Medicine, University of British Columbia.

Valerie To, Director, Research Competitions added, “We are very proud of our peer review process and hugely grateful to our peer reviewers who volunteer their time and expertise to support the development of BC research talent and nurture a world-class research environment in our province”.