Is peer review biased? Female health researchers who applied for grants from Canada’s major health research funder were funded less often than male counterparts because of potential bias, and characteristics of peer reviewers can also affect the result, found a study in the Canadian Medical Association Journal.
Applicants who had not been previously funded also received lower scores, making them less likely to be funded.
Between 2012 and 2014, 11,624 applications were submitted to the Canadian Institutes of Health Research (CIHR) open operating grant competitions. Two-thirds (66 per cent) of applicants were male and 69 per cent were aged 40 years or older. Almost two-thirds of applications (64 per cent) were in basic science, with the remainder from applied science (16.6 per cent clinical, 8.1 per cent health services and 11.3 per cent in population health).
The study, by researchers at McGill, looking at reviewer characteristics, including gender, previous success rates with grants, experience, scientific domain, conflict of interest and more, found that these traits did introduce bias into peer review of grant applications. This bias resulted in lower scores that could place the application in the non-fundable range.
CIHR’s annual investment in health research is about $1 billion a year as of 2018.
Previous studies have found inconsistent evidence of bias, but few studies have analyzed whether reviewer characteristics could potentially bias applications.
“This study confirmed many of the suspected biases in the peer review of operating grant applications and identified important characteristics of peer reviewers that must be considered in application assignment,” writes Dr. Robyn Tamblyn, Scientific Director, CIHR — Institute of Health Services and Policy Research and a senior scientist at the Research Institute of the McGill University Health Centre. “By measuring and controlling for scientific excellence of the applicant, we were able to examine how applicant, application and reviewer characteristics may unduly influence the assessment of operating grant applications.”
The researchers also found that reviewer expertise influenced the application rating, as reviewers with high expertise rated previously successful applicants higher than less experienced applicants.
“We found lower scores for applied science applications, gender inequities in application scores that favoured male applicants who had past funding success rates equivalent to female applicants, particularly in the applied sciences,” write the authors. “Conflicts on the panel, male reviewers only, reviewers with all high expertise, and those whose own research was exclusively in the same scientific domain as the applicant’s conferred positive benefits in application rating.”
They suggest that training of reviewers, policy change and monitoring may help address these biases.
“These findings are important, as securing less funding slows career progression for women and reduces opportunities for publishing and other forms of collaboration, which are criteria for professional advancement,” writes Rosemary Morgan, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, in a related commentary with coauthors. “To understand why this occurs, we must recognize that gender bias within the grant review process is a manifestation of historical and systemic gender bias within academia.”