A Rigorous Peer Review: Why You Should Care
Peer review, it’s a term you’ve probably heard numerous times. But what exactly is it? As the name implies, peer review is the evaluation of work by others in the same field as a means to ensure high quality standards. Two main examples are: 1) peer review used by the NIH and other agencies, including RSRT, to evaluate proposals that are submitted for funding 2) peer review to evaluate manuscripts submitted to journals for publication. A rigorous and unbiased peer review is critical to the scientific process.
At RSRT we implement a peer review process to ensure we spend our supporters’ donations wisely and that our projects move us steadily forward towards the achievement of our mission – a cure!
Over the years I’ve spearheaded review for over 1000 proposals. I learned long ago that the peer review process is only as strong as the reviewers themselves. Besides the obvious criteria that reviewers have strong domain expertise it is also crucial to ensure maximum objectivity to guard against conflicts of interest, or inherent biases, for or against the research being proposed.
RSRT maintains strong relationships with many dozens of scientists and clinicians in academia and industry who willingly volunteer their time and contribute their expertise to help assure the research we fund is optimally designed and resourced.
We typically recruit 4 to 6 reviewers for every proposal and provide one month to submit their frank appraisal. Beyond evaluating the scientific rigor, reviewers provide constructive advice for modifications to improve the proposal. Additional considerations include the strength of the institution where the project will be undertaken, whether investigators have the necessary resources and equipment and are surrounded by the intellectual capital required to succeed. Curriculum vitae of the principal investigator as well as lab members who will play key roles are also reviewed to ensure the team has a proven track record of success. Feedback from our own internal scientific team, Dr. Carpenter, Dr. von Hehn, as well as my own experience through two decades of immersion in the Rett field, are also considered in the analysis. Reviewer comments are anonymously shared with the applicant, rebuttals encouraged, and scientific discourse fostered.
After review, three scenarios occur: 1) consensus among reviewers that the project should be funded as is, 2) consensus that project should not be funded, or 3) somewhere in between.
In many cases reviewers’ comments and recommendations spur an iterative process between the applicant and the RSRT team resulting in a much-strengthened proposal. Projects deemed worthy of funding are taken to our Board of Trustees for discussion and a vote.
Peer review is a process that RSRT takes extremely seriously. And unfortunately it can be undermined in a number of ways including selecting soft reviewers or reviewers who do not have adequate expertise. We seek out tough reviewers who will ask the difficult questions of our applicants up front. We owe this much to every individual suffering from Rett Syndrome and MECP2 disorders, as well as to the families and supporters who fundraise for RSRT.
