一项分析了科学同行评审的有效性的研究提出，同行评审在预测“良好”的论文方面是有效的，但是可能难以识别出卓越的和/或突破性的研究。

Kyle Siler及其同事使用了2003年和2004年提交给3个主要的医学期刊——《内科学年鉴》、《英国医学杂志》和《柳叶刀》——的1008份手稿的一个数据集合，从而评估获得了编辑和同行评审者有不同评价的论文的引用结果的差异。

这组作者发现，编辑退稿的手稿——编辑认为不值得进行同行评审——比在退稿之前发给同行评审的论文在最终发表之后获得了较少的引用数量。此外，在所有被接受和被退稿的手稿中，同行评审者打分较低的手稿在最终发表后获得了相对较少的引用数量。然而，这组作者发现这3份医学期刊拒绝了许多高引用的手稿，包括14篇引用数量最多的手稿，而这14篇手稿中的12篇是被编辑退稿的。

这组作者得出结论说，编辑和同行评审者通常会、但并不总是会做出关于哪些手稿应该发表和退稿的明智的决定，并且提出科学同行评审可能难以识别出非传统的和/或卓越的研究。(来源：生物360)

Measuring the effectiveness of scientific gatekeeping

Abstract  Peer review is the main institution responsible for the evaluation and gestation of scientific research. Although peer review is widely seen as vital to scientific evaluation, anecdotal evidence abounds of gatekeeping mistakes in leading journals, such as rejecting seminal contributions or accepting mediocre submissions. Systematic evidence regarding the effectiveness—or lack thereof—of scientific gatekeeping is scant, largely because access to rejected manuscripts from journals is rarely available. Using a dataset of 1,008 manuscripts submitted to three elite medical journals, we show differences in citation outcomes for articles that received different appraisals from editors and peer reviewers. Among rejected articles, desk-rejected manuscripts, deemed as unworthy of peer review by editors, received fewer citations than those sent for peer review. Among both rejected and accepted articles, manuscripts with lower scores from peer reviewers received relatively fewer citations when they were eventually published. However, hindsight reveals numerous questionable gatekeeping decisions. Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were desk-rejected. This finding raises concerns regarding whether peer review is ill-suited to recognize and gestate the most impactful ideas and research. Despite this finding, results show that in our case studies, on the whole, there was value added in peer review. Editors and peer reviewers generally—but not always—made good decisions regarding the identification and promotion of quality in scientific manuscripts.

原文链接：http://www.pnas.org/content/early/2014/12/17/1418218112.full.pdf+html