Peerless Science

Daryl E. Chubin is Senior Analyst in the Science, Education, and Transportation Program, Office of Technology Assessment, U.S. Congress. Edward J. Hackett is Assistant Professor in the Department of Science and Technology Studies at Rensselaer Polytechnic Institute.

"Peer review is the principle on which the internal governance system of science has traditionally depended. During the past ten years there has been a great deal of evidence suggesting that that assumption is no longer valid (if it ever was), and that many of the strains in the science-government relationship in the US are traceable either to the assumption itself or to the ways it is implemented. Chubin and Hackett examine the assumption and its implementation from different perspectives and explore its role in the science-government relationship."The authors display the ability to take peer review and use it as a springboard to illuminate a central issue of our time: the relationship between the conduct of science and the larger society in the US in the waning years of this century. The writing itself — colorful, interesting, heartfelt — is surprising and refreshing." — William A. Blanpied, National Science Foundation

• Preface and Acknowledgments 1. The Centrality of Peer Review What Is "Peer Review"?Symbolism and ChauvinismApproach to a DialogueFive Axioms about the Culture of ScienceThe Policy ContextStudying Grants Peer ReviewSummary: A Study of Policy and Practice 2. Peer Review in Theory and Practice Peer Review: Origins and Current PracticesPeer Review at the National Institutes of HealthPeer Review at the National Science FoundationTrends in Proposal Volume and Award Rates Criticisms of Peer Review: More Than Sour GrapesRustum Roy's CritiqueRoy Redux Criteria for Evaluating Peer ReviewConclusions 3. Funding Success and Failure Entering the Black BoxAgency-Sponsored StudiesAgency-Funded StudiesIndependent Studies Litigation and Science: The Story of a Social SurveyObtaining the Sample of ScientistsThe AppealLessons from Litigation Empirical Studies of Grants Peer ReviewSurveys of ScientistsSuccess in the Pursuit of Research SupportThe Consequences of FailureAttitudes toward Peer Review Competition for Research SupportUpward Creep and Resubmission: A Protracted ProcessAnomie and Reform Inferences: Peer Review and Conflicts of InterestConclusions 4. Peer Review and the Printed Word The Purposes of Scientific Publication and Journal Peer ReviewNew Burdens"Truth" as Consequence Principles and Practices: The Tensions of Peer ReviewJournal PracticesStudies of Journal Peer ReviewThe Dangers of Studying Journal PracticesWhat the Studies Tell Us An Analysis of Referee CommentaryA Sample of Referee-Editor DiscourseThe Rhetoric of RefereesToward a Theory of Refereeing Vignette" The Sanctity of Journal Peer Review and a Conspiracy of Ignorance Sober ConsiderationsConclusions 5. Scientific Malpractice and the Politics of Knowledge Malpractice DefinedCan Peer Review Help? The Emergence of Public ScienceMalpractice as a Grappling with NormsFraud in Research: The Social Structure of Scapegoating Misconduct and Public ScienceThe Mainstreaming of DisputeOut of the Nursery, Into the Night Earmarking and the Pork BarrelAutonomy, Accountability, and the Politics That Intervene 6. Augmenting Peer Review: The Place of Research Evaluation Bibliometrics as Research Evaluation: The PromiseThe First Generation of Bibliometrics (1961-1974)The Second Generation (1975-Present)Lessons LearnedLessons Applied Evaluative Bibliometrics and BeyondConverging Partial IndicatorsQualitative Scientometrics Challenges and New DirectionsScience Push-Application PullBroker Roles Conclusions 7. Peer Review and Unauthorized Science Policy Impediments and Nagging Issues Reforming Peer Review: Slouching Toward IconoclasmModest Improvements at the MarginChanging Roles and RulesToward a Process of Reform Meta-Analysis, The Science Critic, and Science PolicyThe Science Critic in ActionPolicy Research and the Public Interest Science Policy and the Flywheel of Peer ReviewInventing Tools for Perception and Foresight Appendix: Survey of NCI Applicants Notes Glossary Index