Peer review In An Era Of Evaluation: Understanding The Practice Of Gatekeeping In Academia

Contents
Notes on Contributors
List of Figures
List of Tables
Part I: Peer Review: Introduction and Its Context
	1: Peer Review in Academia
		Introduction
		The Many Face(t)s of Peer-Review Practices
			The Interplay of Primary and Secondary Peer Review
			Sites and Reviewer Tasks and Roles
			Power-making in the Selection of Expertise
			The Academic Genres of Peer Review
			Criteria and the Professional Judgement of Quality
		The Autonomy of a Profession and a Challenged Contract
		Why Study Peer Review?
		State of the Art of Research on Peer Review
		The Structure and Content of the Volume
		References
	2: Hierarchies and Universal Inclusion in Scientific Communities
		Tensions and Contradictions in Contemporary Society
		Universal Inclusion
		Publication as the Elementary ‘Unit-Act’ of the System of Science
		Authorship of Publications as the Form of Inclusion in Science
		Citations as the Internal Structure of Publications
		Reading and Writing in Scientific Communities: The Hierarchy of Authorship
		The Emergence of Peer Review: The Hierarchy of Readership
		The Two Hierarchies: Authorship and Readership
		The Third and Fourth Hierarchy: Hierarchy of Publication Places and Hierarchy of Recruitment for Co-authorship
		Two Modalities of Quality Control in Science
			Anticipatory, Centralized Control by Scientific Elites
			Post-hoc, Decentralized Market Control Based on Universal Inclusion
		Concluding Remarks
		References
	3: “Disciplining” Educational Research in the Twentieth Century
		Introduction
		Reviewers and Authors
			Initial Expectations
			Further Expansion
		Authors and Reviewers
			Community of Peers
			Blind Peer Review
		Papers and References
			Suggestions for Contributors
			Citation Consciousness
		Conclusion
		References
Part II: Peer Review and the Higher Education Evaluation Machinery
	4: Gatekeepers on Campus: Peer Review in Quality Assurance of Higher Education Institutions
		Which Gates Do Peers Keep?
		Quality of Higher Education Institutions
		Quality of Performance and Performance Indicators
		Strengths and Weaknesses of Performance Indicators
		Quality of the Organisation and Peer Review
		Strengths and Weaknesses of Peer Review in Quality Assurance
		Persistent Dilemmas: Instead of a Conclusion
		References
	5: The Many Faces of Peer Review
		Introduction
		Classical Peer Review as One of Several Evaluation Models
		Informed and Standards-Based Peer Review
		Modified Peer Review
		Extended Peer Review
		Peer Review in the Educational Field
		Conclusion and Discussion
		References
	6: Your Brother’s Gatekeeper: How Effects of Evaluation Machineries in Research Are Sometimes Enhanced
		Introduction
		Metrics and Ambiguity
		A Case Study of the Danish Bibliometric Indicator
		Vignettes
		Discussion and Conclusion
		References
	7: Peer Review in Public Administration: The Case of the Swedish Higher Education Authority
		The National System for Quality Assurance of Higher Education in Sweden
		The History
		Critique
		2011: A New System Is Launched
		Peer Review
		Sources
		The Role of the Peers and the Method of Peer Review
			Early Assessments 1997–1998
			Later Assessments 2016–2017
		Grasping the Concept of Result
			The Early Assessments 1997–1998
			Later Assessments 2016–2017
			Collegiality as a Form of Governance
			Peer Review in the National Quality Assurance System
		Sources
			Directions on Assessments
			Reports Describing Whole Quality Assurance Systems
			Guidelines for HEIs and Peers
			Reports on Reviews Conducted Within the Framework of National Systems for Quality Assurance of Higher Education
			Accounts of Government Assignments
			Meta-analyses of the Authority’s/the Authority’s Own Reviews
			Other Reports
			Other
			References
	8: Performance-Based Evaluation Metrics: Influence at the Macro, Meso, and Micro Level
		Introduction
		Algorithmic Historiography and the Birth of Scientometrics
			Bibliometrics for the Evaluation of Research
			The Citation as Mediator: The Performativity of “Being Cited”
			Consequences for Research Policy
			Co-production of Science and Society
		Performance-Based Research Allocation Models at Three Levels
			PRFS at the Macro Level: A Comparison Between the Norwegian and Swedish Systems
			At the Meso Level, Within Universities
			At the Micro Level, Individual Researchers
		Conclusions: The Performativity of Performance-Based Research Funding on Different Levels
		References
	9: Peer Advocacy: Expressions of Loyalty in Peer Review
		Introduction
		Aims and Uses of Peer Review
		Advocacy in Academia
		Cases of Advocacy
			Case 1: National Reviews of Subjects and Programmes
			Case 2: Institutional Review of an Academic Development Unit
			Case 3: External Evaluators Appointed to Promote an Application for University Status
			Case 4: HR Competencies in Academic Appointment Processes
		Discussion
		Conclusions and Recommendations
		References
	10: Is Peer Review Fit for Purpose?
		Introduction
		Methodology
		Refereed Journal Articles
		The Assessment of Doctoral Degrees
		The UK Research Excellence Framework (REF)
		Conclusion
		References
Part III: Specificities of Different Peer-Review Practices
	11: Peer Review in Academic Promotion of Excellent Teachers
		Introduction
		Career and Reward Structure in Swedish Higher Education
		Gatekeeping in the Promotion of ‘Distinguished University Teachers’
			The Process of Promoting Teaching Excellence
			The Mandatory Content of the Application Dossier
			Faculty Guidelines and Candidate Applications
		Judgement and Legitimation in the Making of ‘Distinguished University Teachers’
		The (E)valuation of the ‘Distinguished University Teacher’: The Judgement
			The Scholarly Judgement of Criteria and Content
			The Scholarly Judgement of Evidence
		Justification of the (E)valuation: The Legitimation
			Justification In-between Standardization and Professional Judgement
			Justification Through Mandated Intertextuality
			Justification Through Scholarly Judgement
		The Intersection of Promotion, Peer Review, and Teaching Excellence
			The Promotion of ‘Distinguished University Teachers’: The Same, But Different
			A Latecomer and an Emergent Game Changer?
		References
	12: Peers in Systematic Review: Gate Keeping Understandings of Research in the Field
		Introduction
		Peers in Academic Publishing and Systematic Review
		Analytical Framework
		Method
		Examples of Peers in three Review Processes
			Swedish Institute for Educational Research
			The Norwegian Knowledge Centre for Education
			Danish Clearinghouse for Educational Research (DCU)
			Peers in the Systematic Review Processes
		Concluding Remarks
		References
			Selected Systematic Review Technical Reports Analysed in the Study
			The Example Reports Used for the In-depth Reading Are Underlined
	13: The Decision-Making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome
		Introduction
		Data Sources and Methods
		Analysis of the Grant-Review Practices of RCN
			The Various Models
			Overview of Proposals and Grants
			The Quality Criteria in the Assessments
			Considerations Other than Research Quality in the Assessments
			The Effects of the Various Ranking Processes
			Important Organizational Factors Affecting Ranking
		Conclusions
		Appendix: The Review Criteria in the Different RCN Divisions
			The RCN Divisions’ guidelines listed the following review criteria:
		References
	14: Typecasting in the Recruitment of Full Professors
		Introduction
		The Field of Educational Science(s)
		Recruitment Procedures—A National Context
		Typecasting in the Recruitment of Full Professors
		The Intellectual and Social Organization of Educational Science(s)
			The Emergence of a Heterogeneous Field
			The Selection of Peers
			The Mobilization of Job Categories
			The Intersection Between Institutional Demands and Scientific Values
			Eliminatory Criteria, Signs and Retrospective Judgments
		Conclusive Notes
			Typecasting as a Device of Peer Review in the Formation of Education Science(s)
			The Professor of Education Science(s) in Late Modernity
		References
	15: Assessing Academic Careers: The Peer Review of Professorial Candidates
		Introduction
			Picking the Best: Peer Review in Assessment Procedures
		Analysing Referee Reports
		Judgement Devices and Intellectual Structure
		Merits and Their Assessment: Five Main Themes
		Authorship, or the Reading of By-Lines
			Publication Prestige and the Importance of Articles in ‘Top-Journals’
			Temporality. The Importance of a ‘Positive Trajectory’.
			Racing for the Prize: Reputation Through Awards and Citations
			Boundary Keeping and the Shielding of Academic Markets
		Discussion
		References
	16: Bureaucratic, Professional and Managerial Power in University Tenure Track Recruitment
		Introduction
		Selection and Recruitment of Candidates: A Regulative Perspective
		Recruitment and Tenure Track Systems in Finland
		Collection and Analysis of Data
		Influence and Actors in Recruitment: Bureaucratic, Managerial and Professional Powers
			The Recruitment Process in Tenure Track
		Defining the Position
		Advertisement and Application
		Evaluation
		Decision and Justification
		TUT Findings
		Tenure Call
			Setting up the Working Group
			Evaluation
			Decision and Justification
		UTA Findings
		Tenure Call
			Evaluation
			Decision and Justification
		Conclusions
		References