This observation was written by Caroline Winter, with thanks to Inba Kehoe for her feedback and contributions.
At a glance:
| Title | The Review, Promotion, and Tenure Project |
| Creator | The Scholarly Commuications Lab (ScholCommLab) at Simon Fraser University |
| Publication Date | n/a |
| Keywords | open access, open scholarship, review, promotion, and tenure (RPT) |
What is the Review, Promotion, and Tenure Project?
Review, promotion, and tenure guidelines have been criticized by some as impediments to a transition to open scholarship (see The Film Paywall: The Business of Scholarship and Partner Response to Tri-Agency Statement of Principles on Digital Data Management). The Review, Promotion, and Tenure (RPT) Project analyzes RPT documents from universities in the US and Canada in order to determine how publishing in Open Access (OA) and closed-access venues are incentivized and how this incentivization affects open scholarship.
The RPT Project is based at the ScholCommLab and led by INKE Partnership member Juan Pablo Alperin, who is also Associate Faculty Director of Research of the Public Knowledge Project (PKP) and a member of the Canadian Institute for Studies in Publishing (CISP). The project stemmed from the ScholCommLab’s research on open scholarship, in which RPT guidelines were frequently cited as barriers to open scholarship (Fleerackers 2018; Schimanski and Alperin 2018, 3).
The RPT Project has two phases. In the first phase, the project team analyzed over 850 documents from 129 universities in the US and Canada outlining RPT guidelines at the institution and department level. The study’s method is described in depth in Alperin et al. 2018a, and the dataset has been published (Alperin et al. 2018b). The second phase of the project, currently underway, involves surveying faculty at those institutions about their interpretations of the guidelines and how they affect their work (Fleerackers 2018).
Three publications arising from the first phase of the project highlight its key findings, which include a misalignment between universities’ public missions and the incentive structures outlined in their RPT documents, a recognition of high-quality research as the most significant factor in RPT evaluations and the challenge of measuring quality, and the extent to which metrics such as the Journal Impact Factor (JIF) are referred to in RPT documents, and to what effect.
In the first publication, “How Significant are the Public Dimensions of Faculty Work in Review, Promotion, And Tenure Documents?,” Alperin, Carol Muñoz Nieves, Lesley Schimanski, Gustavo Fischman, Meredith Niles, and Erin McKiernan read the 850 RPT documents gathered as part of the first phase of the project against universities’ stated public missions. They identify “organizational tensions” between these public missions and the types of research outputs that are most valued, based on how those outputs are rewarded (2018a). They find that, of the three dimensions of faculty work—research, teaching, and service—work that benefits the public is most often classified as service and thus valued least for RPT purposes. Work that benefits the academic community specifically—such as publishing journal articles—is valued most. Because OA journal articles fulfill both the public mission of the university and serve the academic community, Alperin et al. identify the lack of incentive for this mode of publishing as a significant “missed opportunity” (2018a, 25).
The second outcome from the RPT project, Schimanski and Alperin’s “The Evaluation of Scholarship in Academic Promotion and Tenure Processes: Past, Present, and Future” (2018), surveys existing literature about RPT procedures at universities in the US and Canada. Like Alperin et al. (2018a), this survey finds that research tends to weigh most heavily in RPT evaluations, even though many faculty spend most of their time on teaching and service. The authors note that more transparency about how research is evaluated could solve some of the problems facing scholarly communication and academia more generally, and to that end, Schimanski and Alperin analyze existing studies of RPT procedures in order to help researchers “understand the explicit rules of the game” (2018, 3). The article points out that, although the perceived quality of research is important for the evaluation process, due to a lack of clear and consistent methods for evaluating quality proxies are often used instead, including the prestige of the publication venue and metrics such as the journal impact factor, even though these are widely acknowledged to be problematic.
Finally, in the third RPT Project publication, “Use of the Journal Impact Factor in Academic Review, Promotion, and Tenure Evaluations,” (2019), McKiernan, Schimanski, Nieves, L. Matthias, Niles, and Alperin investigate the frequency and context in which the Journal Impact Factor (JIF) metric is mentioned in the Project’s corpus of RPT documents. They report that the JIF is widely used as a proxy metric for evaluating research quality, both explicitly and implicitly, even though the evidence suggests no relationship between a publication’s JIF and the quality of the research it publishes. The article echoes calls from DORA and other similar initiatives for more “responsible use of metrics” and the use of alt-metrics for RPT.
Reception of the Project
Inba Kehoe of the University of Victoria Libraries points out that funded research and the publications in scholarly journals are prestigious and dominant in RPT processes because research dollars and numbers of publications are easy to measure and count. Peer review is at the core of this practice and an important filter to recognize academic quality.
As noted on the ScholCommLab’s Media page, the RPT Project’s findings have been covered widely in the academic press, including the Peer J Blog, Times Higher Education, the Chronicle of Higher Education in 2018 and 2019, Inside Higher Ed, and Nature. In a book chapter about public scholarship, 2018–19 University of Victoria Honorary Resident Wikipedian Erin Glass notes that “Connecting to the public is a key part of the mission of higher education.” (1), and cites the RPT project’s findings as important to the conversation about the discrepancy between universities’ missions and practices.
Relevance to Open Scholarship
The findings of the RPT project so far are significant in the context of Open Scholarship as a whole because, as Schimanski and Alperin (2018) put it, “Chief amongst these barriers [to changing and improving academia] are the incentive structures currently in place for faculty career advancement” (3). Although more and more universities are adopting OA practices, RPT guidelines are slow to catch up (Fleerackers 2018). OA is mentioned in only 5% of the RPT documents surveyed and often in a negative way, usually as a warning about predatory OA journals; and none of the RPT documents surveyed encourage faculty to publish their work OA (Alperin et al. 2018a, Fleerackers 2018). Because RPT practices inform where and how faculty publish their work, the project team argues that “RPT reform may be one of the most successful ways to effect change in the academic system” (Schimanski and Alperin 2018, 10). Kehoe contends that open scholarship practices demand a re-imagining of not only what constitutes scholarly work, but appropriate criteria for assessment. A one size fits all model will not work under the circumstances. Finding equitable way to assess and credit publicly engaged scholars, alongside existing scholarly practice will benefit all scholars in higher education.
Works Cited
Alperin, Juan Pablo, Carol Muñoz Nieves, Lesley Schimanski, Gustavo E. Fischman, Meredit T. Niles, and Erin C. McKiernan. 2018a. “How Significant are the Public Dimensions of Faculty Work in Review, Promotion, and Tenure Documents? ELife. https://elifesciences.org/articles/42254.
Alperin, Juan Pablo, Carol Muñoz Nieves, Lesley Schimanski, Erin C. McKiernan, and Meredith T. Niles. 2018b. Terms and Concepts found in Tenure and Promotion Guidelines from the US and Canada, Version 3.0. Harvard Dataverse, https://doi.org/10.7910/DVN/VY4TJE.
F[l]eerackers, Alice. 2018. “Preliminary Findings from the Review, Promotion, and Tenure Study.” ScholCommLab Blog, 30 May 2018, https://www.scholcommlab.ca/2018/05/30/preliminary-findings-from-the-review-promotion-and-tenure-study/.
Glass, Erin Rose, and Micah Vandegrift. “Public Scholarship in Practice and Philosophy” (pre-publication) CORE, Humanities Commons, 2018, http://dx.doi.org/10.17613/g64d-gd16.
McKiernan, Erin C., Carol Muñoz Nieves, L. Matthias, Meredith T. Niles, and Juan Pablo Alperin. 2019. “Use of the Journal Impact Factor in Academic Review, Promotion, and Tenure Evaluations.” PeerJ Preprints 7:e27638v2, 10.7287/peerj.preprints.27638v2.
Schimanski, Lesley A. and Juan Pablo Alperin. 2018. “The Evaluation of Scholarship in Academic Promotion and Tenure Processes: Past, Present, and Future” [version 1; peer reviewed: 2 approved]. F1000Research 7:1605, https://f1000research.com/articles/7-1605/v1.