Risk reform, or remain within the academic monolith?
Sarahanne Field on ECRs stuck between a rock and hard place.
03 January 2023
The science reform movement has been gathering steam during the past decade. Reports of widespread questionable research practices have raised serious doubts about how trustworthy research findings – and our understanding of the world around us, which is built upon them – truly are. Proven cases of research fraud spanning decades-long careers (Verfaellie & McGwin, 2011) and poor reproducibility rates (Freedman et al., 2015) clearly demonstrate the impact of a business-as-usual approach to research.
This epistemic crisis has given birth to a social movement, which is particularly active in the behavioural sciences. Its goal of 'science reform' challenges every aspect of the traditional academic system, and the scientific community is watching as tensions play out between it and 'old academia.' While an emphasis on openness and transparency is a common thread, the reformers are a diverse and multi-faceted group, and opinions about the most fruitful or important approaches to changing science vary widely. Some sub-groups in the movement are pushing for changes to traditional academic systems like publishing. For instance, by developing new platforms and submission formats like ResearchEquals, leading the way in modular publishing, or registered reports, a journal submission type which introduces peer review of a research plan before data are collected.
Other groups focus on academic culture. Consider activist group 0.7 in the Netherlands, who protest unpaid overtime in academia, or the San Francisco Declaration on Research Assessment (with 21,923 signatories to date), which recommends against using journal-based metrics for assessments for promotions and funding, or the Leiden Manifesto which argues for qualitative and discipline-sensitive judgments of academic performance.
"If you do not see eye to eye with your supervisor, I suggest parking the bigger, practical changes you would like to make (e.g., data sharing), and focus instead on small changes you can make within the confines of your current position, and which are outside your supervisor's control."
Some individuals and groups aim to make academia more diverse and inclusive, so that researchers, their subjects, and the knowledge they co-produce most fully represent the colourful, complex landscape of humankind. For instance, many organisations and institutions are changing how they approach conference and workshop planning to allow for asynchronous participation, and to provide grants for childcare for those with children who would otherwise be unable to attend.
Inertia from the old guard
Despite these efforts and their potential benefits, reformers experience inertia from some who are not ready to yield practices and paradigms that have served their careers well. This old guard staunchly pushes back, defending methods and ideology that are from a previous era in science. They give arguments for why they should not have to adopt transparent, rigorous, and inclusive science. Registered reports are not appropriate for master's and graduate programs – they take too much time, and anyway, if you register our study plans before we do the study, we might get scooped! You shouldn't conduct a replication study; you know you won't get any credit for doing something someone else has done before! No… Those results don't support the hypothesis, do they? Let's collect some more data and see if we can get that p-value below 0.05... Fortunately, it is likely that these gatekeepers of academic tradition are in the minority. Unfortunately, it is likely that most of them supervise and mentor early-career researchers (ECRs).
These ECRs are in a difficult position. They are mired in the friction between the scientists they look up to and learn from, and the pull of a system that seems to be slowly, though surely, evolving. They are attracted to a movement which might just help empower and include them in research at the stage they are in now (rather than making them wait until they are tenured and middle-aged before they have credibility in their field). They are starting to realise that some of the old approaches may not yield the most reliable science. Their eyes are opening, and yet many of them have neither the freedom nor support to adopt slower, more transparent, reliable, and valid methods.
Intimidating reform ideologies
Another problematic factor for these ECRs is the reform movement community. Its ideals encompass inclusivity and diversity, and its practices may help improve science, but many would-be members are met with the reality of a community that can be less than welcoming. Some of its members are focused on the practices and tools but are less interested in making the reform culture safe and inclusive by rejecting bullying and exclusionary behavior, or by pushing against the systemic sexism and racism that are endemic in the current system. Others in the movement give the impression that the only way for people to truly participate in reform is to adopt every available practice (Field, 2022). Such things can be off-putting or intimidating, especially for ECRs who lack the freedom and support to reform their entire research pipeline.
"One of the best things about reform initiatives, though, is the number and variety of them. Find what's possible for you for now, find your people within the movement, and branch out when the time comes."
So, what can ECRs do to reform their own practices while staying in the good graces of supervisors that are probably well-meaning? How can they navigate entry into a community that might put up barriers of its own to make participation hard? Below, I share some recommendations for brave ECRs who are willing to work between the traditional system they are rooted in and the reforms they would like to explore.
Reforming your practice as an ECR
First, ECRs should be realistic and take stock of their current situation. It's easy to suggest that ECRs talk with their supervisors and convince them to get on board with new initiatives, but, realistically, this is not possible for many ECRs. They may share difficult dynamics with their supervisors, making communication hard, or might simply have supervisors who are not interested in hearing what they have to say. Some supervisors might be resistant because they are unaware of the benefits of some reforms.
If you do not see eye to eye with your supervisor, I suggest parking the bigger, practical changes you would like to make (e.g., data sharing), and focus instead on small changes you can make within the confines of your current position, and which are outside your supervisor's control. Look for a journal club or Open Science Community at your university (or find one you can join online) which will help you learn about the reforms you are interested in through the literature. My guess is that few supervisors would take issue with their student attending a journal club!
Learning to write transparently and reproducibly and cleaning up your research workflow is a useful skill you can hone somewhat independently of your supervisor. If you are not allowed to share data, perhaps start by pre-registering your study plan and sharing your materials on the Open Science Framework.
If your supervisor is open to talking, but not yet convinced, refer them to the arguments and resources presented in Robson and colleagues' preprint Nudging Open Science (2021). If you are interested in theory, but not sure where to start, I recommend Kathawalla and colleagues' (2021) guide to reform for students and their supervisors. Crüwell and colleagues' annotated reading list on open science practices is another excellent resource (2019). If paying for open access is the problem, upload a preprint, or seek out diamond open access journals who do not charge fees, or ones that waive processing fees (Lawson, 2015).
If your supervisor is open, but you are anxious about joining the reform community, I suggest trying to find your people. Peers within your faculty or discipline that can support your interest in open science and scientific reform that will help you get started. One of the central findings in my work on the reform movement is that there are a variety of different sub-groups within the broader community, each with their own approach and perspectives. If you look around on Twitter (for instance) or join something like the Nowhere Lab (founded by Priya Silverstein), it's likely you will be able to find a sub-group which reflects your priorities and ideals and can support you.
The open science buffet
Open and reform science is a buffet. Adopt what works for you (and your supervisor) until you have got more autonomy to choose for yourself. You do not need to adopt every new initiative that's out there. For one thing, there are way too many, say Whitaker and Guest (2020), and you'll exhaust yourself trying. Moreover, not everything fits everyone's research pipeline or discipline. Reformers and meta scientists are still learning how to reify our ideals, that is, to project the goals of improving science into appropriate practice, and much work has yet to be done to make reform initiatives widespread in the scientific community. For qualitative research especially, this has been a challenge that requires additional consideration (Haven et al., 2020).
Adopting science reform is not always easy or straightforward, and some ECRs face unique challenges as they find themselves torn between tradition and progress. One of the best things about reform initiatives, though, is the number and variety of them. Find what's possible for you for now, find your people within the movement, and branch out when the time comes.
About the author
Dr Sarahanne Field is a postdoctoral researcher at the center for science and technology studies at Leiden University, working on establishing a diverse, cross-disciplinary conceptualization of responsible research practice. She also studies the science reform movement, reflexivity, approaches to replication target selection and the unintended consequences of science reform.
Key sources
Crüwell, S., van Doorn, J., Etz, A., et al. (2019). Seven easy steps to open science: An annotated reading list. Zeitschrift für Psychologie, 227(4), 237–248.
Field, S.M. (2022). Charting the Constellation of Science Reform. OSF Preprints:
Freedman, L.P., Cockburn, I.M. & Simcoe, T.S. (2015). The economics of reproducibility in preclinical research. PLoS biology, 13(6), e1002165.
Haven, T.L., Errington, T.M., Gleditsch, K.S. et al. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods, 19.
Kathawalla, U.K., Silverstein, P. & Syed, M. (2021). Easing into open science: A guide for graduate students and their advisors. Collabra: Psychology, 7(1).
Lawson, S. (2015). Fee Waivers for Open Access Journals. Publications, 3. 155-167.
Robson, S.G., Baum, M.A., Beaudry, J.L., et al. (2021). Nudging Open Science. PsyArXiv.
Verfaellie, M. & McGwin, J. (2011). The case of Diederik Stapel. Psychological Science Agenda.
Whitaker, K. & Guest, O. (2020). # bropenscience is broken science. The Psychologist, 33, 34-37.
https://www.researchequals.com/
