Engaging in Open Science practices as an ECR

In recent years, there has been a push to improve rigour and replicability of results from published studies in the behavioural sciences in response to the 'replication crisis'. Although this issue is by no means unique to psychology, many of the concerns have focused on this discipline and evidence of low replication rates have been consistently found in psychological research (Renkewitz & Heene, 2019). For example, it has been estimated that only 36% of significant results published in psychological journals are replicable (Open Science Collaboration, 2015).

Widespread adoption of open science methodologies has been proposed as a solution to improve the robustness of research findings. Open science is an overarching term which comprises "principles and behaviours that promote transparent, credible, reproducible and accessible science" (Parsons et al., 2022). 

Open science has no formal checklist, with multiple taxonomies proposed, and encompasses a range of activities, such as: sharing of analysis code, data and research materials; pre-prints and open-access publishing; pre-registration of research protocols and/or hypotheses and open peer-review. Importantly, open science is not intended to be prescriptive; relevant structures and processes will vary across different subject areas within psychological research. For example, there remain practical questions about how open data requirements may apply to the use of qualitative methodologies (Prosser et al., 2022).

Open science has been described as a paradigm shift, reflecting the attitude that it is a case of 'not if, but when'. Multiple reporting guidelines now require statements regarding data accessibility and pre-registration (e.g. PRISMA, CONSORT) and there are several major funding bodies (e.g. MQ Mental Health, Wellcome, UKRI) which require or encourage awardees to pursue open science practices, such as sharing pre-prints or publishing open access. 

Whilst some have criticised psychology for being relatively slow to adopt open science methodologies, the number of pre-prints and pre-registrations published or deposited on popular psychology and social science servers (e.g., PsyArXiv) has been rapidly growing (Nosek & Lindsay, 2018). However, despite the large amount of momentum behind the open science movement, there are still considerable steps required to ensure standardisation of these values and practices.

ECRs and Open Science – Barriers

Supporting the future of open science requires identifying current challenges and barriers faced by the individuals pursuing these practices. A large proportion of the open science movement has been propelled by communities of ECRs (Allen & Mehler, 2019), who will also have a crucial role to play in the future trajectory of open science in their role as current or prospective editors, peer-reviewers, lecturers, mentors, research group leaders and principal investigators. 

Numerous challenges to conducting open science are common across different settings and career stages (e.g. time commitment, incentive structures). However, some challenges are exacerbated by being an ECR and some are unique to ECRs, due to the acknowledged precarity of the roles they occupy (e.g., short-term contracts).

Peer and collaborator support

Engaging in open research practices as an ECR relies not only on your own enthusiasm to do so, but also on having wider structural support. For example, when working on a project with more senior researchers, their opinion on the necessity or importance of pre-registration, code sharing, or open data may have a significant impact on whether you can do these things or not. 

Depending on the familiarity of the team with these open science practices, collaborators may also need to access training or support to be able to contribute to the wider project. Even within a team that values open science principles, there may be individual differences in the level of commitment each person wants to, or can, provide, which might conflict with your own ideals. It can help to have these discussions at the inception of a new project, to make your expectations clear.

Time commitment

Open science practices can take up a considerable amount of your time and energy, especially when you are new to them. A common rebuttal to this is that open science practices, such as pre-registration, represent a 'front-loading' exercise (i.e. time will be recouped). However, it has been noted that this is not always the case, and these methods can make research take substantially longer (Allen & Mehler, 2019). 

Based on our experience as ECRs seeking to do open science, we have found that applying multiple open science practices (including data simulation, pre-registration and code quality control) can have the effect of creating larger-scale projects which require more time than traditional research projects. This is especially apparent when applying these methods for the first time, or applying them to a different type of project, as less templates and resources will be available.

For the most part, we would argue that the benefits are worth it. However, as an ECR you might be particularly reliant on others who are evaluating your work (e.g., hiring committees, thesis examiners) to also value the importance of open principles over say, quantity of work produced. Given that academia still works on a 'publish or perish' principle, choosing to focus on openness, transparency and rigour can sometimes feel like working 'against' the current system.

Scooping

Other issues include the pressure in research to be the 'first' to investigate a phenomenon, something which could be threatened by widely sharing your research plans ahead of actually doing them. Concerns that other researchers could 'scoop' a big idea could be partially alleviated by registering work on the Open Science Framework and putting an embargo date to keep the registration private until a certain date. Further, sharing research plans online can actively avoid cases of unintentional overlap in projects, and trust in peers can increase willingness to share (Laine, 2017).  

Resources

Another significant barrier is money. Open access fees for a paper can cost up to £8000, depending on the journal. Some universities have sources of money to cover these fees, which is always worth investigating, but not all do. Particularly for those from low-income countries, this can be a huge barrier to publishing papers open access. However, many other open science practices are completely free, such as pre-registration, sharing code or uploading pre-prints. Open science is not an 'all or nothing' exercise. Beginning by implementing just one open science practice is a great start, and it is entirely possible to choose an option that has no cost.

Not all ECRs will face these structural problems when trying to engage with open science, however these could still be factors to keep in mind before joining a new lab group, or to instigate discussion in your current group.

Personal barriers

Wider structural issues aren't the only type of barrier ECRs might face to doing open research. At an early career stage, you are likely to be learning and trying out new experimental methods, data types, and/or analysis types. You might have no prior experience of these methods, and some things feel easier to learn 'by doing'. 

This isn't always a comfortable position from which to share your research plans, in advance and in detail, with the world. What if the analysis plan that you pre-registered turns out not to make much sense once you've had more training? What if you share your code and someone finds a fatal flaw? What if you have to deviate from your trial protocol on several occasions? 

Even though these situations are common, and often turn out to be an invaluable learning experience, the initial fear of failure or embarrassment can be a real barrier to wanting to engage in these practices. Submitting registered reports that are formally peer-reviewed could increase confidence in your proposed research plans, however this can be a time-consuming addition to a project, and currently not all journals offer registered reports. Other factors such as working in a second language or working without sufficient training or support can exacerbate these issues.

Recommendations

We want to conclude by offering some brief advice on ways to increase opportunities to engage in open science as an ECR, based on our own experiences. These recommendations are not intended as an exhaustive list, simply a starting point for your own open science journey.

• Start early, start small… with incremental open science changes to your workflow, and planning these in advance to allow extra time. For example: (i) do what you can to make your analysis data and code openly accessible on a repository (e.g., GitHub); (ii) create supplementary materials to give detailed information on methodology; (iii) work with collaborators to pre-register a study protocol and track any deviations.
• Join a community: There are multiple national and international communities of researchers interested in open science. Joining these often provides access to free training courses, journal clubs or learning materials (e.g., FORRT, ReproducibiliTEA). These are a great way to network, gain support from other researchers and access resources. There are also online blogs and web pages to follow, which post about up-to-date developments in the open science community (e.g., Centre for Open Science).
• Use free and low-cost alternatives: For example, you can freely deposit pre-registrations on the Open Science Framework (OSF) or pre-prints can be deposited freely on online servers (e.g., PsyArXiv) to achieve green open-access. SHERPA/RoMEO can help you find guidance on journal policies on self-archiving. These versions can then be shared amongst academic and non-academic communities for feedback and engagement (e.g., via Twitter).
• Embrace mistakes: We believe engaging in open science requires a shift towards a 'growth mindset', to contribute to better scientific discussion and also to best support researcher wellbeing and development. Although there is no easy fix for this, we recommend reading Bishop (2018) for further personal exploration of this topic.
• Access training and support: There are various free training resources accessible online which give an overview of different stages of the open science life cycle, or guidance on using specific softwares (e.g., EdX, The Turing Way, The Open Science Training Handbook). Journals sometimes have options available to help support open-access publications. For example, fee-waivers for researchers at specific institutions or from specific regions (e.g.,Research4Life) or institutional funding pots for open-access publications or public engagement activities (e.g., focus groups). Some APA journals offer 'open science badges' to provide supporting evidence for your engagement in open science.

About the authors

Chloe Burke and Rachel Lees are PhD students based in the Addiction and Mental Health Group at the University of Bath. Chloe's research aims to better understand the causal role of cannabis and tobacco use for risk of developing mental ill-health. Rachel's research investigates the development and treatment of cannabis use disorder.

Further information

For a reflection on open science initiatives that can support ECRs engaging in open science, including the Framework for Open and Reproducible Research Training, see the article by Flavio Azevedo in this issue. 

Many of the terms used in this article can be found in the FORRT community glossary (Parsons et al., 2022).

Key sources

Allen, C. & Mehler, D. M. (2019). Open science challenges, benefits and tips in early career and beyond. PLoS Biology, 17(5), e3000246.

Bishop, D. V. (2018). Fallibility in science: Responding to errors in the work of oneself and others. Advances in Methods and Practices in Psychological Science, 1(3), 432-438.

Laine, H. (2017). Afraid of scooping: Case study on researcher strategies against fear of scooping in the context of open science. Data Science Journal.

Nosek, B.A. & Lindsay, D.S. (2018). Preregistration becoming the norm in psychological science. APS Observer, 31(3). https://www.psychologicalscience.org/observer/preregistration-becoming-the-norm-in-psychological-science

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.

Parsons, S., Azevedo, F., Elsherif, M.M., et al. (2022). A community-sourced glossary of open scholarship terms. Nature Human Behaviour, 6(3), 312-318. 

Prosser, A. M. B., Hamshaw, R., Meyer, J., et al. (2022). When open data closes the door: A critical examination of the past, present and the potential future for open data guidelines in journals. British Journal of Social Psychology.

Renkewitz, F. & Heene, M. (2019). The replication crisis and open science in psychology: Methodological challenges and developments. Zeitschrift für Psychologie, 227(4), 233–236.