Moving Psychological Science Forward in Europe

'The scientific world has changed', claimed Professor Daryl O'Connor (University of Leeds, and Chair of the British Psychological Society's Research Board), opening this symposium on 'Moving Psychological Science Forward in Europe'. Since the publication of the Open Science Collaboration's 2015 article in Science, a replication of 100 psychology studies in which only 39 per cent of effects replicated the original findings, psychologists have been forced to think about how to move the field forward and do better science.

'This was a seminal, breathtaking article, but also devastating in its conclusions', said O'Connor, who convened the symposium to bring together key academics and stakeholders involved in the dissemination of psychological knowledge from across Europe. So, how to remove the 'threats to reproducibility'?

O'Connor observed that science is a behaviour, and that given our expertise in behaviour change, psychologists should be well placed to change their own questionable practices such as p-hacking and HARKing. With reference to Susan Michie and colleagues' COM-B model of behaviour change and behaviour change wheel, O'Connor stressed the importance that 'all levels of the behaviour needed to be targeted'.

A theme that cut across the whole symposium was that to truly move the discipline forward we must focus not only on individual researchers, but also change the way that we train psychologists to undertake research, and the incentive structures within the discipline. O'Connor's talk struck an optimistic tone, highlighting many of the initiatives that are already in place to remove threats to reproducibility. These include pre-registration of studies, and registered reports, which are now a publication method across the entire portfolio of British Psychological Society journals. 'We are leading the way, not just for psychological science, but for science more generally'.

The idea that both 'bottom-up' and 'top-down' approaches to science are needed was also a key message which Andrea Abele-Brehm (Universität Erlangen-Nürnberg, Germany) emphasised during her talk. Professor Abele-Brehm highlighted the need to better teach undergraduate students to develop theories, open research practices and methodologies including how to appropriately conduct direct and conceptual replications. But Abele-Brehm also discussed an evaluation of data sharing her and colleagues have conducted (in press in Social Psychology). This showed that Professors have the least positive attitudes towards data sharing of all career stages, but also the least fears – suggesting both opportunities and barriers to changing behaviours in the most senior academics.

Abele-Brehm mentioned that within the discipline there are too many small-scale theories with overlapping constructs, underpowered studies, questionable research practices, insufficient openness and insufficient collaboration across labs. These are all threats to reproducibility. Universities could do more, by making a commitment to open science practices a selection criterion when appointing academic staff. Publishers and research funders could shift incentive structures, to ensure that researchers are required to engage in open science practices.

Professor Remo Job (University of Trento, Italy) also stressed the importance of providing appropriate training in theory development at undergraduate level. He suggested that we needed to shift the mindset of students away from thinking that individual studies provide definitive insight; there needs to be greater emphasis on meta-analysis on undergraduate programmes. Job also stated that 'teaching replication can be exciting and fun, but adequate supervision is crucial'. There also needs to be a shift in teaching from focusing on whether an effect is statistically significant, to theory development and methods.

Rebecca Harkin (Wiley) then spoke about how the publishers are helping researchers to better engage with open research practices. Harkin said that it is now widely acknowledged among publishers and researchers that psychology has a replication crisis: 'the question isn't "why" anymore, it's "how"'. Data sharing is one initiative which is now becoming ubiquitous, but it's a challenge to overcome concerns that researchers have with data sharing, such as fears of being 'scooped'. All journals in the BPS portfolio now 'expect' data sharing, meaning that authors need to provide a data sharing statement stating how their data can be accessed, and provide reasons why it isn't publicly available if it isn't being shared. Harkin also described Wiley's transparent peer-review pilot, which is designed to explore how peer-review can be opened up in a smart, sensible, flexible and friction-free way. Reviews and author responses are published alongside the paper, and reviewers are encouraged to sign their reviews to increase transparency and openness.

The final speaker was The Psychologist's Managing Editor, Dr Jon Sutton. He opened with a quote from the TV series Game of Thrones: 'There's nothing more powerful than a good story'. A particularly potent aspect of such stories is change. Could we help people to tell a story of change about their work, and how the accuracy, openness and accessibility of psychological science can be improved? Sutton highlighted how a lack of clarity in dissemination of science (for example in press releases) can lead to the propagation of unsubstantiated and inaccurate claims.

The Psychologist is providing opportunities for early career researchers to learn how to tell their stories in a simple, engaging and transparent way. 'People often think that writing is profound if they don't understand it, or if they've had to work hard to do so', Sutton claimed, 'but clarity is really important for transparency'. Sutton also discussed how we often hear that open science is a 'renaissance' that we're living through, yet there still isn't a 'culture' of replication. 'Do we need to make the story around open science more fun and engaging, in order to truly encourage researchers to adopt open science practices? Are we missing the importance of stories – who tells them, to whom, why and how – in the open science debate?'

Perhaps Sutton's closing message summed up this whole symposium perfectly: 'We can't just bounce ideas around in a liberal echo chamber – we have to reach people who we are not reaching.' I feel that the conversation around open science is now happening well outside of this 'echo chamber'. Symposia such as this one are highlighting the good work that is already going on around open science, and will likely be a catalyst for sharing and promoting initiatives which will tackle threats to reproducibility. The scientific world in Europe and beyond is indeed changing – rapidly, and for the better.