Tone deaf?

The clamour around the 'tone' of the debate around replication and reproducibility in psychological science grew again in January, following the publication of a Boston Globe piece by Pardis Sabeti titled 'For better science, call off the revolutionaries'. But is anybody listening? And should they be?

Sarbeti, a computational geneticist and professor of biology of Harvard, wrote of her 'hope that scientists of all stripes – but especially social psychologists – will slow down and start approaching one another with greater respect.' Referring to numerous examples from the field of psychology, Sarbeti concludes that 'The attacks on these scientists have become so personal and so threatening that there may be no one in the field willing to speak up on their behalf… revolutions can also lead to a bandwagon effect, in which bullies pile on and bystanders fearfully turn a blind eye.'

In fact, many big-name psychologists immediately weighed in on social media. Daniel T. Gilbert, a Harvard colleague of Sabeti's who has previously called 'Psychology's replication police' 'shameless little bullies', called the piece 'one of the smartest essays about the politics of social psychology that I've ever read', and told me 'Criticism and correction is how science works… Doesn't have to become the politics of personal destruction'. He added that 'the "gotcha" game isn't merely unkind, but can destroy a science by making scientists too fearful to explore. Sciences die by playing it safe, not by getting it wrong.' Steven Pinker (also Harvard) advocated 'Rigor, or course, but put a lid on the aggression and call off the social media hate mobs'. Susan Cain called the piece 'brave'.

Some commenters brought gender into the debate, with Alberto Diaz-Cayeros saying 'it is not a coincidence that men are the attackers (replication advocates), women their targets!' – despite numerous counter-examples on both fronts.

There was a swift and dismissive response from many open science advocates. Chris Chambers (Cardiff University) tweeted: 'First they ignored us. For decades. Then they smirked from their Ivy League thrones & corporate boxes. Then they got desperate & cried "bully", "sexist", "terrorist". Finally they realised they hadn't even lost b/c they were simply not worth fighting.'

At an event the week before, Chambers had said: 'You'll never change lobbyists. They have their own agenda.' I put it to him that if he had been talking about some of the figures in this debate so far, that could partly account for the occasional tone of it: they think they are scientists and that others should try to persuade them, slowly and gently, whereas open science advocates may see some as cautionary examples, to simply be held up to encourage the spread of open practices as mainstream. 'I think that's a good observation,' Chambers replied. 'I don't have time to coddle the unconvincable. Things have progressed to where they are now because we've focused on the people who matter most: funders, journal editors, learned organisations, policy makers, and above all early career researchers.'

Writing in Slate, Daniel Engber pointed out that virtually all of the examples of 'cruel and hurtful rhetoric' cited in Sabeti's piece were actually from a single source (Columbia statistician Andrew Gelman's blog and associated comments), and aimed at just one target (Amy Cuddy). He concludes that 'Not every replication failure ends up in a fight, and most discussions of these issues have been forward-looking. Indeed, social psychology has already done far more to solve its problems – implementing systems at journals and in labs that help prevent the spread of false-positive results – than most other scientific fields.' Christopher Soto, an Associate Professor at Colby College, agreed, writing: 'As a psychologist whose research has been the target of a replication attempt, [Sabeti's] characterization strikes me as inaccurate and unfair.'

Engber also took issue with the 'call off the revolutionaries' aspect of the heading to Sabeti's piece, 'as if a "calling off" could be ordered from the top, decreed by some generalissimo psychologist… the current revolution in psychology, like the one that happened in genomics, isn't under anyone's command.'

I put it to Chambers that if there was any 'split' in this debate worth characterising, it is to do with how 'public' the person having their research examined has been in their dissemination: how 'lofty' is their throne and how much do they therefore feel they have to lose in being (as they see it) fairly publicly and brutally toppled from it? 'Right', Chambers replied. 'And notice how this narrative is all about them, not the public or scientific community. They occasionally insert unsubstantiated fearmongering that critical discourse puts young people off science, but it's really all about them and their own fears.'

That suggests to me a debate that is in need of more data, about what it actually feels like to be the 'target' of failed replication attempts and therefore questions around your whole approach to science. Some have indeed written about this, notably Susan Fiske. But perhaps it's time for a social psychology of the (largely social) psychology replication debate.

It seems, though, that those engaging with the 'tone debate' are unlikely to be the main open science advocates. They have already left it behind. 'Things are moving quickly,' said Professor Dorothy Bishop (University of Oxford). 'We are in an intensely political period,' Chambers has written, 'and whenever you argue about tone you are playing your opponents' game by your opponents' rules. While we tear ourselves sideways worrying about such trivial nonsense, they are smirking from their thrones.'

Chambers has concluded that 'We need to change the way psychological science works, not the way some people talk about psychological science.' He told me: '[These are] death throes. We're approaching the point of no return. (In fact we have probably already passed it). We're witnessing the fall of Rome in real time. Quite something isn't it?'

- At this very moment, the British Psychological Society is hosting its second event on replication and reproducibility. Here's a report / video from the first one; a report on today will follow in due course. In the meantime, find much more on the replication debate in our archive.

- If you have a story about what it feels like to have your work replicated, or not, get in touch on [email protected].