'We need a certain amount of humility'

You published an article around a week ago, on the Unherd website. Can you summarise the main points?
I had been reading a slew of articles by psychologists, or people writing about psychology, making the same kind of point – that people were overestimating the risk of Coronavirus. The world's largest ever quarantine had been imposed in China, Italy had declared a state of emergency, and so I was amazed all these articles were saying 'you're overestimating risk'. The articles were based on decision-making risk psychology type stuff, which is, I think, interesting and all very well in a lab situation and for many other situations. But here a genuinely scary world catastrophe was happening, and yet all these psychologists were saying, 'Oh, no, don't worry about it', making analogies to terrorism and the like. 

So I wrote an article saying this is clearly psychology overstepping its boundaries. I'm not actually that sure what psychology has to contribute right now. Psychology is very good when it stays in its lane. Psychologists will be required to help people deal with the mental health consequences of things like isolation and losing family members, but that's a secondary effect. The really immediate things, the scientific priorities, are things like developing a vaccine, modelling the outcomes, trying to work out which drugs work. I just don't really see a role for psychology in the immediate first response. 

Was there a reaction you were expecting, or hoping for?
Well, the reaction I got was a bit disappointing, focusing on the 'Don't trust the psychologists on coronavirus' headline. 'How can you say that? As a psychologist?' Well, I didn't mean all psychologists, and I acknowledged that some psychological stuff is going to be helpful eventually. 

The response I wanted to get was 'here is a list of specific contributions that psychology has made to this epidemic right now, that are really important and can be on an equal level to the importance of the scientific stuff'. What I got was vague statements about 'psychology is really important', 'psychologists have contributed to this', 'here's a list of blog posts'. There are clearly relevant psychological principles to this, but it's just not clear how they would be practically useful in this terrifying situation.

As somebody who's been involved in collating some of those perspectives, I've seen psychologists like Vaughan Bell reminding people that there is specific research on pandemic response, and that psychologists should draw on that specific research base. And then I think in terms of how psychologists can and are making a contribution; directly with the government and their scientific advisory groups. I thought it was interesting that a couple of aspects of that input – the importance of the collective angle rather than emphasising the individual aspect, and the implications of that for policing tactics – seemed to be picked up in your Twitter thread as something that's obvious and banal. Have some psychologists been so successful in coming up with ideas that are actually quite counterintuitive, having impact with their research and communicating it, that the conclusions start to seem banal and obvious and trite within Psychology? Whereas in fact, to the wider public, they're probably not, and aren't informing the actual government response to the extent that they should be.
I think that's a very good point. But if you read through the advice the government published on their website, it's useful to have it all written down, but I'm just not certain to what extent that is based on scientific understanding and specific studies, and to what extent it's based on thoughtful people thinking about the situation. Maybe having smart people in these groups who are interested in human behaviour and can point to some signs, that's useful. But is it specifically 'this study said you should do this, therefore…?' I just wonder to what extent we can call that psychological science.

With those specifics, in terms of likely public response and policing, I know that is grounded in actual published research papers. But I take your point around smart people who generally think in a psychological way… isn't that the direction the applied arm of the discipline has been heading towards in recent years, i.e. looking to inform a generally psychological way of thinking amongst a wider set of the workforce?
I agree, but the problem comes when a statement is made by a psychologist but there's no link between 'they said this, therefore, we did this'. For instance, the government strategy on herd immunity. It's very, very opaque at the moment where that came from. Some people have said it was Dominic Cummings. David Halpern, a psychology graduate leading the Behavioural Insights Team, seemed to be the first person to actually use the phrase in an interview. And then a couple of days after that, Matt Hancock said, 'No, that was never a strategy. We're not doing that'. Yet the behavioural science document does basically say, 'if you make the argument to people that there is a herd immunity being built up in the population, that might make them more amenable to some of these strategies…'

I have to be honest, that Halpern interview hasn't aged well, but at the time I totally bought it.
Yes, and isn't that the problem, if you're not aware of studies such a strategy is being based on? My friend Saloni Dattani wrote a great article arguing against the herd immunity strategy, with loads of references. And the initial evidence from Sweden, where they're still pursuing something like that, doesn't look good.

It's all so shrouded in uncertainty, with no clear statement. To me, that emphasises the fact that we need to be super careful with the things we say, and yet this was just in the middle of a page as a brief comment [see point 10 here]. 

It's similar with the concept of behavioural 'fatigue' around lockdown measures. It would be useful to trace that back to specific published advice, and to be more certain about what they meant at a time by fatigue. I think most likely is that they were talking in societal and structural terms as well as individual psychological behaviour, saying that the nation will get tired of this and it won't be practical for very long. That may well be true, but everyone seemed to pick up on the idea of fatigue being something more individual and psychological.
I think the argument here is that putting in aggressive measures, right from the very start, might have meant that we didn't need to keep going for as long because it would have stopped the infection spreading quite so much as it has.

Obviously, it's easy with hindsight to look back at strategies. And I personally think the scientific advisors seem to be doing a fantastic job. Isn't there just a risk that psychologists, like everybody else, was caught on the hop by just how serious it was? And that to an extent those early articles were still right, in that the vast majority of us might be impacted by the measures around coronavirus while not actually being at a level of personal risk the media coverage might have us believe? 
Well, as I say, the first article I referenced was substantially after the biggest quarantine in the history of humanity. Even if it was in China, not in the author's own country, we must realise that things are in a global state. 

On your second point, I just worry that lots of articles by esteemed psychologists in reputable publications like the New York Times, contribute to an atmosphere of lots of trained professionals who know about risk telling us that actually, it's not too bad. That might trickle upwards towards the policymaker. And we're talking about a really vital stage in how people respond to an outbreak.

The point you're making is kind of 'with great power comes great responsibility'… that's an interesting angle to open the discussion up a little. The starting sentence in your article was 'psychologists haven't had a great few years'. And then you write about the replication crisis, and the collusion with the US government on the torture programme during the Iraq war, and then the revisionist approach to some of psychologies classic studies like the Stanford Prison Experiment. When I read it, I thought that that whole paragraph could be rewritten as 'psychologists have had a great few years… First, they started getting their house in order over the replication crisis, then psychologists were central in exposing a cover up over torture, then some very clever psychologists have gone back over some classic studies and given interesting new insights into them.' So, in terms of how psychologists are wielding their power, isn't this a golden age for psychology, and we should be proud of all these things
Absolutely. I think we should be proud of the new movement to change and correct things. And, psychologists are developing methods which are now being used in completely different fields that are improving the openness and the transparency of research. But with that does come the realisation that for many years in psychology, the stuff we've been doing, it's been less than reliable. That's still something that we should be concerned about, especially given that it's had a huge cultural impact – think of the Stanford Prison Experiment, Rosenhan's 'being sane in insane places'. There's been scientific fraud, hype of results, biases and whatever else. But then there's a little bit of hope at the end, that it's us discovering this stuff… even though there's been resistance from some of the more established psychologists. There's a lot of early career researchers who are really pushing openness and transparency, and we shouldn't overlook that. 

What needs to change in the 'new normal', if the pandemic does change things? Do you think that this situation could lead to differences in the way psychologists seek to reach out and influence policy?
There's going to be loads of good research on this, but I don't think people should be rushing it out. Anne Scheel wrote a good blog on this. Ask yourself, is psychology really the most urgent thing we need to focus on here? But someday that research can be done in ways that are transparent and open, and the results can hopefully inform the next time.

But I also think we can look back at some of these clear failures to communicate accurately. Does the research we link to in popular articles hold up? Cass Sunstein linked to a Psychological Science paper from 2001 as the only evidence he cited to say we shouldn't be taking this so seriously. We need to do better than that. We need to have a certain amount of humility. We need to have more generalisable studies that we base our predictions or advice on. And maybe it's the case that those studies don't exist. As you say, Vaughan Bell referenced all the studies on SARS and influenza, yet none of them were referenced by the government in any of the discussions I've seen about it. 

It's about tracing a solid gold thread isn't it, through the statement made via the credentials of who is making it to the quality of the research it was grounded on. I agree that intellectual humility is so vital, but I would argue that I'm seeing more and more of that from psychologists… there are an increasing number who are willing to say in media pronouncements 'do you know what, we don't know. And this is, this is why we don't know… People are complicated. The world is complicated. But on the basis of what we do know, this is our best guess at the moment. This is where we need to add to our knowledge.'
Yes. If the evidence isn't there, then you just have to put your hands up and say it's not. 

Of course, there is a huge pressure on us, just like there is to publish articles. Journalists have called me about research on, say, cognitive ageing, and they want an answer, and if I say I'm not sure, either they're not interested or they push for a definitive statement. And that pressure is heightened in a situation like this. It's a terrifying global pandemic, people want to have some sense that they know the answers.

And there are personal rewards, I guess for coming up with those answers, in terms of profile, reputation, book sales…
Yes. All these books have come up with single word titles that promise one quick fix solution. If you read into the details, there's actually a lot more going on, but I think a lot of people get the impression that there's just this one thing that needs to change and it can have this big effect. There's a real pressure on psychologists to come up with these cool, innovative, sometimes counterintuitive ideas. When that's around whether you should worry about global pandemic, I think it's more of an issue. 

I agree there's a need for psychologists to remind other psychologists about that intellectual humility, and to be cautious and evidence-based. But is there a danger that self-policing, that self-flagellation, spills over into something else? I'm constantly seeing phrases such as 'stick to the science', 'stay in line', 'know your limits'. But given how messy sciences and the world is, shouldn't there be tolerance in the research we do and how we respond to people who are making attempts to reach out? It's not always going to be perfect, people are going to get things wrong. But there's also a danger in 'back in your box… We've got nothing to say on this topic', when there is loads of good work that psychologists are doing to try to 'make the world a better place'.
Sure. I agree. We can be too quick to write things off. I don't want to write off psychology. I think psychology has got lots to say. But in this specific situation, this is probably the most serious thing we're all going to encounter in our lifetimes. So there's a much stronger argument for saying 'we're not 100% sure that the research backs this up'. There's research we can be doing, but do we need to be publishing it right now? Do we need to be wary of the same set of incentives that push us towards writing tonnes of low-quality articles in the first place? Do we need to be a bit more humble? Is this research really on solid foundations?

I think you use the phrase 'ready for primetime'.
Yes. Angela Duckworth, talking about an area she has made famous, grit, said the hype is overtaking the science on this. It might not be ready to be implemented in classrooms. I've got an enormous amount of respect for that. By that point, the hype train really had set off. But the idea of grit is far less damaging than the idea of 'Don't worry too much about the Coronavirus'. In this case, all of these arguments are so much more acute because of the danger of saying incorrect things. 

Luckily, some of the psychologists or people writing about it have done a complete 180 degree and said, 'actually, this is really bad, and we need to do more about it' [listen also to this interview with Daniel Kahneman on 'why we underestimated COVID-19']. So, they're not completely insensitive to these problems. But I wish they had been more circumspect. 

- Stuart Ritchie's book, Science Fictions: The Epidemic of Fraud, Bias, Negligence and Hype is published by Penguin in July.

Editor's note: This interview was originally published online on 9 April. For the print edition of The Psychologist, we asked Stuart some extra questions…

What areas of psychology / specific psychologists do you think are 'ready for primetime'? 
This is just one example among many I could think of, but perhaps the area of psychology that has one of the best claims to be truly useful in 'the real world' is one that tends to get forgotten about: occupational psychology, or I/O psychology as Americans call it. In that field they have enormously massive datasets, solid evidence for prediction over time (e.g. of personality or aptitude tests and job performance), replication across many studies, and the thing that the studies are looking at is the real-world thing the area is trying to impact: that is, helping employers predicting who'll do best, be most efficient, be happiest at which kind of job or in which position, which is the best set of psychometric variables that can help us predict that, etc.

Other areas – clinical psychology with therapy for at least some psychiatric disorders, for example – are more relevant at least to the knock-on, secondary or tertiary effects of the pandemic… though we shouldn't underestimate some of the replication issues that my clinical colleagues tell me they're having, too.

How can psychologists best make that judgement, on whether they're risking straying into 'science fictions'?
I think that's where our suggested 'Evidence Readiness Levels' framework comes in, from our recent preprint [https://psyarxiv.com/whds4]. We suggest that, just like NASA does, we rate each piece of psychological scientific evidence as to how rigorously it has been tested, how replicable it is, how much we know it's been used in contexts that are very close to the one we want to use it in now, and so on. Having read through the best current attempt from lots of experts to link up psychological research to the coronavirus pandemic, our judgement is that a great deal of the evidence we have is at the low end of our 'Evidence Readiness Levels' scale. Small-n studies, lab-based research from a small number of countries, unclear replicability, leaps from neuroscientific findings to psychological principles: often generally pretty weak evidence.

All of those studies might be fine starting points for a line of research. But they're not, we'd argue, the sort of thing you'd rely on with a metaphorical gun to your head – and that, with the pandemic, is the situation policymakers find themselves in right now.