Liberating, or locking away, our best tools?

The journal Basic and Applied Social Psychology has banned papers which contain p-values in an effort to reduce the amount of lower-quality research it publishes. Journal editors David Trafimow and Michael Marks have said they believe the ban on the null hypothesis significance testing procedure (NHSTP) will increase the quality of submitted articles by 'liberating authors from the stultified structure of NHSTP thinking thereby eliminating an important obstacle to creative thinking.'

Authors can still submit papers which contain p-values to the journal, but these, along with t-values, F-values and statements about significant differences will have to be removed before any work is published. The journal has also banned confidence intervals and stated that where Bayesian methods (where evidence is expressed in terms of degrees of belief) are used the editors will make case-by-case judgements on whether they will be allowed.

The journal now requires that authors include strong descriptive statistics and encourages the use of larger sample sizes than 'typical in much psychology research'. It also suggests authors present frequency or distributional data where possible.

Reactions to the new guidelines have been mixed among psychologists and academics. Graham Smith (University of Northampton) said he was in favour of changing the ways psychologists analyse their data. Dr Smith added: 'The New Statistics of effects sizes, confidence intervals and meta-analyses have many advantages over significance testing. Significance testing is widely misunderstood and frequently leads to unwarranted conclusions.

'So I understand why some favour banning NHST; a clean break might force researchers to adopt the New Statistics.' However, Smith warned that a previous ban in the Memory & Cognition journal 'didn't bring about lasting change. However, I sympathise with the frustration of supporters of the New Statistics who have essentially won the intellectual argument yet their good advice is ignored by most psychologists.'

Andy Field (University of Sussex) said he thought the flaws in significance testing were more widely known than ever before and that with the replication crisis within psychology it was inevitable that journals, and psychologists more generally, were exploring other options. He added: 'However, scientists tend to analyse data in the way they know how, so change will be slow. I think at the heart of this decision is a feeling that to make people change their ways something radical has to be done. This ban is radical. The positives are that they are encouraging authors to report raw data distributions and think about what their data actually show beyond a p-value. The downside is that their policy seems to be anti-inferential statistics full stop (no NHST, NO CIs and no Bayes unless we decide it's OK on a case-by-case basis).'

Professor Field said he thought banning inferential statistics was a retrograde move. 'In the absence of those principles you are left with very little. In terms of the future, I think psychologists, as scientists, should be thinking more about the appropriate models to fit to our data rather than shoehorn-ing our data into the pair of boots that we happen to like wearing. That doesn't necessarily mean locking the boots in the cupboard, it may be better achieved by a trip to a shoe-shop to see what prettier shoes there are on the market.' Others agreed, with cognitive scientist Jan de Ruiter tweeting that although 'NHST is really problematic', banning all inferential statistics is 'throwing away the baby with the p-value'.

- Coming up the June issue of The Psychologist will be a feature on 'building confidence in confidence intervals', by Dr Graham Smith and Professor Peter Morris.