Repent and reform

For the past few years, the discipline of psychology has been undergoing a period of methodological self-criticism and debate that we usually refer to as the 'replication crisis'. It began in 2011 and went up a gear in 2015 with the publication of the results of the Reproducibility Project that were widely interpreted as showing that perhaps only one in every three psychology discoveries are in fact reliable. Professor Chris Chambers offers the first major book to emerge from this debate.

His premise is that psychology is indeed in a perilous position, and may soon fall by the wayside in the onward march of science generally, eventually becoming little more than a cautionary tale of yet another protoscience that never made it to maturity. The book aims to uncover the root causes of this worrying state of affairs and, as the title implies, identifies seven major contributing factors (or 'sins'). These are: publication biases that demands novel and positive results; the gaming of statistical tests to always achieve statistical significance; our use of low-powered studies and our aversion to replication studies; our 'dog in the manger' attitude to our own empirical data; the problem of outright fraud; closed-access publishing; and our obsession with proxy metrics to quantify quality. As a consequence of this collection of problems, the discipline of psychology has become a system for the mass production of false discoveries: scientific dross is packaged and sold as scientific gold, and we've deprived ourselves of our much celebrated self-correcting mechanism to eventually root out and discard these mistakes.

What I valued most about this book is its thoroughness and the seriousness of its tone. It would be easy to cheapen or sensationalise this topic. After all, the replication crisis has become a popular subject of debate within psychology. It has even permeated into mainstream media where it has sometimes been unfortunately sensationalised (recall, for example, the Independent's August 2015 headline about the reproducibility project: 'Study reveals that a lot of psychology research really is just "psycho-babble"'). Moreover, the debate often plays out in the rough-and-tumble world of social media where the discussion has sometimes become unedifying, to put it mildly. In Chambers's book, the discussion is always detailed and careful. All arguments are backed up by ample evidence. Indeed the book's extensive endnotes are a trove of relevant facts and references, and in themselves serve as an annotated bibliography of the entire replication crisis. There is no attempt to scapegoat and vilify any particular individuals or sub-field or method (for example, the humble p-value is not treated as a whipping boy, as has sometimes happened in the past, but is treated fairly in the context of more general discussion of statistical testing). Even if one did not fully agree with Chambers's diagnosis or prognosis, I don't think anyone could see this book as anything less than a detailed, careful and reasonable analysis of the current state of psychology.

Ultimately, this is not simply a description of a problem, but a call for collective responsibility and action. Its general perspective is that we've collectively created this problematic culture and we should own up to this and then take the necessary steps to reform the field. Those steps to reform are sometimes obvious and could be implemented immediately by any of us (for example, what is stopping any of us from conducting high-powered studies, publicly preregistering our hypotheses and methods, sharing all our empirical data and materials, and so on?), and so all that is required is just a change in our culture and values. Of course, culture can be surprisingly intractable. The dominant tendency of any group or organisation is always in favour of maintaining the status quo. What matters for the future of psychology is how well the advocates of reform can convince the majority to overcome their natural aversion to change. In this respect, this is an important book and one that I hope is widely read and discussed. It marshals all the arguments concerning the inadequacies for our current scientific practices and provides detailed examples of the many options we have for reform, and it does so in a generally positive and constructive manner, without ever being divisive and pugnacious.

- Find lots more in our archive.