Concerning study says psychotherapy research has a problem with undeclared researcher bias

When a good doctor encounters research comparing the effectiveness of drugs A and B, she knows to beware the fact that B was created by the people paying the researchers' salaries. Pharmaceutical industry funding can be complex, but the general principle of declaring financial conflicts of interest is now embedded in medical research culture.

Unfortunately, research into psychological therapies doesn't yet seem to have got its house in order in an equivalent way. That's according to a new open access article in the journal BMJ Open which suggests that, while there is less risk in this field of financially-based conflicts, researchers may be particularly vulnerable to non-financial biases, a problem that hasn't been adequately acknowledged until now.

The research team, led by Klaus Lieb of the University of Mainz, examined 95 systematic and meta-analytic reviews that had evaluated the efficacy of psychological therapies by looking at the weight of evidence across multiple randomised controlled trials. Such reviews are generally used to give a balanced picture of what really works, above and beyond a single trial.

The journals that publish this kind of research tend to be on the look out for financial conflicts of interest that could lead to a bias: for example, if the author of a meta-analysis of Therapy X was a license-holder for that therapy but chose not to declare that interest when reviewing it. Indeed, Lieb and his team found that four out of every five journals they investigated had explicitly asked that such conflicts be declared.

But only a third of the journals asked the same of non-financial conflicts of interest, such as a review author being trained in one specific therapy, or otherwise being a particular advocate of one over others. Lieb and his colleagues argue that this presents a real problem because unlike a pharmaceutical researcher who becomes disenchanted with their favoured drug, "psychotherapy researchers who realise that the effect of the therapy to which they are allied is less beneficial than another therapy cannot easily switch their research programme to another therapy (since they have often been trained in that therapy for many years)". This is especially true if they have been on record as a cheerleader for a particular therapy, or as a critic of its alternatives. Incidentally, at the end of their paper, Lieb and his colleagues act by example, providing a thorough outline of their own therapeutic training and related information.

So these conflicts may be especially pernicious for psychological therapies, and the fact that so few journals ask for declarations of these kind of allegiances is alarming. Even in cases where journals had asked for this information, only four researchers volunteered it.

Perhaps this just means these conflicts are thankfully rare? This seems unlikely. Lieb's team found that 34 of the reviews covered at least one and sometimes many more experimental studies authored by the reviewers themselves: not a crime in of itself, but a sign of at least a potential conflict. What's more, Lieb's group investigated these 34 reviews using a standard protocol to check for signs of "research allegiance", that is whether the article betrayed a belief in the superiority of a treatment. Fifteen of the 34 reviews met these criteria, meaning that one or more of the authors had a vested interest in the conclusions.

Lieb and a co-author, Jan von der Osten-Sacken, also read through anonymised versions of all 95 reviews and labelled a conclusion as "spinned" when they felt it wasn't consistent with the empirical results reported. Based on this, they claimed that reviews they'd identified earlier as having non-financial conflicts also tended to have biased conclusions. However, this link wasn't statistically significant, so at best this tentative finding indicates something to be studied in the future.

Lieb's team emphasise they are not claiming that there is rampant fraud and deliberate misrepresentation. They suspect that "researchers may simply not be seeing the necessity of declaring non-financial interests, or be clear on what this includes". So it's incumbent on journals to outline more explicitly what constitutes a conflict of interest, and request and reveal them with more vigilance. Researchers themselves should take to heart the foundation on which the search for truth is built: to let go of your beliefs and be willing to be wrong.

Further reading

—Conflicts of interest and spin in reviews of psychological therapies: a systematic review

About the author

Alex Fradera (@alexfradera) is Contributing Writer at BPS Research Digest