Ten famous psychology findings that have been difficult to replicate

Every now and again a psychology finding is published that immediately grabs the world's attention and refuses to let go – often it's a result with immediate implications for how we can live more happily and peacefully, or it says something profound about human nature. Said finding then enters the public consciousness, endlessly recycled in pop psychology books and magazine articles.

Unfortunately, sometimes when other researchers have attempted to obtain these same influential findings, they've struggled. This replication problem doesn't just apply to famous findings, nor does it only affect psychological science. And there can be relatively mundane reasons behind failed replications, such as methodological differences from the original or cultural changes since the original was conducted.

But given the public fascination with psychology, and the powerful influence of certain results, it is arguably in the public interest to summarise in one place a collection of some of the most famous findings that have proven tricky to repeat. This is not a list of disproven or dodgy results. It's a snapshot of the difficult, messy process of behavioural science.

Power posing will make you act bolder

Put your hands on your hips, widen your stance. Do you feel bolder now? Harvard psychologist Amy Cuddy and others have published numerous studies that appear to show that our body position can affect our emotional feelings and the way we behave. One of the reasons this line of research has been so influential is because of Cuddy's inspirational TED talk "Your Body Language Shapes Who You Are" which has been viewed many millions of times.

In 2010, in a paper that's already been cited over 450 times, Cuddy and her colleagues Andy Yap and lead author Dana Carney showed that after adopting one of two power poses for just one minute (either legs on desk/hands behind head or feet apart leaning on a desk) participants made riskier bets and had higher testosterone and lower cortisol, compared with participants who spent one minute in a low power pose (sitting with hands in lap or standing embracing oneself).

However, whereas many studies have found expansive postures, such as power poses, affect how we feel, it's been much more difficult to replicate the finding that they affect our actual behaviour or our physiology. For instance, last year researchers led by Eva Ranehill at the University of Zurich attempted to replicate the effects of power posing on risk-taking behaviour with 200 participants (compared with the sample of 42 participants in Cuddy's research) and while participants who adopted power poses said they felt more powerful, they showed no differences in their testosterone or cortisol levels compared with the low-power pose participants, nor were they more willing to make risky bets. "Using a much larger sample size but similar procedures as Carney et al. did, we failed to confirm an effect of power posing," the replication team said.

Cuddy and her colleagues responded with an overview of 33 published studies that have shown psychological and physiological effects of power posing. But in a paper that's forthcoming in Psychological Science Joseph Simmons and Uri Simonsohn at the University of Pennsylvania have conducted a statistical analysis on these 33 published studies that they say shows "the existing evidence is too weak to justify a search for moderators or to advocate for people to engage in power posing to better their lives".

- See also 'A decade of power posing: Where do we stand?'

Smiling will make you feel happier

We know that feeling happy makes us smile, but can smiling make us happy? In 1988, researchers reported that participants found cartoons funnier when they held a pen between their teeth, forcing them to smile, as compared with when they held a pen between their lips, forcing them to pout. The finding appeared to be consistent with the facial-feedback hypothesis – the idea that our facial expression doesn't just reflect our feelings but also affects them – and according to Google Scholar it has been cited nearly 1500 times.

However, a replication attempt published this Summer by 17 independent labs and involving nearly two thousand participants found overall no effect of mouth position on people's rating of the funniness of cartoons. The replication team said their replication failure was "statistically compelling". The lead author of the 1988 study, Fritz Strack, said he was surprised by the null result and he highlighted a number of problems with the replication attempt, including the fact the participants were filmed during the study, which may have made them self-conscious and affected their emotions.

Self-control is a limited resource

One of the most influential psychological theories of modern times is that willpower is akin to a fuel – the more of it you use in one situation, the less you have left over to deal with other demands. Of the many dozens of studies that have demonstrated this principle, known as "ego depletion", one that was recently targeted for replication was published in Psychological Science in 2014 by Chandra Sripada and colleagues – it showed that performance on a task requiring self-control was impaired if it was preceded by an earlier task that also required self-control, but not if it was preceded by a non-demanding task (the study also demonstrated that this apparent effect of depleted self-control was ameliorated by earlier intake of the drug Ritalin, presumably through its neurochemical effects, but this aspect of the study was not part of this year's replication attempt).

The replication effort, by 23 labs and involving nearly two thousand participants, repeated the sequential task procedure used by Sripada, to check whether the basic principle holds true that being drained by one self-control task impairs people's performance on a further test of self-control. In fact, overall, the combined results were not consistent with this idea. "… [I]f there is any [ego depletion] effect, it is close to zero," the replication researchers said. Their finding added to the outcome of a recent meta-analysis of 68 relevant published and 48 unpublished studies that found little support for the idea of willpower as a limited resource.

Responding to the failed replication, Roy Baumeister, who co-discovered and proposed the concept of ego-depletion, said that it was "misguided" and had been a mistake to focus on trying to replicate a study that had used new procedures, rather than more established means, for provoking depleted self-control.

Revising after your exams can improve your earlier performance

If I told you that I was planning to revise for my upcoming exam after the exam, you'd quite understandably probably think I was rather daft. And yet a series of studies published in the prestigious APA publication the Journal of Personality and Social Psychology (currently cited nearly 500 times) appeared to show that such a plan could work – among the bizarre findings that appeared to show established psychological effects working backwards in time, Darly Bem and his colleagues found that spending time learning words after a memory test actually boosted performance on that earlier memory test.

However, when Stuart Ritchie, Richard Wiseman and Chris French each independently attempted to replicate this finding, in each case using Bem's original computer programme and materials, they were unsuccessful. "Our participants were no better at remembering the words they were about to see again than the words they would not, and thus none of our three studies yielded evidence for psychic powers," Ritchie and his colleagues said in a commentary for The Psychologist.

But the controversy hasn't gone away. This year Bem and his colleagues gathered together all the data from 90 experiments from 33 laboratories in 14 countries that have attempted to replicate his surprising findings (including the retrospective effects of learning, but also the other backwards effects he documented). Overall, Bem says the totality of this evidence is overwhelmingly in favour of "pre-cognition" or "psi" being a real phenomenon. "The incompatibility of psi with our current conceptual model of physical reality may say less about psi than about the conceptual model of physical reality that most non-physicists, including psychologists, still take for granted—but which physicists no longer do," Bem and his team concluded.

Exposure to words pertaining to ageing will make you walk more slowly

The idea that our thoughts and behaviour can be influenced by the meaning and connotations of the words, symbols and objects around us, even when we're not paying attention to them, is known as "social priming". The findings in this field are fascinating but they are proving to be the most difficult effects to replicate in psychology.

A modern classic of the social priming literature – cited over 3700 times to date – was published in 1996 and involved participants unscrambling jumbled lists of words to form coherent sentences. When these word lists contained words pertaining to ageing and the elderly, participants on leaving the lab walked out more slowly than when the lists did not contain such words. Presumably the ageing-related words triggered related ideas in the participants' minds that led them to behave in a stereotypically more elderly way.

This makes for a neat story, but unfortunately this specific finding and others in the field have been tricky to repeat. In 2012, researchers at the Université Libre de Bruxelles and the University of Cambridge attempted to obtain the same effect on walking speed but failed. They only achieved the same outcome as the original research when they deliberately manipulated the expectations of the researchers who were interacting with the participants, to make them consistent with the earlier results.

John Bargh, the lead author of the 1996 classic, criticised the replication attempt and outlined a number of reasons why it might have failed, including that the replication researchers may have included too many ageing-related words thereby making the priming effect too obvious.

Cleaning your hands will wash away your guilt

"Out, damned spot!" cries a guilt-ridden Lady Macbeth, obsessively washing her hands in the hope it will clear her conscience. Many research findings have demonstrated the that the link between moral purity and physical cleanliness is more than metaphorical, and that when we're feeling guilty we're motivated to clean ourselves physically.

In one of the earliest examples of the "Macbeth Effect", Chen-Bo Zhong and Katie Liljenquist asked participants to hand-copy an account of either an ethical or an unethical deed (helping or sabotaging a work colleague, respectively), and then asked them to rate the desirability of various products. Those who'd written about an unethical deed rated hygiene-related products more highly, such as soap and toothpaste.

But in 2013, when researchers at the University of Oxford tried three times to replicate this effect with participants from the UK, USA and India, they failed on each occasion. Brian Earp and his colleagues did not claim that there is no link between physical and moral purity, nor did they dismiss the existence of a Macbeth Effect. But they said their replication failures call for a "careful reassessment of the evidence for a real-life 'Macbeth Effect' within the realm of moral psychology." The Oxford University research complemented another study from 2011 that failed to replicate more of Zhong and Liljenquist's findings on the Macbeth Effect, including the idea that physical cleaning reduces guilt and as a result makes people less likely to be altruistic.

Babies are born with the power to imitate

Pick up almost any introductory psychology book and inside you'll read about research conducted in the 1970s that appeared to show that humans are born with the power to imitate, complemented by black and white images of a man sticking his tongue out at a baby, and the tiny baby duly sticking out her tongue in response.

Earlier this year, however, a methodologically rigorous investigation found no evidence to support the idea that newborn babies can imitate. Janine Oostenbroek and her colleagues tested 106 infants four times between the ages of one week and nine weeks. The researcher performed a range of facial movements, actions or sounds for 60 seconds each including tongue protrusions, mouth opening, happy face, sad face, index finger pointing and mmm and eee sounds. Each baby's behaviour during these 60-second periods was filmed and later coded according to which faces, actions or sounds, if any, he or she performed during the different researcher displays.

Whereas many previous studies have compared babies' responses to only two or a few different adult displays, this study was more robust because the researchers checked to see if, for example, the babies were more likely to stick out their tongues when that's what the researcher was doing, as compared with when the researcher was doing any of the 10 other displays or sounds. There was no evidence that newborn babies can reliably imitate faces, actions or sounds. Across all the different displays, actions and sounds, there was no situation in which the babies consistently performed a given facial display, gesture or sound more when the researcher specifically did that same thing, than when the researcher was doing anything else.

Writing on The Conversation website, psychologist Richard Cook and PhD candidate Daniel Yon said the new results appeared to show that "Rather than being born with an innate ability to imitate, it therefore appears that human infants actually learn to imitate." They added that this shouldn't cause concern: "Instead, the new findings … suggest that cultural forces can profoundly shape our psychology. Many of the abilities that define us may not reside in our DNA, but may instead have their roots in the societies around us."

Big brother eyes make us behave more honestly

Imagine there was an honesty box in your coffee room at work, do you think it would make any difference to your behaviour if there was poster of staring eyes on the wall? A hugely influential study published in 2006 suggested that it would – that feeling watched, even by a  picture of eyes rather than an actual person, increases people's honesty. In fact, in the study, donations to the box were an average three-fold larger in the presence of an eye poster rather than a picture of flowers. The finding even inspired the West Midlands Police in the UK to launch a poster campaign featuring staring eyes and the strapline "We've got our eyes on criminals".

Unfortunately, the finding has proven difficult to replicate. For example, in 2011 researchers at University of Bamberg, Germany tested the effects of the same poster materials on a larger sample of participants (138 people compared with 48 in the original), in this case in terms of how the participants said they would behave in various social situations, such as lending study materials to a fellow student who has been off sick. They found no evidence that the Big Brother eyes increased prosocial behaviour.

Furthermore, this year, two meta-analyses combined the data from over 50 studies involving collectively tens of thousands of participants and found no evidence overall that watching eyes boost people's generosity.

Sniffing the "cuddle hormone" will make you more trusting

Oxytocin is neurohormone produced by the hypothalamus in the brain and it's released when we hug or have sex, and there's some evidence that when we sniff it we become more trusting and empathic, hence its various nicknames including the "moral molecule" and "cuddle hormone". For example, in a study published in the prestigious journal Nature, and that's been cited over 2500 times, researchers reported that after ingesting oxytocin nasally, participants were more willing to give money to a stranger in a financial game.

Findings like these have led to a lot of hype – the io9 website went so far as to brand oxytocin "the most amazing molecule in the world". However, the effects of the hormone have proved to be a lot more complex than originally realised. For example, in some contexts it can elicit envy and if someone has an aggressive nature it can even increase their intentions to be violent toward their partner.

Some of the positive effects of oxytocin have also turned out to be difficult to replicate. In 2015 researchers at the Université catholique de Louvain attempted to replicate their own finding that inhaling oxytocin increases participants' trust in a researcher not to open an envelope in which the participants have inserted confidential information. Twice the researchers failed to reproduce this effect leading them to conclude that "nothing can be taken for granted about oxytocin". One possibility they considered is that all previous demonstrations of the effects of intra-nasally administered oxytocin on behaviour have been false positive results. "Even though this is a chilling hypothesis, it is still plausible," they said.

Being reminded of money makes us selfish

Money symbolises materialism and market competition. So powerful are these connotations that when exposed to reminders of money, it changes our mindset to be more selfish and less interested in equality. That's according to several studies, including a 2013 paper, led by Eugene Caruso and co-authored by Kathleen Vohs, that asked participants to complete one of two versions of a questionnaire about their personal details, one featuring a faint background image of a $100 bill. Participants exposed to the money imagery subsequently showed stronger endorsement of  the current political system and less sympathy for victims or socially disadvantaged groups.

However, as part of the "Many Labs" Replication Project, 36 international psych labs attempted to replicate the effect of money reminders on endorsement of the existing political system and only one reported a significant effect. Separately a team led by Doug Rohrer attempted to replicate all of the findings obtained Caruso and colleagues, but with larger sample sizes, and failed in each case. "Although replication failures should be interpreted with caution," Rohrer's team said, "the sheer number of so many high-powered replication failures cast doubt on the money priming effects found by Caruso et al."

In her response Kathleen Vohs highlighted that over ten years, 165 studies from 18 countries have documented psychological effects of money reminders, and that perhaps one factor influencing the results is how important money is to the participants in question – she and her colleagues had tested students at the University of Chicago, an institution renowned for its economics scholars, whereas Rohrer had mostly tested people online via Amazon's Mechanical Turk and The University of California.

"Recently I read a quotation that is not about science but could well be," Vohs concluded. "It said that democracy is valuable because it 'doesn't think of itself as finished or perfect' – and neither does science. It takes many scholars and many attempts to figure out the way the world works."