From the Research Digest, October 2015

When psychologists tried to replicate 100 previously published findings
In Science

After some high-profile and at times acrimonious failures to replicate past landmark findings, psychology as a discipline and scientific community has led the way in trying to find out more about why some scientific findings reproduce and others don't, including instituting reporting practices to improve the reliability of future results. Much of this endeavour is thanks to the Center for Open Science, co-founded by the University of Virginia psychologist Brian Nosek.

In August the Center published its latest large-scale project: an attempt by 270 psychologists to replicate findings from 100 psychology studies published in 2008 in three prestigious journals that cover cognitive and social psychology: Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory and Cognition.

The Reproducibility Project is designed to estimate the 'reproducibility' of psychological findings and complements the Many Labs Replication Project, which published its initial results last year. The new effort aimed to replicate many different prior results to try to establish the distinguishing features of replicable versus unreliable findings – in this sense it was broad and shallow and looking for general rules that apply across the fields studied. By contrast, the Many Labs Project involved many different teams all attempting to replicate a smaller number of past findings – in that sense it was narrow and deep, providing more detailed insights into specific psychological phenomena.

The headline result from the new Reproducibility Project report is that whereas 97 per cent of the original results showed a statistically significant effect, this was reproduced in only 36 per cent of the replication attempts. Some replications found the opposite effect to the one they were trying to recreate. This is despite the fact that the Project went to great lengths to make the replication attempts true to the original studies, including consulting with the original authors.

Just because a finding doesn't replicate doesn't mean the original result was false – there are many possible reasons for a replication failure, including unknown or unavoidable deviations from the original methodology. Overall, however, the results of the Project are likely indications of the biases that researchers and journals show towards producing and publishing positive findings. For example, a survey published a few years ago revealed the questionable practices many researchers use to achieve positive results, and it's well known that journals are less likely to publish negative results.

The Project found that studies that initially reported weaker or more surprising results were less likely to replicate. In contrast, the expertise of the original research team or replication research team was not related to the chances of replication success. Meanwhile, social psychology replications were less than half as likely to achieve a significant finding compared with cognitive psychology replication attempts, but in terms of declines in size of effect both fields showed the same average reduction from original study to replication attempt, to less than half (cognitive psychology studies started out with larger effects, and this is why more of the replications in this area retained statistical significance).

Among the studies that failed to replicate was research on loneliness increasing supernatural beliefs; on conceptual fluency increasing a preference for concrete descriptions (e.g. if I prime you with the name of a city, that increases your conceptual fluency for the city, which supposedly makes you prefer concrete descriptions of that city); and links between people's racial prejudice and their response times to pictures showing people from different ethnic groups alongside guns.

A full list of the findings that the researchers attempted to replicate can be found on the Reproducibility Project website (as can all the data and replication analyses: see https://osf.io/ezcuj).

This may sound like a disappointing development for psychology, but in fact really the opposite is true. Through the Reproducibility Project, psychology and psychologists are blazing a trail, helping shed light on a problem that afflicts all of science, not just psychology. The Project, which was backed by the Association for Psychological Science (publisher of the journal Psychological Science), is a model of constructive collaboration showing how original authors and the authors of replication attempts can work together to further their field. In fact, some investigators on the Project were in the position of being both an original author and a replication researcher.

'The present results suggest there is room to improve reproducibility in psychology,' the authors of the Reproducibility Project concluded. But they added: 'Any temptation to interpret these results as a defeat for psychology, or science more generally, must contend with the fact that this project demonstrates science behaving as it should' – that is, being constantly sceptical of its own explanatory claims and striving for improvement. 'This isn't a pessimistic story', added Brian Nosek in a press conference for the new results. 'The project shows science demonstrating an essential quality, self-correction – a community of researchers volunteered their time to contribute to a large project for which they would receive little individual credit.' cj

To hear about what it was like to take part in the project, and for links to further coverage, see https://thepsychologist.bps.org.uk/reproducibility-project-disaster-or-triumph-psychology

Having strong political skills can be a drawback in the workplace
In Journal of Applied Psychology

If you overheard someone at work refer to you as 'a real political operator', would you feel complimented, or alarmed? The latter turns out to be a sensible reaction, as new research suggests that supervisors and colleagues have less faith in the performance of the highly politically skilled.

Study authors Ingo Zettler and Jonas Lang noted a conundrum in their field: researchers treat political skill as a uniform good, the more the better, yet a meta-analysis found a spotty relationship between more political skill and improved outcomes like job performance. Might deft politicking, however well intentioned, create suspicions in co-workers? Once others lose trust in a politically focused performer, their ability to get things done is stymied. Or perhaps habitually working the angles leads highly skilled individuals to make like Machiavelli and potentially do harm.

Zettler and Lang predicted that thanks to these reasons, those who live and breathe political approaches would actually do worse at their jobs compared with those merely competent in political skill. This prediction was confirmed in two studies. The first, involving on-the-job apprentices, found that the relationship between self-ratings of political skill and supervisors' ratings of their job performance was positively correlated, but only up to a point. Beyond a political skill score of 3.5 on a five-point scale, supervisor ratings flatlined and then began dropping. The second study found the same overall pattern in employees with longer work experience, each rated by a supervisor and also a colleague. This study also found that this 'curvilinear relationship' between political skill and job performance (whereby intermediates in political skill outperformed low- and high-skilled participants) – was most pronounced when the rater was not personally close to the participant. Savviness and bluntness alike can be forgiven by close colleagues – 'that's just how Chris gets things done' – but others are less trusting.

What is it like to be a refugee with psychosis?
In Psychosis

We're in the midst of a 'migrant crisis' as tens of thousands of brave, desperate people seek new lives in Europe, risking life and limb to get here. Amidst the tragedy and controversy, the continued plight of those people who actually make it to relative safety is often forgotten. Unsurprisingly, given all they've endured, refugees often have serious mental health problems, including hallucinations. As an indicator, research published in 2011 reported that 80 per cent of 130 young Somali refugees surveyed in Minnesota had symptoms of psychosis.

Now a timely, heart-rending study published in Psychosis has reported the results of in-depth interviews with seven African refugees or asylum seekers in the UK (aged 26 to 43; one woman), all of whom reported experiencing symptoms of psychosis. The researchers' aim was to gain insight into the 'lived experience' of their participants. This is the first time the first-hand perspective of refugees with psychosis has been documented. 'Such information is crucial for understanding and working with such clients,' the researchers said.

Clinical psychologist John Rhodes and his colleagues analysed the interview transcripts and identified six key, recurring themes in their participants' accounts. The first was bleak agitated immobility – the participants' sense that their lives were going nowhere. One participant likened the feeling to being in a never-ending race. Similarly, Amine (aged 43) said: 'I feel like I'm finished. There's no life, there's no future, there's no anything any more. I think everything is going to become like darkness.'

The second theme was trauma-related voices and visions. These tended to be the sounds or sights of lost relatives or attackers from the past. Belvie (aged 30, female) heard voices of a past torturer, and the torture itself also had a voice: 'Some voice I have it's like from the past. But some of them are not from the past. I don't know. Sometimes it's like a voice of the thing that was done to me when I was back home, when I was tortured. Sometimes I hear the voice of that person.'

Reflecting on the nature of such symptoms, the researchers see them as distinct from the flashbacks associated with post-traumatic stress (PTSD). 'The trauma-related intrusions did not appear to be relived experiences in the classic "PTSD" sense,' they explained, 'but rather to be engrossing and believable perceptions "flavoured" by past trauma.' Such experiences do not fit well with conventional Western psychiatric categories, they argued. Rather than interpreting their participants' hallucinations as indicative of schizophrenia or PTSD, they suggest a neutral description: 'complex trauma with perceptual disturbance'.

The participants also described their powerful feelings of fear and mistrust. Belvie feared a man on a bus was planning to kill her just because he looked at her. All the participants also had a sense of a broken self. 'My emotional state has changed and my personality has changed… I really haven't been alright,' said Frederic (aged 39). They also described the pain of losing everything. 'The degree of loss for these participants is difficult for us to understand,' the researchers said. 'They have lost their worlds. A new location or role does not replace "home", that place this, many of the painful feelings described by the interviewees, such as there being no future, were the same regardless of whether they'd be granted asylum (as four of them had) or whether they were still waiting to hear about their status.

The final theme concerned the attraction of death. Several of the participants described past suicide attempts and the unbearable strain of life. 'The worst part,' said Sando (aged 26), 'is I keep harming myself… and you know knocking my head to the wall, kinda too much stuff in there, you know, I just want to open my head and finish with this.' Yet, the participants also expressed optimism. The researchers described the participants' wish to die 'held in tension with their wish to live and build a purposeful and worthwhile life'. This final theme is important for clinical services, the researchers said, which 'need to recognise that while many [refugees] speak of building a new life, there is an attraction to suicide as escape'. cj

Weird things start to happen when you stare into someone's eyes for 10 minutes
In Psychiatry Research

A psychologist based in Italy says he has found a simple way to induce in healthy people an altered state of consciousness – simply get two individuals to look into each other's eyes for 10 minutes while they are sitting in a dimly lit room. The sensations that ensue resemble mild 'dissociation' – a rather vague psychological term for when people lose their normal connection with reality. It can include feeling like the world is unreal, memory loss, and odd perceptual experiences, such as seeing the world in black and white.

Giovanni Caputo recruited 20 young adults (15 women) to form pairs. Each pair sat in chairs opposite each other, one metre apart, in a large, dimly lit room. Specifically, the lighting level was 0.8 lx, which Caputo says 'allowed detailed perception of the fine face traits but attenuated colour perception'. The participants' task was simply to stare into each other's eyes for 10 minutes, all the while maintaining a neutral facial expression. A control group also sat in a dimly lit room in pairs, but their chairs faced the wall and they stared at the wall. Beforehand both groups were told that the study was going to involve a 'meditative experience with eyes open'.

When the 10 minutes were over the participants filled out three questionnaires: the first was an 18-item test of dissociative states; the other two asked questions about their experience of the other person's face (or their own face in the control group).

The participants in the eye-staring group said they'd had a compelling experience unlike anything they'd felt before. They also scored higher on all three questionnaires than the control group. On the dissociative states test, they gave the strongest ratings to items related to reduced colour intensity, sounds seeming quieter or louder than expected, becoming spaced out, and time seeming to drag on. On the strange-face questionnaire, 90 per cent of the eye-staring group agreed that they'd seen some deformed facial traits, 75 per cent said they'd seen a monster, 50 per cent said they saw aspects of their own face in their partner's face, and 15 per cent said they'd seen a relative's face.

Caputo thinks the facial hallucinations are a kind of rebound effect, as the participants in the eye-staring group returned to 'reality' after dissociating. This is largely speculation and he admits that the study should be considered preliminary. I'd also highlight that while it's true the eye-staring group scored higher than controls on dissociative states, they didn't score any of the items on the scale higher than 2.45, on average, on a five-point scale (where 0 is 'not at all' and 5 would be 'extremely').

We don't know what the crucial elements of the eye-staring exercise were for inducing the described effects (nor why they had these effects). We can infer that low lighting was not the only important element because the control group sat in the same dim room. Other clues come from prior research finding that simply staring at a dot on the wall for a prolonged duration can induce dissociative-like states, as can staring at one's own face in the mirror. However, comparing the questionnaire scores in the current study with those reported in his past research, Caputo says that what he calls 'interpersonal gazing' has a more powerful dissociative effect than staring into a mirror. cj

What do long-distance runners think about?
In International Journal of Sport and Exercise Psychology

Marathon runners are on the road for hours at a time, what on earth goes through their minds? Past investigations have relied on asking runners to remember what they were thinking, but of course that is an unreliable method. Now Ashley Samson and her team have conducted the first ever 'think aloud' investigation of long-distance runners, which involves them verbalising 'everything that passes through your head'.

The researchers recruited 10 amateur long-distance runners (four women) with an average age of 41, all with a habit of running long-distance at least three times a week. All were in training for a half-marathon or longer distance. The runners were given some practice recording their thoughts while on a treadmill. Then they were given the equipment and asked to record their thoughts while out on a real run of at least seven miles.

The researchers ended up with over 18 hours of recordings to analyse, with the runners' thoughts falling into three distinct categories. The majority (40 per cent) of thoughts pertained to pace and distance, showing just how important it is even in a non-competition context for long-distance runners to continually calculate their optimum speed, considering their energy levels and the distance left to cover. This category included thoughts to do with monitoring pace (e.g. 'downhill, don't kill yourself, just cruise'); strategies to maintain pace, such as correcting form (e.g. 'lean and steady, make it a long stride, lean and steady'); and thoughts about altering pace (e.g. '6.50 mile that's alright ... 2 miles to go ... 6.20 that's better').

Making up 32 per cent of all thoughts, the next major category was, perhaps unsurprisingly, pain and discomfort. This included thoughts about injuries (e.g. 'My hips are a little tight. I'm stiff, my feet, my ankles, just killing me this morning'); about the causes of pain and discomfort (e.g. 'Hill, you're a bitch ... it's long and hot'); and thoughts about coping, including motivational strategies (e.g. 'neck and shoulder relax'; 'that sucked but it's going to be an awesome run on the way back').

The final category, making up 28 per cent of all thoughts, pertained to thoughts directed outwards to the environment. This included thoughts about geography, especially those nasty hills, and the weather (e.g. 'I need it to start raining'); admiration for scenery (e.g. 'it's so beautiful, the ocean, the mountains'); thoughts about wildlife (e.g. 'hope I don't see any snakes'), and finally, thoughts about traffic and other runners and cyclists (e.g. 'this is such a fucking busy street. I hate it'; 'ton of bikes out now ... I've been passed by 20 of them').

If you were wondering whether long-distance runners use the time to solve life's dilemmas – relationship troubles or metaphysical conundrums, say – it seems not, at least not in this sample of runners anyway. They're too busy focusing on their performance, bodily sensations and surroundings. Of course, it's likely the participants censored some of their thoughts, so we can't know for sure.

This is the first time long-distance runners' thoughts have been recorded live, and the researchers said there were some specific insights that could be useful to sports psychologists. For example, they noted that nearly all the runners recorded thoughts near the beginning of the run that suggested they were finding it difficult, but things nearly always seemed to get easier as the run progressed. From a practical perspective, it would be interesting if future research using this methodology could identify specific thoughts or thought styles that tend to correlate with better pacing and performance. cj

Digest Digested
Full reports are available at www.bps.org.uk/digest

You might want to avoid pulling a 'duck face' the next time you take a selfie. Research with users of the Chinese Sina Weibo microblogging website found that people who posed with their lips pouted in this exaggerated fashion were judged to be more neurotic and less conscientious than others. Computers in Human Behavior

People who think they have expert knowledge in a given field are particularly prone to 'over-claiming' in that area – that is, saying they are familiar with impossible words or concepts that don't really exist. Psychological Science

A comparison of five-month-olds' sitting ability across six cultures revealed some striking differences. For example, none of the Italian infants studied showed independent sitting compared with 92 per cent of the Cameroonian infants. Journal of Cross-Cultural Psychology

Free personality tests based on the Big Five factors of personality are more reliable than proprietary versions. That's according to an assessment of the tests' 'internal consistency'. Researchers say the reason could be that users of paid-for tests are usually prohibited from changing them, thus preventing any chance of refinement. Journal of Psychology

Gay people's 'coming out' experiences are related to their psychological wellbeing 10 or more years later. Specifically, participants who recalled more negative reactions from a friend or family member tended to be less happy in the present day, an association that was mediated by their having weaker feelings of autonomy in their relationship with that person. Self and Identity

Taking part in a brain-scan experiment appears to change how children think about brains and minds. Eight-year-old children who'd had their own brain scanned two years earlier were more likely than controls to say that dreaming and imagination require both the mind and the brain. Trends in Neuroscience and Education

Teenagers are the most prolific liars, while young adults (aged 18 to 29) are the most skilled. That's according to a test and survey conducted with members of the public aged up to 77 years who visited a science museum in Amsterdam. Acta Psychologica