News, January 2012

Questionable research practices rife?

Questionable research practices are rife among US psychologists, according to research obtained by The Psychologist. Leslie John at Harvard Business School and her colleagues George Lowenstein of Yale and Drazen Prelec of MIT surveyed nearly 6000 academic psychologists in the US. Based on anonymous replies from 2155 of them, it's estimated that one in ten psychologists falsify data, and the majority are guilty of selectively reporting studies that 'worked' (67 per cent), failing to report all dependent measures (74 per cent), continuing to collect data to reach a significant result (71 per cent), reporting unexpected findings as expected (54 per cent), and excluding data post-hoc (58 per cent).
Admissions were higher among cognitive, neuroscience and social subdisciplines, and lower among clinical psychologists. The more questionable practices a psychologist admitted to, the more likely they were to claim such practices were defensible. However, 35 per cent of respondents said they doubted the integrity of their own research.
Surveys of this kind have been published before but this is the firstto incorporate an honesty incentive, which has led to far higher admission rates than previously identified. Participants were told, truthfully, that more money would be donated to charity by the researchers based on an estimate of their honesty. This estimate was computed by comparing the participants' own confessions to questionable practices against the average rate of admission and participants' estimates for how many colleagues indulge in such practices.
In their report on the findings, to be published in the journal Psychological Science, John and her colleagues (psychologists by background) describe the methods they investigated as representing a large 'grey zone' of acceptable practice. These practices 'threaten research integrity and produce unrealistically elegant results that may be difficult to match without engaging in such practices oneself', they concluded. 'This can lead to a "race to the bottom", with questionable research begetting even more questionable research.'
John told us that she has no data on UK research practices but has no reason to believe a survey here would produce different results. She told The Psychologist she'd like to see bodies like the American Psychological Association and the British Psychological Society put their weight behind the reforms needed to reduce the use of questionable research practices.
On the Society's Research Digest blog, which was first to break details of the study, debate centred on the seriousness (or not) of the reported practices, and on whether they say more about the wider publishing culture as opposed to individual ethics. 'Neuroskeptic' thought that 'many of the questionable practices are actually quite hard to avoid doing within the current academic publishing system'. He also queried 'why this kind of methodological self-criticism seems to be focused on psychology', although it is arguably to the discipline's credit that it is. Others highlighted the reforms needed to address the issue: Alex Holcombe (University of Sydney) pointed to his psychfiledrawer.org, set up to act as a repository for failed replication attempts in experimental psychology, which should help counter the publishing bias towards positive results.
The new survey results come hot on the heels of the unfolding investigation into the practices of the disgraced social psychologist Derek Stapel (see December news), and they make for worrying reading when considered alongside a paper 'False-positive psychology' published in Psychological Science in November (free PDF from tinyurl.com/canb33z). In that paper, Joseph Simmons at the Wharton School, University of Pennsylvania and his colleagues used computer simulations and a real example experiment to show how the questionable research practices documented by John's survey can greatly increase the risk of false-positive results (that is, finding significant effects where in fact there is no effect).
For example, they succeeded in demonstrating the logically impossible finding that listening to children's music can reduce participants' actual chronological age. They obtained that result in a between-subjects design by: testing for numerous other outlandish dependent variables, any of which could have been chosen as the 'finding' if significant (including participants' political orientation and how often they refer to the past as 'the good old days'); by increasing the participant pool after failing to find a significant result with fewer participants; and by ensuring that father's age was included as a covariate, ostensibly to control for variation in baseline age across the participants (removing father's age as a covariate rendered the result non-significant).
Each of these factors adds degrees of freedom to the statistical analyses, thus undermining the notion of a nominal false-positive rate of five per cent (i.e. p = .05). However, the reporting rules of many journals wouldn't require the researchers to disclose many of these manipulations, leading the significant finding to appear credible.Based on their demonstrations, Simmons and his colleagues call for the authors of psychology papers to conform to a set of new rules, to be policed by reviewers, including:
I    Decide the rule for data termination prior to beginning data collection and report this rule in the write-up.
I    List all dependent variables.
I    If analysis includes a covariate, report the results with and without the covariate.
I    Report all experimental conditions, including failed manipulations.

These requirements 'pose minimal costs on authors, readers and reviewers,' Simmons' team conclude. 'These solutions will not rid researchers of publication pressures, but they will limit what authors are able to justify as acceptable to others and to themselves. We should embrace these disclosure requirements as if the credibility of our profession depended on them. Because it does.' cj

Public sector strike action 

The public sector strike on 30 November was supported by several major unions, including the Association of Educational Psychologists (64 per cent support for strike action from a 30 per cent response rate) and UNITE (75 per cent support from a 31 per cent response rate).
Dr Khadj Rouf, Consultant Clinical Psychologist and member of the Applied Psychologists Occupational Advisory Committee, Unite the Union, told The Psychologist: 'We should not be afraid to speak out at a time when patient care is under threat, and when the rights of the public sector workforce are being eroded.'
How were services affected? Dr Rouf said: 'As healthcare providers, we care about our patients and clients. There was crisis cover across health services on the day of the strike, so it was very similar to cover provided over Christmas. The action did not compromise patient care. Cuts to public services will have a far worse impact on services and patient care, and for more than one day.'
A member of the Leadership and Management Faculty of the Society's Division of Clinical Psychology reported: 'The general feeling was that not that many were striking and certainly some of the more junior (and often younger) were not always union members. Five out of 12 consultant clinical psychologists went on strike. Of the ones who were on strike, a couple said they felt that it was an important principle whether they would be protected from the changes or not.'
Susan van Scoyoc, Acting Chair of the Society's Standing Committee for Psychologists in Health and Social Care, said: 'SCPHSC, along with other parts of the Society, are aware of the changes taking place throughout the NHS and the public sector generally. We have expressed opinions on how these changes are likely to impact upon the users of such services and these can be seen in the Society responses to various government lead consultations (see www.bps.org.uk/consult). Individual members have differing views on how to respond to these changes but it is clear we all share one overriding concern – to provide the best quality of care to our clients now and in the future.'
I    For more comment from Dr Rouf, see 'Letters'. 

1 in 12 Self-harm
The first population-based study to assess the course of self-harm from adolescence to young adulthood has found that around one in 12 young people self-harm, with the balance skewed towards girls.
Published in The Lancet (see tinyurl.com/cfo52y5), the cohort study was conducted between August 1992 and January 2008 in Victoria, Australia, with participants aged 14–15 at the outset. The researchers, led by Paula Moran (Institute of Psychiatry), chose this period from adolescence as one 'characterised by major changes in health and a steep rise in deaths resulting from self-inflicted injuries'. Risks for self-harm increased substantially across puberty, 'a process that seems to be independent of age' according to the authors. Self-harm during adolescence was independently associated with the presence of depression and anxiety, antisocial behaviour, high-risk alcohol use, cannabis use, and cigarette smoking. Injury to the skin through cutting and burning was the commonest method of self-harm during adolescence, although by young adulthood no one form of self-harm predominated.
There is some good news though: 90 per cent of people who self-harm as adolescents will naturally stop in adulthood. 'Our findings suggest that most adolescent self-harming behaviour resolves spontaneously,' the authors said. 'However, young people who self-harm often have mental health problems that might not resolve without treatment, as evident in the strong relation detected between adolescent anxiety and depression and an increased risk of self-harm in young adulthood.'
Commenting on the age-related decline in self-harm in The Lancet (tinyurl.com/cesa42q), Keith Hawton (University of Oxford) and Rory O'Connor (University of Stirling) considered that as young people move from adolescence to young adulthood, the extent of exposure to peer self-harm might decrease. They also referred to a possibility not addressed by Moran and colleagues: the extent to which clinical interventions might have contributed to the reduction in self-harm. 'The results of Moran and colleagues' study will offer some reassurance to parents of adolescents who self-harm and to health and educational agencies,' Hawton and O'Connor said. 'Clinicians can offer encouragement to both young people who are self-harming and their families.' js

The mystery of music

The latest of the 'Plug in your brain' public lectures at the University of Westminster came in the form of a delightful neuroscience and music mash-up. Neuropsychologist and life-long musician Dr Catherine Loveday collaborated with husband and guitarist Darren Loveday, pianist Anna Tilbrook and soprano Joanne McGahon. Together they moved and entertained the audience with their live performances, as Dr Loveday attempted to 'unravel the mystery' of music's power.
Music is 'fundamental, universal and ubiquitous', Loveday explained. We learn to appreciate music naturally, she said, and relics of bone flutes and other archaeological evidence show its influence through human history. As Tilbrook soothed sore minds with Chopin's 'Fantastie' Impromptu, findings on the health benefits of music flowed onscreen, including its ability to: lower blood pressure; reduce pain in palliative care; improve sleep; alleviate allergies; boost immune function; and reduce depression.
It's not just humans that are affected by music. Dogs are calmed by classical pieces, Loveday said, and they bark to rock songs. Unpublished research suggests slow music can boost the milk yield of cows. 'And my favourite,' Loveday said: 'rats subjected to 24-hours of stress-inducing rock music take longer to heal from their wounds.'
But what exactly is music? Essentially, Loveday said, it's our ear-drums vibrating. It's organised sound. Tilbrook played various versions of 'Happy Birthday' to demonstrate the effects of altering pitch, harmony, tempo, timbre, loudness and dissonance. There needs to be a regular rhythm so that we can predict what comes next and then 'Bam!' – it's the meaningful violations to that pattern that can so move us. 'It's those little changes, those little violations of our expectations that make the heart flutter and cause us to respond,' Loveday said.
A 2004 study documented over 100 ways people described the way that music made them feel, from ecstatic to spiritual. Underlying these emotional responses to music are physical changes to the body and brain. Music affects our cortisol levels (a hormone involved in stress), SigA (an antibody), our endogenous opiates, dopamine (a neurotransmitter implicated in the anticipation of reward) and oxytocin (nicknamed the 'cuddle hormone' because of its role in attachment and bonding). Music can also trigger a chill response; sometimes known as a frisson. 'The chill response is something that  I reckon every single person in this room has had,' Loveday said. 'It's that shiver down the spine that you get when you listen to a particular moment in music, so you're listening to it and it just moves you.'
The chill response has been studied extensively, not least because it's easy to measure and observe. It's associated with changes in skin conductance (because of sweat), heart rate, temperature, breathing and most people can describe and identify when it's happened. The chill response has even been observed in chicks and Loveday took her place at the piano to play the piece that was found to ruffle their features: Pink Floyd's 'Post War Dream'.
What happens in the brain when we're listening to music? Virtually every part of the brain is affected from the cerebellum, involved in movement and rhythm, to the amygdala, associated with emotional learning. Musical enjoyment is correlated with activity in the caudate in the limbic system, which tracks musical anticipation. And peak musical pleasure is associated with nucleus accumbens activity. The fact that music and sex trigger activity in similar parts of the brain shows once again, Loveday said, that music is a 'fundamental human activity'.
Loveday next discussed why music has these effects on us. Some of if it is no doubt learned by association, she said. As Tilbrook demonstrated at the piano, the iconic two-note repeat from the movie Jaws still has the power to unnerve people. There's also an element of mood contagion, including responding to the performer's own emotion. Some of it is the cognitive effects of expectancy violation, discussed earlier. And finally, some of it is primal and innate. This was demonstrated with great power as Joanne McGahon took to the stage to sing 'Vissi d'arte' from Puccini's Tosca, her surging, soaring voice seared with grief. The entranced audience were moved despite not being able to understand a word of the Italian.
'Laughter, screaming and crying… it's all basically a form of music and we're innately primed by it,' Loveday said. 'So we've got these primal, direct effects, but then as we learn the language of music more, our appreciation deepens, we are able to use sounds in more complex ways and the more complex our emotional response becomes.' cj

Educational psychology training 

The future of educational psychology training in England has been secured, at least for the short term. After months of uncertainty, the Department for Education has earmarked £16 million for the duration of the current Spending Review, which runs until 2014/15. These funds will cover trainees' first-year tuition fees and a bursary towards living expenses. Local authorities and other employers will be expected to fund bursaries and other costs for years two and three of the educational psychology doctorate.
The announcement came as the government published its review into the training of educational psychology in England. It discusses alternative training models for educational psychology, including the idea of a 'Family psychologist', which combines elements of educational and clinical psychology training, and the 'Fast-track model', which would reduce training to two years. However, the preferred option is for the existing three-year doctorate model to continue.
The most significant development is the review's call for the creation of a new national steering group to manage the relationship between training and placement providers. This will include overseeing an accreditation process for placement providers, which it's hoped will provide quality and consistency for trainees. The steering group would comprise: 'local authority employers, the profession, training providers, placement providers, the Association of Educational Psychologists, the National Association of Principal Educational Psychologists, the Health Professions Council, the British Psychological Society and the Association of Child Psychologists in Private Practice'. In turn, the management of this steering group will be overseen first by the Children's Workforce Development Council (CWDC) and then from April 2012 by the Teaching Agency.
The new proposals follow a period of grave uncertainty for educational psychology, which sees about 120 students enter training each year at an annual cost of about £10,000 each. The recent model has been for local authorities to make voluntary contributions towards the costs of the 12 educational psychology training providers. However, the recession and other factors has led these contributions to all but dry up. The government has agreed to meet the shortfall for 2012/13, prior to funding all year one costs for the 2013 trainee cohort.
Another issue raised by the review is the fact that there's been no systematic evaluation since 1978 of the educational psychology workforce and the demands placed on it. 'We believe there is a strong argument for undertaking a robust workforce modelling exercise,' the review says. To this end it recommends that the CWDC and then the Teaching Agency undertake regular surveys of the educational psychology workforce and the demands placed on it.
The Children's Minister Sarah Teather said that educational psychologists fulfil a valuable role in their work with children and families in schools, and as part of early intervention projects. 'We want the most vulnerable children, and those who would benefit from extra support, to be able to access the expertise and support of educational psychologists,' she said. '[The £16 million earmarked by the Department of Education] helps to secure the future training of educational psychologists and is part of the work we are doing on the SEN green paper.'
Dr Jane Leadbetter, Chair of the Society's Division of Educational and Child Psychology, told us that she welcomed the findings of the review, 'which confirmed that the current three-year training model, at doctoral level, is a success and is fit for purpose.' She added: 'It is reassuring that funds are being provided to sustain training over the next few years and that a seamless process whereby university time and time spent in educational psychology services will be set up and properly managed. Of concern is the ongoing cuts to local authority services around the country which is having a direct impact upon EP posts and the work that can be undertaken at preventative and systemic levels.' cj
I    The review Developing Sustainable Arrangements for the Initial Training of Educational Psychologists is at tinyurl.com/bpceg6c

A journey in the fast and slow lanes

Anyone with an interest in the foibles of human reasoning has been spoilt over the last decade. A succession of popular books from David Myers' Intuition: Its Powers and Perils (2002) to Jonah Lehrer's The Decisive Moment (2009) have documented the biases and heuristics that shape our attitudes and decisions. Every single one of these books cites the influential work of two psychologists – Daniel Kahneman and Amos Tversky – for which Kahneman was awarded the Nobel Prize for Economics in 2002 (Tverksy died in 1996 and was therefore ineligible for the prize).
Now we get to hear from the pioneer himself: Professor Kahneman of Princeton University has finally published his own popular account of his field: Thinking, Fast and Slow (Allen Lane), described by the New York Times as 'a lucid and profound vision of flawed human reason in a book full of intellectual surprises and self-help value'.
In November, Kahneman promoted his book at the LSE, 'in conversation' with Professor Lord Richard Layard, the architect of the government's Improving Access to Psychological Therapies programme. Layard began by asking what is meant by 'thinking fast and thinking slow'. This is a reference to the idea that we have two forms of mental process, Kahneman said: System 1 and System 2 (these are metaphors, he further explained, rather than literal brain systems).
System 1 operates all the time, and more often than not we're guided by it – such as when we're walking or driving and we don't have to think consciously about what we're doing. System 2 monitors and interprets System 1 and it comes into play when we think effortfully and consciously about a problem. Asked to solve '2 + 2', System 1 would deal with it, producing the answer automatically and without effort. Challenged with '22 x 17' and an answer probably won't come to mind immediately – you have to reflect effortfully on how to solve the problem. 'My standard example,' Kahneman said, 'is that if you have to stop doing [a mental task] when you make a left-turn into traffic, then it's effortful.'
Kahneman said there's much psychological insight to be gained by investigating what System 1 can and can't do. 'It can do wonderful things,' he said, 'but it has strange limitations.' As an example of a useful System 1 skill, he said he was able to determine his wife's mood from the first word she utters on the phone. On the other hand, System 1 produces mistakes when it doesn't have a skilled answer. Here Kahneman gave the example of people's judgements about the likely university grades of a woman who they're told learned to read fluently at age four. An idea will come to their mind instantly (thanks to System 1) based on assumptions about the proportion of people who read at age four and how that correlates with later academic achievement. However, these implicit statistical assumptions are mistaken and neglect many other factors. Consequently, people's predictions about the woman's grades will tend to be far too extreme.
Examples like this show how System 1 treats whatever information it has (the woman's reading precocity in this case) as if that is all the information that matters. Another more striking example: Kahneman cited research from the 1990s that asked some people to say how much they'd be willing to pay for travel insurance against death by any means. Their answers were compared against a second group who were asked how much they'd be willing to pay for insurance specifically against death by terrorism. The terrorism group were willing to pay substantially more money! 'This is absurd,' Kahneman said. 'The way System 1 deals with this question is that there's something you know immediately – how afraid you are. And that's it, you translate your answer from something System 1 produced, an evaluation based on fear and emotion.'
These insights beg the question, Layard said: should the government take our flawed thinking into account in the way that it formulates policy? Kahneman argued that indeed, the outdated economic idea that humans are perfectly rational has had some pernicious consequences for government policy, most of all the idea that people don't need to be protected against their own mistakes. 'People do need protection against their own mistakes,' Kahneman said, 'because they make highly predictable mistakes – including in savings and insurance. Furthermore, they need protection against predators because they will disclose all the relevant information only in small print. Rational agents might read the small print, but people don't.'
What about implications for education, Layard asked. Can people be taught to counteract the flaws in human reasoning? Kahneman confessed that he is a pessimist in this regard, whereas he realises that Layard is an optimist. 'I don't think reading this book will help you,' Kahneman admitted. 'Writing it certainly hasn't helped me!'
So, is there any hope? Kahneman said there was a benefit to be had in introducing a more sophisticated language of gossip. We're all far more conscious of other people's mistakes than our own, he explained, and by providing a more informed terminology for talking about people's errors, our judgements and understanding will improve. 'So there's some hope,' he said, 'but not much.' He added that institutions could improve themselves by avoiding known biases. Layard stepped in to give the example of interviews, which research shows are a highly ineffective selection tool, thanks in large part to the misleading power of first impressions and other prejudices. He said the LSE had done away with student interviews, but that an amazing amount of time continued to be wasted on interviews at universities like Oxford and Cambridge.
Layard moved the discussion onto well-being – a topic that Kahneman has focused on in recent years. In particular his research has shown how a distinction needs to be made between people's overall satisfaction with life, and their (hedonic) moment-by-moment experience of happiness and misery. The two are not the same and don't always correlate. Kahneman said that unlike Layard hewas more concerned with reducing misery than promoting happiness (Layard demurred, saying this was his priority too) and he described the UK's plans to measure citizens' well-being as an 'ambitious effort'. But he fears the levers of government policy probably won't make much difference.
'We're at the beginning of our understanding of well-being,' Kahneman said. 'There are so many empirical questions that we don't know.' For example, there are no doubt medical consequences of well-being, he said, yet we don't currently know whether life satisfaction or hedonic experience is the more important. 'Having answers to these kinds of questions will help philosophers, policy makers… assign relevant weights to the different dimensions, but we're really at the beginning of that journey.' cj
I    Listen to the audio at tinyurl.com/cy5k4nf

For students, North and South
We hear from the Society's Psychology4Students days in Preston (reported by Catherine Loveday, University of Westminster) and Watford (reported by Jon Sutton, Editor)

The University of Central Lancashire was this year's enthusiastic host to the Society's 'Psychology4Students' North lectures. With 350 students attending, the event was once again a sell-out and there was a palpable buzz of excitement in the hall as Society President Carole Allan introduced Mark Wetherell (Northumbria University) to open with his talk on stress and how it affects us.
'I want you to think about a time when you've experienced an acutely stressful event,' said Wetherell. 'How do you feel?' A steady flow of responses came flooding back, illustrating the brain's fast adrenalin-mediated stress mechanism. A similar question about longer-term periods of stress enabled Wetherell to explain the slower, cortisol-activated stress response, which switches off the long-term processes so that resources can be directed to deal with immediate threat. However, if those stresses don't go away, he explained, the long-term processes remain switched off, leading to a whole host of negative consequences including ill-health, insomnia and fertility problems.
Wetherell went on to give a very clear explanation of the dynamics of cortisol and showed data from a range of his experiments, including some work with Ecstasy users that showed how regular chronic use of the drug causes major disruption to cortisol regulation and stress responses. The ecstasy studies certainly intrigued the students but, judging by the reaction of the crowd, Wetherell's trump card was showing photos of his young son on each of the first six days of his life, alongside graphs of Wetherell's own cortisol profiles. 'What a geek!' he said of himself, 'but what a perfect illustration that having a baby messes with male hormones too!'
Next up was Charlie Frowd (University of Central Lancashire) to talk about his award-winning work with the police force, developing a new system for constructing the face of a criminal. He asked the crowd to try and guess the identity of a range of well-known faces, constructed using old photo-fits and the more modern E-FIT system. Frowd's experiments show that recognition of faces from these constructed images is shockingly poor though, under 20 per cent.
In an attempt to improve on these approaches, Frowd and his colleagues have developed a new system, based on the principle that face recognition is far better than recall. His new 'Evo-FIT' system provides an array of faces from which the witness selects the face most like the one they remember seeing. This selection is used to produce a new array and through an iterative process a best likeness is produced. Frowd concluded with impressive statistics that the new system has so far had an arrest rate around 40 per cent and helped to catch some very notorious local criminals.
The morning session was brought to an end by my own lecture, 'Lost in Music' (see p.12), stepping in as a last-minute replacement.
The afternoon started with Deborah Riby (Newcastle University) providing insight into the way in which children with autistic spectrum disorder (ASD)
and Williams syndrome (WS) look at and process faces. Children with ASD are known to have significant difficulties with social communication, Riby explained, whereas children with WS are known to be very socially driven and highly empathic.
Riby and her colleagues have been able to show that these two groups of children process faces very differently. Those with ASD have a greatly reduced face gaze compared to typical children, whereas children with WS spend far longer fixated on the face, in particular the eyes, and fail to look at other non-facial clues or to look away when thinking, as typical children do. This may be why WS children, despite being very sociable, still have significant problems with peer relations. This research, explained Riby, provides huge insight into the exciting and growing field of social neuroscience.
The day ended with another much appreciated replacement, Dave Shaw from Lancaster University, reprising last year's talk (see February 2011) on the importance of psychology in sport.