Was that new Science paper hyped and over-interpreted because of its liberal message?

It would be very concerning if "girls as young as six years old believe that brilliance is a male trait", as The Guardian reported last week, especially if "this view has consequences", as was argued in The Atlantic.

Both stories implied girls' beliefs about gender could be part of the explanation for why relatively few women are found working in fields such as maths, physics, and philosophy.

These news stories, widely shared on social media, were based on a new psychology paper by Lin Bian at the University of Illinois at Urbana-Champaign and colleagues, published in Science, entitled "Gender stereotypes about intellectual ability emerge early and influence children's interests".

The paper reported four studies, which at first appear to have simple, clear-cut conclusions. But a closer look at the data reveals that the results are rather weak, and the researchers' interpretation goes far beyond what their studies have shown.

In one task in the first study, the children were told about a "really, really smart" person, then asked to choose, from a selection of pictures of males and females, who they thought the description was about. At ages 5, 6, and 7, boys tended to pick a male picture, suggesting they linked being male with being smart (at the three ages, 71 per cent, 65 per cent, and 68 per cent of boys showed this "own-gender" bias, respectively).

On the other hand, girls linked being female with being smart 69 per cent, 48 per cent, and 54 per cent of the time at ages 5, 6, and 7, respectively.

Results were similar in a second study, except that this time, the percentage of girls displaying own-gender bias never dropped below 52 per cent. So, at all ages except 6 (and then only in one of two studies), girls were more likely than not to assume that a description of a "really, really smart" person was about a female than about a male.

Stated this way — and although boys' own-gender bias was stronger than girls' — the finding certainly appears less worrying than was implied by much of the media coverage.

Although the stronger "own-gender" bias for boys existed in relation to general "smartness", it wasn't present when the children were asked a more specific question about which gender gets the better school grades. For that question, the girls in fact had a stronger own-gender bias than the boys. So whatever was causing the girls to believe stereotypes about smartness, it wasn't making them believe stereotypes about school performance.

But the big statement in the title of the paper comes from the third study, where the researchers found that boys were more likely than girls to be interested in playing a (fictional) game that they were told was aimed at "really, really smart children" (as opposed to a game for "children who try really, really hard").

The researchers contend that the children's gender stereotypes were the underlying reason for this, arguing that a statistical analysis known as mediation modelling "pinpoint[ed] these stereotyped perceptions… as the underlying mechanism". But, first, they only tested one mediation model with one possible mediator, so nothing was "pinpointed".

Second, the mediation was only partial: differences in the strength of the girls' and boys' average gender-biased stereotypes could only explain a small portion of the differences in their game preferences (moreover, this mediation was only just statistically significant in Study 3, and there was no attempt at a replication in Study 4).

Third, and most critically, this wasn't an experiment: all of the data were purely correlational, and it was therefore impossible for the analysis to have revealed anything about what causes what.

Indeed, unwarranted causal claims abound in the paper, from the title ("…influence children's interests") to the abstract ("…have an immediate effect…") to the concluding paragraph ("…stereotype begins to shape children's interests…"). None of them can be supported by the data presented.

To back up these claims, one would need to arrange an experiment: randomly split children into two groups, intervene to manipulate the stereotypes of one group but not the other, then compare their interests. Nothing even approaching that was done for this paper.

Why would the reviewers and editors at one of the world's top scientific journals allow through a paper with such glaring correlation-causation errors?

One possibility is political bias: these conclusions align with the generally liberal political opinions that many scientists, particularly social psychologists, hold. Perhaps the gatekeepers let their guard down a little because they agreed with the message that the gender disparity in science might have social causes (of course, this is mere speculation on my part).

Political bias might also have been the reason that the paper fails to cite any of the many studies suggesting that there actually is an over-representation of males in the "really, really smart" end of the intelligence distribution (and also the very low end), even if only to argue against them.

We'd all agree that, if young girls have their career ambitions derailed by their own broad-brush stereotypes about the abilities of their sex, this is tragic, as well as damaging to the academic fields that they avoid. But assessing whether this is actually the case (not to mention doing something about it) requires extremely careful science.

As has been noted elsewhere, if one wants a strong basis for one's political position, one should be extra-sceptical of studies that appear to support it. Given the widespread sharing, reporting, and discussion of this interesting yet hyped paper, that scepticism doesn't seem to have been applied here.

Further reading

—Gender stereotypes about intellectual ability emerge early and influence children's interests (pdf)

About the author

Post written by Stuart J. Ritchie for the BPS Research Digest. Stuart is a Research Fellow in the Centre for Cognitive Ageing and Cognitive Epidemiology at the University of Edinburgh. His new book, Intelligence: All That Matters, is available now. Follow him on Twitter: @StuartJRitchie