"They're so focused on the symptoms-checklist view"

The World Health Organisation this week published the ICD-11 (the International Cassification of Diseases), including Gaming Disorder as an addictive behaviour disorder for the first time. In the run-up to its publication researchers in psychology and beyond have spoken out against this decision – pointing especially to the low quality of research the disorder is based on.

Gaming disorder is described by the ICD-11 as a persistent pattern of gaming behaviour, online or offline, to the detriment of a person's everyday life. For a diagnosis someone must have been showing a lack of control over gaming, priority being given to games over other interests and an escalation of gaming within the last 12 months.

We spoke to Professor Andrew Przybylski, Director of Research at the Oxford Internet Institute at the University of Oxford, a researcher who has written extensively on the reasons against including gaming disorder in the ICD-11.

Can you tell me a little about the research base in the area?

The research literature has largely been from survey studies, and the great majority of these fall into one of two camps. In the first camp most of the studies have used what we would call 'samples of convenience': so for example a researcher will create a scale that's meant to tap into disordered gaming or gaming addiction or MMO addiction or online game addiction. There's upwards of 50 video game addiction scales used in the literature… in the absence of the ICD or APA draft guidance on internet gaming disorder, what researchers did was use checklists from disordered gambling or substance use research, take out the word gambling or alcohol and put in the word 'game'. The questionnaires usually also include some other measure that's anchored to disordered gaming. So this can be demographics, to find the prevalence of disordered gaming within certain groups; it could involve specific game titles; and another category is measurements of wellbeing, physical or mental health. Some inference is drawn about gaming addiction or disordered gaming having a possible negative effect on psychosocial functioning of some type.

The second camp of studies use more controlled samples and try to collect panel data which is representative of the population. But in most cases convenience sampling makes up a majority of the studies, nearly all of it is cross sectional. People try to build statistical models to draw, from correlational data, some kind of inference about gaming and disordered gaming in terms of demographics, games, or wellbeing and mental health. In almost all cases it's self-report measures, not a teacher or caregiver providing feedback or insight. This is the main body of work that people look at when they're trying to describe whether or not gaming is a problem. That's easily 90 to 95 per cent of the research.

The thing that has alarmed me most about this body of work isn't that it's based on self-report or the sampling issues – it's the general lack of transparency. Researchers don't share their data, the analysis code isn't available and it's not clear whether or not these are the only measures of wellbeing or gaming addiction which were used. But a lot has changed in terms of what we think of as gold standard research in psychological science. My methodological bugbear is we're creating a new disorder that can be so far behind the ball in terms of what counts as good science.  

From your point of view can we conclude much, reliably and of use, from the literature? 

Absolutely not. From my perspective what we can conclude is the research is done quite poorly and the evidence for this abounds. First – video games are seldom defined, in the US context a lot of the research is guided by Internet Gaming Disorder [included in the DSM-5 as a topic for further study] but I've yet to find a psychiatrist or psychologist or games industry expert who can tell me what an internet game is! Gaming is an activity and we would run into some very similar definitional problems pretty quickly if we were talking about food addiction or sex addiction. When we learn about the nuance and primary roles of sex and food in people's lives it would be really hard just to study the pathological aspect of it without understanding the primary phenomenon.

What's happening is there are all of these logical shortcuts people are taking to avoid having to defend the basic definition. If we were to talk about a game being addictive we'd have to talk about the mechanics of a game. We'd have to talk about something really boring and hard to understand like 'how do different reinforcement schedules in loot boxes relate to people spending more than they want to spend when playing the game Overwatch..!' You'd have to specify a very niche hypothesis.

The problem is because the topic is so flashy and because many of the people who study the topic have a limited methodological toolkit – when you're a hammer the whole world is a nail, right? So they're not actually philosophically or methodologically equipped to study the thing the correct way. The correct way to study it is probably working with games companies, doing big data analysis, looking at problematic patterns then working back to find individuals who are in distress. But that's not the way these researchers ask research questions. They're so focused on the symptoms-checklist view because that's easy to understand.

I hear people invoking the dopamine-reward pathway in the brain often when discussing gaming addiction or disordered gaming – is that just nonsense?

It's nonsense in a few ways but it's important to break the nonsense down. The first and most concrete way it's nonsense is whenever you hear someone talking about anything rewiring the brain that's an immediate red flag – that's not how the brain works. The next one is typically people will talk about squirts of dopamine. In the first instance we'd genuinely hope dopamine was involved, because dopamine is part of the learning system and is part of food and sex and socialising. But you'll hear people say 'it activates dopamine so it's just like cocaine!', but the thing to keep in perspective – and my colleague Chris Ferguson has done some work on this – is the realm of dopamine you're talking about. If you're talking about dopamine that's available for reuptake if you have an activity like eating or sex or video games we're talking levels which are 50 to 100 per cent higher than the normal levels of dopamine in the brain. But if you compare that to something like cocaine or ecstasy or methamphetamine you're talking 14 to 15 times the amount of free-floating dopamine. It's true that it's 1.5 times higher in the series of small-scale studies which have been done, but the magnitude of the difference is really huge.

To play devil's advocate – is there any benefit to have Gaming Disorder in the ICD-11?

I think yes, but there are downsides – it's going to be very embarrassing when people realise how poor the evidence base is, it'll be pretty embarrassing for the WHO. When somebody figures out that, say, half of one per cent of players are addicted, that's still tens of millions of people on the planet. It will also be stigmatising. There's a lot of people who play games to relax, to de-stress, and if they're struggling with other things this will just be another one.

On the positives – I think individuals in the games industry are receptive to studying these kinds of topics, but I don't think there's any institutional appetite for it. Perhaps a carrot and stick of formalised diagnosis may be just the thing that the trade bodies need to get serious about open and robust scientific evidence. I don't think the games companies should be able to pick their researchers and vice versa. I think that in the same way we apply for grants, researchers should apply for data.

There's a profound asymmetry between the amount of data video games makers hold on players and the kinds of data academic scientists have access to. Really great initiatives like the Understanding Society Project and the Institute for Social and Economic Research are just dipping their toes now into linking social media data with these huge cohorts. They're spending tens of millions of pounds on this and they're great projects, but if we could link participants' play behaviours and their identities across this data we could learn much, much more,

What will happen now the ICD-11 has been published?

I think the thing that's important to understand is this is another step in an ongoing process. You're not going to turn up at the doctor's tomorrow and have your GP ask you the four questions suggested by the WHO [which may suggest someone has the disorder] – there'll be an ongoing period of consultation that will last about two years, between WHO member states and feeding back data. At this stage we need to do really, good basic research and we have to be very wary of people who will try to sell us things.

While there's this ambiguity there'll be people with clinics, people selling books and people trying to sell this need and take advantage of the fact this hasn't all been vetted… we need to be on guard. There are entrenched interests that will either try to make this all go away or will try to make a pound off it. The WHO stamp legitimates the whole process and so people, bad actors, and some who think they're doing good are definitely something we need to be wary of. If there were 10 books about gaming addiction published last year you can expect that number to quadruple in the next year.

- Read more on the screen time debate, including links to other resources.