The power of virtual reality

When it comes to psychology, it's hard to think of a more potentially useful tool than virtual reality (VR). Imagine telling the first psychologists that one day we would be able to observe behaviour in any situation we could conjure up, or to deliver interventions remotely using virtual therapists. But has that potential truly been realised?

In a 1999 article for The Psychologist, David Rose and Nigel Foreman reflected on the many possibilities that VR presented at the time. It has endless potential, they said, in psychological research, training, and assessment. However, Rose and Foreman also urged caution – pointing out that psychologists had been slow to embrace the use of VR, and there was research to suggest the 'excitement to accomplishment' ratio of the emerging tech was high.

Early haptic technologies such as 'body suits and data gloves', Rose and Foreman said, needed refinement – an aspect of VR which is still limited even 25 years later. After pointing out that VR had been described as a 'solution looking for a problem' they wrote: 'Despite the current limitations and the issues surrounding its use discussed above, it would seem that VR is poised to answer a number of important research questions in psychology'.

I spoke with researchers and practitioners and discovered that much has changed in VR technology over the last two decades, not just related to the cost of tech and improvements in the resolution of virtual environments. But has VR in psychology found its important problem, or problems, to solve? Is VR practical, effective, scalable and useful?

Potential

Having worked with VR for 20 years, Professor Daniel Freeman (University of Oxford) said he had seen dramatic changes in both hardware and software. 'My first VR lab had a handmade £30,000 headset. Now we use VR headsets that cost just £300 each and don't need any external tracking devices or a computer.'

Although the technology itself has changed considerably, Freeman still hasn't seen its use increase in his own area of research, delivering mental health therapies. Here, VR has shown great promise in the treatment of phobias, social anxiety, panic attacks, post-traumatic stress disorder and generalised anxiety. But Freeman is confident VR will play a key role in the future provision of mental health treatments, 'for two reasons: it can be an inherently therapeutic medium, and it can allow scaling up of provision.'

Freeman tells me this work remains 'at the very early stages. The value of automating therapy is obvious, but there needs to be careful thought about where VR is best used, how its many possibilities can be exploited fully, and its limitations. Technology is not an answer in itself – it has to be applied to the right problems with precision, rigour, and detailed design work with the end users.'

Scale

While Freeman suggested there was still not enough investment in people or resources in the use of VR in psychological research and practice, this hasn't stopped many psychologists from demonstrating the power of VR.

Alongside mental health conditions, VR has been successfully used with patients with physical health problems. Sheffield Hallam researchers Ivan Phelan, Principal Research Fellow and Director of the Impact VR Lab and researcher Dr Penny Furness, have been working together on some fascinating applications of VR.

Phelan, a former software engineer, had been excited by the possibilities VR presented from its very early days. After working with the technology to help people with amputated limbs learn to use prosthetics, Phelan moved into developing VR in pain distraction. Furness, who was previously a nurse on a burns-care ward for 10 years, saw an advert to work with Phelan on a new project with patients in hospital who had suffered burns and the two have collaborated ever since. 'I think sometimes it's hard to find the cutting-edge of things in psychology. It can be hard to find a niche for yourself, but it felt as though I was able to actually contribute to projects which were at the cutting edge of something very important.'

Phelan had seen research from the US Army who had been trialling VR in pain distraction, and as headsets became smaller and more affordable he wanted to explore what impact VR could have. Specifically, he wanted to see whether VR could help burn patients who were having dressings changed – an extremely painful procedure.

After working with former burns patients in the design phase, he developed a relaxing, calming, VR experience. However, he found that the more engaging a VR experience was the better it was for pain distraction – and, of course, different people are distracted by different things.

'We created a competitive game and more of a puzzle-based game. Initially, when we did the trial the staff were a little apprehensive – they were very busy so we tried to set everything up for them and keep out of their way while they worked. But when they started seeing that it was delivering, they were much more open to asking for us to come in for patients. We also had some patients who initially weren't up for using it… they didn't see it as something they'd find useful, but then eventually they started using it and said in hindsight they wished they had from day one. It was really good to see that it was effective – it wasn't removing pain, it just made it more tolerable.'

Phelan said many of the lessons he learned from this work have informed future VR projects. 'If you're watching a movie – unless it's John Wick or something – there's peaks and valleys, and you could see on patients' faces when the valleys came. They would start grimacing with pain again. When we made the game we transitioned from one level to another with a black screen or fade, but that was a mistake really… we should have had it at 100 the whole time.'

The pain distraction, Furness told me, also had another useful side effect for patients. 'The VR did increase patients' tolerance of the pain – there were those waves – but we found that because they were more able to tolerate the pain during fairly extensive dressing changes, we saw that nurses were able to do more active wound care during the time patients used VR compared to what patients would tolerate previously.'

Furness said another finding had surprised them – after using VR patients could experience a rush of pain when they removed the headset. 'One thing nurses could be doing if VR went into burns care on a more regular basis is making sure they're prioritising post-dressing as well as pre-dressing pain relief.'

Furness and Phelan said there was a real appetite among hospital staff for this novel approach to pain distraction. 'We have champions now in the hospitals – they've seen the work we've done and they're approaching us to do new projects.'

Since that time, Furness and Phelan have also taken advantage of more useable, portable VR headsets in work with children's upper-limb rehabilitation at Sheffield Children's Hospital. Phelan explained that following arm injuries some children may experience pain and limited motion. He worked with physiotherapists to find out more about the types of motion children may need to practise. 'The physios told us that moving the arm behind the back, above the head and to the side was important, and I thought that motion looked as if someone was grabbing an arrow from a quiver.'

After this realisation, Phelan and his colleagues developed a fantasy-style game where players had to climb a tower and shoot arrows at gnomes on the ground below. 'We got really good results from that, it was more of a validation that this was this was helping with pain, and it was super engaging for the kids. It's also quite funny – kids were coming in with their parents saying, "Oh I can't move past this point" but once they put on the headset they realised they had no problems with the movement.'

This trial used a standalone headset, and as the research took place during Covid this allowed Phelan and Furness to send children home with the headset. 'This is still being used at Sheffield Children's Hospital and Stanford University Children's Hospital is using it as well. It also recently won an award for Best Health Game at Games for Change.' Phelan said his healthcare work had taken on a life of its own, and increasingly he is moving to create games and tools that can be used without a researcher present.

Furness added that working with children seemed to be a natural home for VR projects like theirs. 'Since then wherever we've had projects where the focus has been on children and young people it's like we're pushing at an open door – although virtual reality can definitely benefit adults. We've been asked to speak about this work a few times and all the professionals who work with children are really excited by this. We're constantly being contacted by people with ideas for future projects.'

Furness and Phelan are currently working on more upper-limb rehabilitation trials with an aim to get medical device regulations in place, as well as a VR intervention for social anxiety and a physiotherapy game for use with patients with Duchenne Muscular Dystrophy.

Impact

As Phelan and Furness have shown, VR has wide potential applications, but there are also those who sound a note of caution around its widespread use.

Dr Domna Banakou, Visiting Assistant Professor of Practice of Interactive Media (NYU Abu Dhabi), agrees that 'one of the most significant advancements in the field of VR and psychology has been the ability to apply this technology in real-life settings. This has allowed both researchers and practitioners to study and address a wide range of psychological issues with patients from a variety of cultural, society, economic and other backgrounds, examine the effectiveness of VR treatment, and test the boundaries of its therapeutic powers to treat a broader range of disorders – in some cases even replacing the therapist – but also to address other social and cognitive processes.'

However, Banakou added that most VR experiences are created by those without psychological expertise. 'It is important for researchers, practitioners, and designers to continue to carefully consider the limitations of VR and to conduct research to better understand its effects and potential risks. There is a need for more interdisciplinary research and collaboration when designing VR experiences and guidelines for good ethical conduct.'

From her own research, Banakou has found that VR can lead to behavioural, attitudinal, and higher-level cognitive changes. 'Being irresponsible in its use can harm research in the field and, worse, those using it. There is a high risk that VR may be used to exploit or manipulate individuals. For example, VR experiences could be designed to manipulate emotions or beliefs, or to exploit vulnerabilities such as phobias or addiction.'

Banakou points to the importance of a sense of touch to make virtual worlds more immersive and believable. 'Although there is extensive research in haptics for VR, we are still a long way from having a technology that can be universally adaptive to VR experiences – same as visual, proprioceptive, and auditory cues are handled nowadays.' Augmented reality – where digital objects can be 'seen' in the real world – and 'super-realism' in computer graphics both have the potential to improve psychological research which uses VR. 'But as this line between what is real and what is not becomes thinner,' Banakou says, 'it raises many important ethical, societal, and legal considerations. For example, in exposing vulnerable people such as children or patients prone to psychosis to sensitive topics impacting their physical and psychological well-being.'

Ethics

The ethics of using VR in research is something Dr Kathryn Francis (Keele University) cares deeply about. Her work, which has explored the ways people react to moral dilemmas in the virtual world, itself raises many ethical questions Francis is ever-mindful of.

Francis was drawn to moral psychology by the ways it combines many theories and methods from various areas of psychology – cognitive, organisational, and social – with philosophy and even anthropology. She first used VR in research during her PhD and later worked with a computer science research assistant to create a moral dilemma in Virtual Reality.

They uncovered an interesting contradiction in the ways people respond to moral dilemmas in the virtual world compared with simply reading a text-based moral dilemma. 'You give people a moral vignette which can involve deciding whether to sacrifice one life to save many lives in a utilitarian way. If you give people this on paper most people really don't want to endorse harm – they find it very aversive to support harming anyone, even if it means that you're going to save a greater number of people. But what you find in virtual reality is almost a complete flip pattern of responses where people are far more willing to carry out a harmful action for the greater good.'

Francis tells me there is still much to be done to understand why this might be the case – but she is interested in exploring the question of whether VR changes what drives decision-making in moral dilemmas. 'In VR you're no longer relying on imagination. When you're reading a text-based vignette you've got to picture every single event that's happening in that situation and the parties who are potentially in danger, but in virtual reality, you're seeing all of that. I think that what might be happening is that the thought of seeing those five people die suddenly becomes something that you're assigning a huge amount of negative value to, you don't want to see that. That negative value then overrides any negativity you had assigned to harming one person. I think it shifts where you're assigning value in the situation and how you're thinking about your actions and the outcomes.'

In one classic moral dilemma, participants are asked whether they would push a rotund man onto tracks into the path of an oncoming train to save the lives of several workers. Would it make a difference if you could feel yourself pushing what felt like a person onto VR train tracks? 

Working with roboticists at Plymouth University, Francis used a large haptic virtual reality robot – 'a large robotic arm that you can move in the space around you. We used that to simulate physical resistance that makes your actions feel real – to generate the sensations that might reflect what it would feel like to carry out actions in those moral dilemmas. We still found this same pattern of responses, which was fascinating.'

Taking it a step further, Francis later worked with a speculative designer called Agi Haines. 'She built a version of somebody's torso out of platinum grade silicone and expandable foam, put heated wiring in it and a t-shirt over the top. If you shut your eyes and put your hand on this torso, it would feel like a human body. We used that as the response device in the virtual reality version of the footbridge dilemma. Instead of just pushing up a plastic joystick, which feels very mechanical, you were then having to physically push something that felt like a human, and it was even weighted for someone that size. Even in this case you still get this big discrepancy and how people respond.'

In this type of work, inevitably ethical questions arise. When VR feels so real, how would someone feel after they have virtually pushed a person onto train tracks to save lives? Francis pointed to work by the US-based philosopher Erick Ramirez who is considering the ethics of VR in moral psychology. 'The value of VR is that we're able to study people's moral actions in the moment of decision making, while protecting their welfare, but in a situation that might have more contextual information and might feel and look a bit more real than a text-based vignette. So that's always been the way that we've thought about the role of VR in moral psychology. But Ramirez started to make some really important points about the ethics of this approach. His argument is that even though individuals aren't being exposed to these life-and-death situations, they're still coming to terms with the fact that they made this particular decision. If they go away thinking, "Oh, is this something I might do in real life?", then that's something that we as researchers need to start thinking about. It's a case of making sure we're debriefing participants properly and that we're considering these things before we design experiments.'

Francis tells me that in the lab, 'we're at a point now where we could make simulations that look incredibly realistic, but often we choose not to… we choose to keep the graphics limited to a certain extent. I wouldn't say that it's a reason not to incorporate VR – it just means we must be responsible researchers when we use it.'

The future

As VR becomes more scalable and accessible, what might be next for this technology? Professor Hugo Spiers (University College London), a researcher of wayfinding, navigation and the built environment, tells me that 'using 3D videos of the real world in VR is really interesting and massively underused.'

Spiers and Dr Robin Mazumder and Professor Colin Ellard (both University of Waterloo) recently published research which used both real-world experiences and immersive 3D videos of high-rise buildings in Central London. They found participants rated the high-rise building environment as less open and friendly and felt less happy and in control compared with being faced with low-rise buildings.

'We couldn't do this kind of research without VR: there's something very different about watching a film of somewhere on a screen compared with being in an immersive 3D world where you can look anywhere. You can see the light changing, you have 3D audio all around you – I think 3D video in VR could be a real game changer.'

Given that VR headsets are increasingly affordable, and comfortable compared with early versions which Spiers used in the late 1990s, he said the potential of VR was enormous. 'There are some amazing experiences you can have in VR. You can use it to visualise data, you can move through a whole data landscape, and those are exciting ways to understand data, but few people have got the headsets to do that, and the effort required compared with the outcome isn't worth it currently. If most people had some way of experiencing these things it would be quite amazing.'

Spiers also pointed to the potential cost and time savings of using VR in other areas – in architecture for example, to iron out any design flaws by walking around buildings which haven't yet been built. He also said VR could be useful in diagnostics. 

'With Alzheimer's, for example, people could be sent a headset and navigate around a 3D environment in VR. It could also feed into the increasing interest in the way built environments affect us – how we feel in them. More and more groups are interested in this challenge of what's the best design and whether it will improve health and wellbeing. I think VR will become a more impressive tool in that domain. I'm quite optimistic about where VR will go.'

A solution in search of a problem?

So, my conversations suggest that Psychologists have indeed found many psychological questions which need VR as the solution – and there are undoubtedly many more awaiting further advances in the technology. Yet problems of scale remain – while researchers have proven how useful and effective it can be, VR is still not mainstream within psychology, healthcare or therapy. There are also ethical questions which may only come more to the fore as the virtual worlds become ever more realistic. Either way, the power of VR, in terms of potential uses and abuses, is clear to see.

Ella Rhodes is a Journalist for The Psychologist.
[email protected]

Image (above): 'Phelan and his colleagues developed a fantasy-style game where players had to climb a tower and shoot arrows at gnomes on the ground below. 'We got really good results from that, it was more of a validation that this was this was helping with pain, and it was super engaging for the kids.'

If you have found other novel uses for VR in your psychological research or practice, let us know at [email protected] or tag us on Twitter/X.