‘We’re here to advocate the responsible use of tests’

We've not met, but I know you've been involved with the BPS for some time, and you're now Chair of the Committee on Test Standards (CTS)?

Yes, since January last year. I've spent quite a lot of my career with the BPS in some shape or form. I was on the Division of Forensic Psychology committee and was asked to join the working group developing the forensic testing standards. That's how I ended up being on CTS, and I've now been on for eight years or more… but always a bit on the periphery, mainly focused on sort of the forensic side, that's my expertise.

Over time, you come into contact with people from other Divisions, and you get a broader awareness of testing issues and testing practice. It has been a great bit of CPD and learning. I've always really enjoyed being part of it. I saw that there was potential to do even more to push the message around the safe use of tests, and I wanted to lead the way with that and further promote the excellent work of the Committee's members.

I'll admit, I don't know a huge amount about the Society's Psychological Testing Centre (PTC) and the CTS.

So, the PTC is the 'testing arm' of the BPS, with several functions including being the interface between the BPS and test users, test takers, test developers and the general public. The PTC operationalises the standards in test use, manages the qualifications and verification processes, and provides independent reviews of tests. When people need any information about testing or tests it's the PTC they'll come into contact with first.

The work of the PTC is driven by the Committee and they're a great source of advice and support. The Committee sets the strategy and priorities and identifies the gaps and opportunities to either improve standards or promote the safe-testing agenda in areas where some awareness-raising is needed. 

So, what areas of psychology are there testing standards for?

Occupational, Educational, Forensic. And it's not only psychologists who use tests. In particular, there are health settings where people may not have had specific training in test use, and there are no standards either. This is where there is a gap. So we've been working to develop a new set of Health and Social Care standards. We did a consultation, across the BPS membership and more widely, but the response was poor. That was hugely disappointing. Obviously, consultations don't always get a lot of responses. We're not sure whether that's just due to timing, how we advertised it, or maybe people are just not as concerned about test use as they perhaps should be. Either way, these new standards have the potential to be quite impactful given they would have the widest reach – covering Clinical, Health, Sport and Exercise etc. We certainly want to engage in discussion with colleagues working in these areas.

Do people think that tests are internally validated before they're published, so if it's a published test, it would have already been reviewed for standards?

So yes, largely a published test will have used a formal test design and development process. However, there are key decisions made during that process which really impact the utility and applicability of the test – as well as its reliability and validity. BPS reviews are based on the EFPA European Test Model. This review structure is so successful in its clarity, that it has gone beyond Europe to be used across the Americas, Africa, the Gulf, Asia, and Australasia.

In terms of who controls access to tests, that's the publishers. We can have our standards, training, CPD, and all of this, but we can't tell the publishers not to sell tests to certain people. We work closely with publishers to get them on board with our testing standards, because only by them advocating the use of standards can we push up the standard of testing delivery more broadly. Ultimately, each publisher has its own criteria, and you can have all sorts of professionals accessing tests via different means, but there's no one really controlling access. The PTC can't. All we can do is advocate that people use our standards, do our Assistant Test User and Test User training in the three domains, and continue with CPD and supervision to ensure they're using tests responsibly. But I don't think we can say that the reach is broad enough, that all this is really happening as it should.

What do we mean by tests?

Traditionally, people consider them to be paper and pen psychometrics: for example, personality questionnaires are something that people are very familiar with. But they come in a lot of different formats now. If you're going to assess cognitive ability, you might have a whole pack of exercises, some of them more practical in how they look and how you use them. Quite often, there's a crossover between what is testing and what is assessment… testing only ever complements the whole assessment process more generally. It's never the be-all and end-all of how you would get to understand someone's presentation. All tests are designed to access certain things you want to know about people that require a particular approach, to gauge that information which you can't get from doing a standard interview with them.

The standards are a set of competency statements that explain what a competent test user should know and be able to do. There is a clear set of test knowledge and understanding at the core (e.g., reliability, validity, norms, errors, cut-offs, etc.). It is in the application where the domains get specific. 

So tests can be approved and test users can be accredited and put on the register of qualified test users, right? And if a test has come through a publisher and their recognised processes, it might be quite a simple approval process?

Definitely… complete the qualification, then affirm your competence every year thereafter, and you'll stay on the register. We need to publicise our test reviews more! As a psychologist, you can go to any publisher, look up a test and think, 'Oh, that looks appropriate for what I need to use it for'. That initial information is likely to be within the manual. But what you really need is someone to independently review that test, to give an unbiased and practical perspective on it. Our reviewers, who are experienced people in testing, have categories to follow and criteria for what makes a good test.

Everything from how it looks, how it feels, how it was constructed, how it was developed, the norms that we use, whether any adjustments are needed, all of those things. They rate every aspect of the test across the board and provide feedback to the publisher. Practitioners can then access the reviews and consider if a test is quite good for some things but is falling down slightly on others… maybe there aren't enough norms for a particular group, or maybe it's only used in clinical settings, but you work in a forensic setting, that kind of thing.

What we want is for the publishers to submit more tests to us. There's an over-representation of occupational and educational tests, not many forensic ones. Perhaps publishers are anxious about getting a not-so-positive review, but there are ways of managing that. A review may allow for further development of the test, so it can be improved. It's an important process, but I don't know how many people are aware that we do offer that as part of CTS. So we want and need more diversity of tests!

I know you've also edited a book around diversity in forensic testing, and challenging bias. Just unpick that for me: what needs to change?

We know about biases and injustices in the criminal justice system, but how do we incorporate that understanding into how we work with our client group? It's a client group who are vulnerable, who come from certain socioeconomic groups and ethnicities, and they find themselves in our system partly due to systemic bias across society. How do you marry that up with the fact that you need to then assess someone with a test that's been perhaps developed from a Western viewpoint, as most tests are? That is really problematic.

So, the book covers everything from supervision to bias in test construction, testing with different populations for whom there are different issues, concerns and complications… working with someone who's committed a terrorist offence is very different to working with someone who has offended against a child, for example. It's understanding some of those differences and how you would assess and use tests to develop a fuller understanding of that behaviour.

There are broader societal issues as well… it's not just a book about testing. Working on the book has given me real impetus for what CTS needs to be doing in relation to this, and hence it features strongly in our strategy and priorities. 

So tell me more about your day job… I'm always very aware people are serving the Society on top of that.

I work in safety. I feel strongly about making prisons safer. Anything that relates to suicide, violence or self-harm… and I do a lot around influencing policy and working with colleagues around understanding how we can best make use of the evidence to make prisons safer. Supporting colleagues in the field who are working with prisoners who are in crisis, we do a lot of that kind of work. I coordinate many different types of projects, support the Young Adults Strategy, and of course, I'm part of a psychological testing working group! At the moment I'm also doing a Doctorate exploring potential case management approaches which could be used in the safety space, so I'm very busy studying as well as trying to do my CTS work!

A quote about 'people are in prison as punishment, not for punishment' sparked a real shift in my thinking around prisons.

I think all practitioners need to treat people with humanity. I believe that's what most psychologists will try to do in their own particular areas. We know that punishment in and of itself doesn't work. This is about the bigger picture – how we get the best outcomes for individuals and for society. You have to be able to really reach out to that person. That goes against some of the narratives that might be in the media, for sure.

I really like trying to make a difference, and I apply that to all that I do. The testing area can be a bit of a difficult one to navigate sometimes, and a lot of harm can be done by testing. The way to prevent that is to support people, to try to offer guidance and facilitate discussion around the uses of testing. That dialogue needs to happen across all areas of psychology. There is best practice that applies across the board, and there's a lot to capitalise on. But we need to raise the profile of testing, and the importance of good testing. Not everyone has to use tests, but if you are going to, we're here to advocate the responsible use of tests, and to assist practitioners everywhere to do that in their day job in the best way they can.

What do you need to ensure that happens?

We want to hear from people who have concerns or issues about the use of tests. We have a strategy now which is helping us to prioritise our work: over the next five years to really push on areas around diversity and inclusion in particular, as well as national standards more generally. We'll be seeking expressions of interest in being co-opted onto the Committee. It's underrepresented across certain groups, for example, ethnicity and disability, so we want to redress that. We're going to be developing guidance documents for different aspects of testing, and we need people with a breadth of expertise for that. We are a diverse group of people in terms of our experience, but we need other people's knowledge and experience to help us truly land what we're doing. If any practitioners out there believe we're on the right lines and want to get involved they need to get in touch!