Psychological assessment and ‘the lost clinicians’

It is just over 40 years since Monte Shapiro retired from the Institute of Psychiatry where, until three years earlier, he had been the head of the psychology service to the Maudsley Hospital and the clinical training course. He made contributions both to research and to practice, but the heart of his work was directed at understanding the individually distinctive experiences of patients.

This preoccupation endeared him to practitioners as well as patients, and it shaped his views on routine clinical assessment. He would suggest to his trainees that while clinical researchers might tend to study single psychological 'constructs' (disorders, traits, drives, attributes and so on), clinical practitioners needed to be interested in all the distressing phenomena of a psychological kind that each patient brought. While he didn't use the word himself, I will refer to these phenomena as 'content' here.

Documenting concerns in a person's own words

Much at odds with prevailing practice at the time, Shapiro developed a systematic method for capturing such content. This, 'for want of a better name' (Shapiro, 1961, p.151), he called the Personal Questionnaire (PQ). The name stuck. There have been a few similar procedures. Bilsbury and Richman (2002) comprehensively reviewed their theory and practice in a distinguished but little-known monograph. Shapiro's method, though, might well be the most familiar, at least in the UK, and it was the one I was drawn to throughout my career.

Rather than talk about 'administering the PQ', the collaboration between practitioner and patient makes it preferable to talk about 'constructing a personal questionnaire'. The questionnaire is constructed from the purposeful conversation that precedes it. Here is an example of such a set of items for a young woman with a small child living with her parents:

Difficulty stopping washing my face once I've started
Feeling ashamed of myself after washing it
Feeling of anxiety if I don't wash my face
Tendency to snap at my family after I've been washing
Sense of myself as just existing
Concern that there is nothing I know how to do okay
Uncertainty about how to deal with my mum and dad when they interfere
Sense of not knowing how I feel about anything
Feeling of being dirty when I fetch my daughter from the nursery
Concern that other parents know that I'm less together than I appear

What makes this kind of assessment of someone's distress so striking? The material is not just personal in the words chosen but contextual in the way it takes hold of a person's situation in life, and not simply their state of mind. What is produced is a set of items and its value has much to do with what those items contribute both individually and collectively, which will vary from person to person but include some of the following:

• the identification of a number of fairly discrete areas of distress

• a set of problems indicative of some sort of disorder, or condition

• a number of issues that are the result of a disorder rather than part of a disorder (material illustrative of a person's predicament)

• the better understanding of particular concerns by the way other concerns shed light on them

• items that provide options, choices of ways in, a means of planning and prioritising, or hints that point a way forward

• a safeguard against what might otherwise be overlooked

• a sense of a person's position in relation to how they see their difficulties and what brought them about, and occasionally their readiness to work with you

• material that needs explaining or understanding in a psychological formulation

• the experience for a patient of being carefully listened to and perhaps better understood than hitherto.

A road not taken

You might view personal questionnaires and similarly descriptive, phenomenological procedures as a continuation of what counselling and clinical psychologists usually do anyway in their routine clinical work. We ask a new patient why they have come to see us, or what the person's problem is. We note the answer and possibly write it down in the person's own words. In principle, we have only to repeat the question a number of times in a 'Are there other reasons why you have come?' or a 'What else is a problem?' kind of way to create a set. There is an irony of the so-near-yet-so-far sort to reflect on later here.

The material generated this way is experiential and subjective, representative rather than exhaustive of a personally distressing state of affairs. At the time Shapiro did most of his work with personal questionnaires – the sixties and seventies – it was quite radical to press on with something that, while systematic, was also quite conversational in the way these lists of concerns were created. It was radical too, in the sense that it was far from a typical view of what was objective and desirable – far at least from what systematically assembled assessment data usually looked like then.

However, now there seems to be something much more immediate and appealing for practitioners about such material. Beyond the argument for accuracy and relevance, there are the ethical considerations that remind the clinician that it is perhaps the client's or patient's distress that is central rather than the one or more diagnostic conditions that seem best to fit. This focus on distress is evident in Shapiro's writings (see, for example, Shapiro, 1975a). But one finds the same emphasis in a recent psychological textbook (Cromby et al., 2013); and it was expressed again in Peter Kinderman's address as President of the British Psychological Society in 2017.  

Yet personal questionnaires have never caught on. Nor procedures like them. If they had a heyday, it was in the 1970s. Today, there is recognition that distress is central to matters, but there seems to be no urgency at large to find a way to capture the content of it directly and systematically. Why have the ideas on which personal questionnaires and similar procedures were based failed to take hold? Has it been due to circumstantial factors? Bill Yule in Clinical Psychology in Britain: Historical Perspectives (CPIB) (Hall et al., 2015) suggested that Shapiro's writing failed to make an impact. 'He kept up a steady flow of thoughtful papers on the role of the clinical psychologist, but the published corpus of his work is relatively sparse and does not do justice to his thinking' (CPIB, p.133). This is true of his writings on personal questionnaires, where we have only a brief preliminary article and a faded typescript to turn to for practical instruction (Shapiro, 1961; Shapiro 1975b). Elsewhere in CPIB the suggestion is made that there came a moment when the research interests of the profession as a whole turned from assessment to treatment. The inference might be that here were ideas deserving to be researched that for one reason or another – reasons not to do with the inherent merit of the ideas – failed to be communicated effectively or explored sufficiently, and so never came to fruition.

To measure or not to measure?

Might something else be involved? Years ago, Alvan Feinstein (1967) made a distinction, rather like Shapiro's, between practitioners and researchers. He contrasted 'clinicians' and 'psychosocial scientists'. He saw the psychosocial scientists as happy to do what Lord Kelvin told them to do and just get on with it and measure, but he saw the clinicians as lost. When it came to assessment, either they measured what they judged could be measured rather than what ought to be measured; or, despite knowing that there were times when they ought not to be measuring, they measured nonetheless; or, believing their activities to be too human and too complex for science, they failed to measure at all.

Today's scientist practitioners seem in principle better able to reconcile the roles of psychosocial scientist and clinician. But they remain under great pressure to measure, especially if by 'measure' one includes not just formal assessment on psychiatric and the better known psychological constructs. One needs also to consider the looser assessment procedures and criteria involved in such tasks as establishing how far someone is suffering a severe mental health disorder or a person's degree of suitability for a particular psychotherapy. These too imply metrics, even if crude ones.

This pressure might be one way of understanding the usual outcome of the debate about whether to prioritise person-based idiographic or population-based nomothetic assessment. The two compete for the limited time available for assessment. Given the conflicting demands – and idiographic assessment is indeed much slower than nomothetic – the case for routinely using personal questionnaires suffers. The specificity and sensitivity on offer may seem a luxury (Green, 2016). There hasn't up to now been sufficient reason to overturn the historic judgment of Beutler and Crago (1983) on the inclusion of personalised outcome measures in routine evaluation: that they are justifiable only 'from an idealistic perspective' (p.483).

There is an impression to be had here of idiographic procedures like personal questionnaires as essentially supplementary in the sense of filling in and adding on, providing greater specificity and sensitivity, and thereby refining what has gone before in terms of measurement.  If one has only this impression, one falls short of understanding what was potentially on offer to Feinstein's lost clinicians who struggled in their measuring to find the path to securing a hold on their patients' experiences and meanings. To hold only this impression is to misunderstand the true nature of personal questionnaires. At heart, personal questionnaires are not measurement procedures; they are sampling ones.

Sampling is needed where the wish is to address content rather than engage with psychological constructs. The issue surfaced as long as 70 years ago in a substantial book by Florence Goodenough on mental testing, published in the year of the Boulder Conference on the future of clinical psychology (Goodenough, 1949). It was 'an elementary fact', she said, that sampling involved considerations of 'time and circumstance' (p.101). Psychological constructs needed to transcend such considerations. She makes no mention of content; and yet, when she later described 'the well-nigh infinite variety of the problems the clinical psychologist is called upon to solve', or when she said 'the clinical psychologist cannot choose his problem; it is thrust upon him' (p.469), she understands the difficulty psychologists face in dealing with material that is highly personal and contextualised, and utterly steeped in 'time and circumstance'. She laments the fact, acknowledges the issue, but declines to address it.

Assessing content then and now

Shapiro does address it. What he did that was so bold and radical was to tackle time and circumstance head on and ask of practitioners that they be test constructors every time they met a new patient. He encouraged practitioners to plunge in and explore the content of the patient's concerns in their own words; and do it systematically. In contrast to Goodenough, and those who were to follow her in the search for 'better' psychological constructs, he was seeking to find direct expression of what the patient was finding difficult rather than proceeding via reconstructions of it in professional language.

This keenness to be personal, contextual, and to employ ordinary language suggests more a wish to describe than to explain; or, to put it another way, the emphasis of his new procedure was on determining what was to be explained or understood rather than on the explaining or understanding of it. In modern vocabulary that might be better expressed as emphasis on what is to be formulated rather than on the formulating of it. I sense Shapiro would have agreed with Clare Crellin's (1988) observation that it makes little sense to formulate without first listening to the patient's account of their difficulties. To take no account of these is 'neither logical or scientific' and therefore to be avoided unless one's view of formulation is purely a 'technological' one (p.26).

Assuming though that formulations are not done in a formulaic, purely technological fashion, they have in common with personal questionnaires a recognisably bottom-up quality. Both present material that practitioner and patient need to make sense of. However, in Shapiro's time things were much more top-down. This can be seen in his writing. Notwithstanding all that might be said about the radical nature of his procedure, reading Shapiro's original article, one sees how he operated within the diagnostic framework and supporting phenomenology of psychiatry. Rather than speak broadly of a person's concerns or difficulties, he chose to speak of 'symptoms' (Shapiro, 1961). Fourteen years later, he is still careful to describe the content of the personal questionnaire interview as covering 'quite familiar psychiatric ground' (Shapiro, 1975b, p.8). And it isn't surprising, given Shapiro's other interests, that the role proposed for personal questionnaires in his final paper is not their potential use in routine clinical assessment but in reinvigorating research into psychiatric depression (Shapiro, 1989).

The response of clinical psychologists to Shapiro at the time also reflected something of the age. The fine work undertaken by Patrick Phillips and David Mulhall in the 1970s looked at the scaling, scoring and administration of 'the PQ' not at what it was that should be being scored, scaled and administered in 'personal questionnaires'. Their research was eminently practical and psychometric rather than exploratory and investigative. But now, in this more bottom-up, patient-centred era, questions about what goes into a personal questionnaire seem much more central and immediate. We no longer have to speak of 'self-report' as an aspect of something more important or fundamental; instead, we are able to talk directly in terms of a patient's 'personal experience'. This way of expressing things is softer and less entangled in the jargon of assessment theory, even if the distinctions of assessment theory remain key. It is important to think about the content validity of constructs, but also important to think about content in its own right.

Where does this leave Feinstein's lost clinicians? The case outlined to them is for taking on Shapiro and beginning new psychological assessments by sampling, putting any measuring aside for later. It is for defying Lord Kelvin and reversing Beutler and Crago. The irony is that actually many of us probably do first ask what brings the patient to see us. (But then probably hesitate and change course.) The argument here is for persistence in identifying a person's concerns systematically, confident in the quality of the material that will be generated. It requires a conceptual step to distinguish content from constructs; and there is a further practical one involved in starting to focus professional attention on interview coverage and what to sample in approaching the matter of psychological content (Chalkley, 2015).

- Jack Chalkley is a former NHS clinical psychologist who worked in Bath, Bristol and Torbay in adult mental health and recovery services over some 30 years.
[email protected]

References

Beutler, E.L. & Crago, M. (1983). Self-report measures of psychotherapy outcome. In M.J. Lambert et al. (Eds). The Measurement of Psychotherapy Outcome in Research and Evaluation. New York: Wiley.

Bilsbury, C.D. & Richman, A. (2002). A staging approach to measuring patient-centred subjective outcomes. Acta Psychiatrica Scadinavica, 106 (Supplementum 414).

Chalkley, J. (2015). The Content of Psychological Distress. London: Macmillan.

Crellin, C. (1988). Origins and social contexts of the term 'formulation' in psychological case-reports. Clinical Psychology Forum, 112, 18-27.

Cromby, J., Harper, D. & Reavey, P. (2013). Psychology, Mental Health and Distress. Basingstoke, Hampshire: Palgrave Macmillan. 

Feinstein, A.R. (1967). Clinical Judgment. Huntington, New York: Robert E. Krieger Publishing Company.

Goodenough, F.L. (1949). Mental Testing: Its History, Principles and Applications. New York: Rinehart & Company.

Green, D. (2016). Making the case for using personalized outcome measures to track progress in psychotherapy. European Journal of Psychotherapy and Counselling, 18, 39-57.

Hall, J., Pilgrim, D. & Turpin, G. (2015). Clinical Psychology in Britain: Historical Perspectives. Leicester: The British Psychological Society.

Kinderman, P. (2017). Psychology in action, not thinking about itself. The Psychologist, June, 50-54. 

Shapiro, M.B. (1961). A method of measuring changes specific to the individual psychiatric patient. British Journal of Medical Psychology, 34, 151-155.

Shapiro, M.B. (1975a). The requirements and implications of a systematic science of psychopathology. Bulletin of the British Psychological Society, 28, 149-155.

Shapiro, M.B. (1975b). The Assessment of Psychological Dysfunctions: A Manual with Its Rationale and Applications. Parts One and Two. Unpublished Typescript, Institute of Psychiatry, University of London.

Shapiro, M.B. (1989). A phenomenon-oriented strategy in depression research. British Journal of Clinical Psychology, 28, 289-305.