How should we supervise qualitative projects?

There is a 'shortfall in numbers of highly skilled qualitative researchers' says the Economic and Social Research Council (2004). What is psychology doing about it? The Society's revised syllabus (BPS, 2002) states that students should be able to collect and analyse qualitative (non-numerical) data. The Quality Assessment Agency (QAA, 2002) also specifies that psychology should cover qualitative methods. Therefore, in time, psychology graduates should have the expertise the ESRC needs.

But if qualitative methods are to be included in the mainstream psychology curriculum, we need to find and maintain a high standard of supervision in this specialised field. At the moment many departments may have only one expert in qualitative methods. However, there is a growing demand for supervision of qualitative projects (Elliott et al., 1999; Krahn et al., 1995). Guidelines could help the lone supervisor benefit from others' experience. Guidelines could also provide a template for departments beginning to make qualitative projects available to their students.

Parker (2004) offers three overarching criteria for good research designed for supervisors of undergraduate qualitative projects:
l    grounding in existing research;
l    coherence of argument; and
l    accessibility of presentation.

But to supervise and produce projects of this nature requires guidance and consistency, and to that end we produced a handout for our own qualitative project students at Leeds (Madill et al., 2001). This made us realise we had different opinions about, for example, the amount of data students should collect. It seemed a good time to ask our colleagues across the UK to help define good practice. It also seemed democratic to ask undergraduates about their experience of doing qualitative research.

We hosted a one-day workshop on 'developing guidelines for the supervision of undergraduate qualitative research in psychology', funded by a grant from the Learning and Teaching Support Network for Psychology (now the Higher Education Academy Psychology Network). Supervisors from all over the UK took part: 55 of them, representing 36 different institutions. We content-analysed audiotapes of their small group discussions, and discussions amongst 12 of our own students who had just completed their qualitative project. From this material we produced the evaluation criteria shown in Table 1 and a set of procedural guidelines.

[SEE PDF VERSION FOR TABLE 1]

Guidelines for the supervision of qualitative projects

Preparation
l    Prepare students for the labour-intensive nature of qualitative research and help them time-manage the phases of their project.
l    Research questions should have some social relevance and originality.
l    When recommending a particular qualitative method, consider the demand on the supervisor, the theoretical background required, and the time demand on the student (see Table 1).
l    Provide access to previous high-quality qualitative projects and indicate examples of relevant published qualitative research.
l    Consider using staff with experience in qualitative research as project consultants and/or limiting the types of qualitative method offered in order to use elements of group supervision.

Data collection
l    Where access to participants is difficult or inappropriate, consider using archive material (including media texts). The selection and sifting of these should be substantial enough to be considered a form of data collection.
l    When deciding how much (interview) data students should collect, refer to guidelines associated with particular methods (see Table 1).
l    Require students to notify someone of their whereabouts when collecting data outside university premises.
l    Have informed consent obtained before and after data collection and, if appropriate, again once the transcript has been approved by the participant.
l    If interviewing, require students to conduct a pilot in order to check the student's reaction to the research topic, their interpersonal sensitivity, and skills in using an enquiring technique.

Post data collection
l    Check an early sample transcript for anonymisation. Participants could be invited to do this, with the right to withdraw potentially identifying details.
l    Analysis should move beyond description, not reflecting too closely the questions asked of participants, and there should be a serious effort to be reflexive.
l    Reports should show sophisticated understanding of the differences between qualitative and quantitative research, ground the method theoretically and epistemologically, be written in the first person where appropriate, and develop a coherent narrative about the research as a whole.
l    After the project has been marked, monitor the destruction of non-anonymised data, audio-tapes, and files, and the return of signed consent forms to the department for confidential storage.

Evaluation of methods

Table 1 allows student and supervisor to evaluate the demands of a particular method on four relevant criteria. These demands can be weighed against the resources available, such as time and training.

Supervisors agreed that a small data set would be fine for methods requiring detailed analysis, such as conversation analysis (Drew, 2003). More data would be needed for methods providing a pre-given analytic structure, such as attributional coding (Stratton, 1997). The minimum amounts of data shown are suggestions based on experience of allowing students to complete their project on time while demonstrating competence in the method used. Cross-institutional guidelines like these should reassure supervisors concerned that examiners might baulk at the seemingly small amount of data used.

Supervisors thought their job was particularly demanding due to the lack of prior training students had in qualitative data collection and analysis. Our participating students agreed that they felt underprepared for their project. Supervisors had to offer a lot of guidance and overcome common misconceptions. For example, some students presented hypothesis-testing designs inappropriate to qualitative research.

Most qualitative approaches have a strong theoretical basis. For example, free association narrative interviewing (Hollway & Jefferson, 2000) draws heavily on psychoanalytic theory. The student needs to understand the theoretical premises of a method in order to apply it well. Interpretative phenomenological analysis (IPA: Smith & Osborn, 2004) and grounded theory (Strauss & Corbin, 1998) may be exceptions. They offer procedures for extracting themes from textual data which might be applied without too much theoretical overlay.  

The students found transcription and analysis very time-consuming. The methods which avoid transcription, such as repertory grid analysis (Fransella & Bannister, 1967), may be less labour intensive. The thematic analysis required by IPA may also be less time-consuming than other more detailed approaches to analysis, such as discourse analysis (Potter & Wetherell, 1987).

Supervisor and student comments were also collated into procedural guidelines (see box above), giving suggestions on preparation for the project, data collection, analysis and write-up. These guidelines provide a reference point and source of ideas for supervisors. They are not prescriptive or definitive. We agree with Reicher (2000) that 'there are basic differences amongst qualitative methods which render a common standard of excellence difficult or even impossible to achieve' (p.5). We also acknowledge Hollway's (2002) warning that 'qualitative methods need more theoretical development – both in terms of an epistemology and an ontology – before teachers (and researchers) in qualitative psychology could be ready to set guidelines' (p.1). However, our recommendations are about good practice in supervision and are intended to be general, pragmatic and used flexibly.

Research environment

So far, recommendations have focused on the tasks of student and supervisor. Students feel more satisfied with their research and work more effectively when their tasks are clear, but they also value a supportive research environment and opportunities to influence their work (Swager, 1997). Supervisors should offer educational guidance, but good meetings also include personal support that allows students to own their research (McMichael, 1992). This is understandable as some undergraduates have a huge personal investment in the project they select (Wilkinson, 1994). This challenges us to see project supervision as a form of mentoring.
Parker (2004) helps us understand the mentoring process. He identifies three core principles for aiding student performance:
l    Apprenticeship: help the student learn the language and traditions of the research area;
l    Scholarship: encourage the student to argue well in support or against positions within the field;
l    Innovation: nurture the student towards creating something novel.

Ethical issues

The workshop revealed different opinions about participant and student vulnerability. Some argued that participant distress in a research interview is not necessarily harmful. Some thought that vulnerable individuals, such as those diagnosed with a mental illness, should not be exposed to novice researchers. Some were concerned about students they suspected of using their project as therapy (for example, the emaciated student wanting to study eating disorders).
A widely accepted suggestion was that students should conduct a pilot interview. This would allow the supervisor to check the student's reaction to the research topic, their interpersonal sensitivity, and skills in using an enquiring technique. However, supervisors were concerned about their ability to manage interpersonal issues, such as counselling a student away from a research topic. Mentoring students towards a reflexive account of their involvement in the production and analysis of their material also requires a great deal of skill and sensitivity that is unlikely to have been taught in any course.

Issues for discussion

Our recommendations need further development, and we can already see several issues that need more discussion. Many of these draw on important and complex debates in qualitative research. For example, Hollway (2002) highlights how the amount of data a student collects depends on 'the research question, the method, the type of analysis, the status of the theories being used, the mode of and constraints upon generalisability' (pp.6–7). We therefore need creative ideas to refine our guidelines on amount of data collected. We also need to extend recommendations to data other than interviews.Our participant supervisors were concerned about their workload, given the popularity of qualitative projects. Our guidelines suggest using group supervision to help manage workload where there are few suitably qualified staff. Limiting the number of methods offered may be more controversial.

We believe it is worthwhile to produce guidelines for the supervision of undergraduate qualitative research in psychology. One useful outcome will be greater parity in the demands made of undergraduates in different psychology departments. We hope the recommendations presented here will stimulate discussion. We invite constructive comments through the letters page of The Psychologist and at the JISCmail site (see weblinks) which includes an extended report of this work.
 
- Dr Anna Madill, Brendan Gough, Dr Rebecca Lawton and Professor Peter Stratton are at the Institute of Psychological Sciences, University of Leeds. E-mail: [email protected].

Weblinks

JISCmail discussion list: www.jiscmail.ac.uk/lists/QUALITATIVEPROJECTS.html
Leeds University Qualitative Psychology Group: www.psyc.leeds.ac.uk/research/qual
Forum Qualitative Social Research: www.qualitative-research.net
Discourse Analysis Online: extra.shu.ac.uk/daol

References

British Psychological Society (2002).
Revised syllabus for the Qualifying Examination. Membership and Qualifications Board, Board of Examiners for the Qualifying Examination. Leicester: Author.
Drew, P. (2003). Conversation analysis. In J.A. Smith (Ed.) Qualitative psychology:
A practical guide to research methods (pp.132–158). London: Sage.
Economic and Social Research Council (2004). Demonstrator scheme for qualitative data sharing and research archiving. Retrieved 21 March 2005 from www.esrc.ac.uk/ESRCContent/
researchfunding/QUADS_Spec.asp
Elliott, R., Fischer, C.T. & Rennie, D.L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38, 215–229.
Fransella, F. & Bannister, D.A. (1967). Validation of the repertory grid technique as a measure of political construing. Acta Psychologica, 26, 97–106.
Hollway, W. (2002, April). Teaching qualitative research to undergraduate psychologists. Keynote address. LTSN workshop, School of Psychology, University of Leeds.
Hollway, W. & Jefferson, T. (2000). Doing qualitative research differently: Free association, narrative, and the interview method. London: Sage.
Krahn, G.L., Hohn, M.F. & Kime, C. (1995). Incorporating qualitative approaches into clinical child psychology research. Journal of Clinical Child Psychology, 24, 204–213.
Madill, A., Stratton, P., Gough, B., Hugh-Jones, S. & Lawton, R. (2001). Doing qualitative projects: Some advice. School of Psychology, University of Leeds. Retrieved 21 March 2005 from www.psyc.leeds.ac.uk/research/qual/
QualProjectsAdvice.doc
McMichael, P. (1992). Tales of the unexpected: Supervisors' and students' perspectives on short-term projects and dissertations. Educational Studies, 18, 299–310.
Parker, I. (2004). Criteria for qualitative research in psychology. Qualitative Research in Psychology, 1, 95–106.
Potter, J. & Wetherell, M. (1987). Discourse and social psychology: Beyond attitudes and behaviour. London: Sage.
Quality Assurance Agency for Higher Education (2002). Subject benchmark statements: Academic standards – Psychology. Gloucester: Author.
Reicher, S. (2000). Against methodolatry: Some comments on Elliott, Fischer
and Rennie. British Journal of Clinical Psychology, 39, 1–6.
Smith, J.A. & Osborn, M. (2004). Interpretative phenomenological analysis. In G. Breakwell (Ed.) Doing social psychology (pp.229–254). Oxford: Blackwell.
Stratton, P. (1997). Attributional coding
of interview data: Meeting the needs
of long-haul passengers. In N. Hayes (Ed.) Doing qualitative analysis in psychology (pp.115–142). Hove: Psychology Press.
Strauss, A. & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd edn). London: Sage.
Swager, S.L. (1997). Faculty/student interaction in an undergraduate research program: Task and interpersonal elements. Dissertation Abstracts International, 58(2-A), Aug 1997, 0400, US: University Microfilms International.
Wilkinson, S. (1994). Editor's introduction. Passionate projects: Students' views. Feminism and Psychology, 4, 425–429.