Breaking out of the silo mentality

This article is part of a special feature from the August issue: see also

Learning from the lifeworld

Crossing into the digital realm

Rhetoric and resistance

Let a thousand flowers bloom

One on one… with Victoria Clarke 

A silo, in which systems are unable to operate with any other systems, is perhaps best epitomised within psychology by the notion of the 'paradigm wars' (Oakley, 1999). These arose out of the practices that saw quantitative research methods used separately from qualitative methods. At the height of the wars, users of each type of method went so far as to criticise the other approach, arguing that theirs was the most justifiable in the advancement of the understanding of human behaviour.

With the outbreak of a 'fragile peace' (Bryman, 2006) some reconciliation between the users of both types of methods took place, and the rise of mixed-methods research developed rapidly in psychology. From its origins as a research paradigm that combines one quantitative method with one qualitative method – in which the qualitative method was originally most often a secondary method used to triangulate or inform larger-scale more generalisable research – mixed-methods research has now evolved to include the mixing of more than one method with others in multi-method research (e.g. Brewer & Hunter, 1989) and the prioritising of qualitatively oriented research questions in qualitatively driven mixed-methods research (e.g. Hesse-Biber, 2010; Mason, 2006).

These methods have sought to place the research question back at the centre of psychological inquiry and avoid the research emphasis being placed on method (a process termed 'methodolatry' by Curt, 1994). With such developments questions about epistemological and ontological (in)coherence have been raised by researchers asking whether and how different views of knowledge and its acquisition can be combined. One response to these questions is found in the development of pluralistic approaches to research. Developing simultaneously in the UK and the USA (e.g. Frost et al., 2010; Johnson & Stefurak, 2014), forms of pluralism that include methodological pluralism (Frost, 2009), analytical pluralism (Barnes et al., 2014), interpretative pluralism (Coyle, 2010) and dialectical pluralism (Johnson, 2012) all adopt the view that human experience is multidimensional and multi-ontological, that its exploration can be better served by combining methods to address the research question in many ways, and that embracing the differences that different paradigms bring can help us better understand the complexities of human experience and interaction.

In this article we briefly consider the history of methods in psychology to consider how they led to a silo mentality. We will consider the ways in which mixed methods and pluralistic research address some of the concerns about epistemology and ontology and show how they offer a flexible and functional disciplined approach to research into human behaviour.

What is the meaning of science?

The growth and dominance of experimental methods to understand behaviour in psychology was embraced by behaviourists such as John B. Watson as a response to perceived limitations of introspection for scientific pursuit. The science of experience and culture was left behind as psychology focused on observable and measureable behaviour. This provided satisfying ways to place paradigms of epistemological assumptions about how valid scientific knowledge can be gathered, drawn largely from the natural sciences, at the fore of psychological research, giving it recognisable status and acceptance. The dominance of the scientific approach became the consensus amongst psychology researchers as the best way to understand human behaviour. A new concept of science as applied to human behaviour was developed and adopted into the mainstream. However, in time, and largely spearheaded by the advent of feminist critique of the underlying assumptions of reality (ontology) and the validity of scientific knowledge (epistemology), the beginnings of a scientific revolution took hold. The dissenting and marginalised voices began to be raised and led to the 'turn to language' in psychology.

The turn to language

The cognitive revolution had promised a move away from behaviour towards a more meaningful examination of the human subject. Jerome Bruner (1990) and others were disgruntled with the artificial intelligence and information-processing models that came to dominate cognitive psychology; they were limited by their experimental methods and failed to ask the bigger questions about the nature of human experience. This prompted a shift towards the examination of discourse, our means of communication, the bedrock of our social existence (well charted in qualitative methods textbooks, e.g. Banister et al., 1994; Smith et al., 1995; Willig, 2008) .

In the US this movement was led by a cry for 'new paradigm' research that was inspired by humanistics and phenomenology (e.g. Giorgi, 1970; Reason & Rowan, 1981). In the UK critical psychology and discourse analysis took to the fore (e.g. Gough et al., 2013; Potter & Wetherell, 1987). The result was a call for psychological research with people (rather than subjecting them to tests and observing them) that might give voice to participants and improve their lot in the world. This emancipatory goal created a political agenda for research and represented a backlash against the 'us–them' divide that had become evident in experimental psychology; psychology was accused of ethnocentrism, and critical psychologists demanded that the power imbalance between researcher and researched be broken down, or at least recognised for what it was (Stainton Rogers, 2003).

For psychology this meant taking a critical stance towards the study of human phenomena, building awareness of the researcher's role in constructing the data beyond 'experimenter effects', prioritising the participant's voice, and accepting the coexistence of multiple meanings attributed to the same event, state, or text (e.g. Finlay & Gough, 2003; Shaw, 2010; Smith, 2008). In brief, this meant the rejection of positivism and objectivism. Some researchers engaged with postmodernism, some with social constructionism, and others with humanistics, existentialism and phenomenology. For all it meant a focus on language.

Muddying the waters between principle and method

As a political movement the turn to language initiated a sea-change in the way research is conducted and participants treated. Funding bodies now expect to see how participants or service users will be involved in the development, running, evaluation and dissemination of research projects. Furthermore, it is a requirement for any study involving human participants to be considered by an ethics committee (in the heyday of social psychology experiments, ethical issues were often bypassed or forgotten). An extension to this is the current focus on impact, which means researchers need to demonstrate the impact their work will have and how this will manifest in the everyday lives of people in the real world. These changes in the principles of research have gone beyond method and are far-reaching. In some ways, they are the legacy of the turn to language.

The turn to language has also become synonymous with the growth of qualitative methods in psychology. The link is understandable because with the turn to language came a focus on quality over quantity (i.e. a focus on examining the meanings of textual data instead of the statistical analysis of numerical data). Using participants' own words followed the principle of working with them and giving them voice. However, what it has also done is to muddy the waters between a shift in principles and the use of (quantitative or qualitative) methods. The terms 'qualitative methods' and 'qualitative research' came to signify more than a type of data or method; indeed, as stated above, qualitative methods were sometimes referred to as 'new paradigm' research. The implication of this is that any research adopting an emancipatory or collaborative inquiry approach was expected to use qualitative methods in principle; ergo any study using qualitative methods was assumed to fall within this 'new paradigm' of research. While for a good many years this happened to be the case, the two were not fundamentally connected, nor were they mutually dependent. Nevertheless, the useful shorthand, 'qualitative psychology', has gained ground and is now in popular use.

A more disciplined inquiry

Talking of paradigms and epistemology is too 'heavy' for most people, but bear with us while we attempt to demonstrate how (a) mixing up a type of data (qualitative) with a discipline (psychology) and (b) muddying the waters between principles (emancipation, giving voice) and methods (turn to language) has led to a fundamental misunderstanding in psychology, and thus of mixed methods. Kuhn's (1970) theory of scientific revolution, notes that it is the paradigm that dictates all subsequent research decisions. Hiles's (2014) model of disciplined inquiry also begins with paradigm, followed by strategy, method, analysis and critical evaluation; and it emphasises the significance of the research question. The formulation of the research question is of paramount importance in any research project; decisions that follow should be guided by that question rather than an arguably arbitrary preference for quantitative or qualitative data. Indeed, as Hiles (2014) argued, focusing on the distinction between 'quantitative' and 'qualitative' research is a red herring; it is flawed logic. Putting such emphasis on the type of data misses the point that it is the strategy that is adopted – the logic of inquiry – that impacts on research design. Also note that the paradigm will guide the strategy taken; which will in turn denote which methods in the 'toolkit' are fit for purpose; and the data generated that will determine what method of analysis to use; and all of the above will lead to particular decisions with regard to critical evaluation. The discipline within this model of inquiry is manifest in this directional flow of decisions. Following this model, the label, 'qualitative psychology' becomes illogical because it prioritises the type of data gathered and its method of analysis.

Mixed methods in psychology

The idea of mixed methods in psychology only became viable once qualitative methods of generating and analysing data had become more acceptable in psychology. Cynically, we could consider this as a 'validation' of qualitative methods by using them with quantitative methods. Combining methods within and across paradigms allowed research questions to be asked about how human beings talk and practise themselves into particular subject positions and what those positions might consist of. It soon became clear that the plurality of methods available within qualitative methodology could lead to a toolkit approach where the most appropriate methods are selected for the research question, such as is seen in the pragmatism approach. However, the significance of epistemological allegiance in psychology meant instead that researchers still opted for their preferred method, one that fitted with their worldview, and moved forward by always using that method. Consequently, some qualitative researchers in psychology became known for the method they use rather than their subject of interest.

Despite this, advocates for increasing qualitative outcome research (e.g. McLeod, 2005) argue that the use of qualitative inquiry encourages questioning and deconstruction of taken-for-granted concepts such as 'outcome' and 'change'. They suggest that instead of seeking evidence based on traditional natural scientific designs and concepts, qualitative research allows for creativity not possible in quantitative work alone (e.g. Mason, 2006) and so also allows for in-depth enhanced insight into human experience. This has led to an increased use of qualitative approaches alongside traditional quantitative approaches to bring multidimensional research strategies to research questions of lived experience and individual realities (e.g. Bryman, 2007). It has also led to the emergence of pluralistic approaches. These allow not only for the mixing across paradigms but also within them.

Pluralism

In its simplest terms pluralism denotes diversity – of beliefs, practices, views or opinions. When applied to the conduct of research, pluralism suggests the mixing of paradigms, data, and/or analysis techniques to promote engagement with diversity, to enter into personal and methodological dialogue to promote and foster understanding of research inquiry and outcomes (Frost, 2011).

Combining analytical tools that emerge from different paradigms means that assumptions about the nature of reality and the knowledge being sought influence the type of research questions asked and the way data is interrogated. Thus pluralism advocates mixing paradigms within and across approaches in order to reduce the likelihood of reductionism of the data or the meanings within it, to bring different vantage points to the research. The range of methods available to researchers allows for visual, verbal, technological and observational datasets to be combined with each other and/or with measured statistical data. Data are transformed within a theoretical and intersubjective framework that results in the construction of personal and collective perspectives on lived experience and social worlds. 'Dialectical pluralism' (Johnson & Stefurak, 2014) actively seeks difference across positivist and interpretivist paradigms by explicitly incorporating stakeholders' and researchers' epistemological and social/political values to guide the research. It aims to combine important ideas from competing paradigms and multiple values into a new socially agreed upon whole (Johnson & Stefurak, 2014). By considering how each method works alone and with other methods, pluralistic approaches set up dialogue across methods rather than putting barriers between them.

Concluding remarks

Moving beyond the silo mentality of qualitative vs. quantitative methods prompts psychologists to work across difference and to work with diversity, in the recognition that human experience is not confined to one way of seeing, understanding and making sense of the world (see boxed research examples). Mixed-methods research goes some way towards this by offering ways to design research that are both nomothetic and idiographic. Pluralistic research offers the opportunity to gain more holistic, in-depth insight by bringing a range of perspectives, each of which is valued in relation to the research question.

It is important however to recognise the tension inherent in the desire to be open and inclusive to practice and methods whilst also needing to avoid an ill-disciplined 'anything goes' approach. Clear theoretical foundations that link the selected methods to the focus of the inquiry are key. Ross (2012) suggests developing a 'pluralism of pluralisms' within the counselling field that will minimise the risk of closing down inclusivity. Perhaps mixed methods and pluralistic researchers can do the same and consider ways of holding together multiple accounts in theoretically consistent ways.

Challenges to researchers include staying with the messiness of these approaches and resisting the urge to tidy up what is found into neat packages that present only some of what is experienced.  This means developing confidence to present what is closer to the dynamism, chaos and untidiness of human life. Or to put it another way, to accept that 'loose ends do not have to mean frayed ends' (Rodriguez & Frost, 2015) when striving to break out of the research silos of psychology.

Box - Pluralistic method, multiple selves

Katsiaficas, D., Futch, V.A., Fine, M. & Sirin, S.R. (2011). Everyday hyphens: Exploring youth identities with methodological and analytic pluralism. Qualitative Research in Psychology, 8(2), 120–139.

A pluralistic narrative analysis approach was employed in a two-year study to explore the enactment of multidimensional and dynamic identities amongst three adolescent women. Participants were selected from a wider sample of urban youth, attending different schools in New York City. Their ethnic backgrounds differed: two were recent immigrants, one from Tibet and one from China, and the third was born to native parents of black, Indian and Italian background. Each was interviewed about their social, academic and emotional engagement, given sentence completion tasks, and asked to create identity (Years 1 and 2) and learning maps (Year 1). The maps invited participants to make visible their selves across place, relations and time (Sirin et al., 2010). The pluralistic methods aimed to gather evidence that better documented social and academic engagement.

The materials were analysed within person, across time, and through the data sources, using a set of theoretically driven codes drawn from the 'hyphenated selves' framework, in which identity is regarded as joined, and separated, by history, socio-political context, geography, biography, longing and loss (Fine, 1994). The analytic dialogue across data was further supported by interpretative dialogues between researchers in which they used their theoretical and methodological differences as an additional resource.  

The study found tensions of loss, longing and disconnect in representations of self. The texts provided descriptions of the young women's lives, the drawings showed them as double selves, split into a happy and sad face, a relational devoted self surrounded by friends, and a worldly transnational stick figure, and a single blossoming flower. Interpretations enabled understanding of struggles against silencing and advocacy, towards bridging family obligations and language challenges outside it, and of resistance and engagement within different contexts.

The pluralistic methods enabled identification of a core dynamic of desire/struggle in the adolescents' development of identities, within emotionally charged constructions of the self.

Box - Balancing out nicely

Muller, I., Kirby, S. & Yardley, L. (2015). The therapeutic relationship in telephone-delivered support for people undertaking rehabilitation: A mixed-methods interaction analysis. Disability and Rehabilitation, 37(12), 1060–1065.  

This mixed-methods study was conducted within one arm of a trial that tested rehabilitation treatment for people with chronic dizziness. A randomised controlled trial within applied health research offers opportunities for a number of elements to come together in pursuit of a particular objective. It is especially useful to employ a number of methods when the intervention being tested is complex.

The intervention reported in this paper involved providing patients with a booklet including cognitive and behavioural strategies for balance retraining and therapeutic support by telephone. Telephone sessions conducted were recorded and transcribed for analysis. The Rota Interaction Analysis System (RIAS) was used to categorise utterances against 43 predetermined mutually exclusive communication strategies. These were then coded to form three composite categories of communication within the medical encounter: relationship building, therapist dominance, and person-centredness. The outcome measure for the therapy was working alliance based on subscales relating to goal, task, and bond (using the Working Alliance Inventory – WAI-S). Bivariate correlations were examined to establish whether there was a relationship between these variables. Person-centredness was significantly correlated with goal and bond.

The qualitative component of the study explored the content of therapy rated to have high and low person-centredness. An inductive thematic analysis was conducted and identified differences between high and low person-centred sessions in the following themes: unrelated friendly chat, therapist encouragement, therapist reassurance of no harm, therapist reacting to participant cues, therapist not reacting to participant cues, and participant concerns. Therapist sessions with high person-centredness involved more responsiveness to participant cues, more reassurance, and a friendly feel. Low person-centred sessions did not involve unrelated chat, encouragement, and therapists were likely to be unresponsive to participant verbal cues.

Here, both methods used are needed to answer the research question about effectiveness of communication strategies in developing a therapeutic relationship. Implications are drawn from both elements of the study, and it is difficult to see how these findings could have been offered without using qualitative and quantitative methods.

- Rachel Shaw is Senior Lecturer in Psychology at Aston University
[email protected]

- Nollaig Frost is Senior Lecturer at Middlesex University London
[email protected]

References

Banister, P., Burman, E., Parker, I. et al. (1994). Qualitative methods in psychology. Maidenhead: Open University Press.
Barnes, J., Caddick, N., Clarke, N.J. et al. (2014). Methodological pluralism in qualitative research. Qualitative Methods in Psych Bulletin, 17, 35–41.
Brewer, J. & Hunter, A. (1989). Multimethod research. Sage Library of Social Research, Vol. 175. Thousand Oaks, CA: Sage.
Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.
Bryman, A. (2006). Integrating quantitative and qualitative research. Qualitative Research, 6(1), 97–113.
Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8–22.
Coyle, A. (2010). Qualitative research and anomalous experience. Qualitative Research in Psychology, 7(1), 79–83.
Curt, B.C. (1994). Textuality and tectonics. Buckingham: Open University Press.
Fine, M. (1994). Working the hyphens. In N.K Denzin & Y.S. Lincoln (Eds.) Handbook of qualitative research (pp.70–82). Thousand Oaks, CA: Sage.
Finlay, L. & Gough, B. (Eds.) (2003). Reflexivity. Oxford: Blackwell.
Frost, N. (2009). 'Do you know what I mean?': The use of a pluralistic narrative analysis approach in the interpretation of an interview. Qualitative Research, 9(1), 9–29.
Frost, N.A. (2011). Qualitative research in psychology. Buckingham: Open University Press.
Frost, N., Nolas, S.M., Brooks-Gordon, B. et al. (2010). Pluralism in qualitative research. Qualitative research, 10(4), 441–460.
Giorgi, A. (1970). Psychology as a human science. New York: Harper & Row.
Gough, B., McFadden, M. & McDonald, M. (2013). Critical social psychology: An introduction (2nd edn). Basingstoke: Palgrave Macmillan.
Hiles, D.R. (2014). Qualitative inquiry, mixed methods and the logic of scientific inquiry. Qualitative Methods in Psychology Bulletin, 17, 49–62.
Hesse-Biber, S.N. (2010). Mixed methods research. New York: Guilford Press.
Johnson, R.B. & Stefurak, T. (2014). Dialectical pluralism. Qualitative Methods in Psychology Bulletin, 17, 63–69.
Kuhn, T.S. (1970). The structure of scientific revolutions (2nd edn). Chicago: University of Chicago Press.
Mason, J. (2006). Mixing methods in a qualitatively driven way. Qualitative Research, 6(1), 9–25.
McLeod, J. (2005). Counselling and psychotherapy as cultural work. In L.T. Hoshmand (Ed.) Culture, psychotherapy and counselling. Thousand Oaks, CA: Sage.
Oakley, A. (1999). Paradigm wars. International Journal of Social Research Methodology, 2(3), 247–254.
Potter, J. & Wetherell, M. (1987). Discourse and social psychology. London: Sage.
Reason, P. & Rowan, J. (Eds.) (1981). Human inquiry. New York: Wiley.
Rodriguez, D. & Frost, N.A. (2015, May). A methodological reflection on the application of qualitative pluralistic research to couple relationships. Paper presented at the Society for Qualitative Inquiry in Psychology Second Annual Conference, City University of New York.
Ross, A. (2012). The new pluralism. European Journal of Psychotherapy & Counselling, 14(1), 113–119.
Shaw, R.L. (2010). Embedding reflexivity within experiential qualitative psychology. Qualitative Research in Psychology, 7(3), 233–243.
Sirin, S.R., Katsiaficas, D. & Volpe, V.V. (2010). Identity mapping. Intnl Society of the Study of Behavioral Development Bulletin, 2, 22–26.
Smith, J.A. (Ed.) (2008). Qualitative psychology (2nd edn). London: Sage.
Smith, J.A., Harré, R. & van Langenhove, L. (1995). Rethinking methods in psychology. London: Sage.
Stainton Rogers, W. (2003). Social psychology. Maidenhead: OUP.
Willig, C. (2008). Introducing qualitative research in psychology. Maidenhead: OUP.