‘Scientific research, in my mind, is still a human activity at heart’

Your book tackles how Psychologists go about studying concepts they don't have a proper grip on… or, more formally, 'how to analyse the investigative processes in areas of research that are characterised by a high degree of conceptual openness and epistemic uncertainty'. Can we start with an example?

In my book, I use memory as my main example, focusing on implicit memory and short-term (or working) memory. So, for example, I say that researchers first identified implicit memory as a type of memory revealed through priming tests, because the results of such tests were functionally dissociated from those on regular recall- or recognition tests. 'Functionally dissociated' means that under certain circumstances people did poorly on 'normal' memory tests while doing well on priming (or implicit) tests, leading researchers to posit the existence of implicit memory as a distinct kind of memory. Now, having reasons to posit a previously unknown kind of memory is not the same as already having a well-articulated concept of this new kind of memory. This is to be expected at an early point on the research process. That's what I mean by the expression 'high degree of conceptual openness and epistemic uncertainty'.

Let me emphasise that this expression is not intended as a criticism of psychologists. While the expression 'not having a proper grip' may sound as if I were diagnosing some kind of deficit, I think it is in fact a very normal part of research, perhaps even a motivating factor. Scientists do research on specific questions or objects precisely because of what they don't know. My aim in the book is to analyse how psychologists can generate novel insights and develop their concepts despite the epistemic uncertainties they operate under.

Reading the book as a Psychologist though, it does feel pretty damning to strip it all back to 'What kinds of things are psychological kinds?' As you write, 'scientists take their work to be about something real out there'. Does it surprise you that, in your view, Psychologists still haven't really got to grips with such fundamental stuff?

I am curious as to why it feels damning to you as a psychologist that I address questions about the nature of the psychological subject matter. It was not intended as damning at all! On the contrary, I have a lot of respect for the theoretical and methodological ingenuity of many psychologists. If anything, my book criticises a lot of traditional work in the philosophy of science, which – I argue – often fails to acknowledge the epistemic situation of practicing researchers. Philosophy of science often relies on the fiction that theory building has already been completed. Consequently, a lot of philosophical work is focused on an analysis of theory testing. By contrast, I try to give an account of a situation where theory-building and conceptual development has not been completed. I claim that rather than testing theories, psychologists are often concerned with the exploration and description of objects of research, such as implicit memory or working memory. 

To be clear: I think that's a good thing. A lot of the book focuses on the role of 'operational definitions' in this process. By this I mean preliminary conceptualisations of the objects of research (such as implicit memory) that are based on existing measurement instruments (such as priming tests). I am interested in how researchers use such 'definitions' to generate more knowledge about their objects.

Only in chapter 6, which I think you're referring to in your question, do I try to take a step back and think more broadly about the question of what psychological kinds are. In doing so, I connect to debates within philosophy of science about so-called 'natural kinds'. These debates are concerned with questions such as whether the objects we study scientifically exist independently of us, whether they have some kind of essence that science tries to uncover. My argument in this chapter is that psychological kinds do not have mind-independent essences. I am not suggesting that all psychologists need to engage with those philosophical debates – though I am gratified to see more psychologists become interested in them in recent years! 

I should mention that I started out being trained in psychology and I think I realised that I kept asking questions about 'the fundamental stuff'. This is how I ended up studying the history of psychology and in the philosophy of science. Again, I don't necessarily think it's a deficit if practicing researchers in psychology don't do that. Doing empirical research is extremely time-consuming, hard, work. But I do think that my work connects to real concerns of researchers, and I welcome conversations with them.

You explain that Psychologists are not unaware of these issues. Giving accounts of some foundational figures, you say 'All three were wary of the potential pitfalls of positing phenomena, entities, or processes that went beyond the conceptual resources of a purely observational language. At the same time, their research questions prompted them to use terminology that expressed such concepts.' With foundations based in sand, is this now too late for Psychology to fix?

I am not sure I would put such a negative spin on it. In chapter 2, I analyse three 'fundamental figures' from the 1920s and 1930s – S.S. Stevens, Edward Tolman, and Clark Hull. I chose these three figures because they are sometimes identified as early proponents of a position called 'operationism' (the position is sometimes also called operationalism). Operationism ties the meanings of psychological concepts to specific measurement instruments (e.g., priming tests for implicit memory). For a long time, operationism had a really bad reputation within philosophy, because it seemed naïve to say that, for example, implicit memory is 'nothing but' what's measured by priming tests. In my book, I present a more charitable analysis of operationism, according to which most psychologists, even in the early days of operationism, did not actually believe anything this naïve. I argue, rather, that they used operational definitions as tools to study their objects of research. Applied to the (more recent) case of implicit memory: If you have a hunch that that there might be an interesting object to study (e.g., implicit memory), it seems perfectly legitimate to use a specific measure (e.g., priming tests) in your efforts to find out more about this object. I argue in the book that this can, ultimately also give rise to different measurement procedures, which allows researchers to keep exploring their objects in novel ways.

Regarding your formulation that psychology's foundations are based on sand: I am not so worried about this. I don't think that any discipline has secure and unchanging empirical foundations. A method that was once thought to be reliable and to deliver secure knowledge may later be criticised and replaced with a different one. But isn't that a sign that some epistemic progress was made, despite the shaky foundations? So, no: While I do see problems in current psychology, I don't see the problems as being grounded in foundations of sand. And I think that there are many smart people in psychology doing interesting work.

Talk to me about the distinction between an 'object' of research and a 'phenomenon'.

In my book, I use the term 'object of research' to refer to an overarching object, such as implicit memory or working memory. In philosophy of science, many people use the term 'phenomenon' to refer to such an overarching object. For example, they might say: 'I am interested in the phenomenon of memory'. While there is nothing wrong with using the term 'phenomenon' in this way, I think it glosses over an important difference: It seems plausible to me (and is also borne out by how psychologists talk) that implicit memory consists of a number of different phenomena. There are behavioural phenomena (such as specific patterns of responses on priming tests), there are phenomena having to do with conscious experience or lack thereof (such as the fact that implicit memory is involves lack of awareness), and there are cognitive phenomena (having to do with storage and retrieval). Once we understand that many objects of psychological research are composed of multiple phenomena, it makes sense to conceive of the research process as one where psychologists try to describe the various phenomena to gain a fuller conceptual understanding of their objects.

Your main focus is experimental cognitive psychology, but could you level these charges at areas right across Psychology?

Yes, absolutely. I would like to suggest that my analyses can be extended to other areas of psychology as well.

Have you had any reaction yet from Psychologists?

I have mentioned this research to psychologists over the years. I think many were sympathetic to the idea that operational definitions can play a productive function in research, but I haven't discussed this at great length with any psychologists yet. I look forward to receiving feedback from psychologists.

In the last few years, I became somewhat involved in some of the debates coming out of the replication crisis. I am on record for saying that 'replication is overrated', which is a provocative thing to say. This assessment was informed by the work I was doing on my book. I think that in a situation of epistemic uncertainty and conceptual openness, replicability will only tell you so much: you might have a perfect replication, but the data might still be confounded by an uncontrolled variable. Conversely, you might have an experiment that's really hard to replicate, but this does not mean that one should abandon the research in question. Failure to replicate might indicate something about the research object that's worth exploring further. I have had some pushback from psychologists about these points, but I have also had many positive responses from psychologists who agree with me.

So what you're saying is actually pretty positive in terms of the replication crisis in Psychology… we needn't get massively hung up on the methods, it's more about the conceptual apparatus. Is that right? Is that potentially easier to fix? 

I think there are many legitimate concerns that have been raised in response to the replication crisis, in particular concerns about the prevalence of questionable research practices. While I am not condoning the use of practices like p-hacking, I don't think replication problems can be reduced to questionable research practices. From my perspective, it has to do with the complexity of the subject matter and with the high degree of conceptual openness and epistemic uncertainty that I already mentioned. This means that from my perspective the focus should be on conceptual and methodological issues, not just on procedural ones.

I don't think that this will make the problems easier to fix. To me it's more a matter of which problems we choose to focus on. Surely, following procedures cannot be an end in itself.

If you were a Psychologist, would you be embracing AI as an at least partial solution to some of the problems you raise?

I think that AI can be a useful tool for research, for example to process data or to model alternative scenarios. However, scientific research, in my mind, is still a human activity at heart. This means that even if we use AI to process data, human scientists still need to make sense of the results. My book is an attempt to spell out how they go about doing this.

One of the hopes that have been placed in AI is that it can help us transcend our human biases. So, from that perspective, one might expect AI to enable psychologists to move beyond the concepts that inform their operational definitions. I am skeptical of this vision of the neutrality of AI, because AI is always going to be trained on data that reflect our current reality.

As already mentioned earlier, I am also skeptical of the idea that there are completely human-independent and a-historical facts about psychological kinds. And lastly, I think it's the job of psychological research to answer to real-life concerns. All of this suggests to me that human researchers have to remain an important part of the picture as people who work with, but also critically reflect on, existing concepts of the psychological subject matter.

What next for you in terms of writing? Will you be sticking with Psychology?

Yes, I plan to continue on topics in the philosophy and history of psychology. I have begun working on a project about the scientific study and measurement of personality.

Operationism in Psychology: An Epistemology of Exploration is published by the University of Chicago Press.

• Photo credit: John Carson