How real people communicate

As a scientist of conversation, people often ask me about many aspects of human communication. Some questions draw upon commonly held myths about the way we speak. For example, if I am showing an audience how customer service works over the telephone, people ask about 'body language', and the limits of the voice-only mode. They ask about the relative status of talking versus what our bodies do to communicate. And their questions often reveal a presupposition about the answer. Body language, it is assumed, has primacy over words. Our words transmit one message, but our bodies leak another. Actions speak louder than words.

Myths about how we talk originate in language itself. The word 'talk' suffers from being something basic, ordinary, and simple. It has hundreds of synonyms to characterise and evaluate its use: we 'banter' and 'chat'; we 'prattle', 'natter' and 'yap'. Conversation is also characterised in myriad ways: we engage in 'small talk', 'discussion', 'gossip', and 'tête-à-têtes'.

The notion that talk is secondary to something else – to action – forms the basis of many idioms, proverbs and phrases. The first recorded use of the English proverb 'Actions speak louder than words' was in 1628. It's shared across all major world languages, including Brazilian Portuguese, Finnish, Russian, Swedish, Persian, Chinese Mandarin and Irish. Related phrases include the American colloquialism, 'to talk the talk', first used in 1906. The words of someone who 'talks the talk' are just rhetoric and without substance; someone who 'walks the walk' supports their rhetoric with action. Likewise, the American proverb from around the same era, 'talk is cheap' – actually a shortened version of commonly used idioms such as 'talk is cheap but it takes money to buy whisky' – denotes that you do not believe that someone will in fact do what they are saying they will do. There are other sayings: we 'talk a mile a minute', and until we are 'blue in the face'.

Talk becomes disconnected from action

Almost 400 years after its first recorded use, the phrase 'actions speak louder than words' continues to capture the public imagination. It figures repeatedly in advertising, art, culture and literature. A recent exhibition at London's Halcyon Gallery was promoted with the poster 'Actions not Words'. Aaron Reynold's 'Effin' Birds' shouts the question on Twitter, 'Can we stop talking and actually fucking do something?' And Amazon advertises its web services with the strapline, 'While talkers talk, builders build'. The idea, then, drives our understanding of talk towards communicative practices other than talk as the place to find out what people are really doing. Somehow, then, talk is not action. Talk is disconnected from action.

For many years, implicitly if not explicitly, much of psychology maintains this separation between action and talk. Perhaps even more importantly, much of psychology – and the social sciences more generally – treats language as a mere window onto the real scenery, of the mind. Talk is the means by which we can access people's underlying thoughts, emotions, personalities, attitudes and psyche. Of course, there is a massive branch of cognitive psychology dedicated to understanding the (typical and atypical) development of language and communication. There is also a great deal of work dedicated to understanding brain function, brain damage and language. But, overall, the mundane use of language is not of interest to psychologists.

However, in the late 1980s, a branch of psychology called 'discursive psychology' was invented by Derek Edwards and Jonathan Potter. The aim of discursive psychology was to encourage psychologists to treat language seriously. They argued that language – in the form of real language in use – should be something of interest to psychologists. Discursive psychologists were influenced by the language philosophers Ludwig Wittgenstein, John Austin, and John Searle. For these writers, words do things – they create 'speech acts'. Austin wrote that saying is doing – as one utters the words, 'I name this ship the Queen Mary' the ship is named. The action is done.

How do these ideas about talk and action get used in conversation? And how might they cause problems for successful communication? Some years ago, I was approached by the police about a research project. They had heard about my work and wanted to know if conversation analysis could be applied to hostage and crisis negotiations. When a person threatens their own life or someone else's, police negotiators attend the scene to try to talk the person in crisis off the roof. They try to persuade them to live, not die.

The negotiations are recorded routinely, and Rein Sikveland and I started to analyse the harrowing recordings. The aim of our research was to identify words and phrases that made it more likely that the person in crisis would keep talking. We wanted to see what worked to make it more likely that the person in crisis would keep taking turns. Every time the person took a turn, they chose life – they did not jump.

Crisis negotiations are often very long encounters – hours or even days. We immediately adopted a spiral metaphor to understand them. At the top of the spiral is the negotiator, the person in crisis, and life. At the bottom of the spiral is the negotiator, the person in crisis, and death. Across the entire negotiation, with each turn at talk, the person in crisis moves up or down the spiral. Another useful analogy when thinking about these negotiations is a football match. Whatever the outcome of a match – win, draw or lose – the players, manager, coach and fans will scrutinise every pass of the ball. They will determine every successful and failed pass of the ball. A team can only win, and keep winning, if players land each pass with the next player. Every pass matters.

First, negotiators must get a person in crisis to talk to them. Negotiators ask regularly if they can talk to the person in crisis. They describe the negotiation process as 'talking'. In the 'natural laboratory' provided by the police recordings, the effectiveness or otherwise of these turns reveals itself in the next turn.

01           Negotiator:     Can we talk about how you are.

02                                               (0.5)

03           P in C:              No:, I don't want to ta:lk,

Persons in crisis frequently resisted requests to talk to the negotiator. In Example 1, the person in crisis hangs up the phone connection shortly after this exchange.

In the next case, another person in crisis similarly resists a request to 'talk'.

01           Negotiator:     Moona,=hTalk- talk to me about these

02                                   immigration [p r o b l e m s.    ]

03           P in C:                                     [(I don't) want to] talk

04                                   to you guys now,

            -

05           P in C:              That's my LAST word man. <That's my last word. 

Not only does the person in crisis resist talking, he recruits a near-idiomatic phrase about talking – 'that's my last word' – to emphasise his commitment to not talking.

In our third example, the person in crisis also resists talking. He draws explicitly on the familiar cultural idiom that talk is just talk and does not do anything. The negotiator, in turn, is attempting to resist the idiom.

01           Negotiator:     You kept saying earlier about action:s rather than

02                                   wo:rds, .hh this i- this is genuine action that we can

03                                   [give you.  ]

04           P in C:              [It's not ge]nuine action man, you're just talking.

05                                               (0.4)

06           Negotiator:     Why don't [you think it-]

07           P in C:                                 [It's talking.   ]=You ain't done anything.

This example, and others like it, shows us that when negotiators ask or suggest that the person in crisis should talk; they open up a slot for resistance. They provide a space in the interaction for the person in crisis to use the idiom. Even though in the act of taking this turn the person in crisis is choosing life – not jumping – the lines of communication are temporarily fraught. The very thing that negotiators need persons in crisis to do – keep talking – is now framed as something useless.

So what word – if not 'talk' – will engage persons in crisis to talk? What kind of football pass will land with the next player? Which words will result in the person in crisis and negotiator moving up the spiral, towards life? Example 4 contains the answer.

01           Negotiator:     And I wanna get- and I wanna come down and I wanna

02                                   speak to you, .hhh [and see if we can sort this out.]

03           P in C:                                               [Miss.    I'm    so    scared.           ]= I stabbed

04                                   myself in the neck. And … 

In the natural laboratory of real talk, we can identify what works. How real people communicate, or at least how they say they would like to! When negotiators frame the proposed activity as speak – not talk – the person in crisis starts to… talk. At line 3, the person in crisis starts to talk before the negotiator has even completed their first turn. And in the final example, the person in crisis makes a request to 'communicate' also using the verb 'speak'.

01           Negotiator:     You said for the sake of your children.=Why don't you

02                                   come down for them.

03           P in C:              I've already spoke. I'm spea- I wanna speak to her.

04                                   Not you.

05           Negotiator:     You want to speak to who:?

06                                               (0.9)

07           P in C:              THAT police officer.

We would never know that the verb 'speak' is more engaging of persons in crisis than the verb 'talk', unless we analysed real interaction. The words are near synonyms. We tend to believe that persons in crisis will either talk or not because of some other psychological motivation – their personality, the strength of their suicidal intent, and so on. In short, factors which precede the talk.

But there are no equivalent idioms about 'speak' – we do not say, for instance, 'he can speak the speak but can he walk the walk?' Persons in crisis do not say, 'I don't want to speak', or 'it's just speak' or, more grammatically, 'it's just speaking'. 'Talk' is a noun and a verb – 'speak' is not. When negotiators suggest that they 'speak', they do not open up a slot for resistance. We are pushed and pulled around by language more than we realise, and we can see how when we look.

It remains the case that psychologists do not, generally, study how real people communicate. Instead, to find out about talk, we run experiments about it, ask people to report on it, or simulate it. Plenty of commentators have noticed this gap in our knowledge. For example, over a decade ago, the American psychologist Roy Baumeister wrote in a high-ranking journal, Perspectives on Psychological Science, that while psychology calls itself the 'science of behaviour', much of psychology never studies behaviour directly. Instead, psychologists rely on questionnaires and other proxies for studying actual social life, becoming the 'science of self-reports and finger movement'. For Baumeister, 'psychology pays remarkably little attention to the important things that people do'.

Ten years later, another psychologist working in America, Matthias Mehl, made the following assessment of psychology.

Laypersons often think of psychologists as professional people watchers. It is ironic, then, that naturalistic observation, as a methodology, has a remarkably thin history in our field. In contrast to ethologists (and researchers working with infants), psychologists are in the privileged position to be able to obtain valuable data by simply questioning their subjects. At the same time, there are clear limitations to what self-reports can assess. …the psychological scientist's tool kit also needs a method to directly observe human behavior in daily life. …naturalistic observation can bring behavioral data collection to where moment-to-moment behavior naturally happens.

It is probably evidence of the lack of interdisciplinarity in academia – despite all sorts of interventions and efforts – that Baumeister's and Mehl's evaluations of psychology remain correct. It might be because psychologists lack imagination when they conceive their research. And I speak as a psychologist. Yet conversation analysis (rooted in sociology, not psychology) has been producing knowledge about what people do for over half a century. Some of this research is among the most cited in academia, including 'hard science' disciplines. A lot of people need to start talking!

I hope that this special feature of The Psychologist will open a conversation. We have several articles on seemingly mundane things that, when we pay attention, might reveal some fairly fundamental truths about how we interact. From lapses and sighs to burping, from expressing emotion to talking to tech, from mumbling to complaining, listen in as we consider how real people communicate.

- Elizabeth Stokoe is Professor of Social Interaction at Loughborough University
[email protected]

Read a transcript of her 2016 Latitude Festival appearance for us. Our editor Jon Sutton will be helping Professor Stokoe launch her new book, 'Talk', at Waterstones Nottingham on 27 November. You can read an extract, 'Do women and men talk differently?'

Further reading
Sikveland, R.O. & Stokoe, E. (forthcoming). Verbs of engagement:  Managing resistance in negotiations with suicidal people in crisis.
Stokoe, E. (2018). Talk: The Science of Conversation. London: Little, Brown.

 

The problem with facial expressions

José-Miguel Fernández-Dols argues that expressions are tools rather than signs

Consider the 'facial expressions' displayed on nursery walls, in cartoons, in clinical tests or in psychology textbooks. Static, disembodied faces that supposedly have only one function: to express basic emotions, irrespective of the context in which they are displayed. Today these expressions are great cultural icons, transmitted to worldwide audiences through Pixar movies, TV series and the emotional-intelligence business. But do they really exist? Are they representative of how real people communicate?

No. Only in your nightmares will you run into disembodied faces with still expressions. Actual people display changeable, animated faces, and we always perceive these muscular movements in combination with the movements of the rest of the body and the events in the surrounding context. Scientific studies, such as those published by Hillel Aviezer and his collaborators, show that deciphering the emotional meaning of a facial display in a highly emotional situation (tennis players after winning or losing a crucial match) can be very difficult or even impossible without contextual information.

Even emojis, those schematic versions of facial expressions of basic emotion, provide an example of the strong interaction between facial expression and context. Users do not typically use them to convey a category of emotion but to add subtle affective or motivational nuances to their message. A smiling face besides the message 'you are late' does not mean 'happiness' but 'I am feeling friendly anyway'. A frowning face besides the message 'I don't know' does not mean 'anger', it may mean, for example, 'having a hard time making up my mind'. It could be argued, then, that millions of social media users have naturally developed a more accurate description of the actual role of facial expressions than have psychologists and neuroscientists. We threw the baby out with the bathwater.

In fact, new lines of research have confirmed that the static expressions promoted by the classic version of the basic emotion theory (BET) should not be so popular. Our recent studies in the Trobriand Islands and Mozambique (much of it with Carlos Crivelli) did not find the pattern of recognition observed in literate Western societies. Furthermore, our meta-analysis of the studies that tested the predicted coherence between expression and emotions such as anger, disgust, fear, sadness, surprise or happiness has actually found a surprisingly weak link between emotion and expression.

Today a significant number of researchers envision or support alternative approaches to the understanding of what we call 'facial expressions' (see my 2017 book with James Russell, The Science of Facial Expression). We're inspired not only by a growing amount of empirical evidence, but also by the conceptual problems behind the assumption that the static, disembodied, posed expressions of basic emotion are frequent, natural signatures of our emotions. Let me focus on just one of these conceptual problems.

Facial expressions as pornography

A Martian comes to Earth: their equivalent of a behavioural scientist, and Mars happens to have its own Darwin and an evolutionary theory that is practically identical to ours.

On landing, the Martian researcher looks for behavioural categories that seem to be salient for human beings. She assumes that their relevance most probably hints at its important role for the species' survival. After some googling and mindreading, she finds an interesting target for her inquiries: sex. What is its adaptive function? Our Martian is on the academic tenure track and can't spend terrestrial years observing the causes and consequences of sex on an everyday basis. So she decides to run a 'recognition study'. Her rationale is that if the term 'sex' is so prevalent, maybe the most relevant contribution of sex to human survival is not found in sex itself, but in its 'recognition' as a signal. So she googles 'sex', and voilà! She gets a never-ending series of images, from the thriving earthly pornography industry.

Taking a representative sample of these pictures – which have a high degree of uniformity in some interesting features – she combines them with other neutral pictures, and eventually she asks people, all over the world, which ones are depicting sex. She finds out that the pornographic acrobatics of naked, young and attractive individuals are 'sex'. Actually, much to her delight, she finds that a lot of people can categorise these acrobatics not just as 'sex', but into a number of consistent categories, which she calls 'categories of basic sex' (I won't go into the details). She concludes that 'basic categories of sex' are universally recognised.

At this point, she concludes that if sex has a repertory of universal signals, its main evolutionary function must be the expression of sex. Human survival must depend on the recognition of different combinations of acrobatic, naked bodies as belonging to particular categories of basic sex. But there's a problem: try as she might, she just can't find many naked, slender, good-looking humans performing these categories of sex in conspicuous ways 'in real life'. There must be display rules, she concludes.

Our Martian friend goes back home with a new, attractive, unfalsifiable theory about the evolutionary function of sex in humans: it is a set of innate signals that can be universally recognised! Human sexual intelligence and adaptive fit depend on how well people code and decode all these basic categories of sex. The theory will make her famous, but leave her clueless about the main evolutionary function of sex.

Substitute facial expressions for sex, and this might be a pretty accurate description of what has happened to this field. The original choice of the universal expressions of basic emotion by Tomkins and his disciples was not backed by any empirical study of the prevalence of such expressions in emotional situations (and this remains true). Tomkins' selection was probably inspired by the facial expressions that seemed most salient in the already fully fledged Hollywood industry at the time he published the first sketch of BET.

Universal agreement on the labelling of an icon does not mean that the icon is the represented behaviour; most importantly, it does not mean that the evolutionary function of the behaviour represented by this catchy icon is communication. Stretching the metaphor, facial expressions of basic emotion are catchy, cognitively alluring icons of emotion, but their widespread recognition by Western audiences does not mean that they are the actual facial behaviour observed in emotional episodes. Nor does it mean that the main evolutionary function of the represented facial behaviour is communication.

So what?

I'm not saying that facial expressions of basic emotion cannot have a role in psychology. Although evidence is mixed, they remain potentially useful as diagnostic devices in the same way that people's responses to our Martian's pornographic icons could be, and actually are, highly diagnostic of different human tendencies or representational processes. Artificial icons can detect the natural normal or abnormal functioning of the human brain even better than natural signals. And facial expressions are useful resources for the short-term and long-term success of our interactions. (Our Martian explorer should have assumed this, taken a lot of less slender and agile pairs of humans, observed them for a while, and waited for nine months.)

But we do need a new look at facial expression – one that emphasises what we do with our faces, rather than what is meant by them. Let's consider this via the study of that quintessential facial expression: the smile. It has become synonymous with happiness: even scientists treat the two as interchangeable. But what is really behind a smile? And what is the facial display of people who are happy?

Believe it or not, in the BET golden age of the 1980s and 1990s, psychology did not offer us many answers to this question. Of course, we knew that people – well, Western people – say that smiles are happiness, but systematic information about what actually people display on their face in situations in which they claim to be happy was scarce. A few scientific mavericks were raising questions at that time, but psychologists were increasingly uninterested in human ethology and the question was not taken seriously. In 1979 Kraut and Johnston published a paper in which they showed that bowlers, pedestrians and hockey fans' smiles were predicted by social interaction, rather than happiness. The study, inspired by a long tradition of observational ethological studies, was received with indifference or even hostility by a vast majority of its audience. This foundational work received the Golden Fleece Award, a forerunner of the Ig Nobel awards but with a political agenda: it was aimed at exposing frivolous spending of public research funds.

Fortunately, other maverick scientists were willing to consider the forbidden question. Jim Russell, on the methodological side, and Alan Fridlund, on the theoretical side, joined forces to write an important book, Human Facial Expression, which became a sort of heretical, quasi-forbidden book for BET followers. Fridlund's point was strongly counterintuitive: facial behaviours, including smiles, are nature's tricks to engage people in interaction; they are a powerful tool for influencing others but do not signal a uniform, universal message.

Why do we display smiles when we are alone? Because we need to interact with real or imagined others: consider Cast Away, one of Tom Hanks's most famous movies, in which our desert island hero develops a moving relationship with a volleyball with a face drawn on it.

To make things more complicated (and fascinating), sometimes we are happy and do not smile at all; sometimes we are not happy but we smile. We display a surprisingly large repertory of facial behaviours that make sense in context. I have been studying happy (and unhappy) people for years, either in the lab or in natural settings, and my conclusion is always the same: spontaneous, 'true' smiles are neither necessary nor sufficient signals of happiness or enjoyment. Bullfighters, for example, report fear before the bullfight but happiness while they confront the animal's charges; surprisingly, this happiness is associated with a characteristic expression (funnel lips: García-Higuera et al., 2015). The enjoyment of having sex is associated with a grimace that recalls an expression of pain (Fernández-Dols et al., 2011). Extremely happy athletes cry out of happiness (Fernández-Dols & Ruiz-Belda, 1995). On the other hand, people smile not only when they are happy; they also smile when they are amused (but not necessarily happy; see Gervais & Wilson, 2005), when they are embarrassed (LaFrance & Hecht, 1995) or even when they are anxious (e.g. when sexually harassed during a job interview; see Woodzicka & LaFrance, 2001).

The final outcome of this accumulation of scientific evidence is an open rather than a closed typology of facial expressions. Whereas the current normative closed categories of 'true expressions of basic emotion' have rigid boundaries, open typologies are generative and accommodate new variables in order to provide a mapping of the correspondences between emotional episodes and spontaneous facial movements. This approach is already used in artificial intelligence by some developers who do not use the expressions-of-basic-emotion closed categories, but context-dependent dimensions extracted from actual interaction. Thus expressions are recognised to have multiple functions and more natural appearances (e.g. Castillo et al., 2014; Lang et al., 2013).

There is an urgent need to develop this new mapping of the facial expression from a fresh, open-minded perspective – not only for theoretical but also for practical, everyday reasons. We should stop teaching children that the only intelligent way of expressing their emotions is through a limited repertory of six or seven facial displays! This approach is utterly simplistic or even deleterious. Let me finish with a telling real story. In 2013, a mother in Spain was accused of murdering her daughter. The case had a parallel popular trial in the media and a decisive 'proof' of the mother's guilt was found in her expression during a conversation with her lawyer, while the police searched her house. Some TV stations broadcast her smiles in that conversation, and some supposed experts cited them as proof of the mother's perverse character: she was happy. After the trial, it was known that her lawyer had been telling her funny anecdotes about her late father, a popular endearing character, while trying to relieve her anxiety. Only her lawyer seemed to know that humour can be an effective coping strategy in the face of negative emotions, and that expressions are mainly about and for others: whether a defendant's dad, or a castaway's volleyball.

- José-Miguel Fernández-Dols is a professor of psychology at Universidad Autónoma de Madrid  
[email protected]

References

Aviezer, H., Trope, Y. & Todorov, A. (2012). Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science, 338(6111), 1225–1229.
Castillo, S., Wallraven, C. & Cunningham, D.W. (2014). The semantic space for facial communication. Computer Animation and Virtual Worlds, 25(3–4), 223–231.
Crivelli, C. & Fridlund, A.J. (2018). Facial displays are tools for social influence. Trends in Cognitive Science, 22, 388–399.
Crivelli, C., Jarillo, S., Russell, J.A. & Fernández-Dols, J.M. (2016). Reading emotions from faces in two indigenous societies. Journal of Experimental Psychology: General, 145, 830–843.
Crivelli, C., Russell, J.A., Jarillo, S. & Fernández-Dols, J.M. (2016). The fear gasping face as a threat display in a Melanesian society. Proceedings of the National Academy of Sciences, USA, 113, 12403–12407.
Crivelli, C., Russell, J.A., Jarillo, S. & Fernández-Dols, J.M. (2017). Recognizing Spontaneous Facial Expressions of Emotion in a Small-Scale Society of Papua New Guinea. Emotion, 17, 337–347.
Duran, J.I., Reisenzein, R. & Fernández-Dols, J.M. (2017). Coherence between emotion and facial expression: A research synthesis. In J.M. Fernández-Dols & J.A. Russell (Eds.) The science of facial expression (pp.107–129). New York: Oxford University Press.
Fernández-Dols, J.M. (2017). Natural facial expression: A view from social constructionism and pragmatics. In J.M. Fernández-Dols & J.A. Russell (Eds.) The science of facial expression (pp.457–475). New York: Oxford University Press.
Fernández-Dols, J.M., Carrera, P. & Crivelli, C. (2011.). Facial behavior while experiencing sexual excitement. Journal of Nonverbal Behavior, 35, 63–71.
Fernández-Dols, J.M. & Ruiz-Belda, M.A. (1995). Expression of emotion versus expressions of emotions: Everyday conceptions about spontaneous facial behavior. In J.A. Russell, J.M. Fernández-Dols, A.S.R. Manstead, & J.C. Wellenkamp (Eds.) Everyday conceptions of emotion (pp.505–521). Dordrecht: Kluwer.
Fernández-Dols, J.M. & Russell, J.A. (Eds.) (2017). The science of facial expression. New York: Oxford University Press.
Fridlund, A.J. (1994). Human facial expression: An evolutionary view. San Diego, CA: Academic Press.
Garcia-Higuera, J.A., Crivelli, C. & Fernández-Dols, J.M. (2015). Facial expressions during an extremely intense emotional situation: Toreros' lip funnel. Social Science Information, 54, 439–454.
Gendron, M., Roberson, D., van der Vyver, J.M. & Barrett, L.F. (2014). Perceptions of emotion from facial expressions are not culturally universal: Evidence from a remote culture. Emotion, 14, 251–262.
Gervais, M. & Wilson, D.S. (2005). The evolution and functions of laughter and humor: A synthetic approach. Quarterly Review of Biology, 80, 395–430.
Kraut, R.E. & Johnston, R.E. (1979). Social and emotional messages of smiling: An ethological approach. Journal of Personality and Social Psychology, 37, 1539–1553.
LaFrance, M. & Hecht, M.A. (1995). Why smiles generate leniency. Personality and Social Psychology Bulletin, 21, 207–214.
Lang, C., Wachsmuth, S., Hanheide, M. & Wersing, H. (2013). Facial communicative signal interpretation in human–robot interaction by discriminative video susequence selection. In Proceedings of the IEEE International Conference on Robotics and Automation (pp.170–177). Karlsruhe: IEEE.
Russell, J.A. & Fernández-Dols, J.M. (Eds.) (1997). The psychology of facial expression. Cambridge: Cambridge University Press.
Woodzicka, J.A. & LaFrance, M. (2001). Real versus imaged gender harassment. Journal of Social Issues, 57, 15–30.

 

Repair

Saul Albert analyses how conversation is continually corrected

Imagine a society in which we all communicate telepathically, so that there are never any misunderstandings, misspeakings or mishearings. Whenever we wanted to do something complicated that needed lots of people to cooperate, our actions would be coordinated in a smooth, uninterrupted flow. The problem with this idea, apart from it sounding like the plot of a 1950s B-movie, is that if there were a misunderstanding, it would be nearly impossible to tell. How would we know, and be secure in our knowledge, that people weren't just going along with whatever others were doing without really engaging with and understanding one another? If we couldn't detect and fix failures of communication, how would we even recognise success?

Counterintuitive as it may seem, it is our miscommunications, hesitations, disfluencies, quizzical looks, and words like 'huh?' that provide the best evidence for mutual understanding. If we look at recordings of how real people talk in everyday life it's surprising how much of what they do involves dealing with problems of speaking, hearing and understanding as they emerge. In conversation analysis, we call the methods people use to fix these kinds of problems 'repair'. The way that repair works is very systematically organised, which makes sense, given that repair procedures are the basic tools we must be able to use in order to understand each other.

The simplest way to categorise the repair procedures that have been discovered so far is to divide them up in terms of who initiates the repair (by seeing that there's trouble and pointing it out), and who completes the repair (by fixing the source of the trouble so we can move on). Any speaker (self) or recipient (other) of a 'trouble source' can initiate and/or complete a repair, so we can fit any repair into one of four main categories (Schegloff et al., 1977). Each of these types of repair is organised differently, with different uses and social outcomes.

'Self-initiated self-repair' is when the speaker identifies and completes repair on their own talk. This is by far the most common form of repair, which makes sense since if the trouble source is coming out of the speaker's mouth, they will probably have the first chance to identify and fix it. You can identify this type of repair in any conversation if you listen for when people cut off then re-start their stream of speech.

If you record and replay a self-initiated self-repair slowly, you'll often find that the speaker will have deleted, added or changed something they were in the process of saying. This happens often if someone is choosing their words carefully, so studying this kind of self-repair can show us how people deal with sensitive social situations (Lerner, 2013; Mandelbaum, 2016). 'Self-initiated other-repair' is when the speaker identifies a problem in their own speech but then relies on someone else to fix it. For example, if a speaker is struggling to remember a name they can self-initiate other-repair by cutting off and saying 'er… you know, whatsisname, er…'. As long as the recipient has heard enough to get the gist, they'll suggest candidate 'repair solutions' until they fix the problem together and move on with the conversation. Such displays of forgetfulness are very useful tools for achieving all kinds of social outcomes: for example, inviting others to 'fill in the blanks' when telling a shared story (Goodwin, 1987, 2004).

'Other-initiated self-repair' is when a recipient notices some problem in the speaker's talk, and points it out, but then leaves it for the speaker to complete the repair themselves. This kind of repair initiation, in response to a first turn, is designed to catch problems of speaking, hearing and understanding just after they occur. It also provides many useful techniques for dealing with these three distinct types of communication problem. For example, if someone is speaking and their recipient says 'huh?', the speaker will usually produce a full repeat of their last turn. 'Huh', which seems to be an almost universal word for the other-initiation of self-repair across languages (Dingemanse et al., 2013), is an example of an 'open class' repair initiator (Drew, 1997) since it doesn't give any clues about the cause of the recipient's problem. When speakers do the usual full repeat after a 'huh?', this treats it as a problem of hearing. Contrast this with category-specific other-initiations of repair such as 'who?', 'where?' or 'when?', which specify the precise kind of thing that needs to be fixed.

The speaker can fix the problem and move on with a partial repeat of the prior turn, targeting only the specific trouble-source. The most help a recipient can give to the speaker in terms of fixing the trouble at hand is by offering a candidate hearing or understanding, which the speaker can then confirm or deny.

Finally, 'other-initiated other-repair' is the least common of the four main repair types since it involves the recipient unilaterally identifying and fixing the speaker's talk, which may be socially inappropriate. The situations where it is common for recipients to initiate and complete repair can help explain why it's otherwise quite unusual. For example, other-initiated other-repair is common for parents correcting their child's speech, or when native speakers correct the speech and pronunciation of second-language learners (Hosoda, 2006). Most often, however, recipients use other-initiated self-repair to allow speakers the opportunity to correct their own talk.

These systematic variations in the structure of repair in different situations show why repair is such a useful tool for securing and maintaining mutual understanding in social interaction. The choices people make about the kind of repair procedures to use in any given situation not only demonstrate their understanding of talk, but also their understanding of the social situation. This is why Schegloff (1992) describes 'third position repair' (a form of self-initiated self-repair where a recipient's response reveals that they have misunderstood the speaker's initial turn) as 'the last structurally provided defense of intersubjectivity in conversation'. When misunderstandings pass unnoticed through the ongoing flow of turns and sequences at talk, we may never discover or deal with the fact that any kind of miscommunication has occurred. Telepathy aside, without repair and the strong evidence it provides for the communicative success of our interactions, the complexity and coordination of our societies would be almost impossible to imagine.

If you would like to find out more about repair, check out our lab's open access review of the conversation analysis literature (Albert & de Ruiter, 2018), which includes open data recordings of the various forms of repair I've described.

- Saul Albert works at the Tufts Human Interaction Laboratory, Medford, Massachusetts
[email protected]

Key sources

Albert, S. & de Ruiter, J.P. (2018). Repair: The interface between interaction and cognition. Topics in Cognitive Science, 10(2), 279–313.
Dingemanse, M., Torreira, F. & Enfield, N.J. (2013). Is 'Huh?' a universal word? Conversational infrastructure and the convergent evolution of linguistic items. PLoS ONE, 8(11), e78273.
Drew, P. (1997). 'Open' class repair initiators in response to sequential sources of troubles in conversation. Journal of Pragmatics, 28(1), 69–101.
Goodwin, C. (1987). Forgetfulness as an interactive resource. Social Psychology Quarterly, 50(2), 115–130.
Goodwin, C. (2004). A competent speaker who can't speak: The social life of aphasia. Journal of Linguistic Anthropology, 14(2), 151–170.
Hosoda, Y. (2006). Repair and relevance of differential language expertise in second language conversations. Applied Linguistics, 27(1), 25–50.
Lerner, G.H. (2013). On the place of hesitating in delicate formulations: A turn-constructional infrastructure for collaborative indiscretion. In M. Hayashi, G. Raymond & J. Sidnell (Eds.) Conversational repair and human understanding (pp.95–134). Cambridge: Cambridge University Press.
Mandelbaum, J. (2016). Delicate matters. In J.D. Robinson (Ed.) Accountability in social interaction (pp.108–137). Oxford: Oxford University Press.
Schegloff, E.A. (1992). Repair after next turn: The last structurally provided defense of intersubjectivity in conversation. American Journal of Sociology, 97(5), 1295–1345.
Schegloff, E.A., Jefferson, G. & Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language, 53(2), 361–382.

 

Talking with Alexa

Stuart Reeves and Martin Porcheron listen in to the 'conversation'.

The idea of a 'smart personal assistant' that you can speak to in your home is no longer the stuff of science fiction. Apple Homepod, Google Home, the Amazon Echo and more are all vying for this role. They are sold (in their millions) as household helpers that let you perform various tasks naturally by just talking to them, whether that's asking them for information, helping out with the cooking by guiding you through a recipe, putting on some music, doing some shopping or just telling the time.

If you own one of these devices, though, you'll know that the reality is a bit different. Often they don't seem to hear what we say, and when they do respond, the response often betrays a significant lack of understanding of what we really mean. There are now many videos available online of inexplicable interactions recorded by owners of these devices. Interaction with them is a little 'messy'.

The field of human–computer interaction (HCI), which has strong historical roots in psychology and its application to computer interfaces, is actively exploring not only the role of these new devices in our home life but also how they might be better designed to take the complexity of conversation into account. As HCI researchers, we think that taking a human-centred approach, by looking at the precise details of how people actually use language to get things done, will help us better understand the interactional 'mess' and work out how to improve the design of these systems. In our research we find that users of voice-based assistants often work very hard to integrate them into the social setting and deal with the various problems they encounter in use.

Our group – along with Joel Fischer and Sarah Sharples – have been doing some empirical work looking at the Amazon Echo, marketed as a voice-based personal assistant that uses the Alexa Voice Service. We did fieldwork by collecting audio recordings from five households each deployed with an Echo for a month, capturing what participants said to the device but also the conversations they had before, alongside and after moments of interaction with the device. Informed by a conversation analysis approach to make sense of this corpus of hundreds of hours of recorded audio data from the home, we have been developing descriptions of the various methods people use to organise their talk with and around the Echo into a coherent conversation.

Let's take just one example. Nikos and Isabel are at a New Year's party and they are trying to get Alexa to play some suitable music.

Nikos      Alexa

(2.6)

Isabel    play some New Year's music

        (1.8)

Alexa     here's a station for jazz music,

instrumental jazz (1.4) ((music starts playing))

(4.3)

We have many such examples (180+) of householders' extended 'conversations' with Alexa. Several observations can be made from this short fragment that illustrate features we repeatedly find in these exchanges.

First, we can spot a form of use that is never depicted in the adverts: Nikos addresses Alexa with the wake word 'Alexa', but then after a pause, Isabel takes over with her own instruction. It is a form of 'speaker selection' (see Gene Lerner's work), but very different to human conversation.

Nikos      Alexa

(2.6)

We see this kind of collaboration (and sometimes 'competition') between users of Alexa frequently in our data. The home is a social environment and offers of help (both explicit and implicit) emerge frequently to smooth things along (see Kendrick & Drew, 2015). There is a politics to the control of the device that is worked out as part of the life of the home.

Having been asked for 'some New Year's music', Alexa responds.

Alexa     here's a station for jazz music,

instrumental jazz (1.4) ((music starts playing))

(4.3)

Isabel    Alexa this is not what we wanted ((laughs))

This response is treated negatively by Isabel. There are three interesting things about this.

Firstly, it turns out that Alexa's response is the result of a speech transcription error (we know this from logs). But the potential mismatch between what was said by Isabel ('New Year's music') and what has been captured by the device is never revealed to users; no hesitancy or uncertainty is displayed in the response from Alexa (e.g. a question format could be employed, 'did you want to listen to jazz music?'). Competent conversationalists routinely perform remedial action to repair emerging misunderstandings between themselves and others (see Saul Albert's article in this collection, and Schegloff et al., 1977). But voice-driven devices seem poorly designed to live in a world of constant verbal 'fixing'. As a result, it is users of them who are constantly seeking to repair various sense-making problems that are encountered.

The second aspect is about Isabel's negative assessment of Alexa's response and the music being 'not what we wanted' (and her laughter). The category 'New Year's music' turns on various socially shared (and culturally situated) assumptions about what constitutes relevant music to play; as conversationalists we work with the complexity of categorisation routinely (Schegloff, 2007). It is not a genre or artist or song Isabel is asking for (which happen to work readily as search keywords).

Thirdly, Isabel laughingly says 'this is not what we wanted', which she addresses notionally to Alexa but also deftly acts as a joke for the others to join in with. We see frequent uses of the Echo as a prop for shared jokes, often involving utterances ostensibly addressed to the device. The role of the tech as a resource for such things is largely absent from demos or sales pitches for voice interfaces, perhaps because doing irony with the device as a prop might be perceived as undermining for a marketing campaign (since it often turns on making the device look 'stupid').

Something interesting happens next. Nikos tries to stop the music playing with 'shut up', but Isabel then chides him with a third-person 'apology' ironically addressing the device.

Nikos      Alexa (1.1) shut up!

Isabel    hey! (0.7) Alexa, Nikos apologises for being so rude

This is another feature we repeatedly see: normative moral order – the shared, agreed sets of ways of acting against which we are held to account – is not somehow suspended when addressing the voice assistant. What is said to the device is necessarily often said around others. In other words, you are accountable for what you say, even to a computer. The Echo, like its counterparts, is sold as a device to live in the home. In doing so it becomes embedded into the fabric of that home, including the established and expected organisation of social conduct. Thus, conduct designed for the device is nevertheless socially implicated conduct. It's important not to get confused here, however. Isabel is not somehow apologising to the device but rather offering an analysis of Nikos's behaviour that is accountable to a particular normative moral order ('being polite').

The final part of this exchange sees Isabel's 'apology' being responded to.

Isabel    hey! (0.7) Alexa, Nikos apologises for being so rude

Alexa      hi there

(3.4) ((music is still playing))

Nikos      Alexa stop stop

There seems to be little sequential coherence between this response and what Isabel said (or Alexa's prior actions, like playing some jazz). This forms a break in the illusion of what the device is doing. Alexa's 'conversation' with the user is really just set of attempts by the device to fulfil 'commands' that it has likely 'heard'. At best, voice devices may have a sense of 'state', connecting one utterance by a user to a prior one. However, these are still fairly limited exercises in 'slot-filling' for a set of possible paths (rather like following a simple recipe). For users, however, there is ongoing context being built up all the time and a rich set of implied meanings (e.g. categorisations) that can be used as resources for 'next moves' in the conversation. For Alexa that tracking of and response to the always-building context is severely impoverished, and users must thus work around the limitation all the time. We can see this when Nikos – reformulating his prior command, 'Alexa (1.1) shut up!' – utters 'Alexa stop stop'. Nikos does not treat Alexa's greeting 'hi there' as a greeting at all (i.e. there is no paired greeting from him e.g. 'hi Alexa'). Instead he carries on with his command to 'stop'.

Some concluding remarks. Research into how we talk is catching up with the latest developments in 'conversational' interfaces and personal assistants as they become more widespread in everyday life – both via disciplinary hybrids, such as our use of conversation analysis in HCI, and in conversation analysis itself beginning to examine the organisation of non-human (and human/non-human) interaction (e.g. see Federico, 2013, and Pika et al., 2018).

Our recent work suggests that many of these new AI-driven systems are designed to support 'conversations' with people. But the reality of their use is that they tend to display significant difficulty with many routine but deeply critical aspects of talk that have been mostly overlooked by speech technology research (which tends to focus on technologically driven advances). That said, we nevertheless see users of voice-based interfaces going to significant lengths to repair breaks in interaction, sense-making, and often in the course of doing so, innovating possibly novel conversational forms that research into human language and communication has yet to document fully.

Of course, our study was limited to one month of use. What remains unclear is how long people will tolerate such interactional clunkiness and whether this leads either to permanent abandonment of these new voice-based personal assistants or to increasingly novel ways of speaking that encompass new forms of device-oriented language—new ways that 'real people communicate' that (much like adapting to a mouse and keyboard) are simply accommodations people must develop to get by.

- Stuart Reeves is in the Mixed Reality Lab, School of Computer Science, University of Nottingham
[email protected]

- Martin Porcheron is at the University of Nottingham

Key sources

Federico, R. (2013). Sequence organization and timing of bonobo mother–infant interactions. Interaction Studies, 14, 160–189.
Kendrick, K.H. & Drew, P. (2015). Recruitment: Offers, requests, and the organization of assistance in interaction. Research on Language and Social Interaction, 49(1), 1–19.
Lerner, G.H. (2003). Selecting next speaker: The context-sensitive operation of a context-free organization. Language in Society, 32, 177–201.
Pika, S., Wilkinson, R., Kendrick, K.H. & Vernes, S.C. (2018). Taking turns: Bridging the gap between human and animal communication. Proceedings of the Royal Society B, 285(1880), 20180598.
Porcheron, M., Fischer, J.E., Reeves, S. & Sharples, S. (2018). Voice interfaces in everyday life. In Proceedings of the 2018 ACM Conference on Human Factors in Computing Systems, CHI '18, ACM, New York.
Schegloff, E. (2007). A tutorial on membership categorization. Journal of Pragmatics, 39(3), 462–482.
Schegloff, E., Jefferson, G. & Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language, 53(2), 361–382.

 

Analysing burp sequences

Alexa Hepburn and Jenny Mandelbaum dig in to a family mealtime interaction.

Many of us can recall the guilty pleasure of childhood burping competitions, or the shameful embarrassment of an unintentional burp in a public setting. However, as we will show, closer investigation of children's mealtime burps allows us a fascinating glimpse into how families orient to both appropriate behaviour and rebellion. 'Real people' burp, and burps carry communicative information.

As part of a larger study on parents' socialising practices in family mealtime interaction, we examined children's (apparently) intentional and unintentional burps in early childhood (ages 3–5), later childhood (age 9), and teenhood (age 16), along with parental responses to these burps.

While this study is preliminary, based on conversation analysis of a collection of 21 burp sequences that occur spontaneously in 59 field recordings of family dinners, we find that parents' initial responses to burps are calibrated to two related issues – how intentionally produced the burp is, and the burpers' displays of contrition. Our analysis also shows that even the youngest children can be highly sophisticated in understanding parental reprimands, whereas older children may exploit the inferential work required of them to flout their parents. We can illustrate this with a couple of contrasting examples.

In our first extract, Cindy, aged 9, and her parents are eating dinner together. We join them as Dad assesses a new dish that Mom has prepared. In line 6, Cindy produces a loud burp, and immediately raises her hand to her mouth, as if to 'cover' the burp. This sharp movement of her hand displays surprise at the burp, perhaps marking it as an inadvertent eruption. The transcript shows elements associated with the coordination and prosodic delivery of talk, and key interactional non-vocal activity.

1.  B1 Stew Dinner AH revised

01    DAD:  this is [no:t (.) one of my favorite dish]es.

02    CIN:          [ I wan' another ro:ll please.   ]   

03    MOM:  °Mmhm.°

04    DAD:  But it's not ba:d. Iddis: (.) [it's like [soup.   ]

05    DAD:                                [((moves  g[lass))

06 -> CIN:                                           [((burp))]=

07    CIN:  =((ha[nd to mouth)) [hns­ih hnih [hih hnih

08    DAD:       [It's very clo:[se to soup.

09    MOM:       [ (("looks" to [Cin and ))  [((raises eyebrows))
10    MOM:                                   [<Ni:ce ma:nners.>]=

11    CIN:  =[hih hih.//((laughs through hand))

12    MOM:  =[((gaze to rolls))

While Cindy is still in the process of raising her hand to her mouth, Mom in line 9 turns her head towards her, and gives her 'the look', a rather stern sustained gaze, as described by Kidwell (2005). While this is happening, Cindy in line 7 begins to laugh. Mom then produces '<Nice ma:nners.>' in line 10. The delivery of Mom's turn proposes the turn as an ironic or joking censure rather than a serious one. In this case then, after Cindy begins to produce a display of contrition, her burp is acknowledged as a social infraction, but more or less in passing; it is not taken up as a serious matter for reproof.

In contrast, in Extract 2, we see a burp that appears to be deliberately exposed and over-produced. While Mom is producing agreement with her son Kevin in line 1, in line 2 we see her daughter Karen (age 16), lift her head into what could be construed as a 'pre-burp' position. We join the extract after Mom, Kevin 22 and Daniel, 20 have been discussing how different sides of the brain function.

B2_16_FCSp14_DM_23-33

()

01    MOM:    That's what I'm [saying.

02 -> KAR:                    [((Lifts head up))

03    KEV:    Yeah.

04    MOM:    [Is- is it bec-]

05 -> KAR:    [  ((Burp))    ]=

06    MOM:    =°Karen, ((gaze to Karen))   

07    MOM:    (0.2)//((sustains gaze to Karen))

08    KAR:    MO:[M.

09    MOM:       [((Mom returns gaze to her food))] // (0.3)

10    KAR:       [((Reaches to serve herself))    ]

11    MOM:    ((returns her gaze to Karen/giving her "the look"))

12    MOM:    Hhh ((sigh, sustaining 'the look' at Karen))

13            [(1.5)

14    MOM:    [((gazes to Karen, then returns gaze to her food))

15    MOM:    Where are your [ ma[(h)nn(h)ers. ]

16    MOM:                   [((s[haking head))]

17    KAR:                       [  eh  heh    ]

18    MOM:    (0.3)/(( continues to shake her head))

19    KAR:    °Oh shit°/((drops some food on her napkin))

20            (1.5)

21    MOM:    'T's interesting, Kevin ((prior topic continues))

In line 5 Karen emits an open-mouthed burp, produced in overlap with Mom's turn in line 4. Immediately upon completion of Karen's burp, Mom looks over at her and says her name, 'Karen'. While the format of Mom's turn is a summons, the prosody, in combination with the 'look' at lines 6 and 7 immediately after the burp, makes Mom's turn hearable as initiating a censure.

These kinds of subtle sanctioning actions invite children to inspect their conduct for an infraction, find it, and then acknowledge its transgressive character (Hepburn & Potter, 2011). However, in response to Mom's turn, in line 8 Karen does a return summons to Mom in a loud, exaggerated way, 'MO:M', challenging her to acknowledge her own wrongdoing. Producing an exaggerated turn of the same format as Mom's may be understood as a way of refusing to acknowledge transgressive conduct, and in this way implements defiance. In line 19 Mom produces a hearable sigh, while still giving Karen 'the look', but Karen sustains her attention to the food she is serving herself. In line 5 we see a more explicit attempt by Mom to get Karen to acknowledge and atone for her behaviour with 'Where are your ma(h)nn(h)ers.', produced with a head shake. Prior talk resumes after this, without Karen having acknowledged the burp.

Here then we see a burp exposed apparently intentionally, the burper prompted to acknowledge and show contrition for the burp but instead resisting increasingly overt attempts to get her to acknowledge wrongdoing. In this way, Karen subverts the asymmetry that is characteristic of parent–child interaction, in which parenting comes with elevated rights to direct and sanction behaviour. In pursuing a display of contrition after Karen's counter-summons, Mom deletes the relevance of Karen's summons, and her entitlement to do it, and resumes her sanctioning project.

These contrasting extracts illustrate some of our broader analytic findings: when a child burper displays contrition, like Cindy in our first extract, parents overlook or respond to the burp with a very mild or joking censure. However, older children like Karen can exploit the inferential work required by covert admonishment techniques such as the sustained gaze or summons. It seems then that it's not burping alone that is transgressive, rather it is the intentional exposure of burping, and/or failure to suppress or remediate its occurrence. Interestingly, parental responses seem calibrated to these issues of intentionality and contrition irrespective of the age of the child. However, we are cautious about making definitive claims about developmental differences on the basis of our current corpus.

Burping in our US and UK mealtimes therefore has a number of important features. Like other bodily eruptions it is purportedly unintentional and offensive, and therefore constitutes a social infraction. Perhaps for these reasons it is attractive to children; its transgressive character renders it funny, and its involuntary character provides defeasibility, and an opportunity to flout parental attempts at controlling behaviour. So, while seemingly inconsequential and uncontrollable, mealtime burping appears to be an environment in which children can learn to 'fly below the radar', offering them both a training ground in proper conduct and a resource for implementing and communicating defiance.

- Alexa Hepburn and Jenny Mandelbaum are at Rutgers University

Key sources

Hepburn, A. & Potter, J. (2011). Threats: Power, family mealtimes, and social influence. British Journal of Social Psychology, 50(1), 99–120.
Kidwell, M. (2005). Gaze as social control: How very young children differentiate 'the look' from a 'mere look' by their adult caregivers. Research on Language and Social Interaction, 38(4), 417–449.

 

Silence is not always golden

Elliott Hoey on… umm… …awkward silences

Regularly enough in conversation people come to a good enough ending to whatever they're talking about and then the conversation lapses into silence. Sometimes this silence is completely fine, comfortable even. Like when you're with family or intimates, eating breakfast at home or driving in the car. But other times, these silences feel awkward, ungainly and deeply uncomfortable. Think of this happening on a first date or when chatting with your boss. What's behind these awkward silences? What makes some lapses in talk distressing?

The sociologist of everyday life Erving Goffman described awkward silences as moments of interaction consciousness, or a kind of alienation from social interaction:
A participant in talk may become consciously concerned to an improper degree with the way in which the interaction, qua interaction, is proceeding… Once individuals enter a conversation they are obliged to continue it until they have the kind of basis for withdrawing that will neutralize the potentially offensive implications of taking leave of others. While engaged in the interaction it will be necessary for them to have subjects at hand to talk about that fit the occasion and yet provide content enough to keep the talk going; in other words, safe supplies are needed. What we call 'small talk' serves this purpose. When individuals use up their small talk, they find themselves officially lodged in a state of talk but with nothing to talk about; interaction consciousness experienced as a 'painful silence' is the typical consequence. (Goffman, 1967, pp.119–120)

This painful silence likely resonates with our own experiences in conversations. When prompted to think about these moments, people report feeling intensely aware of their own and others' social behaviour, and having uncertainty about what conduct is appropriate for the situation (Clegg, 2012a, 2012b; McLaughlin & Cody, 1982). Goffman (1956) thought moments like these were critical for understanding social behaviour, and indeed enshrined embarrassment as the central emotion for social life. Researchers since have similarly situated embarrassment in the social situation from which it arises, rather than primarily as an individual phenomenon.

But what are the particular social circumstances that give rise to the awkwardness we feel during some silences? We can approach this question by looking at instances of social interaction to see what people actually do. In my research, I've looked at over 500 instances where conversations lapse into silence to see how people behave around those moments and what they make of them. Awkward silences, when they do occur, emerge from participants' self-awareness when managing their own ambiguous involvement in interaction. And this is something generated by general interactional processes. The way that we get 'ambiguous involvement in interaction' is from the mechanisms underlying social interaction itself. Specifically, awkwardness of this sort can be linked to turn-taking in conversation, and how one turn relates to the next.

Who speaks next?

Harvey Sacks, the founder of conversation analysis, noted that 'under the more general rule of the turn-taking techniques we're dealing with, silence is a terrible thing. The turn-taking rules say that somebody should be talking all the time; not more than one person, but somebody' (1992, p.225). Whenever a speaker gets to the end of their speaking turn, other participants get the chance to speak in a particular order. If someone was selected to speak next, they then get the floor. If no one was specifically called out to speak next, then anyone can self-select and take the floor. And if no one else chooses to self-select, then the same speaker can go on speaking.

But an important part of this turn-taking machinery is that everyone has the option to speak. When everyone forgoes the option to speak, then a lapse can develop, consisting of 'rounds of possible self-selection' (Sacks et al., 1974). The awkwardness of some lapses is bound up with a tension or ambiguity around who is obligated to speak right at that moment. It is every participant's right to not speak, but if every participant avails themselves of that right, who is to speak next? Whose right to remain silent, so to speak, prevails here? The tension embodied by these rounds of possible self-selection connects awkwardness to turn-taking.

While the absence of self-selection is the mechanism that gets you the silence, the main driver of awkwardness is bound up with how participants relate one turn to the next, or 'sequence organization' (Schegloff, 2007). Sequences are a series of linked actions or turns that hang together in the formation of coherent courses of action. Participants build up sequences together turn by turn, and advance with some course of action or topic until it's adequately complete or until there's nothing left to say on the matter. At this juncture – that is, where a sequence is possible complete – participants have the opportunity to launch some new course of action, return to something they were talking about before, or otherwise take up some matter that's different from what they were just doing. But it's also possible that at this juncture no one does anything. The choice to not continue with what you were doing before and not proceed to anything new lets the conversation lapse. So a silence can emerge in the place where one sequence has come to an end and before the next one has begun.

Another thing about sequences is that they ordinarily lend intelligibility to whatever's going on in interaction. And this intelligbility relates to the feelings of ambiguity and uncertainty characteristic of awkward silences. The turn-by-turn sequential organisation of talk creates and maintains an 'architecture of intersubjectivity' (Deppermann, 2015; Heritage, 1984; Schegloff, 1992), which lets participants share an understanding of 'what's going on right now'. But if participants are stuck in the juncture between one sequence and the next, then 'what's going on right now' isn't supported by that architecture. In the absence of some next-thing-to-do, participants are left somewhat unmoored from the intersubjective grounds ordinarily supplied by sequences. Sociologist Christian Heath describes the situation in the following way:

Individuals become increasingly aware of their own actions during the episode of embarrassment; the self attention emerging as they attempt to deal with the local configuration of action. The emotion, experience of the situation and its heightened sensitivity, emerging as the individuals progressively attend to the production of their own actions; the emotional experience deriving from their perception of and involvement in the action in which they are engaged. … [Embarrassment emerges] in circumstances in which the nature of the individual's involvement in interaction is at issue or ambiguous … [where] persons are found in each other's immediate co-presence whilst lacking a mutually co-ordinated activity to which they are committed. (Heath, 1988, pp.156–157)

This ambiguity, conflict and heightened sensitivity to action emerges from the sequential organisation of interaction. Lapses in conversation are places for the management of multiple courses of action. They are silences that invite participants to consider a range of things: what just happened in the previous sequence, what it implies (if anything), how the previous course of action began, whether the previous course of action can be reciprocated, whether there are any unresolved matters to take up, what news there may be to deliver, what updates there may be to solicit from others, what sites of interest there are in the environment, and so on (Hoey, in press-a).

These sorts of questions often help participants come up with something to talk about, and in this way they can be seen as a countervailing force to potential awkward silences. At the same time, though, they may serve to fluster a cognitively burdened participant. If the participant is unprepared to address these questions, their collective preponderance might be felt as unwieldy. Even though answering these questions offers a way out of the silence, a participant can feel encumbered by the range of possible things to talk about and the abundance of directions to go with the interaction. The heightened sensitivity to potential courses of action may derive from feeling compelled to weigh the merit of each one and feeling pressured to select from among them, all while the clock is still ticking.

What does this silence mean?

Perhaps another aspect of the awkwardness of lapses has to do with participants' being exposed to the interactional machinery of which they are ordinarily kept unaware, and seeing their place within it at that moment. Questions about what's happened so far, what remains to be done, etc., lay bare the organisation of conversation to a participant, who may become conscious of the local configuration of action, and self-conscious of their particular position in it. The participant becomes aware of the immediate circumstances while still embedded in them and still responsible for them. A lapse in talk may reveal to participants just how fragile and precarious conversation is, and it may implicate the participants in their failure to sustain it. Compounding this is the reciprocal nature of embarrassment – feeling embarrassed, knowing others may be feeling the same way, and nakedly being embarrassed before one another.

This moment of ambiguity in selecting from among different relevant things to talk about can easily shade into embarrassment. For participants who are officially committed to having a conversation, lapses rupture what should be continuously sustained turn-by-turn talk. Lapses place participants in a situation where there is a palpable absence of conversation. This overlong discontinuity in conversation can not only expose participants' collective failure to coordinate turn-transfer but can also belie the interaction itself by implying some uncomfortable things. A lapse in conversation can invite consideration of potentially delicate matters, such as who my co-participants are, who they are to me, what this silence says about me, what it says about them, what it says about us, what it means for our relationship, how I might be seen as 'being silent' (cf. Sacks, 1992, p.101), how the silence reflects on what just happened, how it reflects on the interaction as a whole, and so forth (cf. Koudenberg et al., 2017). And conversely, being able to swiftly arrive at answers to these questions and have confidence in them is probably how certain silences come to be felt as comfortable rather than awkward.

Escaping embarrassment

It is possible to detect in participants' lapse behaviours some indication of this embarrassment. In the lapses I've looked it, we often see things like scratching, stretching, eating, drinking, sighing, yawning and other forms of momentary disengagement (Hoey, 2015, in press-b). These behaviours offer a kind of 'escape hatch' from interaction. They let participants exhibit unpreparedness for speaking and, through that display, a disinclination to speak next.

Psychologists and ethologists have examined very similar self-directed conduct in humans and non-human primates. 'Displacement activities' like scratching, yawning, face/mouth-touching, lip-biting/licking, and hand fumbling have been linked to situations of social tension, anxiety and uncertainty, and correlate with increased autonomic arousal (Troisi, 2002). Similarly, gesture researchers have associated 'body manipulator' behaviours with anxiety, guilt, conflict, stress and discomfort (Ekman, 1977; LeCompte, 1981). Not only are these displacement activities and body manipulations virtually identical to the disengagement behaviours regularly seen in lapses, but reports of awkward situations similarly support the link between such behaviours and anxiety, stress and uncertainty (Clegg 2012a, 2012b).

The value of these behaviours in a lapse is that they allow you to be engaged in the interaction, but involved with something else and therefore a less-than-eligible candidate for next speaker. You can avoid the multitude of questions posed above, or conversely, buy time while considering your route out.

- Elliott Hoey is a postdoctoral researcher at the University of Basel
[email protected]

References

Clegg, J.W. (2012a). Stranger situations: Examining a self-regulatory model of socially awkward encounters. Group Processes & Intergroup Relations,15(6), 693–712.

Clegg, J.W. (2012b). The importance of feeling awkward: A dialogical narrative phenomenology of socially awkward situations. Qualitative Research in Psychology, 9(3), 262–278.

Deppermann, A. (2015). Retrospection and understanding in interaction. In A. Deppermann & S. Günther (Eds.) Temporality in interaction (pp.57–94). Amsterdam: John Benjamins Publishing.

Ekman, P. (1977). Biological and cultural contributions to body and facial movement. In J. Blacking (Ed.) The anthropology of the body. London: Academic Press.

Goffman, E. (1956). Embarrassment and social organization. American Journal of Sociology, 62(3), 264–271.

Goffman, E. (1967). Interaction ritual: Essays in face to face behavior. Garden City, NY: Doubleday.

Heath, C. (1988). Embarrassment and interactional organization. In P. Drew & A. Wootton (Eds.) Erving Goffman: Exploring the interaction order (pp.136–160). Cambridge: Polity Press.

Heritage, J. (1984). Garfinkel and ethnomethodology. Cambridge: Polity Press.

Hoey, E.M.(2015). Lapses: How people arrive at, and deal with, discontinuities in talk. Research on Language and Social Interaction, 48(4), 430–453.

Hoey, E. M. (in press-a). How speakers continue with talk after a lapse in conversation. Research on Language and Social Interaction.

Hoey, E. M. (in press-b). Drinking for speaking: The multimodal organization of drinking in conversation. Social Interaction: Video-Based Studies of Human Sociality.

Koudenberg, N., Postmes, T. & Gordijn, E.H. (2017). Beyond content of conversation: the role of conversational form in the emergence and regulation of social structure. Personality and Social Psychology Review, 21(1), 50–71.

LeCompte, W.A. (1981). The ecology of anxiety: Situational stress and rate of self-stimulation in Turkey. Journal of Personality and Social Psychology, 40, 712–721.

McLaughlin, M.L. & Cody, M.J. (1982). Awkward silences: Behavioral antecedents and consequences of the conversational lapse. Human Communication Research, 8(4), 299–316.

Sacks, H. (1992). Lectures on conversation. Oxford: Blackwell.

Sacks, H., Schegloff, E.A, & Jefferson, G. (1974). A simplest systematics for the organization of turn-taking in conversation. Language, 50(4), 696–735.

Schegloff, E.A. (1992). Repair after next turn: The last structurally provided defense of intersubjectivity in conversation. American Journal of Sociology, 97(5), 1295–1345.

Schegloff, E.A. (2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge: Cambridge University Press.

Troisi, A. (2002). Displacement activities as a behavioral measure of stress in nonhuman primates and human subjects. Stress, 5(1), 47–54.

 

Complaints

Emily Hofstetter on how much better we are at complaining than we think.

Complaints are ubiquitous in everyday life. We tell complaints to our friends and family, we complain to the doctor about our symptoms, and we complain to public services to get the bins collected. Despite each of us complaining multiple times a day in a variety of settings, little research in psychology investigates how we actually accomplish complaints in real time, and for what purpose.

We often imagine complaints to be vitriolic diatribes: an image fuelled recently, perhaps, by interest in internet trolls. Self-help manuals (and the internet) abound with advice concerning complaints, such as how to avoid them, how a manager should deal with them, and how to make them in an effective, 'constructive' way. All this advice assumes we are not already excellent, practised complainers, and this is false. We are highly sensitive to how and to whom we complain. All you have to do is look at real conversations and examine real, naturally occurring complaints. Conversation analysts have been doing just that fordecades, showing how deftly humans manage the social challenges associated with making complaints.

For example, we must appear to be disinterested and reasonable complainers, without a personal stake in the matter, or else our complaint may be treated as characteristic of a personal vendetta (Stokoe & Hepburn, 2005). We must also be careful to avoid being seen as people who complain all the time, lest we be treated as 'whingers' or 'moaners' (Edwards, 2005). Patients cannot visit their doctor about any old thing – in presenting their symptoms, they regularly make special effort to show that their current concern is serious, not personally motivated, and not imagined (Heritage & Robinson, 2006). Further, callers to hotlines typically try to show they have attempted to help themselves before calling (Edwards & Stokoe, 2007), suggesting one only has the right to complain after taking reasonable precautions.

My own research shows two examples of the trickiness of complaining. The first involves visiting the local MP surgery (see Hofstetter, 2016; Hofstetter & Stokoe, 2015). Citizens can visit their local Member of Parliament to discuss a wide variety of concerns. It is typically assumed that people visit to argue about politics. However, there is a problem! Presenting concerns about national-level policy amounts to complaining directly to the person about whom you are complaining. This kind of complaint is very rare, and most conversation data shows that it is more common and more acceptable to complain about someone who is not present (see Heinemann, 2009). How do constituents get around this?

For starters, few present political concerns – only 12 per cent of the encounters in the dataset involved presenting a complaint that clearly involved national politics. Of the remaining conversations, we can look at the data to see how carefully the constituents managed the challenge of complaining directly to a person implicated in the complaint. The constituent (C2) in the extract below is speaking to the MP and the caseworker (CW) about difficulties in receiving a carer's allowance. She has had to return to work in addition to caring for an ill partner and arrives at a For starters, few present political concerns – only 12 per cent of the encounters in the dataset involved presenting a complaint that clearly involved national politics. Of the remaining conversations, we can look at the data to see how carefully the constituents managed the challenge of complaining directly to a person implicated in the complaint. The constituent (C2) in the extract below is speaking to the MP and the caseworker (CW) about difficulties in receiving a carer's allowance. She has had to return to work in addition to caring for an ill partner and arrives at a complaint about the recent change in pension age.

Extract 1: The big black hole

1  C2:      I've had to go back to working five days,

2  CW:      R:ight o[kay.      Yea:h.         Yeah.

3  C2:              [Ahheh heh ba(h)si(h)c'ly cause °I

4           haven't got money coming° [in   an:'    er:  ]

5  CW:                                [Yeah o'course yeah]

6  C2:      °°You know:,°°  '[mean I'm s]ixty,

7  CW:                       [<M  k a y ]

8                 (0.4)

9  C2:      Nearly, an' (.) obviously our pension's been

10          hmoved [hhnow hasn't it, (h)eh(h)eh

11 MP:             [Yeah:,  I know::,  I know:.

12 C2:      [hhhhhahhh     ]

13 MP:      [I know- Well::] >unfortunately< [as you kn]ow=

14 C2:                                       [You know,]

15 MP:      =everyone's- having ta- you know,

16 C2:      Mm,

17                (0.3)

18 MP:      We're trying to sort out er:, you know the big

19          black hole, an:d everybody's °°impacting on        

20          (         )°°

21 C2:      Yeah

22 MP:      °°(      ) Enough.°° [So,

23 C2:                           [That's right. Yeah.

24 MP:      °An:' you know and er but basically, (0.4)

25          Difficult- difficult choices.

26 C2:      Yea[h.

27 MP:         [An' pensions has been- has been one of

28          them,

29 C2:      Mm.

30 MP:      So, er:m,

31 C2:      But I say this is just

Having to go back to work full time is already a problem, because of needing to be at home caring for her partner. This issue might have been more manageable if they could have relied upon a pension, or a soon-to-arrive pension. However, the government recently moved the pension age forward for her age bracket, moving that possible relief out of sight. The MP was part of the government that enacted that policy. Note, though, that C2 does not state the MP's involvement in the policy. In fact, C2 omits any mention of the government at all. The pension has 'been hmoved' (lines 9–10) by an unstated force. By stating the issue in this way, C2 avoids implicating the MP directly in the complaint, and thus also avoids breaking the interactional norm. C2 also uses other conversational strategies that have been documented to mitigate the potential aggressiveness of this complaint, such as using laughter particles (lines 10, 12; see Potter & Hepburn, 2010).

The MP's response to the complaint demonstrates his interpretation of the complaint as political through references to the government sorting out the budget ('we're trying to sort out… the big black hole', lines 18–19). His response also helps reduce his agency in the decision, both by treating the decision as forced (by the black hole), and the omission of who exactly made the 'Difficult- difficult choices' (line 25). C2 accepts the response enough to move the conversation forward ('But I say…', line 31), and the topic is abandoned.

When looking at thousands of these turns in the constituency office setting, it becomes clear that speakers prioritise these interactional rules over any potential issue (Hofstetter & Stokoe, in press-a). Constituents bring very dire situations, including life-altering income problems such as the above, but they never complain about the MP's role in political changes that have brought about their grievance. Despite the high stakes in getting their complaint addressed, constituents still find ways to complain while following the interactional norms (Hofstetter & Stokoe, in press-b), rather than arguably more 'direct' practices.

However, there is one place that I am studying where complaints are regularly directed at the person who has committed the transgression: during board game play (Hofstetter & Robles, in press; Robles & Hofstetter, 2017). Examining such sequences can help provide a contrast to the rare occurrences elsewhere. In the following game, players try to take each other's pieces. To do so, they make formations – as if playing checkers, but with the requirement that a player makes a 'square' shape or some such before taking all the pieces surrounding it. As a result, if one player can successfully disrupt another player's shape before it is finished, then they get a big advantage. The next extract begins after Adam does just such a move to John, taking an important one of John's pieces.

Extract 2: What the game is all about

1  Joh:     ahhhhhhh °man:::,° (0.8) hHo:::#

2                 (1.0)

3  Joh:     You have no idea how much that impacted [me.°

4  Ad:                                              [t(h)ahh (0.4)

5           °I idon't actually,°

6                 (0.9)

7  Joh:     .hhhhh ahhh

8                 (1.6)

9  Joh:     °°I was gonna do° hso many cool things°

10 Ad:      I know- >well that's what I felt like the first two turns too,

11 Joh:     Ye(h)ah,

12                (0.9)

13 Joh:     .hh No# that's good. (0.4) That's good.

John gives an extended set of complaints, through pain-like moan sounds (lines 1, 7), to describing the extremity of the event (line 3), to the denial of his clever plans (line 9). Adam denies knowledge of the effect of his move (line 5), and also points to earlier complainable situations he had faced (line 10), both of which suggest that John's complaint, and/or its extremity, is unwarranted, and at least that Adam is not accountable to the degree John implies. Ultimately John backtracks, saying 'that's good' (line 13): Adam's move is acceptable, and in making the utterance, accepted.

Complaint sequences as above are unusual because John is expressing his complaint directly to Adam (note, though, that John still phrases the transgressive action as 'that', rather than 'you' in line 3, both of which were possible). The result is that both Adam and John end up negotiating how 'problematic' Adam's action was. It is common in these data for the complainer to back down after these discussions, and the game action is never undone. Even more remarkable is that when complaint sequences are entirely absent after an adverse game event (like taking someone's piece), long discussions (too long to reproduce here) often ensue that pursue information about how the adverse event affected the player! In other words, the complaint appears to be necessary – an indication that the game action has had the intended competitive effect. A lack of complaint suggests either a lack of successful effect, or a lack of genuine participation and investment in the game. So, despite the interactional challenges, some complaining may be central to competitive game play. The interaction (rather than the game actions) provides the feedback players need to successfully construct the competitive environment.

It may be tempting to dismiss the games as low-stakes, imaginary scenarios where interactional rules would not apply anyway, but this is not the case. Players follow other interactional rules very carefully, such as delicately navigating when to give advice to a less experienced player (or to avoid it and let them learn or act independently). The game refocuses attention on collaboratively completing the competition. To compete effectively involves showing effort, interest and engagement with the competition itself, and that requires demonstrating that an adverse event was indeed adverse. Complaints are an effective way to accomplish this display. Note that this also means complaints can be used in a subversive way, misdirecting players with an exaggerated complaint and exploiting the misdirection later.

In summary, despite popular suggestions to the contrary, we are very good at navigating the tricky territory that complaining brings. Complaints even have a central role in managing certain interactions, such as service encounters, but also, perhaps surprisingly, competitive play. The ingredients of complaining, as seen in conversational data, structure these interactions in specific ways. Complaining is so commonplace it can be easily overlooked, but it is an essential component to getting things done in these social settings. In looking at data such as the above, we can see what real complaint strategies achieve.

- Emily Hofstetter is at the Universities of  Loughborough and Linköping
[email protected]

Key sources

Edwards, D. (2005). Moaning, whinging and laughing: The subjective side of complaints. Discourse Studies, 7(1), 5–29.
Edwards, D. & Stokoe, E. (2007). Self-help in calls for help with problem neighbours. Research on Language & Social Interaction, 40(1), 9-32.
Heinemann, T. (2009). Participation and exclusion in third party complaints. Journal of Pragmatics, 41, 2435–2451.
Heritage, J. & Robinson, J. (2006). Accounting for the visit: Giving reasons for seeking medical care. In J. Heritage & D. Maynard (Eds.) Communication in medical care: Interactions between primary care physicians and patients (pp.48–85). Cambridge: Cambridge University Press.
Hofstetter, E. (2016). Citizens getting help: Interactions at the constituency office [Doctoral dissertation]. Loughborough University.
Potter, J. & Hepburn, A. (2010). Putting aspiration into words: 'Laugh particles', managing descriptive trouble, and modulating action. Journal of Pragmatics, 42(6), 1543–1555.
Hofstetter, E. & Robles, J. (in press). Manipulation in board game interaction: Being a sporting player. Symbolic Interaction.
Hofstetter, E. & Stokoe, E. (2015) Offers of assistance in politician-constituent interactions. Discourse Studies, 17(6), 724–751.
Hofstetter, E. & Stokoe, E. (in press-a). Making 'politics' relevant: How constituents and a member of parliament raise political topics at constituency surgeries. In G. Horan & M. Kranert (Eds.) Doing politics: Discursivity, performativity and mediation in political discourse (pp.127–150). Amsterdam: John Benjamins.
Hofstetter, E. & Stokoe, E. (in press-b). Getting service at the constituency office: Analyzing citizens' encounters with their Member of Parliament. Text & Talk.
Robles, J. & Hofstetter, E. (2017). Moral dimensions of ascribing intention in board game interaction. Paper presented at National Communication Association Convention, Dallas, Texas.
Stokoe, E. & Hepburn, A (2005). 'You can hear a lot through the walls': Noise formulations in neighbour complaints. Discourse & Society, 16(5),647–673.