The glue that binds memory

The classic example of verbal short-term memory is keeping a telephone number in mind as we dial it, but this cognitive ability plays a critical role in many other everyday activities including mental arithmetic, verbal reasoning, learning new vocabulary and overcoming difficulties in language comprehension. A major research effort in cognitive psychology seeks to understand this ability that we all have, to retain verbatim verbal information for brief periods.

Much of the research on this important cognitive function has focused on the performance of normal participants but investigations of brain-damaged participants can also produce startling new insights. This article will describe some recent studies of people with semantic dementia that have contributed to our understanding of the mechanisms underpinning normal verbal short-term memory.

Verbal short-term memory and language processing

One of the big debates about verbal short-term memory concerns the relationship between this ability and our permanent store of linguistic knowledge. According to the highly influential working memory model, a dedicated short-term memory store called the 'phonological loop' underpins verbal short-term memory (Baddeley, 1986). This loop was proposed to have a time-based capacity of a few seconds and to be separate from long-term language representations. A major problem with this model is that it does not predict that the capacity of verbal short-term memory will change according to the type of material that a person is asked to retain. Verbal short-term memory is usually measured by asking people to listen to a string of words and then repeat them in the correct order: this task is called 'immediate serial recall'. People are able to repeat about 12 words when they form a meaningful sentence (try, for example, The mouse made a nest lined with dry leaves under our floor), but only five or six words chosen at random (e.g. axe, night, table, yellow, road, jelly). Recall is even poorer for lists of made-up 'nonwords' such as fol, baig, coit, raysh (Hulme et al., 1991). These differences demonstrate that multiple levels of linguistic knowledge play a role in verbal short-term memory.

Although it is now widely accepted that long-term linguistic knowledge contributes to verbal short-term memory, there is considerable disagreement about (1) how this comes about, (2) whether linguistic knowledge is involved from the outset of immediate serial recall or at a late stage, and (3) the differing roles played by knowledge of the sound sequences (phonology) and meanings (semantics) of words. According to one view, the contents of the phonological loop are compared with long-term representations of the sounds of familiar words, allowing incomplete/inaccurate memory traces to be restored during recall. For example, a list like democracy, crocodile, microwave may become dem_cracy, cro_odile, microw_ after some forgetting. Missing segments might then be replaced by looking up similar sounding words in long-term memory (Baddeley et al., 1998; Hulme et al., 1991; Schweickert, 1993). Although this process is based on long-term phonological, as opposed to semantic, knowledge, information about word meaning might contribute to the selection of words for reconstruction (Poirier & Saint-Aubin, 1995).

In an alternative conception, verbal short-term memory reflects temporary activation of the same representations that underpin the production and comprehension of language (Martin & Saffran, 1997; Patterson et al., 1994). By this view, there is very little separation between long-term and short-term memory; therefore linguistic knowledge is intimately involved in all stages of immediate serial recall. As many models of language processing suggest that word meaning and phonology are highly interactive, this view predicts that knowledge of word meaning will boost phonological processing. In tasks like immediate serial recall, this will help to prevent decay of the short-term memory trace.

One way of exploring the relationship between knowledge of word meaning and verbal short-term memory is to examine people who have impoverished comprehension of words despite good phonology. How does this loss of knowledge affect the integrity of items in the short-term memory trace?  

Lessons from semantic dementia

Semantic dementia is a relatively rare neurodegenerative condition in which brain cells die in the anterior temporal lobes bilaterally – that is, the cortical tissue underneath the temples on both sides of the brain (Hodges et al., 1992; Snowden et al., 1989). Brain scans show that this part of the brain becomes severely atrophied. People with semantic dementia have a specific and progressive loss of knowledge about things, including words, pictures, objects, environmental sounds and faces (Bozeat et al., 2000; Rogers et al., 2004). This impairment of semantic memory is very debilitating in everyday life because people struggle to understand what is said to them, to remember the names of objects and to recognise the significance of items and people they encounter.

Despite these severe problems, however, people with semantic dementia are not globally demented and retain considerable insight into their condition. They are well oriented in time and space, so they do not believe themselves to be in a previous era or wander off and get lost. They have good episodic memory for recent events, unlike people with Alzheimer's disease, and can talk knowledgeably about what they were doing last weekend or recent sporting events, at least within the confines of their impoverished vocabulary. They have preserved non-verbal reasoning. In addition, many aspects of their language remain intact: their speech is fluent and they produce well-formed words and sentences, free from grammatical or phonological errors (e.g. cad for cat). People with semantic dementia might therefore be characterised as being able to speak but having little to say.

This striking dissociation – highly impoverished semantic memory but intact phonology and syntax – allows us to study the contribution of word meaning to verbal short-term memory. Crucially for research in this area, people with semantic dementia can show poor knowledge of particular concepts whilst retaining a better understanding of other items. Individuals show highly consistent performance when semantic tests are repeated and pass or fail the same items when knowledge is probed using different types of tests. By comparing the results of several different semantic tests, it is possible to select items that a particular semantically impaired participant understands well and poorly. For example, someone might be asked to name pictures and produce verbal definitions for the same items: words that can still be named and defined correctly would be classified as 'known', whereas 'semantically degraded' items would be neither named nor defined. As it is not necessary to have a complete understanding of an item in order to pass a particular semantic test, the meaning of known words may still be somewhat impoverished; similarly, some people may retain a vague understanding of degraded items. Nevertheless, these two sets of items should receive differing levels of semantic support in verbal short-term memory tasks. The two-word sets in these experiments must also be matched for variables that are known to affect immediate serial recall, such as word frequency and length.

Studies using this methodology have demonstrated that people with semantic dementia have poorer immediate serial recall for words that they no longer fully understand compared with better known words (Jefferies et al., 2005; Jefferies, Patterson et al., 2004; Knott et al., 1997; Patterson et al., 1994). This finding is consistent with research showing that normal participants have better recall of more meaningful items, like sentences and real words vs. nonwords. In addition, individuals with semantic dementia show a revealing pattern of errors for degraded words – the segments belonging to one word frequently migrate and recombine with the segments of other list items, producing spoonerisms. For example, the word list seat, ice, knock might be recalled as eat, nice, sock. Here the phonological elements of the items have been retained but produced in the wrong combinations. Healthy people rarely make these sorts of errors in word recall; instead they typically recall whole items out of sequence (e.g. seat, ice, knock – seat, knock, ice). People with semantic dementia show a more normal pattern of errors when recalling words that are still relatively well understood.

These distinctive phoneme migration errors provide insight into the mechanisms underlying verbal short-term memory. They suggest that word meaning might play an important role in keeping the elements of familiar words together in the right configuration. This is sometimes called the 'semantic glue hypothesis' (Patterson et al., 1994). According to this notion, strong associative links are formed between phonology and semantics because activation in these two systems co-occurs during language processing. This allows input from the semantic system to provide a boost to patterns of phonological activation that correspond with a particular meaning.

Numbers and words

It seems that the semantic glue hypothesis may be able to explain one puzzling fact about semantic dementia: the observation that digit strings, such as 2, 7, 3, 4, 1, 9 are repeated normally, whereas word recall is impaired (Jefferies, Patterson et al., 2004). There are, of course, lots of differences between the numbers 1–9 and non-number words which might affect verbal short-term memory: numbers are very frequent in the language, related in meaning and drawn from a highly restricted set of potential items, making it necessary to repeat them again and again in immediate serial recall experiments.

However, controlling these characteristics does not eliminate the effect: even when multi-digit number words such as thirteen and seventy are matched for frequency, imageability and word length to a restricted set of words drawn from a very tight semantic category such as face parts, people with semantic dementia still show a considerable recall advantage for number words (Jefferies, Patterson et al., 2004). We directly compared the comprehension of these number and non-number words using tasks such as picture naming and word-picture matching (depicting numbers as dot clusters) and found that impaired participants' superior immediate serial recall of number words went hand-in-hand with better understanding of number words. This might reflect the fact that the inferior parietal cortex represents the magnitude meaning of numbers – i.e. their position on the mental number line. These representations are unlikely to be affected by the temporal lobe atrophy in semantic dementia, and so number words remain more meaningful than other words to people with this condition (Butterworth et al., 2001; Cappelletti et al., 2001).

Associations and dissociations

So far, so good for the semantic glue hypothesis! However, researchers who remain sceptical of this view have pointed out that some people with semantic dementia have failed to show the predicted recall difference between known and semantically degraded words (see McCarthy & Warrington, 2001). Does
the existence of these people imply that semantic memory does not make a critical contribution to verbal short-term memory after all? This might be the conclusion according to standard neuropsychological practice, in which dissociations between tasks are taken as evidence for separable brain mechanisms. By this view, the observed link between poor comprehension and phoneme migration errors does not constitute good evidence that word meaning helps to bind the phonemes of items together in immediate serial recall, because spurious associations occur for abilities that are underpinned by neighbouring brain regions. Individual cases who show an impairment of semantic memory without effects on verbal short-term memory, on the other hand, are thought to demonstrate that these abilities can be isolated.

Around half of those with semantic dementia examined previously on immediate serial recall failed to show differences between known and degraded words (although all of them made frequent phoneme migration errors). We assessed the recall of known/degraded words using a variety of different methods in order to explore the possibility that methodological factors can explain much of this inconsistency in results (Jefferies, Jones et al., 2004). One factor – set size – stood out as being critical in determining the degree of the recall advantage for known words. Set size refers to the number of items used to construct lists for immediatE serial recall. When set size is small, items are frequently repeated, giving participants the opportunity to become familiar with their phonological forms and potentially reducing the impact of word meaning on phonological stability. Not only did this variable affect the size of the known/ degraded difference in our experiments but, in addition, there was a highly significant correlation of .73 between set size and study outcome for previous research.

Ruling out alternative explanations

In order to strengthen the case for semantic glue, it is necessary to rule out alternative views – such as the notion that phoneme migration errors result from separable impairments to phonological processing or lexical representations of word forms (Knott et al., 1997; McCarthy & Warrington, 2001). Support from word meaning might be especially critical for those with additional phonological problems who struggle to process/maintain the sounds of items. By this view, it is conceivable that a healthy phonological system is sufficient for immediate serial recall and that semantic memory does not play an essential role in normal recall.

To examine this possibility, we tested six people with semantic dementia, ranging in severity from mild to severe, on a variety of phonological processing measures – for example, rhyme judgement and phonological segmentation (take the first sound away from teach ' each). Known/degraded recall differences were observed for every participant, regardless of whether subtle phonological processing deficits were also detected. The more impaired cases did have some difficulty with nonword repetition/recall but this did not appear to be indicative of additional phonological impairment as nonwords derived from known words were repeated more accurately than items that were phonologically similar to semantically degraded words. For example, if a particular individual understood the word strawberry but not kangaroo, the nonword gawberry would be recalled better than pangeroo. This finding suggests that the semantic contribution to verbal short-term memory is pervasive: we repeat nonwords partly through a process of analogy with real words.

This study also obtained evidence to suggest that the known/degraded recall difference reflects a central semantic impairment and not damage to separable lexical representations of familiar words. Participants attempted to reproduce line drawings of the known and degraded items used in the immediate serial recall task after a short delay. Their copies of the semantically degraded items were poorer, preserving significantly fewer of the visual features. Given this item consistency for word and picture recall, it is difficult to argue that the known/degraded differences in immediate serial recall reflected the contribution of word-specific lexical knowledge.

Phoneme migrations: A normal phenomenon

The conclusion that semantic information helps to bind the phonological segments of words together in verbal short-term memory sits comfortably with the view that this capacity emerges from interactions between semantic and phonological representations that underlie language processing more generally. If so, semantic binding should be evident in the recall of normal participants.

As noted above, healthy subjects rarely make phoneme migration errors when recalling lists of words. They do, however, recall the phonemes of nonwords in the wrong combinations; for example, the items gir, vang, kus might be recalled as vir, kang, gus (Treiman & Danis, 1988). This could reflect the lack of semantic glue for meaningless items. An alternative possibility, however, is that both word and nonword phonemes migrate as the short-term memory trace decays: lexical knowledge may then allow the correct phonological sequence to be reconstructed for words but not nonwords.

To address this issue, we used a novel technique to induce phoneme migration errors for real words in healthy participants (Jefferies et al., 2006). People were asked to remember mixed lists composed of both words and nonwords in a random order (e.g. cowt, home, face, bal, wine). The positions of the words/nonwords kept changing, making the structure of the lists unpredictable. In addition, the lists contained different numbers of words/nonwords. We found that the phonemes of words were more likely to migrate when there were more nonwords in the lists. The free-floating nonword phonemes may have acted as 'free radicals' that damaged word recall because of their tendency to recombine with elements of other list items. Moreover, the nonword phonemes were more likely to be recalled in the correct order when they were presented in lists containing lots of words, perhaps because the opportunity for migrations was reduced when the other list items were more tightly bound together. Word variables such as frequency in the language (dog vs. rat) and the ease of forming a mental image (rat vs. zone) also affected the rate of phoneme migration errors in these mixed lists. These findings indicate that (1) long-term linguistic knowledge affects phoneme binding in normal verbal short-term memory; (2) these effects occur even when participants have no prior knowledge about which items are words to facilitate reconstruction; and (3) the extent of binding for individual items affects the stability of the entire list – therefore, the contribution of linguistic knowledge cannot simply come about through the reconstruction of specific stimuli.

Timing matters

Competing theories make different predictions about when linguistic knowledge should influence verbal short-term memory. If this ability reflects ongoing activation within the language system, prior knowledge will be intrinsic to its function and effects will be seen from encoding onwards. Alternatively, if linguistic knowledge is used to reconstruct incomplete/inaccurate phonological information at a late stage in immediate serial recall, memory for words and nonwords should be equivalent in recognition tasks which bypass this process (Gathercole et al., 2001).

Indeed, linguistic knowledge is reported to have little or no effect on 'matching span', in which people listen to two lists of items in quick succession and decide if they are the same or different. The standard version of this task requires people to detect changes in the order of items (Gathercole et al., 2001). However, we found much greater effects of lexical/semantic knowledge when participants were asked to spot changes in phoneme order (Jefferies et al., in press); for example are these lists the same or different? – bag, rock, sun, hall and bag, sock, run, hall. In addition, people found it much harder to detect word phoneme movements when words were mixed up with nonwords. This suggests that lexical/semantic knowledge contributes to phonological binding prior to recall.

Conclusions

Convergent evidence from people with semantic dementia and normal participants supports the view that phonological binding in verbal short-term memory is aided by long-term knowledge of both word meaning and the sounds of familiar words. This 'semantic glue' seems to be of critical importance even in normal recall. This conclusion has far-reaching implications: it helps us predict the problems that language-impaired stroke or dementia patients are likely to encounter, as well as providing valuable insights into normal cognitive function.

- Dr Elizabeth Jefferies is in the School of Psychological Sciences, University of Manchester. E-mail: [email protected].

Weblinks

Fun short-term memory games.

Spoonerisms.

References 

Baddeley, A.D. (1986). Working memory. Oxford: Oxford University Press.
Baddeley, A.D., Gathercole, S.E. & Papagno, C. (1998). The phonological loop as a language learning device. Psychological Review, 105, 158–173.
Bozeat, S., Lambon, Ralph, M.A., Patterson, K., Garrard, P. & Hodges, J.R. (2000). Non-verbal semantic impairment in semantic dementia. Neuropsychologia, 38, 1207–1215.
Butterworth, B., Cappelletti, M. & Kopelman, M. (2001). Category specificity in reading and writing. Nature Neuroscience, 4, 784–786.
Cappelletti, M., Butterworth, B. & Kopelman, M. (2001). Spared numerical abilities in a case of semantic dementia. Neuropsychologia, 39, 1224–1239.
Gathercole, S.E., Pickering, S.J., Hall, M. & Peaker, S.M. (2001). Dissociable lexical and phonological influences on serial recognition and serial recall. Quarterly Journal of Experimental Psychology, 54A, 1–30.
Hodges, J.R., Patterson, K., Oxbury, S. & Funnell, E. (1992). Semantic dementia: Progressive fluent aphasia with temporal lobe atrophy. Brain, 115, 1783–1806.
Hulme, C., Maughan, S., Brown, G.D.A. (1991). Memory for familiar and unfamiliar words. Journal of Memory and Language, 30, 685–701.
Jefferies, E., Frankish, C. & Lambon Ralph, M.A. (2006). Lexical and semantic binding in verbal short-term memory. Journal of Memory and Language, 54, 81–98.
Jefferies, E., Frankish, C. & Lambon Ralph, M.A. (in press). Lexical and semantic influences on item and order memory in immediate serial recognition. Quarterly Journal of Experimental Psychology.
Jefferies, E., Jones, R., Bateman, D. & Lambon Ralph, M.A. (2004). When does word meaning affect immediate serial recall in semantic dementia? Cognitive, Affective and Behavioral Neuroscience, 4, 20–42.
Jefferies, E., Jones, R.W., Bateman, D. & Lambon Ralph, M.A. (2005). A semantic contribution to nonword recall? Cognitive Neuropsychology, 22, 183–212.
Jefferies, E., Patterson, K., Jones, R.W., Bateman, D. & Lambon Ralph, M.A. (2004). A category-specific advantage for numbers in verbal short-term memory. Neuropsychologia, 42, 639–660.
Knott, R., Patterson, K. & Hodges, J.R. (1997). Lexical and semantic binding effects in short-term memory. Cognitive Neuropsychology, 14, 1165–1216.
Martin, N. & Saffran, E.M. (1997). Language and auditory-verbal short-term memory impairments. Cognitive Neuropsychology, 14, 641–682.
McCarthy, R.A. & Warrington, E.K. (2001). Repeating without semantics: Surface dysphasia? Neurocase, 7, 77–87.
Patterson, K., Graham, N. & Hodges, J.R. (1994). The impact of semantic memory loss on phonological representations. Journal of Cognitive Neuroscience, 6, 57–69.
Poirier, M. & Saint-Aubin, J. (1995). Memory for related and unrelated words: Further evidence on the influence of semantic factors in immediate serial recall. Quarterly Journal of Experimental Psychology, 48A, 384–404.
Rogers, T.T., Lambon Ralph, M.A., Hodges, J.R. & Patterson, K. (2004). Natural selection: The impact of semantic impairment on lexical and object decision. Cognitive Neuropsychology, 21, 331–352.
Schweickert, R. (1993). A multinomial processing tree model for degradation and redintegration in immediate recall. Memory and Cognition, 21, 168–175.
Snowden, J.S., Goulding, P.J. & Neary, D. (1989). Semantic dementia: A form of circumscribed cerebral atrophy. Behavioural Neurology, 2, 167–182.
Treiman, R. & Danis, C. (1988). Short-term memory errors for spoken syllables are affected by the linguistic structure of the syllables. Journal of Experimental Psychology: Learning, Memory and Cognition, 14, 145–152.