The surprising world of synaesthesia

As a boy growing up in London, James Wannerton (below) would travel by the underground to school. He could taste his way along the route. 'Piccadilly Circus tasted of the peanuts and goo you get inside a Picnic bar. Bond Street tasted of a tangy aerosol spray. I liked Tottenham Court Road; it tasted of breakfast. The word "Tottenham" tasted of sausage, the "Court" tasted of egg and the "Road" tasted of toast,' he tells me.

James experiences a rare type of synaesthesia, a condition in which sensory input from one cognitive stream gives rise to sensory input from another unstimulated cognitive stream (Cytowic, 1989). The word comes from the Greek for 'joint sensation' – as opposed to the more familiar term anaesthesia, which is Greek for 'no sensation'. Synaesthesia is involuntary, the different associations and senses generally remain stable over time, and associations are often unique to the individual. It is thought to affect at least 4.4 per cent of the general population (Simner et al., 2006).  

There are over 60 known types of synaesthesia. James's lexical-gustatory form leads words to be experienced as strong tastes. One of the most common types is grapheme-colour synaesthesia, which is when letters and numbers are tinged with colour. Letters can represent different tones of colour, as can whole words. Chromaesthesia is another common form, and involves the association of sound with colour – musicians Pharrell Williams, Mary J. Blige and Lady Gaga all claim to have it. Authors and artists including Nabokov and Van Gogh are also said to have experienced forms of synaesthesia.

But it's not just famous musicians and artists who experience synaesthesia. One example is JB, a teenager from New York who has synaesthesia and eidetic memory. When JB was three years of age, he was able to recall the script of the movie Shrek word for word. JB's mother tells me she didn't realise he had synaesthesia until he reported seeing numbers in blue. Is JB's profound memory merely incidental, or is it potentially linked to his synaesthesia? And, if that is the case, could having decent memory be one of the advantages of the condition? Are there other benefits to having synaesthesia? Before answering these questions, we must offer some context into how this fascinating phenomenon was first recognised as a psychological condition.

A brief history

The first scientific reports on synaesthesia emerged in 1812 (Jewanski et al., 2009). German physician Georg Sachs described his own feelings of synaesthesia in a dissertation on his and his sister's albinism. Two pages of the thesis described some of his feelings of synaesthesia; he claimed to experience coloured words, sequences and music. Regrettably, Sachs did not attempt to explain why he was experiencing those feelings.

Although there were other instances of synaesthesia recorded later in the 19th century, academic interest waned with the rise of behaviourism in the 1930s. The theory of behaviourism postulates that all behaviour is explained by conditioning or experiences in one's immediate environment. Researchers began to focus more on external influences, as it was widely assumed that internal feelings and thoughts were not measurable.

Synaesthesia wasn't more widely recognised as a condition until the 1980s. American neurologist Richard E. Cytowic attended a dinner party where he saw someone cooking a chicken sauce. The chef tasted the sauce and said that it tasted 'wrong' and that it 'needed more points' on it. Cytowic questioned the chef and found out that the chef experienced shapes on his hand whenever he tasted food. Cytowic was intrigued by this, and started researching the phenomenon we now know as synaesthesia.

At about the same time in the UK, psychologist Professor Simon Baron-Cohen came across an interview of a painter called Elizabeth Pulford. Pulford (EP) said she experienced words and music in colour and asked whether anyone was interested in studying her. Baron-Cohen got in touch and went on to write several papers on EP (e.g. Baron-Cohen et al., 1987), identifying a synaesthesia that was both genuine and stable. In 1995 Baron-Cohen and his colleagues then proved that synaesthesia was a real neurological condition using fMRI scans on six synaesthetes and six non-synaesthetes (controls). The scans showed brain activity in the part of the brain associated with vision when sound occurred, even when the participants were blindfolded. This only occurred in the brains of the synaesthetes, and not the controls (Paulesu et al., 1995). These findings encouraged other members of the scientific community, particularly neuroscientists, to focus their research on synaesthesia.

Synaesthesia and touch

As fascinating a condition as it is, synaesthesia can be a hindrance to the individual. Carolyn Hart, who works as a massage therapist at Twitter's main headquarters in San Francisco, has a rare type known as mirror-touch synaesthesia. Someone with mirror-touch synaesthesia involuntarily feels the same sensation another person feels. Carolyn tells me about her experiences: 'My earliest memory of it was when I was about three years old. We had a dog that broke her leg in front of me. I remember that at the moment I saw that fracture I felt pain. It's been with me as long as I remember.'

Carolyn's pain is experienced instantaneously, before she has time to think. It doesn't matter how she feels about the person or animal in pain – she has to see the image or object before she feels a tactile sense. 'Sometimes, when I am engrossed in a movie or watching an athletic event, I will involuntarily move my body in ways I see the people on the screen moving. I don't watch a lot of action movies because they are too synaesthetically stimulating for me. Often I'll really tense my muscles because I feel like I'm running along with the actor who is running from the bad guy.'

Carolyn tells me she saw an article in Time magazine recently about the downing of the MH17 plane. 'There was an image of a person strapped to an airline seat in the middle of a wheat field where the body landed. The body was intact and there was no bleeding, cut, or rupture'. At first, Hart didn't react, as there was no visible bleeding, but when she saw that the man's leg was at an impossible angle, it triggered her synaesthesia. She experienced a shooting, electrical pain, from the back of her hips all the way down to the front of her legs and through the back of her arms.

Although Carolyn's synaesthesia can sometimes be a problem, it also can be of great help – especially with her job. 'When I touch people, I tend to feel it in my own body where I'm working on them. It's very pleasant; it's almost like I'm massaging myself. It's not quite as intense as my sense of pain,' she says. 'My fatigue while I'm working gets mitigated by the fact I feel really good physically when I'm massaging. There's a pleasurable component to my work that is beyond simple job satisfaction. I find my work interesting; I enjoy the personal interaction.'

Carolyn's mirror-touch synaesthesia allows her to easily palpate her patient's injuries and knots. She recently left a job she was at for eight years where she was the most requested massage therapist in the 20-year history of the business. Her synaesthesia has allowed her to perceive sense in ways many of her peers cannot.

It also helps her memorise her appointments with clients. 'In my head, I can pull up my entire calendar for months into the future. I don't need to write down my appointments. They are colour-coded in three-dimensional space,' she says. Although synaesthesia can make some mundane tasks difficult, what if we were to focus on the potential benefits and use them to enhance day-to-day life?

Synaesthesia and memory

In the same way as JB's, Carolyn's remarkable memory is unlikely to be incidental. One of the leading experts in synaesthesia and memory is Dr Nicolas Rothen, who is based at the University of Sussex (where he has worked with Professor Jamie Ward, who wrote a 'State of the art' piece on synaesthesia for this publication in 2003). Rothen has written papers on a wide range of different topics around synaesthesia, including how it is linked to artistry and to higher cognitive functions.

In his review paper, Rothen has had to study the parts of the brain associated with vision. The 'Two Streams' hypothesis proposes that the visual system is made up of the dorsal pathway and the ventral pathway. The dorsal pathway is involved in guidance, actions and where objects are in space while the ventral stream is associated with object recognition (Goodale & Milner, 1992).Nicolas discusses some of the trends he has found in his experiments. 'As a group, synaesthetes tend to perform better on memory tasks than non-synaesthetes. More so on tasks involving colour, abstract patterns or words, but not as much on things like spatial location,' he says. 'For example, in grapheme-colour synaesthesia, there is an increased sensitivity in the ventral-visual pathway.'

That leads to a performance advantage in certain types of memory tasks. 'That's the reason why synaesthetes often have an advantage in tasks involving high spatial frequency information, such as words, colour and abstract patterns but not for spatial information, location and sound – those are functions that are mainly located in the dorsal path.'

In music

Some of the memory benefits of synaesthesia can be transferred into creative disciplines. Daniel McBride is a second-year student at the Royal College of Music in London and experiences synaesthesia. After starting piano lessons aged 16, he learned very quickly, and within only seven months, he was performing professionally. 'I'd never write down any of the songs I'd play. I could memorise everything from the colours and patterns I'd see when playing them,' he says.

'Being at college introduced me to a lot of music, some of which used non-traditional triads. Sometimes I'd listen to music and be hypnotised by the colours I'd see,' he says. In college Daniel is often given music projects to work on – it's the style in which he plays them that is influenced by his synaesthesia. 'I clearly see a change in colour when I hear one note and hear it rise slowly in pitch. Because of that, I really like listening to Middle Eastern music. That stimulates me the most.'

Studies that have looked into synaesthesia and creativity have found promising results. For example, one study took a sample of 82 synaesthetes and got them to answer creativity questionnaires and say how much time they engaged in art. There was a significant tendency for synaesthetes to spend more time engaging in creative disciplines, relative to non-synaesthetes. Different degrees of creativity were also linked to the type of synaesthesia experienced (Ward et al. 2008). For example, people who had chromaesthesia were more likely to play musical instruments than other synaesthetes. These findings imply that if scientists are able to figure out a way to teach people chromaesthesia, it may enhance how well people learn to read and compose music.

Daniel has recently got into modern dance music. 'With classical music I experience many different colours, but as house music is more repetitive, I often see one solid colour appear throughout the tracks. There's one track I like at the moment by Koan Sound, which starts as an intense yellow and by the end of the song turns into a cascade of blue. The harp sound coming in at the end makes it a dark blue.'Part of my listening experience is seeing what colours the musical notes evoke. It's positive, as I get really into it and it's part of my performance,' he says. 'When I'm composing, in my head I can imagine what colours I would see if I was to play it. Composing is months of work condensed into four minutes of playing time. While I'm writing I imagine the colours I see – and when I finally hear it I am amazed.'

Teaching synaesthesia

It is clear that McBride's synaesthetic associations help him with his music, but where do his associations come from? A study by Witthoft and Winawer (2013) may help answer this question. They tested 11 colour-grapheme synaesthetes and found they had surprisingly similar colour-grapheme pairings (graphemes are the smallest unit of a written word that has meaning). But, with synaesthesia being thought to be predominantly a hereditary condition, why was this the case? The researchers pointed to a surprising answer:  the colours they saw matched those of a well-known set of Fisher-Price magnets, which 10 of the 11 participants recalled owning when they were younger. This suggested that environmental associations learnt in childhood had a strong effect on synaesthetic symptoms.

If synaesthesia is partly determined by your perceptual environment, could it be possible to teach certain aspects of the condition? In her lab at the University of East London, research fellow Dr Clare Jonas trains non-synaesthetes for a week to help them associate letters with colours. She does this by continuously exposing a person to a letter associated with a particular colour. After training them, she gives them word lists containing achromatic (colourless) words, words with colours congruent to the colours they learned, and words with colours incongruent with the colours they learned.

Jonas often finds that the non-synaesthetes start to behave in a way synaesthetes do – they are often more likely to remember congruent and achromatic words, but find it difficult to remember incongruent words, relative to non-synaesthetes who hadn't been trained. These outcomes suggest that certain aspects of synaesthesia could be taught. Nicolas Rothen's work also supports this theory – he reviewed all of the research in the area and found that some aspects of synaesthesia were transferrable to non-synaesthetes (Rothen & Meier, 2014). However, a lot of the effects depended on the intensity and duration of the training and the intrinsic motivation of the participant to take part in the experiment.

If aspects of synaesthesia can be taught, does this mean there is potential to augment memory in non-synaesthetes by teaching them synaesthesia? 'At the moment we know that synaethesia has benefits for memory in young adults, but we don't know whether it has a protective effect on memory as we get older,' Jonas says. 'Memory declining later in synaesthetes would be an obvious prediction. If that is the case, synaethesia training could be used to either protect or improve the memory of older people.'

Jonas's method of teaching synaesthesia isn't the only method that might work. Olympia Colizoli, an Assistant Professor in Brain and Cognition at the University of Amsterdam, adopts a more passive approach when training her non-synaesthetes. She has them read books with repeating differently coloured letters to prime them into future synaesthetic associations. In one of her experiments, she coloured the letters a, e, s and t in the book, but left the other letters in black. After training, she flashed letters of the alphabet quickly and asked participants to identify the letter's matching colour. In the test Cozoli purposely showed some of the coloured letters in a different colour to what they were in the book. She found that it took people who had been trained (or taught aspects of synaesthesia) longer to identify those colours than those who hadn't been taught – they experienced interference in reaction time known as the Stroop effect (Stroop, 1935). This would have occurred because of the involuntary associations non-synaesthetes formed after being trained. The results showed that this method of training is successful, at least in the short term (Colizoli et al., 2014).

Other, more recent research suggests that taught synaesthesia can have long-lasting effects. A paper published by Rothen and his colleagues found that non-synaesthetes were able to learn synaesthetic characteristics that remained present when they left the lab. Results such as these, along with other those of other training studies, imply that synaesthesia isn't just a hereditary condition, but is shaped by exposure to certain environmental factors during our early development. Other research supports the environmental theory too, finding that coloured synaesthetic associations can be blunted by negative moods (Kay et al., 2014).

Implications for treating disorders

Researcher and installation artist Barbara Ryan has gone a step further by formulating ideas to help synaesthetes with neurodegenerative disorders cope better. A grapheme-colour synaesthete herself, she tells me: 'I worked with a person who could no longer use a telephone – she had to wait for everyone to phone her. Although she was losing the ability to read language, replacing people's names with colours or characters helped her remember what names matched with what numbers.' Ryan believes that a lot can be done with synaesthesia and memory. 'I actually think that synaesthetic techniques could be used to help with disorders like dyslexia. I think synaesthetic techniques can be used as a learning aid in some cases. A friend of mine is dyslexic and has trouble with certain letters, so I asked him what letters he had problems with. I then asked him what kind of associations he had with those letters. Initially, he didn't think he had any, but I told him to take his time and think about it. I then formatted the text so that the letters he had trouble with were now in colour, and straight away he could read it in a way he couldn't before. They became recognisable. Recognition is a lot quicker with synaesthesia, as you have more than one mode of stimulation coming in.'

Synaesthesia doesn't just have potential benefits for memory and creativity – some believe that synaesthetic interventions can help treat mental health disorders. Ian Jordan, from Ayr, Scotland, describes himself as 'an optician with a difference'. He employs synaesthetic methods to try to reduce the effects of different multisensory disorders, such as autism, dyslexia and ADHD. 'We use and monitor synaesthesia in a lot of interventions,' he says. 'Sensory processing disorders are often synonymous with synaesthesia. We think that tinnitus in some ways is a synaesthetic condition. If you change the visual input significantly, you can tune the sound out in the hearing for around 60 per cent of people. It's a synaesthetic effect that probably hasn't been recognised as one.'

'Professionals need to be more aware of synaesthesia,' Ian tells me. 'They need to be trained to understand and work with it. Many opticians haven't even heard of it. We need to have opticians, occupational psychologists and therapists working together as a minimum.'

Nicolas Rothen gives his predictions for the future. 'In the last decade, people were concerned about showing that synaesthesia was a real phenomenon, but now, people are looking into what the effects of synaesthesia are on higher cognitive functions. What are the advantages and disadvantages of synaesthesia? Is it linked to conditions like schizophrenia or autism?' Clearly, we still have a lot to learn.

Jack Dutton is a freelance writer
[email protected]

References 

Baron-Cohen, S., Wyke, M. & Binnie, C. (1987). Hearing words and seeing colours: An experimental investigation of a case of synaesthesia. Perception, 16, 761–767.
Colizoli, O., Murre, J.M.J. & Rouw, R. (2014). Training synesthetic letter-color associations by reading in color. Journal of Visualized Experiments, 84, e50893. doi:10.3791/50893
Cytowic, R.E. (1989). Synaesthesia: A union of the senses. Berlin: Springer.
Cytowic R.E., Eagleman D.M. & Nabokov D. (2009). Wednesday is indigo blue: Discovering the brain of synesthesia. Cambridge, MA: MIT Press.
Goodale, M.A. & Milner, A.D. (1992). Separate visual pathways for perception and action. Trends in Neuroscience, 15, 20–25.
Jewanski, J., Day, S.A. & Ward, J. (2009). A colorful albino: The first documented case of synaesthesia, by Georg Tobias Ludwig Sachs in 1812'. Journal of the History of the Neurosciences,18(3), 293–303
Kay, C.L., Carmichael, D.A., Ruffell, H.E. & Simner, J. (2014). Colour fluctuations in grapheme-colour synaesthesia: The effect of clinical and non-clinical mood changes. British Journal of Psychology [Advance online publication]. doi:10.1111/bjop.12102
Paulesu, E., Harrison, J.E., Baron-Cohen, S. et al. (1995). The physiology of coloured hearing: A PET activation study of colour-word synaesthesia. Brain, 118, 661–676,
Rothen, N. & Meier, B. (2014). Acquiring synaesthesia: Insights from training studies. Frontiers in Human Neuroscience, 8, 109.
Simner, J., Mulvenna, C., Sagiv, N. et al. (2006). Synaesthesia: The prevalence of atypical cross-modal experiences. Perception, 35(8), 1024–1033.
Stroop, J.R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, 643–662.
Ward, J. (2013). State of the art: Synaesthesia. The Psychologist, 16(4), 196–199.
Ward, J., Thompson-Lake, D., Ely, R. & Kaminski, F. (2008). Synaesthesia, creativity and art: What is the link? British Journal of Psychology, 99, 127–141.
Witthoft N. & Winawer J. (2013). Learning, memory, and synesthesia.