The knowledge illusion

Steven Sloman and Philip Fernbach's new book The Knowledge Illusion: Why We Never Think Alone is published on 6 April by Macmillan. Here, we run two exclusive extracts. 

The benefits and perils of the hive mind

In the preface to Saint Joan, his play about Joan of Arc, the teenager whose visions of saints and archangels stirred soldiers into battle early in the fifteenth century, George Bernard Shaw makes a surprisingly compelling argument that following Joan of Arc's mystical visions was at least as rational as following a modern-day general into today's battlefield full of highly technological and incomprehensible weapons of war. His argument is that the warrior of the twentieth century was driven as much by faith as the warrior of the fifteenth century:

In the Middle Ages people believed that the earth was flat, for which they had at least the evidence of their senses: we believe it to be round, not because as many as one per cent of us could give the physical reasons for so quaint a belief, but because modern science has convinced us that nothing that is obvious is true, and that everything that is magical, improbable, extraordinary, gigantic, microscopic, heartless, or outrageous is scientific.

Hyperbole, for sure, but it is remarkable how much we depend on what we're told to get by in the modern world. So little of what happens to us is understood through direct sensory experience. From the alarm that wakes us up, to the toilet that we wander to, to the smartphone that we turn on (before or after our visit to the bathroom), to the coffee machine that welcomes us into the kitchen, to the tap that we use to fill the coffee machine, nothing is completely within our conceptual grasp. But we use these tools; we even rely on them, because they work (except when they don't and our life goes a little out of balance). We can thank the experts who created them, for we are dependent on their know-how. We have faith in the masters of modern technology after years of successfully using their devices. But when those devices fail, when the cable service goes out or the drain emits brown sludge, we're rudely reminded of just how little we know about the conveniences of modern life.

The knowledge illusion occurs because we live in a community of knowledge and we fail to distinguish the knowledge that is in our heads from the knowledge outside of it. We think the knowledge we have about how things work sits inside our skulls when in fact we're drawing a lot of it from the environment and from other people. This is as much a feature of cognition as it is a bug. The world and our community house most of our knowledge base. A lot of human understanding consists simply of awareness that the knowledge is out there. Sophisticated understanding usually consists of knowing where to find it. Only the truly erudite actually have the knowledge available in their own memories.

The knowledge illusion is the flip side of what economists call the curse of knowledge. When we know about something, we find it hard to imagine that someone else doesn't know it. If we tap out a tune, we're sometimes shocked that others don't recognize it. It seems so obvious; after all, we can hear it in our heads. If we know the answer to a general knowledge question (who starred in TheSound of Music?), we have a tendency to expect others to know the answer too. The curse of knowledge sometimes comes in the form of a hindsight bias. If our team just won a big game or our candidate just won an election, then we feel like we knew it all along and others should have expected that outcome too. The curse of knowledge is that we tend to think what is in our heads is in the heads of others. In the knowledge illusion, we tend to think what is in others' heads is in our heads. In both cases, we fail to discern who knows what.

Because we live inside a hive mind, relying heavily on others and the environment to store our knowledge, most of what is in our heads is quite superficial. We can get away with that superficiality most of the time because other people don't expect us to know more; after all, their knowledge is superficial too. We get by because a division of cognitive labor exists that divides responsibility for different aspects of knowledge across a community.

The division of cognitive labor is fundamental to the way cognition evolved and the way it works today. The ability to share knowledge across a community is what has allowed us go to the moon, to build cars and freeways, to make milk shakes and movies, to veg out in front of the TV, to do everything that we can do by virtue of living in society. The division of cognitive labor makes the difference between the comfort and safety of living in society and of being alone in the wild.

But there are also downsides when we rely on others to hold knowledge for us. Most of you reading this book are probably familiar with Alice (of Wonderland fame), but few people today actually read the Lewis Carroll novels that introduced her to the world. Many know Alice indirectly, through movies, cartoons, and TV shows, not through the unique and mind-bending experience of reading Carroll's marvelous books. If we don't know calculus, we can't understand the beauty of imagining time disappearing by letting it shrink into a moment and how that relates to the tangent of a curve. We can't see what Newton saw that made him so important that the authorities buried him in Westminster Abbey. That's one cost of living in a community of knowledge: We miss out on those things that we know only through the knowledge and experience of others.

There are also more dangerous consequences. Because we confuse the knowledge in our heads with the knowledge we have access to, we are largely unaware of how little we understand. We live with the belief that we understand more than we do. As we will explore in the rest of the book, many of society's most pressing problems stem from this illusion.

Real super intelligence

Because computers don't have the ability to share intentionality and we see no evidence that they are on the road to getting it, we're not too worried about an evil superintelligence that will pursue its own goals at the expense of humankind. Indeed, we don't see any kind of superintelligence on the horizon. Machines without the basic human ability to share attention and goals will never be able to read our minds and outsmart us because they won't even be able to under- stand us.

But there is a sense in which technology is enabling superintelligence. The web has provided smart new tools for us to use, like GPS devices and operating systems that talk to us. But one of the most useful forms of web applications turns people themselves into tools. Crowdsourcing applications have created broader and more dynamic communities of knowledge than ever before by aggregating the knowledge and skills of large numbers of people. Crowd-sourcing is the critical provider of information to sites and apps that integrate knowledge from different experiences, locations, and knowledge bases. Yelp and Amazon crowdsource reviews for services and products. Waze crowdsources maps of traffic conditions from the input of individual drivers on the road. Then there are sites like Reddit that allow users to ask questions and encourage anyone to provide an answer.

When done right, crowdsourcing is the best way available to take advantage of expertise in the community. It gets as many people as possible involved in pursuing a goal. This means experts are more likely to offer their input. In the end, that's the best way to achieve an objective: to make use of expertise. Yelp integrates knowledge from restaurant goers who purport to know how good a restaurant is, and Reddit tries to identify who has the most expertise to answer someone's question. So crowdsourcing works best when those with the most expertise have sufficient incentive to participate in the community.

Crowdsourcing creates intelligent machines, but not through AI wizardry. Their intelligence doesn't come from a deep understanding of the best way to reason or through immense computing power. Their intelligence derives from making use of the community. Waze guides you through traffic by integrating the reports of thousands of individuals who know a lot about the traffic conditions where they happen to be located. The advance here is not in intelligence in the conventional sense. It is in the power of connecting people.

One of the big problems facing entrepreneurs using crowdsourcing is how to incentivize experts to contribute. Money is only one incentive. Experts also thrive on being right, often more than on financial incentives. Witness the explosive growth of Wikipedia. It doesn't pay contributors a cent. Nor did the Oxford English Dictionarywhen it first started gathering quotations to identify the meanings of words in 1857. The OED appealed to volunteers to read through countless English texts, and it is still doing so. Most experts love the opportunity to demonstrate their expertise, especially when their contribution is acknowledged. Contributing to the community of knowledge is in our collaborative nature.

Each of us has our little window on the world, a little bit of knowledge that we have access to. Crowdsourcing is a way of looking through tens, hundreds, and sometimes thousands of windows simultaneously. But crowdsourcing works only when it provides access to expertise. Without expertise, it can be useless and even detrimental. Pallokerho-35 (PK-35) is a Finnish soccer club. A few years ago, the team invited fans to participate in decisions regarding recruiting, training, and even game tactics. They voted using cell phones. The outcome was disastrous. The team did poorly, the coach was fired, and the experiment came to an abrupt end. For a crowdsourcing scheme to work, it's not enough just to have a big community; the community needs to have the necessary expertise.

Sometimes expertise is only apparent. Average user ratings on Amazon.com are not all they're cracked up to be. Their ratings do not correlate very well with the evaluations of true experts and are too high for beloved brands and high-priced items. Many consumers just do not have the expertise to accurately assess the quality of technical products like digital cameras and kitchen appliances.

But crowdsourcing can be effective. This was first revealed by Francis Galton in 1907 in a paper entitled Vox Populi (The Wisdom of Crowds). He reports on a contest held at a farmers' fair in Plymouth, England, to guess the weight of a fat ox. The contest was open to everyone willing to pay a small fee with the hope of making a guess close enough to the ox's true weight to win a prize. Guesses came from experts like butchers and farmers as well as the general public. As Galton says, "The average competitor was probably as well fitted for making a just estimate of the dressed weight of the ox, as an average voter is of judging the merits of most political issues on which he votes, and the variety among the voters to judge justly was probably much the same in either case." Galton managed to obtain the tickets with the estimates and counted 787 legible ones. He found that the average guess was within 1 percent of the ox's true weight of 1,198 pounds. When it comes to judging weight, the crowd shows some wisdom. In Galton's understated prose, "This result is, I think, more creditable to the trustworthiness of a democratic judgment than might have been expected." We may individually suffer from the knowledge illusion, but the expertise that's present in a crowd can overcome individual biases.

Crowdsourcing has been much ballyhooed in the business world. It is typically invoked to explain the success of sites like Wikipedia. A group of economists has sung the praises of a type of crowdsourcing called a prediction market. In a prediction market, people make bets about what will happen in the future. The amount that the crowd is willing to bet on a particular outcome is used to estimate the probability of that outcome. People are motivated to bet because the best predictor wins a prize like money or prestige. Experts are especially motivated because they are more likely to know what will happen than novices, and so experts tend to have an outsize influence on the market. Many government agencies and private companies have used prediction markets to make guesses about what will happen in domestic elections, international affairs, and business environments. They often enjoy more success than traditional forecasting methods.

Crowdsourcing is actually a primitive method of employing the community of knowledge compared to what's on the horizon. Web developers are just beginning to develop applications that allow communities to form dynamically to solve specific problems. The idea of these applications is to make collaboration simple by allowing teams of experts from around the world to come together temporarily to work on specific projects. A number of problems must be solved before these applications go mainstream: Experts must be induced to participate; methods must be designed to choose the right set of experts for any particular problem; cognitive labor needs to be divided up effectively; and methods need to be developed to fairly distribute both the risk and the rewards associated with each project. The success of these applications for collaboration depends on how well these problems are solved.

Platforms to support this kind of decentralized collaborative activity are just coming into being with futuristic names like Ethereum, Sensorica, and Colony. Ethereum is inspired by the success of Bitcoin, an Internet currency that is decentralized, not administered by any single entity. The information about who owns how much Bitcoin is stored in a public ledger of transactions called a block chain. A block chain is a sophisticated technology for maintaining a record of all transactions that is updated and stored across the network of Bitcoin users. Distributing the ledger of transactions across the network is a good way to prevent mistakes and cheating. Ethereum uses a block chain method to allow collaboration via decentralized agreement of everyone involved in a project. The assumption is that information is safer if it is distributed across the community, preventing any one individual from having too much access. Access to information means control, and the goal of these collaboration platforms is to foster equity: to allow everyone to contribute according to their ability and to reap rewards according to their merit. The goal is to allow communities to divide up cognitive labor for any sort of project in a safe and secure way. Once one of these platforms becomes popular, it will create an entirely new way of doing business. If it becomes the norm to do business through teams that are constantly changing alliances of experts, we will have to redefine the notion of a company. It could lead to an entirely new kind of economy.

- Buy the book, and find out more about Steven Sloman and Philip Fernbach.

Our thanks to Macmillan for permissions.