Language Simulates Perception

One of the pleasures of this blog has been/is the occasional encounter with an important book. The top 4 were probably:

• Bastard Tongues by Derek Bickerton,
• Empires of the Word by Nicholas Ostler,
• Finding Our Tongues by Dean Falk,
• Origins of Human Communications by Michael Tomasello.

To this distinguished list I want to add Louder than Words: the new science of how the mind makes meaning by Benjamin K. Bergen. It is as important a book as I have found on this blog. Maybe it is even one of the most important I've read in a lifetime of reading about language. A statement like that, of course, tips my hand. This book vindicates a great deal that I have argued over the years on this blog: words work by piloting attention; language is a tool for sharing perceptions; to start speaking our ancestors did not have to get smarter, just more social.

It is not that Bergen says these things. He is not concerned with why we speak or what it accomplishes. How we manage it is his subject. But that's what makes the book so pleasing. It is a report from an experimental scientist full of data that happens to agree (more or less) with my own lines of inquiry.

Bergen is an associate professor of cognitive science at the University of California (San Diego) and a protégé of George Lakoff, a cognitive linguist at UC Berkeley. Lakoff is co-author of a fine book that appeared in 1980, Metaphors We Live By. I read the book more than 30 years ago, but as I recall it's thesis is that metaphors are critical to understanding any abstract statement. Metaphors are more than rhetorical devices. Without them we would only be able to speak about purely concrete subjects.

It might seem a little surprising, therefore, that most of Bergen's new book is about how we understand purely concrete sentences like The polar bear hid his nose. The approach turns out to have good sense behind it. It provides a very clear foundation for the assertion that concrete statements are understood by simulating the perceptions and motions they describe. Metaphors build on that perceptual layer. Abstract statements then build on the metaphorical layer. Bergen discusses all three types of statements but his evidence is richest when he reports his many experiments on concrete sentences.

Lakoff and Bergen have a common enemy, the theory of Mentalese, most thoroughly explored by Stephen Pinker but assumed by many others as well.

Mentalese defines language as a system of symbols organized by syntactic rules. Its great strength is that this definition exactly matches computer languages. To program a machine to use a language, all you need is a dictionary and a grammar. The famous Chomsky hierarchy of languages seems to take it for granted that natural languages (human languages) fall somewhere among this listing of types of machine languages.

Bergen's thesis rejects mentalese. In place of a dictionary, it places perceptions and motor activity. Take the sentence The polar bear hid its nose. To interpret it the mentalese way, a machine or brain looks up the various words in the dictionary and uses syntactical rules to divine the abstract relationships between the words. Bergen says no, as we read the sentence we activate the very same neurons in the brain that we use when we see a polar bear or a picture of a polar bear. For the nose, we active our nose perceiving neurons. For the verb to hide, we activate the motor neurons used to hide a nose (by putting a paw/hand in front of the nose).

There are three reasons (at least) for preferring Bergen's explanation over the mentalese account. First of all, there is the fact that brain imagery confirms that brains do work the way Bergen says they do. I won't linger over this point as I have made it myself many times on this blog.

Second, Bergen's account goes a long way to explaining all the stuff we understand all the unstated material in a sentence. Polar bears are white and operate against a white background, but their nose is black, so they might want to hide their noses to achieve even greater invisibility. We may never have thought about that point before, but once made we can see the way it works. There is no way to understand the role of whiteness in the mentalese method of turning to a dictionary, unless the information is specifically included in the dictionary. Even if we grant that the dictionary does include all this color detail, we see the problem. It would be nice to understand how we can get more out of language than the words seem to say rather than merely explain it all away with the guess that we already knew what was implied. Language would be much less useful if it could not bring us news.

Third, Bergen has performed hundreds of experiments to support his point. The book reports on over two hundred experiments by Bergen, his graduate students, and likeminded investigators. The method gives the book a certain text-book tedium, but the effect is to break down all resistance and say okay, okay, I believe you.

The experiments fall into several general categories. There are the ones where reading a word or sentence disrupts your ability to interpret a picture. It sounds crazy, but reading the word juggle slows your ability to interpret a stick drawing of a man checking his watch. Why should that be? One explanation might be that when you read the word juggle you activate motor neurons in a way that contradicts the way you would use your hands if you were to check your watch. That explanation might sound far fetched if the experiment were not one of many in which such word/visualization interferences occur. There are also the opposite kinds of experiment where words and images support each other. Then there are the metaphorical effects like rating a politician as being more serious, if you are holding something heavy in your hand.

The point of all these experiments is to see how far Bergen can push the idea that meaning is a simulation of the perception and motor activities carried out in the brain. He does a good job of it, although as is usual in the world of hypothesis and experiment, he makes a firmer case against his rival than in proving his own theory.

Mentalese is on the ropes. It cannot account for these experiments. The best it seems to manage is 'so what?' All this interference, support, and metaphorical effect does not prove that simulation determines meaning. To which the retort is no, but it sure says something is going on that Pinker, Chomsky, et al. did not expect.

Perhaps Berger's biggest problem is that putting meaning on perception—something this blog has done for years—still doesn't answer the question of the mechanics of meaning. Perception is as big a mystery as language. We see a polar bear and know it is a polar bear, how does that happen? I've never had to worry about that issue on this blog because, however it works, the answer is older than human origins. As waiters say, "It's not my table." But someday the puzzle has to be faced.

As far as this blog's direct concern—language origins—goes, the implications are important. The most commonly published date for language origins is 100 to 50 thousand years ago. That is said despite the fact that humans have all sorts of physical adaptations to speech that cannot possibly be so recent. It also ignores the evidence that genus Homo has been cooperative for millions of years and that until very recent times, the Homo brain has for some reason or other been getting bigger and bigger. So what's the evidence that language is a recent invention?

Mentalese.

If you accept the mentalese definition of language as a system of structured symbols, it is reasonable to think that the species did not use language until it started using symbols. When was that? Some evidence points to a few hundred thousand years ago, but not everybody accepts that. It is hard to deny the archaeological evidence that by 50,000 years ago symbols were to be found.

But if the mentalese definition is wrong, then there is no reason to stick with symbols as language's sine qua non. Concrete remarks could be very old indeed, with metaphors and abstractions coming later. So, of course, I welcome this account of Dr. Bergen's many experiments.

I will follow up this review with further posts.

Comments

[Vanityfea.blogspot.com]

Language. Language is not a block, it's made of levels. And if the evolutionary perspective is right, which of course it must be, it does not spring into the world fully made; it has to be made. So, the symbolic systems we are familiar with has to be built, even if rests on an evolutionary / biological / neurological basis of perception. The symbolic nature of language cannot be bypassed, and it is indeed language's sine qua non if we are speaking of fully formed language - although this research will help to understand its nature and origin, as well as the biological basis of protolanguage.