A second attack on the importance of recursive processes in producing sentences took place today (Saturday March 15, 2008) at the Evolang conference in Barcelona. A few hours after Derek Bickerton argued that recursive processes are not needed to generate sentences, Joris Bleys argued in a workshop that even if they are needed they require no special evolutionary jumps to appear. Recursion (the transformation of preliminary phrases into final sentences by embedding some preliminary phrases into others) can simply be a side-effect of trying to keep linguistic knowledge as simple as possible. If the Bickerton and Bleys papers stand there seems to be no syntactical reason why language should be limited to humans.
Noam Chomsky has repeatedly argued that it is because of their ability to embed phrases into other phrases that language can be proven mathematically to be capable of generating an infinite number of different sentences, despite the fact that they depend on a finite lexicon and a finite number of syntactical rules. For example, in This is the trap that caught the mouse that lived in the house that jack built, we can keep on embedding new phrases indefinitely: This is the cheese that lay in the trap that caught the mouse that lived in the house that Jack built. In a famous paper by Marc Hauser, Noam Chomsky, and Tecumseh Fitch (available here) the authors proposed that the appearance of this capacity to produce infinite variety with finite materials marks a “discontinuity” in the capacity to communicate. They asserted:
The evolutionary puzzle … lies in working out how we got from there [communication without recursion] to here [communication with recursion], given this apparent discontinuity. [p. 1570]
This idea of an evolutionary discontinuity was attacked here in Barcelona by Joris Bleys, a student under Luc Steels who Bleys credits as the source of the idea tested in his paper. Both Bleys and Steels study artificial intelligence and try to model language learning on computers. The simulation where they tested their side-effect hypothesis used “agents” trying to communicate about a simple “world” consisting of blocks, pyramids, balls. The agents are either speakers (“producer”) and listeners (“interpreter”). A producer tries to indicate a particular object and an interpreter tries to identify the referenced object.
The agents begin with a shared semantics about the world and its objects but they start without any syntactic rules for expressing the meanings they need to communicate. The rules they evolve introduce a new step in the process of getting from meaning to sentence (on the part of the producer) or from sentence to meaning (on the part of the interpreter). The two are said to have communicated successfully when the interpreter can identify the object the producer intended.
The semantic systems that underlie Bleys’ evolving syntax were presented at the conference on Friday afternoon in papers by Joachim de Beule and Wouter van der Broeck. They are too technical and artificial to be covered in this detail on this blog, but drafts of the papers are available on line (de Beule here and van der Broedk here). For purposes of this report it is enough to know that the evolving rules link syntactic categories with semantic units.
Although the agents start without any syntax, they do have rules for constructing ways to produce and interpret sentences. Simplifying the semantics a bit, we have two rules for constructing the syntax that produce or parse sentences:
If possible, assign a meaning to a syntactic category; otherwise create a new category. If the first meaning to be expressed is ball, no category for it exists. So the system creates category A and assigns the semantics of ball to it. If another meaning with the same general semantics, e.g., pyramid, is to be expressed it can be given the same category.
Whenever possible, reuse syntactic rules; otherwise create a new rule. After creating category A, the system still has no rule for using it. It generates one, S[entence] = A. Now it can generate or interpret sentences, ball or pyramid, as appropriate.
Bleys’ computer simulation shows that these two rules, combined with a shared semantics, are enough to develop a complex syntax and, eventually, recursive procedures.
Suppose the meaning to be expressed is a ball. The first rule tells us we can assign ball to category A, but we need to create a new category to express the article a. Call that new category D. We don’t have a rule for generating strings that include categories A and D, so the system generates a rule that gives us S = D + A (a + ball).
Suppose we need to express the meaning a big ball. We must create a third category, F and create a third rule for using it. Semantically, a and big are independent of each other so there is no need to put them together, but big is a property of ball so we will need a rule to show that relationship. The system generates S =F + A (big + ball). Will we need yet a fourth rule for expressing the full meaning?
That further rule could generate either a ball big ball [(D+A)+(F+A)] or big ball a ball [(F+A)+(D+A)].
Remember, however, that semantically big ball go together while a big don't go together. By treating big ball as a syntactic unit the system creates a hierarchy of rules using only the existing rules: D + A (a + ball)in which A = F + A (ball = big + ball), giving us the sentence a big ball. (The figure below shows the hierarchy of rules for generating this sentence.)
This embedding categories inside themselves allows us to generate a seemingly infinite number of house-that-Jack-built type sentences: a big ball… a big round ball… a big round orange ball… a big round orange ugly ball. The rule is truly recursive and the system has generated these processes simply by following their inbuilt semantics and their tendency for keeping the number of syntactic rules as small as possible.
As is always the case with artificial intelligence simulations, they can leave observers with a sense of having watched a magic trick. The rabbit must have already been somewhere in the hat. In this case the recursion was already implicit in the semantics. Bleys conceded the point but insisted that he had provided “a proof of concept” that recursion can be a side effect of limiting the number of rules that language-users require while allowing infinite expression. The discontinuity cited by Hauser, Chomsky, and Fitch does not stand.