Chomsky: Generative & Universal Grammar

  • Noam Chomsky is perhaps better known for his political views than his revolutionary contribution to linguistics, but it that contribution which I intend to explore here.

Beginning with Syntactic Structures in 1957, the book which lay the groundwork for his theory of transformational grammar.

Chomsky’s  ”Three Models for the Description of Language” (1955) outlined three possible candidates for the construction of sentences, finite state grammar, phase structure grammar, and transformational grammar.

Finite state grammar

The simplest model for generating sentences is the finite state grammar, or Markov model, which Steven Pinker illustrates with this example of a word-chain device in The Language Instinct (1994):

From this scheme we can construct a potentially infinite set of sentences:

A girl eats ice cream

The happy dog eats hot dogs

The happy happy boy eats candy

Finite state grammars are those which consist of a finite number of states (Si) with transition symbols, aij, and a set C ={(Si; Sj)} of connected states. Transition symbols may be phonemes (the smallest unit of sound employed to form meaningful contrasts between utterances, e.g. bat or beat), morphemes (the smallest conceptual meaningful component of a word, or other linguistic unit, that has semantic meaning, e.g. unmicrowaveability), or words. These elements are, of course, the focus of Saussurian linguistics.

 As this grammar evolves from state to state it produces strings of ‘concatenated‘ symbols aij that form all the sentences of a finite-state language LG.

This is, in fact, how automated messages work: the automated announcer telling you that  your train is running late draws from a pre-recorded list of train times, destinations, and excuses, to construct meaningful sentences:

The / ten / twenty / six / train / to / Manchester Airport / has been delayed by /approximately / twenty / minutes / due to / staffing shortages

The lists from which the system can draw its words and phrases can, of course, be enormous, and could encompass the entire dictionary if one wished: in order the create realistic sounding sentences one could weight the choices of words according to the probability of word combinations occuring in real speech, so ‘liquid’ or ‘gas’ would be much more likely to follow the word ‘colourless’ than would the word ‘green’.

A first-order process is where the probability affecting the choice of a word is based on the proceding word, a second order process is where the probabilities depend on the two proceding words, etc, this series being designated the nth  order approximation. This grammar decides the probability of a given word occurring conditioned on the n –1 previous words that were observed: as n increases the sentences formed begin to resemble proper English sentences.

There are three basic problems with finite state devices though. Consider the following examples from Chomsky’s article:

colorless green ideas sleep furiously

furiously sleep ideas green colorless

The probability of these words following each other is virtually nil – what would it mean for something to be both colourless and green? How could anything sleep ‘furiously’? The nth order approximation cannot distinguish between grammatically correct – if meaningless – sentences like the first, and grammatically incorrect – and equally meaningless – sentences like the second.

Secondly, finite state systems theories with a problem: in English, ‘either’ must always, eventually, be followed in a sentence by ‘or’, and ‘if’ must always be followed by ‘then’:

Either the happy girl eats ice cream or the girl eats candy

If the girl eats ice cream, then the boy eats hot dogs

We would never say:

Either the happy girl eats ice cream, then the happy boy eats hot dogs

If the happy girl eats ice cream, or the happy boy eats hot dogs

You might try illustrating the generative schema like this (again taken from Pinker’s book):

The problem is that finite state grammars are  memoryless; that is, that each state depends only on the current state and not on the sequence of states that preceded it. The system has no way of remembering if the sentence started with ‘either’ or ‘if’ when it comes to chosing from ‘or’ or ‘then’.

And when we reach more complex embedded sentences like

If either the girl eats ice cream or the girl eats candy, then the boy eats hot dogs

Either if the girl eats ice cream then the boy eats ice cream, or ifthe girl eats ice cream then the boy eats candy

You could try creating a Markov model


Another finite-state grammar that could be used for the English language is the nth order approximation.

Saussurian linguistics cannot distinguish between sentences which share almost identical grammatical structure, for example the sentences:

John is easy to please

John is eager to please 

These sentences share the same sequence of nouncopulaadjectiveinfinitiveverb but in the first John functions as the direct object of the verb to please; i.e. it is easy for someone to please John. In the second sentence John functions as the subject of the verb to please; i.e. John is eager that he please someone else. That this is a difference in syntax is demonstrated by the ease with which the second sentence allows us to form the noun-phrase John’s eagerness to please, but not John’s easiness to please out of the first.

Another example is I like her cooking contains no ambiguous words and has a very simple superficial grammatical structure (nounverbpossessive pronounnoun) but is ambiguous: It can mean I like what she cooks, I like the way she cooks, or even, if you are a nutcase, I like the fact that she is being cooked.

”Twas brillig, and the slithy toves
Did gyre and gimble in the wabe;
All mimsy were the borogoves,
And the mome raths outgrabe.”

– Lewis Caroll,  Through the Looking-Glass, and What Alice Found There

Prostetnic Vogon Jeltz throwing his mean callous heartless exterior into sharp relief.”Oh freddled gruntbuggly thy micturations are to me
As plurdled gabbleblotchits on a lurgid bee.
Groop I implore thee my foonting turlingdromes
And hooptiously drangle me with crinkly bindlewurdles,
Or I will rend thee in the gobberwarts with my
blurglecrucheon, see if I don’t!”

– Douglas Adams, The Hitch-Hikers Guide to the Galaxy

Phase structure grammar

Phase structure grammar

Transformational generative grammar

Transformational generative grammar

Generative Grammar

Generative grammar

Universal Grammar

Chomsky argued that language is an innate human capacity, and that the properties of a generative grammar arise from an innate universal grammar.

Chomsky vs. Empiricism


  • Adams, Douglas (1979) The Hitch-Hikers Guide to the Galaxy
  • Barsky, Robert F. (1998) Chomsky: A Life of Dissent
  • Burgess, Anthony (1992) A Mouthful of Air: Language and Languages, Especially English
  • Carroll, John B. (ed., 1956) Language, Thought, and Reality: Selected Writings of Benjamin Lee Whorf
  • Carroll, Lewis (1872) Through the Looking-Glass, and What Alice Found There
  • Casti, John L. (1990) Paradigms Lost
  • Chomsky, Noam (1957) Syntactic Structures
  • Chomsky, Noam (1959) A Review of B. F. Skinner’s Verbal Behavior in Language, 35, No. 1 (1959); reprinted in Jakobovits, Leon A. and Murray S. Miron (eds., 1967) Readings in the Psychology of Language
  • Chomsky, Noam (1972) Language and Mind
  • Chomsky, Noam (1988) Language and Problems of Language: The Managua Lectures
  • Hawkes, Terrence (1977) Structuralism and Semiotics
  • Jakobovits, Leon A. and Murray S. Miron (eds., 1967) Readings in the Psychology of Language
  • Kharbouch, Alaa and Zahi Karam ”Three Models for the Description of Language” (pdf)
  • Pinker, Steven (1994) The Language Instinct: The New Science of Language and Mind
  • Pinker, Steven (2007) The Stuff of Thought: Language as a Window into Human Nature
  • Saussure, Ferdinand de (1983) Course in General Linguistics
  • Tallis, Raymond (2nd ed. 1995) Not Saussure: A Critique of Post-Saussurean Literary Theory
  • Tallis, Raymond (2nd ed. 1998) In Defense of Realism
  • Valentin, Volosinov (1929) Marxism and the Philosophy of Language
  • Whorf, Benjamin Lee (1956) in John B. Carroll (ed.) Language, Thought, and Reality: Selected Writings of Benjamin Lee Whorf