Structure and Meaning in Language

Communication, Signification, and Language

Communication and signification are general topics, with relatively basic theories and principles. Shannon’s Mathematical Theory of Communication is one paper. Charles Sanders Peirce developed a comprehensive theory of signs, categorizing them into icons, indices, and symbols based on their relationship to the objects they represent. You can learn the basics in a day.

Charles Sanders Peirce identified three types of signs:

Icons

Resemble what they represent — a photo, a map, onomatopoeia (buzz)

Indices

Causally connected to their referent — smoke → fire; pointing

Symbols

Arbitrary, conventional — most words; red = stop

Human language is overwhelmingly symbolic. This is extraordinary! No other species uses symbols at scale. And we’ve been doing it for a long time:

• Ochre engravings at Blombos Cave, South Africa: ~75,000 BCE — geometric patterns, possibly symbolic
• Shell beads used as ornaments: ~100,000 BCE — symbols of identity, group membership
• Cave paintings (Lascaux, Chauvet): ~30,000–17,000 BCE — iconic + possibly symbolic

Symbols are fascinating because they have no intrinsic connection to their referent. They require (a) shared convention, (b) memory, (c) the ability to hold “word” and “meaning” as linked but separate. The power is unlimited generativity — you can make a symbol for anything, even abstractions: justice, infinity, recursion, love, LLM, the meaning of life, etc.

But where do symbols come from? How does a word hook onto the world? Philosophers call this the symbol grounding problem. Harnad (1990) says symbols in a closed formal system only refer to other symbols—they’re not grounded. Note that a dictionary defines words with more words. There is no base word. Humans ground symbols through perception and action

Language is far more specialized and does not admit to a single theory. Sure, it uses signs for communication. But it is so much more. A fair working definition is a structured system of communication, consisting of utterances that can be spoken, signed, or written, formed via compositional rules (or guidelines) agreed upon by users of the language. The interesting thing about this definition is that it covers the structure part but not the meaning part. In addition to syntax, we need semantics. And pragmatics.

Grammar

Words and word-like entities are combined (arranged) according to rules to form larger units like phrases and sentences. The rules constitute a grammar. Where do the rules come from? How do we know if an utterance is well-formed?

Rules or Guidelines?

In programming languages, rules are explicit and prescriptive. In spoken natural language, we tend to treat them as guidelines, but in writing, they are often more rigidly followed.

A grammar will tell you where all the components of an utterance should go—based on the category (e.g., for English: noun, verb, adjective, noun phrase, verb phrase, subject, predicate, adverbial clause, prepositional phrase, etc.) of the word or phrase—of the components, but it won’t tell you what the utterance means. Classic examples are:

• "The cat the dog the rat chased killed ate the malt" — grammatical, incomprehensible
• "Colorless green ideas sleep furiously" (Chomsky) — grammatical, semantically absurd

Kinds of Grammars

There are many different approaches to capturing our idea of a grammar.

Dependency Grammar

Dependency grammar focuses on the relationships between words in a sentence, where one word (the head) governs the others (dependents). The structure is represented as a tree with directed edges from heads to dependents, emphasizing the syntactic functions and hierarchical organization of words.

TODO EXAMPLES

Constituency Grammar

Constituency grammar, in contrast, emphasizes the grouping of words into constituents or phrases, which function as single units within a hierarchical structure. The structure is typically represented as a tree where nodes correspond to constituents, highlighting the nested and recursive nature of language.

Construction Grammar

Construction grammar posits that knowledge of language consists of a collection of form-meaning pairings, known as constructions. These constructions range from simple words to complex syntactic patterns, and meaning is derived from the interaction of these constructions rather than from abstract rules alone.

Operator Grammar

Operator grammar is a theory that focuses on the combinatory properties of operators (typically verbs) and their arguments. It emphasizes the rules governing how operators can combine with other elements to form grammatical sentences, often highlighting the constraints and regularities in argument structure.

Transformational Grammar

Transformational grammar, developed by Noam Chomsky, posits that sentences have an underlying deep structure that can be transformed into various surface structures through a set of transformational rules. This approach emphasizes the generative aspect of grammar and the relationship between syntax and meaning.

Parsing

Technically, the grammar composes utterances into hierarchical structures, even though they tend to be written or spoken as temporal one-dimensional sequences.

Recursion

Recursion is the property of language that allows rules to be applied repeatedly, embedding structures within structures. This is what enables sentences to be infinitely long and complex, even with a finite set of rules and vocabulary.

Here’s a grammar that demonstrates recursion:

We can generate sentences as in this example:

  S
  NP VP
  DET NOUN RP VP VP
  the NOUN RP VP VP
  the dog RP VP VP
  the dog that VP VP
  the dog that SV S VP
  the dog that thought S VP
  the dog that thought NP VP VP
  the dog that thought PN VP VP
  the dog that thought grace VP VP
  the dog that thought grace TV NP VP
  the dog that thought grace hit NP VP
  the dog that thought grace hit PN VP
  the dog that thought grace hit alan VP
  the dog that thought grace hit alan DV NP PP
  the dog that thought grace hit alan threw NP PP
  .
  .
  .
  the dog that thought grace hit alan threw the new blue toy to the fast rat

Recursion can allow sentences to become arbitrarily long and complex:

  S
  NP VP
  NP SV S
  NP SV NP VP
  NP SV NP SV S
  NP SV NP SV NP VP
  NP SV NP SV NP SV S
  .
  .
  .
  she dreamed she dreamed she dreamed . . . she dreamed the dog swam

Language Acquisition

Chomsky thought language acquisition is too fast and input too impoverished for learning alone, putting forth the Poverty of the Stimulus argument: children know grammatical rules they’ve never seen exemplified. So he proposed a kind of innate Language Acquisition Device (LAD) for what became known as Universal Grammar.

Many opposing views exist. Tomasello and Christiansen note that general learning mechanisms + social interaction are sufficient. The computer simulations of Kirby and colleagues support this view, showing that cultural transmission can lead to the emergence of structured language over generations.

Perhaps you’ve heard of pidgins (rudimentary contact languages with no native speakers and minimal grammar) or creoles (pidgins that children acquire natively, with a spontaneously developed full grammar).

You should read about how deaf children in 1980s Nicaragua invented a full sign language in one generation. Themselves. With no adults to mimic.

There seems to be a lot of evidence that given the right social conditions, grammar emerges.

• Compression and communication in the cultural evolution of linguistic structure
• Syntax out of Learning: The Cultural Evolution of Structured Communication in a Population of Induction Algorithms
• Syntax without Natural Selection: How compositionality emerges from vocabulary in a population of learners
• Innateness and culture in the evolution of language
• Culture and biology in the origins of linguistic structure
• Language as Shaped by the Brain
• On how “early syntax” came about
• Cultural evolution creates the statistical structure of language
• Linguistic structure from a bottleneck on sequential information processing
• The Evolution and Emergence of Language: How Did Humans Acquire Language?
• Linguistic Structure from a Bottleneck on Sequential Information Processing
• Compositional Syntax From Cultural Transmission

The Emergence of Syntax

Discuss the fact that LLMs acquire language from stimulus alone, with no innate LAD, and they do remarkably well. LLMs learn grammar implicitly from the data, without explicit instruction. They can generate grammatically correct sentences, but they don’t have an explicit representation of grammatical rules like humans do. How is this possible? Are transformers just big enough to brute-force what evolution gave us for free? Or do we have something else?

More Opposition to Universal Grammar

The fact that syntax is emergent, together with recent research on the learnability of language, the success of large language models, new knowledge of the brain, and the wide variety among structure and meaning in human languages, challenge the notion of a Universal Grammar. Although originally compelling, the need for innate language learning mechanisms are no longer needed to explain language acquisition.

Grammars for Constructed Languages

TODO

Recall Practice

Here are some questions useful for your spaced repetition learning. Many of the answers are not found on this page. Some will have popped up in lecture. Others will require you to do your own research.

1. What is Chomsky’s poverty of the stimulus argument?
   Children acquire grammatical rules they have never seen exemplified in their input, which suggests that at least some linguistic knowledge is innate (the Language Acquisition Device). The stimulus is too impoverished for pure learning to account for the result.
2. What does the emergence of Nicaraguan Sign Language demonstrate?
   That grammar can emerge spontaneously from social conditions alone — deaf children in 1980s Nicaragua, without an adult sign language model, invented a full grammatical sign language within one generation.
3. What is the difference between the strong and weak versions of the Sapir-Whorf hypothesis? Which is supported empirically?
   Strong (linguistic determinism): language determines thought — we cannot think what we cannot say. Now largely discredited. Weak (linguistic relativity): language influences habitual thought. This is supported empirically, e.g., color term differences affecting discrimination speed.
4. What are the three types of signs identified by Charles Sanders Peirce?
   Icons (resemble what they represent), Indices (causally connected to their referent), Symbols (arbitrary, conventional).

Summary