Introduction to Computational

Linguistics
Eleni Miltsakaki
AUTH
Fall 2005-Lecture 6

1
 What’s the plan for today?

• Peer-to-peer tutorial on
– Computational linguistics
– Grammars and parsing
– TAG
– LFG
– HPSG
• Questions about homework
• On-line processing of syntactic ambiguity in
adults and children
2
Slides to guide you review tutorial

3
 What is computational linguistics?

• A discipline between Linguistics and

Computer Science

Concerned with the computational aspects of

human language processing

Has theoretical and applied components

(explain)
4
 Why is language hard for
computers?
AMBIGUITY! (GIVE EXAMPLES OF SYNTACTIC/SEMANTIC etc
AMBIGUITIES)
• Natural languages are massively ambiguous at all levels of processing (but
humans don’t even notice…)

• To resolve ambiguity, humans employ not only a detailed knowledge of the

language -- sounds, phonological rules, grammar, lexicon etc -- but also:
– Detailed knowledge of the world (e.g. knowing that apples can have bruises but
not smiles, or that snow falls but London does not).
– The ability to follow a 'story', by connecting up sentences to form a continuous
whole, inferring missing parts.
– The ability to infer what a speaker meant, even if he/she did not actually say it.

• It is these factors that make NLs so difficult to process by computer -- but

therefore so fascinating to study.

5
 Grammars and parsing
• What is syntactic parsing
– Determining the syntactic structure of a sentence

• Basic steps
– Identify sentence boundaries
– Identify what part of speech is each word
– Identify syntactic relations

• Tree representation

John ate the pizza

(S (NP (N John))
(VP (V ate)
(NP (Det the)
(N cat))))

6
 How to construct a tree
• To construct a tree of an English sentence you need to know which structure are legal in
English

• Rewrite rules
– Describe what tree structures are allowed in the language
NP==> N
NP==> Det NP
VP==> V
VP ==> V NP
S ==> NP VP

S
==> NP VP
==> N VP
==> John VP
==> John V NP
==> John ate NP
==> John ate Det N
==> John ate the N
==> John ate the pizza 7
 Chomsky’s Hierarchy
• Containment hierarchy of classes of formal grammars that generate formal
languages

• Type 0: unrestricted, include all formal grammars

– Any string of terminals and non-terminals to any string of terminals and non-
terminals

• Type 1: context sensitive

– A any string of terminals and non-terminals

• Type 2: context free (the theoretical basis for the syntax of most
programming languages)
– A a, A Ba

• Type 3: regular grammars

– Aa

8
 Tree adjoining grammar
• Introduced by Joshi, Levy & Takahashi (1975) and Joshi (1985)

• Linguistically motivated
– Tree generating grammar (generates tree structures not just strings)
• Example: I want him to leave, I promised him to leave
– Allows factoring recursion from the statement of linguistic constraints
(dependencies), thus simplifying linguistic description (Kroch & Joshi
1985)

• Formally motivated
– A (new) class of grammars that describe mildly context sensitive
languages (Joshi et al 1991)

9
 TAG formalism

• Concepts: lexicalization and locality/recursion

• Who do you like t?
• Who does John think that you like t?
• Who does John think that Mary said that you like t?
• Elementary objects: initial trees and auxiliary
trees
• Operations: substitution and adjunction
– Adjunction

10
11
Adjunction

12
Adjunction

13
Derived and derivation trees

14
 Lexical Functional Grammar
• First introduced by Kaplan & Bresnan (1982)
• Two parallel levels of syntactic representation

– Constituent structure (c-structure)

– Functional structure (f-structure)

• C-structures have the form of context-free phrase

structure trees
• F-structures are sets of pairs of attributes and values;
attributes may be features, such as tense and gender, or
functions, such as subject and object.

15
LFG example

16
 Head-driven Phrase Structure
Grammar
• aka HPSG
• HPSG home: http://hpsg.stanford.edu/

17
18
 Feature structure in HPSG

• A feature structure is a set of pairs of the form

[ATTRIBUTE value]

• ATTRIBUTE: an element of the set of features

named ATT in the grammar (e.g., case, person
etc)

• ‘value’ can be atomic (a string) or another feature

structure
19
Examples of feature structures

20
 Feature types

• Feature structures are

of a certain type,
written in italics

• Features are
organized in
hierarchies
21
 Valence and grammar rules
• Complements are specified as
complex categories in the
lexical representation

• There are also specific rules

for head complement
combinations

22
 Representation of valence in
feature descriptions
A lexical entry consists of:

23
 Head feature principle

• In a headed structure,
the head features of
the mother are
identical to the head
features of the head
daughter

24
 Linguistic generalizations in the
type hierarchy
• Types are arranged in a hierarchy
• The most general type is at the top
• Information about properties of an object of a certain
type are specified in the definition of the type
• Subtypes inherit these properties
• Like an encyclopedic entry
• The upper part of the hierarchy is relevant to all
languages (universal grammar)
• More specific types maybe specific for classes of
languages or just one language
25
A simple example

26
END OF REVIEW SLIDES

27
 Today’s question

• How do humans (adults and children)

process syntactic ambiguity?

28
 Trueswell et al 1999

• “The kindergarten-path effect: Studying on

line sentence processing in young
children”, in Cognition (1999)

29
 The garden-path theory

• At points of syntactic ambiguity the syntactically

simplest alternative is chosen: e.g. minimal
attachment
(e.g., Frazier and Rayner 1982, Ferreira and Clifton 1986)

• However, it has been shown that non-syntactic

sources of information can mediate garden-path
effects
(e.g., Altmann and Steedman 1988, Tanenhaus et al 1995)

30
 Referential principle

• Example: if two thieves are evoked in the

context and then we hear
Ann hit the thief with…
we prefer the NP-attachment reading

(Crain & Steedman 1985)

31
 Experiment 1

• Methodology: eye-tracking
• Participants: 16 5-year-old children
• Material:
– Put the frog on the napkin in the box
(ambiguous between DESTINATION and
MODIFIER)
– Put the frog that’s on the napkin in the box
(unambiguous)
32
Head mounted eye tracker

33
1 and 2 referent context

34
Unambiguous

35
 Analysis

• Percentage of trials with eye-fixation to

INCORRECT DESTINATION (I.e. the
empty napkin)

36
37
 Results

• VP-attachment preference for children: 5-year

olds prefer to interpret the ambiguous ‘on the
napkin’ as destination regardless of referential
context
• Children are insensitive to the “Referential
Principle”
• They don’t ‘recover’ from initial interpretation
• In the 2-referent ambiguous condition they
picked the Target animal at chance
38
 Experiment 2

• Participants: 12 adults
• Same material
• Same methodology

39
40
 Results

• Adults experienced garden path in the 1-

referent ambiguous condition only

41
42
 Conclusions

• Adults and children differ in how they handle

temporary syntactic ambiguity

– Adults resolve ambiguity according to the Referential

Principle: modifier in 2-referent context, destination in
1-referent context

– Children are insensitive to the Referential Principle:

They resolve the ambiguity to the VP-attachment
interpretation, i.e., destination
43
 Explanation of VP-attachment
preference in children

• Minimal attachment?
• Lexical frequency?

44