BIT/BCS:ARTIFICIAL

INTELLIGENCE

Lecture PowerPoints

1
Understanding natural languages
II

2
 Topics Covered
• Syntactic Processing
– Transformational grammars
– Case grammars
– Conceptual Dependency
– Conceptual Parsing

3
 Transformational grammars (1)
• Context free grammars (CFGs) are not powerful enough to
describe the structure of English (or other language).
• CFGs are not even powerful enough to define the syntax of
programming languages.
– They can define most of it but not all.
• Chomsky developed a theory of language called
transformation grammar (TG).
• He attempted to account for the human language ability in
terms of
– a deep structure,
– a surface structure
– and a set of transformation rules that provide a mapping between
these two levels.

4
 Transformational grammars (2)
• Thus active and passive forms of a sentence correspond
to a single deep structure
– and are produced from it by the application of two different
sequences of transformation rules.
• So “John hit Mary.” and “Mary was hit by John.”( different
but one meaning) map to a single deep structure
• Central to this theory is an explication of transformation
rules and the convention which governs their application
in the derivation of natural language (see Figure 11).
• The sequences of transformations, T1, T2 map the deep
structure of some agent acting on some object to the
active and the passive surface structures.
5
 S1 S2

NP VP NP VP

NP
V aux
John Mary V prep NP

hit Mary was

hit by John

T1 T2

(JOHN agent) (HIT action) (MARY object)

Figure 11: One deep structure and two surface structures.

6
 Transformational grammars (3)
• The implication was that the meaning was only in the
deep structure
• Transformations should be meaning preserving
• But problems with this concept have necessitated a
revision that admits that meaning is in both levels.

7
 Case grammars (1)
• Case grammars provide a different approach to the
problem of how syntactic and semantic interpretation can
be combined.
• They were developed by the American linguist Charles J.
Fillmore in 1968.
• Grammar rules are written to describe syntactic rather
than semantic regularities.
• The structures the rules produce correspond to semantic
relations rather than to strictly syntactic ones.
• As an example, consider the two sentences and the
simplified forms of their conventional parse trees shown in
Figure 12.
8
 S S

NP VP NP VP

NP PP
V V

Susan printed the file. The file was printed by Susan.

Figure 12: Syntactic parses of an active and a passive sentence.

9
 Case grammars (2)
• Using a case grammar, the interpretations of the two
sentences would both be
(printed (agent Susan)
(object File))
• Now consider the two sentences shown in Figure 13
• The syntactic structures of these two sentences are
almost identical.
– In one case, “Mother” is the subject of “baked”, while in the other
“the pie” is the subject.
• The relationship between Mother and baking is very
different from that between pie and baking.

10
 S S

VP NP VP
NP

PP PP
V V

for three hours. The pie baked for three hours.

Mother baked

Figure 13: Syntactic parses of two similar sentences.

11
 Case grammars (3)
• A case grammar analysis of these two sentences reflects
this difference.
• The first sentence would be interpreted as
(baked (agent Mother)
(timeperiod 3-hours))
• The second sentence would be interpreted as
(baked (object Pie)
(timeperiod 3-hours))
• In these representations, the semantic roles of “mother”
and “the pie” are made explicit.
• The cases used by a case grammar describe relationships
between verbs and their arguments.
12
 Case grammars (4)
• This contrasts with the grammatical notion of a surface case, as
exhibited , for example, in English by the distinction “I”
(normative case) and “me” (objective case).
• A given grammatical, or surface case can indicate a variety of
semantic, or deep cases.
• Some obvious deep cases:
(A) Agent – Instigator of the action (typically animate)
(I) Instrument – Cause of the event or object used in causing the event
(typically inanimate)
(D) Dative – Entity affected by the action (typically animate)
(F) Factitive – Object or being resulting from the event
(L) Locative – Place of the event
(S) Source – place from which something moves
(G) Goal – place to which something moves
13
 Case grammars (5)
(B) Beneficiary – Being on whose behalf the event occurred
(typically animate)
(T) Time – time at which the event occurred
(O) Object – entity that is acted upon or that changes, the most
general case
• The process of parsing into a case representation is
heavily directed by the lexical entries associated with
each verb.
• Figure 14 shows examples of a few such entries.
– Optional cases are indicated in parentheses.
• Languages have rules for mapping from underlying case
structures to surface syntax forms.
14
open [_ _ O(I) (A)]
The door opened.
John opened the door.
The wind opened the door.
John opened the door with a chisel.

die [_ _ D]
John died.

kill [ _ _ D(I) A]
Bill killed John.
Bill killed John with a knife.

run [ _ _ A]
John ran.

want [ _ _ A O]
John wanted some ice cream.
John wanted Mary to go to the store.

Figure 14: Some verb case frames.

15
 Conceptual Dependency (CD)

• CD theory was developed by Schank in 1973 to 1975

to represent the meaning of NL sentences.
− It helps in drawing inferences
− It is independent of the language
• CD representation of a sentence is not built using
words in the sentence rather built using conceptual
primitives which give the intended meanings of
words.
• CD provides structures and specific set of
primitives from which representation can be built.
 Primitive Acts of CD theory
• ATRANS Transfer of an abstract relationship (i.e. give)
• PTRANS Transfer of the physical location of an object (e.g.,
go)
• PROPEL Application of physical force to an object (e.g. push)
• MOVE Movement of a body part by its owner (e.g. kick)
• GRASPGrasping of an object by an action (e.g. throw)
• INGEST Ingesting of an object by an animal (e.g. eat)
• EXPEL Expulsion of something from the body of an animal
(e.g. cry)
• MTRANS Transfer of mental information (e.g. tell)
• MBUILD Building new information out of old (e.g decide)
• SPEAK Producing of sounds (e.g. say)
• ATTEND Focusing of a sense organ toward a stimulus
(e.g. listen)
 Conceptual category

• There are four conceptual categories

− ACT Actions {one of the CD primitives}

− PP Objects {picture producers}
− AA Modifiers of actions {action aiders}
− PA Modifiers of PP’s {picture aiders}
 Example
• I gave a book to the man. CD representation is as
follows:

P O R man (to)
I  ATRANS  book
I (from)

• It should be noted that this representation is same for

different saying with same meaning. For example
− I gave the man a book,
− The man got book from me,
− The book was given to man by me etc.
 Few conventions

• Arrows indicate directions of dependency

• Double arrow indicates two way link between actor
and action.
O – for the object case relation
R – for the recipient case relation
P – for past tense
D - destination
 Some of Conceptualizations of CD

• Dependency structures are themselves

conceptualization and can serve as components of
larger dependency structures.
• The dependencies among conceptualization
correspond to semantic relations among the
underlying concepts.
• We will list the most important ones allowed by CD.
 Rule 1: PP  ACT
• It describes the relationship between an actor and the
event he or she causes.
− This is a two-way dependency, since neither actor
nor event can be considered primary.
− The letter P in the dependency link indicates past
tense.
• Example: John ran
P
CD Rep: John  PTRANS
 Rule 2: ACT  PP
• It describes the relationship between a ACT and a PP
(object) of ACT.
− The direction of the arrow is toward the ACT since
the context of the specific ACT determines the
meaning of the object relation.

• Example: John pushed the bike

O

CD Rep: John  PROPEL  bike

 Rule 3: PP  PP
• It describes the relationship between two PP’s, one of
which belongs to the set defined by the other.

• Example: John is doctor

CD Rep: John  doctor

 Rule 4: PP  PP
• It describes the relationship between two PP’s, one of
which provides a particular kind of information about
the other.
− The three most common types of information to be
provided in this way are possession ( shown as POSS-
BY), location (shown as LOC), and physical containment
(shown as CONT).
− The direction of the arrow is again toward the concept
being described.

• Example: John’s dog

poss-by
CD Rep dog  John
 Rule 5: PP  PA
• It describes the relationship between a PP and a PA
that is asserted to describe it.
− PA represents states of PP such as height, health
etc.

• Example: John is fat

CD Rep John  weight (> 80)

 Rule 6: PP  PA
• It describes the relationship between a PP and an
attribute that already has been predicated of it.
− Direction is towards PP being described.

• Example: Smart John

CD Rep John  smart

 R PP (to)
Rule 7:ACT 
 PP (from)
• It describes the relationship between an ACT and the
source and the recipient of the ACT

• Example: John took the book from Mary

R  John
CD Rep: John  ATRANS 
O   Mary
book
  PA
Rule 8:PP
 PA
• It describes the relationship that describes the change
in state.

• Example: Tree grows

 size > C
CD Rep: Tree 
 size = C
  {x}
Rule 9:
 {y}
• It describes the relationship between one
conceptualization and another that causes it.
− Here {x} is causes {y} i.e., if x then y

• Example: Bill shot Bob

{x} : Bill shot Bob

{y} : Bob’s health is poor

  {x}
Rule 10: 
 {y}
• It describes the relationship between one
conceptualization with another that is happening at the
time of the first.
– Here {y} is happening while {x} is in progress.

• Example: While going home I saw a snake

I am going home


I saw a snake
 Generation of CD representations
Sentences CD Representations
p o d ?
Jenny cried Jenny  EXPEL  tears
eyes
poss-by 
Jenny

p d India
Mike went to India Mike  PTRANS
? (source is unknown)
Mary read a novel p o d CP(Mary)
Mary  MTRANS  info
novel
 i (instrument)

p o d novel
Mary  ATTEND  eyes
?
Sentence CD Representation

o r One
Since One  INGEST  durgs
drugs can Mouth
kill, I
stopped. c
health = -10
One
health > -10

tfp o r I
I  INGEST  durgs
mouth
Sentence CD Representation

John  Do1
p o r Mike
John warned John  MTRANS
Mike with dire John
consequence.

f
State bad
 poss-by
Mike

Poss-by
r memory  Mike
Mike  MTRANS
o John
Mike  Do2

Mike
 poss-by
John  Do1 State bad
f
Inferences Associated with Primitive
Act (1)
• General inferences are stored with each primitive Act
thus reducing the number of inferences that need to
be stored explicitly with each concept.
• For example, from a sentence “John killed Mike”, we
can infer that “Mike is dead”.
• Let us take another example of primitive Act INGEST.
• The following inferences can be associated with it.
– The object ingested is no longer available in its original form.
– If object is eatable, then the actor has less hunger.
– If object is toxic, then the actor’s heath is bad.
– The physical position of object has changed. So PTRANS is
inferred.
•
 Inferences Associated with
Primitive Act (2)
• Example: The verbs {give, take, steal, donate}
involve a transfer of ownership of an object.
– If any of them occurs, then inferences about who now has
the object and who once had the object may be important.
– In a CD representation, these possible inferences can be
stated once and associated with the primitive ACT
“ATRANS”.
• Consider another sentence “Bill threatened John with
a broken nose”
– Sentence interpretation is that Bill informed John that he
(Bill) will do something to break john’s nose.
– Bill did (said) so in order that John will believe that if he
(john) does some other thing (different from what Bill
wanted) then Bill will break John’s nose.
Problems with CD Representation (1)
• It is difficult to
– construct original sentence from its corresponding CD
representation.
– CD representation can be used as a general model for
knowledge representation, because this theory is based on
representation of events as well as all the information
related to events.
• Rules are to be carefully designed for each primitive
action in order to obtain semantically correct
interpretation.
Problems with CD Representation (2)
• Many verbs may fall under different primitive ACTs,
and it becomes difficult to find correct primitive in the
given context.
• The CD representation becomes complex requiring
lot of storage for many simple actions.
• For example, the sentence “John bet Mike that Indian
cricket team will win incoming world cup” will require
huge CD structure.
 Conceptual Parsing

• Conceptual parsing is required for generating CD

representation from source sentences in natural
language.
• The main steps involved in CD parsing are as
follows:
− Syntactic processor extracts main verb and noun along with
syntactic category of the verb (transitive or intransitive) from
the sentence.
− Conceptual processor then makes use of verbACT
dictionary.
− Once the correct entry from dictionary is chosen, CD
processor analyses the rest of sentence looking for
arguments for empty slots of the verb.
− CD processor examines possible interpretation in a well-
defined order.
 Example (1)

• Case1: Handling of ‘with PP’ phrase by CD processor

and formulating strategies to disambiguate the
meanings.
– Type1: John broke the door with hammer non animate
– Type2: John broke the door with Mike animate
• Rule 1: If PP in ‘with PP’ phrase is non-animate and CD
Act requires instrument then the sentence is of Type1,
where PP (hammer) is resolved to instrument.
• Rule 2: If PP in ‘with PP’ phrase is animate and CD Act
requires instrument then the sentence is of Type2, where
PP (Mike) is resolved as co-actor.
 Example (2)

• Case2: If PPs in both the sentences are non-

animate, then they have to be resolved using
semantic lexicon.
– Type3: John went to the garden with flowers
– Type4: John went to the garden with bag
• In Type3, non-animate noun ‘flowers’ is part of
garden, whereas in Type4, non-animate ‘bag’ is
some object not related to garden.
• Such association of word senses could be found in
Word-Net and then disambiguation is possible.
• Here noun ‘bag’ is treated as possession by John.
 Example (3)

• Case 3: If PPs in the sentences are animate, then

they have to be resolved using semantic lexicon and
context.
• Consider the following examples.
– Type5: John went to the garden with Mike
– Type6: John went to the garden with butterflies
– Type7: John went to the garden with dog
• In these sentences, Mike, butterflies and dog are
animate PPs and can be resolved as follows:
• Mike is easily resolved to co-actor of John as both
are human and have similar characteristics.
 Example (4)

• Word-Net can be used to check if butterfly and

garden has some common sense.
• Dog is still ambiguous.
• It may be treated as possession of actor or may be a
part of garden as animals many wonder in garden.
• Such situations can be further resolved by
considering the context of sentences.
• We can use semantic lexicon dictionary to resolve
some of the ambiguities.