NATURAL LANGUAGE PROCESSING

Dr. Ahmed El-Bialy

 Why Natural Language Processing ?

 Huge amounts of data  Classify text into categories

 Index and search large texts
 Internet = at least 20  Automatic translation
billions pages  Speech understanding
 Intranet  Understand phone conversations
 Applications for  Information extraction
Extract useful information from resumes
processing large


 Automatic summarization
amounts of texts  Condense 1 book into 1 page
require NLP expertise  Question answering
 Knowledge acquisition
 Text generations / dialogues
Where does it fit in the CS taxonomy?
Computers

Databases Artificial Intelligence Algorithms Networking

Robotics Natural Language Processing Search

Information Machine Language

Retrieval Translation Analysis

Semantics Parsing
 NLP

aims at :
 making computers talk 

 endowing computers with the linguistics ability of

humans
 Dialog system
Fiction
 Conversational agent – system dialog

HAL 9000 -- A space odyssey C3PO -- Star Wars

 Dialog system
reality

 E-commerce: AINI

• a chatterbot integrated with

3D animated agent character
• Improve customer services
• Reduce customer reliance on
human operator
Dialog system
reality

 E-teaching : autotutor
(http://www.autotutor.org/what/what.htm )
 Intelligent tutoring system that helps student learn by
holding a conversationnal in natural language
 Animated agent : synthesis speech, intonation, facial
expressions, and gestures
 demo (from 2002)
Machine translation
 Automatically translate a
document from one language
to another
 Very useful on the web
 Far from solved problem
The Dream
 It’d be great if machines could
 Process our email (usefully)
 Translate languages accurately
 Help us manage, summarize, and
aggregate information
 Use speech as a UI (when needed)
 Talk to us / listen to us
 But they can’t:
 Language is complex, ambiguous,
flexible, and subtle
 Good solutions need linguistics
and machine learning knowledge
 So:

9/34
 The mystery
 What’s now impossible for computers (and any other
species) to do is effortless for humans

✕ ✕ ✓
10/34
 What is NLP?

 Fundamental goal: deep understand of broad language

 Not just string processing or keyword matching!

11/34
Role of Knowledge in Language Understanding

 Understanding ≠ word meanings

 Inference  Speaker goal & assumptions
 Non Monotonic Reasoning
 Word meanings are not enough
 What is Natural Language Processing
(NLP)
13/34

 Computers use (analyze, understand, generate) natural language

 Text Processing
 Lexical: tokenization, part of speech, head, lemmas
 Parsing and chunking
 Semantic tagging: semantic role, word sense
 Certain expressions: named entities
 Discourse: coreference, discourse segments

 Speech Processing
 Phonetic transcription
 Segmentation (punctuations)
 Prosody
 The NLP Problem & Symbolic Approach
14/34

Discourse

Pragmatics

Semantics

Syntax

**we can go up, down and up and

Morphology
down and combine steps too!!
**every step is equally complex
Vocabulary
 Prosody
 Phonology
 Morphology
 Syntax
 Semantics
 Pragmatics: “Do you know the time?”
 World Knowledge
 Parsing
Stages of language Analysis
 Parsing: analysis syntactic structure of
sentences  parse tree
 Requires: Language syntax,
morphology, some semantics
 Semantic Interpolation
 Requires: words meanings and linguistic
structure
 Contextual/ World Knowledge
Syntax
 The description of the legal structures in a language is called a grammar.
 Given a sentence, we use the grammar to find the legal structures for a sentence.
This process is called parsing the sentence.
 The result is one or more parse trees, such as the one shown here, which indicates
that the sentence can be broken down into two constituents, a noun phrase and a
verb phrase. The verb phrase, in turn, is composed of another verb phrase followed
by a prepositional phrase, etc.
 Our attempt to understand sentences will be based on assigning meaning to the
individual constituents and then combining them to construct the meaning of the
sentence. So, in this sense, the constituent phrases are the atoms of meaning.
 A grammar is typically written as a set of rewrite rules such as the ones shown
here in blue. Boldface symbols, such as S, NP and VP, are known as non-terminal
symbols, in that they can be further re-written. The non-bold-face symbols, such as
John, the and boy, are the words of the language -also known as the terminal
symbols.
Specification and parsing using Context free
grammar

 Alphabet: V a set of symbols (digits and letters)

 Sentence: x over V a string of finite length
 Length of x: = │x│ = # symbols
eg . V={a, b} so len(x) on V = 2 {aa, ab, bb, ba}
and if len(x)=3 {aaa, aab, abb,aba, …….}
 Concatination: of x with y, where x,y are sent. over V is xy
and │xy│= │x│ +│y│
 Empty string  is string with len =0
 V* closure of V: infinite set of all sent over V including 
 V+= V* - 
Specification and parsing using Context free
grammar

 eg . V={a, b}
 V* = { , a, b, aa, ab, ba, aaa, ……….}
 V+={ a, b, aa, ab, ba, aaa, ……….}

 Any Language is a set of sentences over alphabet

and subset of V*
 Informal Language include some rules for generating
the language.
Context-free grammars
• G = (N, , S, P)
• N is finite set of nonterminals (variables)
•  is finite set of terminals (constants)
• S is the start symbol (one of the nonterminals)
• P is rules/productions of the form X  , where X is a nonterminal
and  is a sequence of terminals and nonterminals (possibly an empty
sequence)
• A grammar G generates a language model L.
• N=
• V= N  
Context-free grammars
• eg. N= {S} ={a,b} P= {S aS, S b}
• apply to get aaaab

g1 g1 g1 g1 g2

• S  aS  aaS  aaaS  aaaaS  aaaab

Context-free grammars
 eg “ahmed ate the food with a spoon”
 S  NP VP  NP  ahmed
 NP  art N  N  food
 NP  NP PP  N  spoon
 VP  V NP  V  ate
 VP  VP PP  P  with
 PP  P NP  art  the
 art  a
Nonterminals Terminals
 S : sent.  ahmed
 NP: noun phrase  food
 VP: verb phrase  spoon
 art: articulation  ate
 N: noun  with
 PP: preposition phrase  the
 V: verb  a
 P: preposition
 What is Parsing?
S  NP VP NP  ahmed
NP  art N S N  food
NP  NP PP N  spoon
VP  V NP V  ate
VP  VP PP NP VP P  with
PP  P NP art  the
art  a
VP PP

V NP P NP

Det
art N Det N
art

ahmed ate the food with a spoon

 Top Down Space
25

S  NP VP
S  Aux NP VP
SVP
VP  V
VP  V NP
NP  PropN
NP  art N
NP  NP PP
VP  VP PP
PP  P NP

art art

12/08/2021
 Bottom-Up Parsing
26

 Of course, we also want trees that cover the input

words. So we might also start with trees that link
up with the words in the right way.
 Then work your way up from there to larger and
larger trees.
 Bottom-Up Search
27
 Bottom-Up Search
28
 Bottom-Up Search
29
 Bottom-Up Search
30
 Bottom-Up Search
31
Transition Network Parsers
 Grammar is a finite state machine
 Each network= transition diagram for each
nontermenal
 Each arc = transition or terminal
 Nodes are states
 Top down parse
Transition Network Parsers
 Here’s a very simple example
S  NP VP My dog ate a frog.
VP  VERB NP
NP  ART NOUN NOUN  frog | dog
NP  POSS NOUN ART  a
VERB  ate
POSS  my
Parse tree

S
The tree notation is difficult to compute
with directly, so we can convert the
representation into more useful:
NP VP
(S (NP (POSS my)
(NOUN dog))
(VP (VERB ate)
POSS NOUN VERB NP (NP (ART a)
(NOUN frog))))

ART NOUN

my dog ate a frog

Transition Network

NP VP
S S0 S1 S2

VERB NP
VP VP0 VP1 VP2

ART NOUN ART

NP NP0 NP1 NP2
NOUN
NP0 NP1 NP2
POSS NOUN
NP NP3 NP4 NP5
POSS
ART ART
NOUN VERB NOUN
S0 S1 S2 S3 S4 S5

POSS POSS
When the lexicon gets really big, drawing them takes forever!
 Transition Network
NP VP
S

V NP
VP

art
N
NP
frog
poss ate
N
V
dog

my
a
art poss
Transition Network
NP VP
S

art N V NP
NP VP

poss ate art N

frog V NP
my
N poss
poss frog
dog a
art N
dog
Why do we want recursive rules in our grammar?

 Natural languages allow us to express an infinite

range of ideas using a finite set of rules and symbols.
 The boy drowned.
 The boy with the raft drowned.
 The boy with the raft near the island drowned.
 The boy with the raft near the island in the ocean drowned.
 The boy ....
A sample recursive transition network(1)

The boy broke the window with a hammer.

Transition Network Paresers
Thank You