Module 1

Introduction to NLP

Weightage 8(Theory)

By Priyanka Bhoir
Languag
es
◦Human to
Human

◦Machine to
machine
Natural
Language
◦ Languages
that are
spoken
naturally in
human being
 Natural Language Processing

 Natural Language Processing is the ability of computer program to understand human

language as it is spoken.

 NLP is a part of Computer Science, Human language, and Artificial Intelligence. It is the
technology that is used by machines to understand, analyze, manipulate, and interpret
human's languages.
 Applications

Spell Spam Detection: Spam detection is

Checker used to detect unwanted e-mails getting to a
user's inbox.
Question Answering:
Question Answering focuses on building systems that automatically answer the questions asked by
humans in a natural language.
Text
Prediction
Search Engine Automatic Reply in
Optimization(SEO) mail
 Information extraction : It is the process of extracting information
◦ Grammar from unstructured textual sources to enable finding entities as well
as classifying and storing them in a database.
correction
Sentiment
Analysis
Machine Translation
Machine translation is used to translate text or speech from one natural
language to another natural language.
Example: Google Translator
Chatbo
t Machine
Translation
Google Voice based assistant-Google
Duplex
 INPUT Text OUTPUT
(Document (Text/
/Paragraph Rating
/Sentense /
/word} Graph
/
Audio)

NLP
System
 Speech
Audio
Processing Text
(Document OUTPUT
/Paragraph (Text/
/Sen tense Rating
/word} /Graph)

◦NLP
System
 Speech
Audio
Processing Text
(Document OUTPUT
/Paragraph (Text/
/Sen tense Rating
OCR /word} /Graph)
Image Processing

NLP
System
Steps in NLP
Morphological
Analysis Word-Level
Analysis
Syntactic
analysis Sentence-Level
Analysis
Semantic Sentence-Level
Analysis
Analysis
Discourse Sentence-Level
Analysis Analysis
Pragmatic Sentence-Level
Analysis Analysis
Steps in NLP 1.
Tokenization
Morphological John ate the pizza ! !
Analysis
Syntactic John
ate the pizza !
analysis
!
Semantic 2. Stop Word
Analysis Removal
Discourse John
Analysis
ate
Pragmatic
Analysis pizz
a
Steps in NLP
Morphological 3.
Analysis Stemming
Stemming is a process of reducing
Syntactic words into its base form (Root
form/stem form)
analysis John-
>John Ate
Semantic -> eat
Analysis Pizza-
>Pizza
Discourse
Analysis

Pragmatic
Analysis
Steps in NLP
Morphological 4. N-Gram Language
Analysis Model
1- John Ate the Pizz
Syntactic gram- a
analysis
Bigram John At the Pizza
Semantic - e
Analysis Trigram Joh Ate Pizza
- n the
Discourse
Analysis 4-Gram John Ate the Pizza
-
Pragmatic
Analysis
John Ate the
Steps in NLP
Morphological
Analysis
Syntactic
analysis
Semantic
Analysis

Discourse

Analysis
Steps in NLP
Morphological
Analysis John Ate the Apple

Syntactic
analysis
Ate the Apple John
Semantic
Analysis

Discourse

Analysis
Steps in NLP John Ate the Apple

Morphological
Analysis
Ate the Apple John
Syntactic
analysis
Semantic
Analysis

Discourse

Analysis
Steps in NLP
Morphological
Analysis She drank Some
Milk
Syntactic
analysis She drank Some
books
Semantic
Analysis
Discourse
Analysis

Pragmatic
Analysis
Steps in NLP She drank Some
Milk
Morphological
She drank Some books
Analysis
Drink Book
Syntactic
Verb Noun
analysis
Swallow, Imbibe A set of written sheets
Semantic
Analysis To take in A long written
or receive or printed
Discourse To join in a literary
A record of a
Analysis toast business's
financial
Pragmatic transactions
Nou Ver
Analysis n bo reserve,
A liquid T
suitable for register
swallowing
Steps in NLP She drank Some
Milk

Morphological
Analysis She drank Some
books
Syntactic
analysis
Semantic
Analysis
Discourse
Analysis

Pragmatic
Analysis
Steps in NLP
Morphological
Analysis • Monkeys Eat
Banana, when
Syntactic they Wake up.
analysis Who is they
here?
Semantic
-Monkey
Analysis
• Monkeys eat
Discourse Banana,
Analysis when they are ripe.
Who is they
Pragmatic here?
Analysis -Banana
Steps in NLP
Morphological
Analysis Close the
Door
-
Syntactic
Order
analysis Please Close the
Door
Semantic -
Analysis Request ,affirmatio
n
Discourse
Analysis
Pragmatic
Analysis
1. Lexical Analysis and Morphological
The first phase of NLP is the Lexical Analysis. This phase scans the source code as a stream of
characters and converts it into meaningful lexemes. It divides the whole text into paragraphs,
sentences, and words.

2. Syntactic Analysis (Parsing)

Syntactic Analysis is used to check grammar, word arrangements, and shows the relationship
among the words.
Example: Agra goes to the Poonam
In the real world, Agra goes to the Poonam, does not make any sense, so this sentence is
rejected by the Syntactic analyzer.

3. Semantic Analysis
Semantic analysis is concerned with the meaning representation. It mainly focuses on the literal
meaning of words, phrases, and sentences.

4. Discourse Integration
Discourse Integration depends upon the sentences that proceeds it and also invokes the
meaning of the sentences that follow it.

5. Pragmatic Analysis
Pragmatic is the fifth and last phase of NLP. It helps you to discover the intended effect by
applying a set of rules that characterize cooperative dialogues.
For Example: "Open the door" is interpreted as a request instead of an order.
Ambiguities
◦Word ,Phrase or Sentence is ambiguous if it
has more than one meaning
Ambiguities in
NLP
Morphological/
◦Word category
Lexical Ambiguity disambiguity
◦Word has more than
one
meaning / category
Syntactic
Ambiguity ◦E.g.
Semantic ◦ Book – Noun –Textbook/Novel
Ambiguity ◦ Book – Verb - Book
ticket/seat
Discourse
◦ Bank-Noun –Financial institute
Ambiguity
◦ Bank-Noun - River Bank
Pragmatic ◦ Bank- Verb- Banking
Ambiguity Transaction
 Ambiguity in sentence
Ambiguities in structure

NLP
Morphological
• Specify the possible arrangements
of words in sentence used
Ambiguity • Grammar or rules are ambiguous

Syntactic
Ambiguity
Semantic
Ambiguity

Discourse

Ambiguity
Ambiguities in
NLP Word Sense
Morphological Disambiguity
Ambiguity • Each word in the
Syntactic sentence having more
Ambiguity than one meaning
• If a sentence has 10
Semantic words each having 3
Ambiguity meaning associated
• No. of possible
Discourse
Ambiguity interpretation
3*3* 3*3* 3*3* 3*3* 3*3=59049
Pragmatic
Ambiguity Ram is in a terrible state
(state: mental state
State:maharashtra,gujrat etc)
Ambiguities in Anaphoric Ambiguity
• Monkeys Eat Banana, when
NLP they Wake up.
Morphological
Ambiguity
Who is they
Syntactic here?
Ambiguity -Monkey
• Monkeys eat Banana, when
Semantic they are ripe.
Ambiguity
Discourse Who is they
Ambiguity here?
-Banana
Pragmatic Identify When, Where, by
Ambiguity whom occurrence was said
Ambiguities in
NLP
Morphological
Ambiguity Understanding speakers
intention.
Syntactic Eg. You are Late

Ambiguity • Informative
• Criticizing
Semantic
Ambiguity
Discourse
Ambiguity

Pragmatic
Ambiguity
Challenges in
NLP
◦ Elongated words

◦ I am sooooo sorry.
◦ It was toooo yummy.

◦ Shortcuts

◦ Pls Please
- By the
◦ BTW way What
- OK
◦ Wat
-
◦K-
Challenges in
NLP
◦Emoji
s
Challenges in
NLP
◦Mix Use of Languages

◦ I liked that movie. Salman khan ka acting

was Lajabab. Too good. जबरदस्त.
Challenges in
NLP
◦Ellipsis
◦ Peter worked hard and passed exam, Kevin
too.

◦ Interpretation
◦ Kevin worked hard
◦ Kevin passed exam
◦ Kevin worked hard also he passed exam
Challenges in
NLP
◦Punctuational Ambiguity

◦Women, without her man, is

nothing.

◦Women! without her, man is

nothing.
Student
Projects
◦ Car Rating and Sentiment
analysis
Student
Projects
◦ Online Paper
Assessment
 History of NLP :
http://psych.fullerton.edu/mbirnbaum/psych101/eliza.htm
Thank You