Natural Language

Processing (CS735E01) – 7 th

Semester
Mithun B N
Asst. Prof
 Unit 1:
Introduction to
Natural Language Processing
Agenda
• The Study of language
• Applications of Natural language understanding
• Evaluating Language understanding system
• The different levels of language analysis
• Representation and Understanding
The study of Language
• Language is on the fundamental aspects of human behaviour.
• It is a crucial component of our lives.
• It could be written or spoken form:
• Written form would serve as long term record for knowledge from one
generation to the next.
• Spoken form serves as our primary means of coordinating our day-to-day
behaviour with others.
The study of Language (Contd.,) Goals of
NLP research
• Create computational models of language
• Able to specify models that approach human performance in the
linguistic tasks.
• Linguistic tasks are: reading, writing, hearing and speaking.
• Computational model are useful for scientific and practical purposes
• Scientific purpose – nature of linguistic communication
• Practical purpose – enabling effective human machine interaction
The study of Language (Contd.,)
Different academic disciplines
• The linguist – structure of the language
• The Philosopher – how words mean anything at all and not
• The Psycholinguists – process of human language production and
comprehension
• The Computational Linguists – computational theory of language
Applications of Natural Language
Processing
• Text based application
• Dialogue based application
Text Based Applications
• Processing of written text – books, newspapers, reports, manuals,
email messages and so on.
• Finding appropriate documents on certain topics from database of
texts
• Extracting information from messages or articles on certain topics
• Translating documents from one language to another
• Summarizing texts for certain purposes
Text Based Applications (Contd.,)
• All may not use NLP, articles can be retrieved by simple matching
technique.
• Machine translation systems built on pattern matching - fails as it is
inability to use an understanding of content to disambiguate word
senses and sentence meanings
• Instead, produce a representation of meaning of each sentence in one
language and then producing a sentence in the other language that
realizes the same meaning
Dialogue Based applications
• Involve human machine communication
• Question-answering systems, where natural language is used to query
a database
• Automated customer service over the telephone
• Tutoring systems, where the maching interacts with a student
• Spoken language control of the machine
• General cooperative problem solving system

• Important to distinguish the problems of speech recognition from the

problems of language understanding
 A Dialogue with Eliza
User ELIZA
Men are all alike In what way?
They are always bugging us about something or Can you think of a specific example?
other
Well, my boy friend made me come here Your boy friend made you come here
He says I am depresses much of the time I am sorry to hear you are depressed
It’s true, I am unhappy Do you think coming here will help you not to be
unhappy?
I need some help that much seems certain What would it mean to you if you got some help

Perhaps I could learn to get along with my mother Tell me more about your family

My mother takes care of me Who else in your family takes care of you?
My father Your father
You are like my father in some ways What resemblance do you see?
 The Different Levels of Language
Analysis
• Considerable knowledge about the structure of the language
• what words are, how words are combined to sentences, their meaning.
Different forms of knowledge relevant for NLP:
• Phonetic and phonological knowledge: how words are related to
sounds
• Morphological Knowledge: how words are constructed from
more basic meaning units called morphemes (primitive unit of
meaning in a language)
• Syntactic knowledge: how words can be put together to form
correct sentences and structural roles
• Semantic knowledge: what words mean and how these
meanings combine in sentences to form sentence meanings
The Different Levels of Language Analysis

• Pragmatic knowledge: concerns how sentence are used in

different situations and it affects interpretation of the
sentence.
• Discourse Knowledge: concerns how the immediately preceding
sentence affect the interpretation of the next sentence.
• World knowledge: includes the general knowledge about the
structure of the world that language users must have in order to.
Example: syntax, semantics and pragmatics
1. Language is one of the fundamental aspects of human behaviour
and is a crucial component of our lives
2. Green frogs have large noses
3. Green ideas have large noses
4. Large have green ideas nose.
Sentence 1 is reasonable; has syntax semantics and pragmatics.
Sentence 2 is well formed syntactically and semantically but not
pragmatically.
Sentence 3 is semantically and pragmatically ill formed.
Sentence 4 is unintelligible

If I ask you where are you going and you reply – “ I go store”
Representation and Understanding
• Most words have multiple meanings – called as senses.
• Cook – has a sense as a verb and a sense as a noun
• Dish – as a noun and as a verb

• Useful representation languages have two properties:

• representation must be precise and unambiguous.
• representation should capture intutitive structure of the NL sentences that it
represents
Representation and Understanding:
Syntax – Representing Sentence
Structure
• Syntactic structure of a sentence indicates the way that words in the
sentence are related to each other.
• NLP should be able to understand ill-formed sentences.
• Example:
• John sold the book to mary
• The book was sold to mary by john
• *After it fell in the river, John sold Mary the book.
• After it fell in the river, the book was sold to mary by john
• *John are in the corner
• *John put the book
Representation and Understanding:
Syntax – Representing Sentence
Structure
• Flying planes are dangerous
• Flying planes is dangerous

Languages are based on the notion of CFG, representing sentence

structure in terms of what phrases are subpars of other phrases.
Representation and Understanding:
The Logical Form
• The structure of a sentence doen’t reflect its meaning.
• The intended meaning of a sentence depends on the situation in
which the sentence is produced.
• Context dependent meaning and context independent meaning.
• The representation of the context independent meeing of a sentence
is called its logical form.

Jack invited Mary to the Halloween ball.

The Organization of Natural Language
Understanding
• Syntactic structure and logical form is called as Parser
• Is uses knowledge about word and word meanings (the lexicon) and a
set of rules defining the legal structures (the grammar) in order to
assign a syntactic structure and logical form to an input sentence

• Visiting relatives can be trying

• Visiting museums can be trying

• Above two sentences have identical syntactic structure, and are

syntactically ambiguous
• First sentence might be relatives who are visiting you or the event of you
visiting relatives
• Both of these alternatives are semantically valid and you would need to
determine the appropriate sense by using the contextual mechanism
• Second sentence has only one possible semantic interpretation, since
museums are not objects that can visit other people, rather they must be
visited.
• If syntactic and semantic processing are combined the system will be able
to detect the semantic anomaly as soon as it interprets the phrase
visiting museums
• The grammar that can be used to identify the structure of a given
sentence or to realize a structure of words is called as bidirectional
grammar
Linguistic Background: An Outline of English Syntax
Words:
• Word – basic unit of linguistic structure.
• Morphology – concerns the construction of words from more basic
components.
• Two ways of forming words:
• Inflectional – use a root from of a word and add suffix to make appropriate
form. Verbs are the best example: give gives giving given (shares same
basic meaning.
• Derivational – derivation of new words from other forms. New words may be
completely different categories. Friend friendly friendliness
Words (Contd.,)
• Traditionally words are classified into different categories based on
their uses.
• First – word’s contribution to the meaning of the phrase that contains it
• Second – the actual syntactic structure in which the word play a role
• Noun and Adjective
• Noun – identify the basic type of object
• Adjective – qualify the object, concept of place
• The green book. Green books
• That green is lighter than the other
• Modifiers can be noun modifier and adjective modifier.
Words (Contd.,)
• Four main classes of words
• Nouns
• Verbs
• Adjectives
• Adverbs
• Other classes
• Articles
• Pronouns
• Prepositions
• Particles
• Quantifiers
• Conjunctions and so on.
Words (Contd.,)
• Head of the Phrase: A word in any of four open classes:
• Noun Phrase:
• The elephant
• An elderly elephant
• The angry elephant killed two men
• Adjective Phrase
• Thirsty
• Very thirsty
• Thirsty made him to do so
Words (Contd.,)
• Sometimes head requires additional phrases following it to express
the desired meaning.
• Example: ‘put’ cannot form a verb phrase in isolation.
• John put - cannot be a sentence.
• Instead, john put dog in the house.
• Sentence can be completed using complement
• The phrase or set of phrases needed to complete the meaning of
head is called the complement.
• In the above example, house is complement.
 Words (Contd.,)
Noun Phrases Verb Phrases
The president of the company Looked up the chimney
His desire to succeed Believed that the world was flat
Several challenges from the opposing Ate the pizza
team

Adjective Phrases Adverbial Phrases

It is easy to assemble Rapidly like a bat
I am happy that he won the prize Intermittently throughout the day
He is angry as a hippo Inside the house
The elements of simple noun
phrases
• Noun phrases (NPs) are used to refer to things: objects, places,
concepts, events, qualities and so on.
• The simplest NP consists of a single pronoun. Pronouns can refer to
physical objects, to objects, to qualities and sometimes takes any
modifiers.
• It hid under the rug
• Once I opened the door, I regretted it for months
• He was so angry, but he didn’t show it.
• He who hesitates is lost. (pronouns taking modifiers very rarely)
The elements of simple noun
phrases
• Proper noun is also used.
• Head of noun phrase is usually a common noun.
• Nouns divide into two main classes:
• Count nouns – describe specific objects or sets of objects
• Mass nouns – describe composites or substances
• Noun phrase may contain specifiers and qualifiers preceding the
head.
• Qualifiers describe the general class of objects identified by the head.
• Specifiers describe how many such objects are described.
The elements of simple noun
phrases
• Specifiers are constructed out of
• Ordinals like first and second
• Cardinals like one and two
• Determiners
• Determiners has following divisions
• Articles – a, an and the.
• Demonstratives – this, that, these and those.
• Possessives – noun phrases followed by the suffix ‘s. Ajay’s . Possessive
pronouns are her, my and whose.
• Wh-determiners – words used in questions, which and what.
• Quantifying determiners – some, every, most so, any, both and half.
 The elements of simple noun
phrases
• A simple noun phrase may have at most one determiner, one ordinal and
one cardinal.
• A sentence may have all three – first three contestants.
• Sentence may contain few quantifying determiners – few songs we knew
• The qualifiers in a noun phrase occur after the specifiers (if any) and before
the head. They consist of adjectives and nouns being used as modifiers.
• Adjectives – words that attribute qualities to objects yet do not refer to the qualities
themselves. Angry is an adjective that attributes the quality of anger to something
• Noun modifiers – mass or count nouns used to modify another noun. Cook book or
the ceiling paint can.
The elements of simple noun
phrases
• Pronouns take forms based on :
• Person – first, second and third.
• Gender – masculine, feminine and neutral.
• In some languages nouns are classified by their gender (French)
• Pronouns distinguish number, person, gender and case (possessive,
subject or object)
The elements of simple noun
phrases
Pronouns as subject

Number First person Second person Third person

Singular I You He, she it
Plural We You they

Pronouns possessiveness
Number First person Second person Third person
Singular My Your His, her, its
Plural our Your Their

Number First person Second person Third person

Singular Me You Him, her, it
Plural us you them
Verb Phrases and Simple sentences
• A sentence is used to assert, query or command.
• Mood – The way a sentence is used
Mood Example
Declarative or assertion The cat is sleeping
Yes/no question Is the cat sleeping?
Wh-question What is sleeping? Or which cat is
sleeping?
Imperative (or command) Shoot the cat!
Verb Phrases and Simple sentences
• Verbs can be divided into several different classes:
• The auxiliary verbs – be, do and have
• The modal verbs – will, can and could
• The main verbs – eat, ran and believe