NLP

Explain the various stages of Natural Language processing

1. Lexical and Morphological Analysis
• Tokenization: Breaking text into smaller pieces like words or sentences.
• Lemmatization and Stemming: Reducing words to their base form (e.g., "running" to "run").
• Stopwords Removal: Removing common words that don’t add much meaning (e.g., "the",
"and").
• Morphological Analysis: Understanding the structure of words, focusing on morphemes
(smallest meaningful parts of a word, like prefixes or suffixes).
2. Syntactic Analysis (Parsing)
• Sentence Structure: Understanding how words are arranged in a sentence to form a
meaningful structure.
• POS Tagging: Labeling each word with its part of speech (noun, verb, etc.).
• Syntax Checking: Ensuring the sentence follows grammatical rules.
3. Semantic Analysis
• Meaning of Words and Sentences: Understanding the meaning of words in context.
• Named Entity Recognition (NER): Identifying people, places, or organizations.
• Word Sense Disambiguation: Determining the correct meaning of ambiguous words (e.g.,
"bank" as a financial institution or riverbank).
4. Discourse Integration
• Contextual Understanding: Understanding how sentences relate to each other in a larger
text or conversation.
• Anaphora Resolution: Figuring out what pronouns refer to (e.g., "She" refers to "Taylor").
• Understanding Connections: Ensuring the overall flow and meaning of a text are coherent.
5. Pragmatic Analysis
• Inferred Meaning: Going beyond the literal meaning to understand the deeper intentions or
emotions.
• Context and Tone: Recognizing that words can have different meanings based on how
they’re used (e.g., "Hello" can be a greeting, expression of surprise, or even frustration).
• Intentions and Implications: Understanding what the speaker truly means in a given
situation.

Explain the applications of Natural Language processing

Natural Language Processing (NLP) has a wide range of applications across various fields. Here are
some of the key areas where NLP is applied:
1. Chatbots and Virtual Assistants: NLP is used to develop chatbots and virtual assistants like
Siri, Alexa, and Google Assistant. These systems understand and respond to user queries in
natural language.
2. Sentiment Analysis: Businesses and organizations use NLP to analyze customer feedback,
social media posts, and reviews to gauge public sentiment. This helps in understanding
customer satisfaction, brand perception, and trends.
3. Machine Translation: NLP is central to translating text between different languages.
Applications like Google Translate and DeepL use NLP algorithms to convert one language
into another.
 4. Speech Recognition: NLP techniques are employed in converting spoken language into text.
This is used in voice-to-text applications, transcription services, and voice-activated controls.
5. Text Summarization: NLP is used to condense large volumes of text into shorter, coherent
summaries. It’s useful in applications like news aggregation, research paper summarization,
and content curation.
6. Information Extraction: NLP helps in extracting structured information from unstructured
text, such as extracting key phrases, names, dates, and locations from documents.
7. Automatic Text Generation: NLP enables automatic generation of text, like in creative
writing, coding assistants, and automatic content creation tools. Examples include AI writing
assistants like Grammarly or tools like GPT-3, which can generate articles, reports, and
scripts.
8. Question Answering Systems: NLP is used to develop systems that can understand and
answer user questions. These are found in customer support systems, search engines, and
specialized databases.
9. Text Classification: NLP helps in categorizing text into predefined labels. It’s widely used in
spam detection, content filtering, and topic classification.
10. Named Entity Recognition (NER): NLP identifies and classifies entities such as names of
people, organizations, locations, and more within a text. This is useful in information retrieval
and organizing content.
11. Content Recommendation Systems: NLP can analyze user preferences and recommend
content such as books, movies, or articles based on natural language queries or past
behavior.
12. Healthcare: NLP is used to extract relevant information from clinical records, enabling better
decision-making and improving patient care. It also aids in the development of virtual health
assistants.

Illustrate the concept of tokenization and stemming in Natural Language

processing
Tokenization and stemming are two fundamental concepts in Natural Language Processing (NLP).
Both are used to process text and help computers understand and manipulate human language.
Here's a detailed explanation of each:
1. Tokenization
Tokenization is the process of breaking down text into smaller units, called tokens. These tokens can
be words, phrases, or even sentences, depending on the level of tokenization. The goal of
tokenization is to simplify text into manageable chunks for further analysis.
Types of Tokenization:
• Word Tokenization: The text is split into individual words. For example, the sentence "I love
programming" would be tokenized into ["I", "love", "programming"].
• Sentence Tokenization: The text is split into individual sentences. For example, the
paragraph "I love programming. It's fun." would be tokenized into ["I love programming.",
"It's fun."].
Example of Tokenization: Input:
"ChatGPT is amazing at natural language processing."
Word-level tokenization would result in the following tokens:
["ChatGPT", "is", "amazing", "at", "natural", "language", "processing"]
Sentence-level tokenization might split a longer paragraph into individual sentences.
Tokenization is a key step in NLP because it helps computers process the text into units that can be
analyzed further.
2. Stemming
Stemming is the process of reducing words to their root or base form, which helps in grouping similar
words together. The idea is to remove suffixes (like -ing, -ed, -es) and reduce words to their "stem" or
core meaning. Stemming can be helpful for applications like information retrieval, where you want to
match words based on their root.
Example of Stemming:
• "running" → "run"
• "plays" → "play"
• "happily" → "happi"
Different stemming algorithms can be used, such as the Porter Stemmer, which is one of the most
common. However, stemming can sometimes lead to non-dictionary words, because it just removes
affixes and does not check if the root word is valid.
Example of Stemming Process: Input:
"I am enjoying playing football and running."
After stemming:
["I", "am", "enjoy", "play", "footbal", "and", "run"]
Notice that "enjoying" was reduced to "enjoy," "playing" to "play," and "football" became "footbal"
(which is not a valid word, but the stemmer typically doesn’t guarantee dictionary correctness).

Lemmatization
Lemmatization is a process in Natural Language Processing (NLP) that, like stemming, reduces words
to their base or root form. However, unlike stemming, which simply strips off suffixes, lemmatization
uses vocabulary and morphological analysis to find the correct base or dictionary form of a word,
called a lemma.
The key difference between lemmatization and stemming is that lemmatization ensures that the
output is a valid word in the language, while stemming may produce non-existent words.
Lemmatization takes into account the word's meaning and its use in context to derive the correct
lemma.
Example of Lemmatization:
• "running" → "run" (the base form of the verb)
• "better" → "good" (since "better" is the comparative form of "good")
• "leaves" → "leaf" (the singular form of "leaves")
• "went" → "go" (the base form of the verb in the past tense)
• "dogs" → "dog" (the singular form of the noun)
Lemmatization vs. Stemming:
• Stemming: Strips off suffixes, may lead to non-dictionary forms.
o "running" → "run" (correct), "better" → "bett" (incorrect).
• Lemmatization: Reduces the word to its proper base form based on its meaning and part of
speech.
o "running" → "run" (correct), "better" → "good" (correct).
How Lemmatization Works:
Lemmatization involves a deeper understanding of the word and its context. It typically uses a
lemmatizer, which is often based on:
• Part-of-speech (POS) tagging: The lemma depends on whether the word is a noun, verb,
adjective, etc.
 o For example, the lemma of "running" (verb) is "run," but the lemma of "running"
(noun) could be "running" (as the action of running).
Example of Lemmatization in Action:
• Input Text:
"The cats are running swiftly across the field."
• After Lemmatization:
"The cat be run swiftly across the field."
The words "cats" and "running" are lemmatized to "cat" and "run," and "are" is reduced to "be," the
base form of the verb.

Open Class Words:

These are words that can easily grow as the language changes. New words can be added to these
categories anytime.
• Examples:
o Nouns: dog, cat, computer, teacher.
o Verbs: run, jump, talk, create.
o Adjectives: tall, happy, beautiful.
o Adverbs: quickly, very, well.
Closed Class Words:
These words don’t change much. There is a fixed set, and no new words are added to these
categories.
• Examples:
o Pronouns: he, she, it, they.
o Prepositions: in, on, at, with.
o Conjunctions: and, but, or, because.
o Determiners: the, a, this, that.

Explain inflectional and derivational morphology with an example (VVIP)

Morphology is the study of the structure of words, including how they are formed and how they can
change. Two key types of morphology are inflectional morphology and derivational morphology.
Both deal with how words change, but they serve different purposes.
1. Inflectional Morphology
Inflectional morphology deals with changes in a wordl that affect its grammatical function, such as
tense, number, gender, case, or person. Inflection doesn’t change the core meaning or the part of
speech of the word; it just shows grammatical relationships.
• Example:
o Word: "walk"
▪ Inflected form: "walks" (third-person singular)
▪ Inflected form: "walked" (past tense)
▪ Inflected form: "walking" (present participle)
In this case, the base word "walk" is inflected to show different tenses or aspects of the verb, but the
core meaning of the word (the action of walking) remains the same.
2. Derivational Morphology
Derivational morphology involves adding prefixes or suffixes to a word to create a new word with a
different meaning or a different part of speech. This process can change the grammatical category
(e.g., from a noun to a verb) or alter the meaning.
• Example:
 o Word: "happy" (adjective)
▪ Derived form: "unhappy" (adjective, with a changed meaning, negating the
original meaning)
▪ Derived form: "happiness" (noun, changing the part of speech from an
adjective to a noun)
▪ Derived form: "happily" (adverb, changing the part of speech to describe
how something is done)
In this case, by adding different prefixes and suffixes, we can change the word "happy" into
"unhappy," "happiness," or "happily," each with a different meaning or grammatical role.
Summary of Differences:
• Inflectional morphology changes a word to express grammatical features without changing
its meaning or part of speech.
o Example: "cats" (plural of "cat").
• Derivational morphology creates new words with new meanings or new parts of speech.
o Example: "friend" → "friendship" (noun, different meaning).

Explain the role of FSA in morphological analysis? (2022)

FSA in Morphological Analysis (in Simple Terms)
A Finite State Automaton (FSA) is a tool used to break down words into smaller parts called
morphemes (the smallest meaningful units like prefixes, roots, and suffixes). In morphological
analysis, FSA helps figure out how words are formed and understand their structure.
How FSA Works in Simple Terms:
1. States as Word Parts:
o The FSA has different states. Each state represents a part of a word. For example,
one state might represent the base word (like "play"), and another state could
represent a suffix (like "-ed" for past tense).
2. Transitions as Word Rules:
o The transitions between states are like rules for how the word parts fit together. For
example, if you add "-ed" to "play," the FSA moves from the "play" state to the
"played" state.
3. Breaking Down Words:
o The FSA takes a word like "played" and reads it one letter at a time. It follows the
states and transitions:
▪ Starts with "play" (root).
▪ Then adds "-ed" (suffix).
▪ It finishes when it reaches an "accepting" state, meaning the word is
correctly understood.
Example:
For the word "played":
1. The FSA starts at the root "play."
2. It sees the suffix "-ed" and moves to the next state.
3. The FSA ends at the "played" state, showing the word is correctly formed from the root
"play" and the suffix "-ed."
Why It’s Useful:
• Fast and Simple: FSAs are quick and easy to use, which makes them great for analyzing many
words at once.
 • Helps with Word Forms: FSA helps understand how different forms of a word (like "play" and
"played") are created by adding suffixes or prefixes.

Illustrate the working of Porter stemmer algorithm

The Porter Stemmer Algorithm is a widely used algorithm in Natural Language Processing (NLP) for
stemming, which means reducing words to their base or root form. The goal is to simplify the word
forms without losing their meaning, and the Porter Stemmer does this by applying a set of rules to
strip common suffixes from words.
Key Concepts:
• Stemming: The process of removing prefixes or suffixes to get the root form of the word.
• Porter Stemmer: An algorithm developed by Martin Porter in 1980, which applies a set of
rules (consisting of conditional steps) to the word to reduce it to its stem.
How the Porter Stemmer Works:
The Porter Stemmer follows a series of steps to remove suffixes in a manner that makes sense for the
word’s structure and meaning. These steps include grouping suffixes and applying rules depending
on the context of the word. Let’s break down how it works using an example:
Example: Word "running"
1. Step 1: Remove common suffixes (e.g., "ing", "es", "ed")
o Rule: If a word ends with "ing", remove the "ing".
o Word: "running" → "run" (because "running" ends in "ing", it removes "ing")
2. Step 2: Handle the plural and possessive forms
o Rule: If the word ends with "s", and the word before "s" is a valid word, remove the
"s".
o For example, "cats" would become "cat", but since "run" is already a valid word, no
change is needed here.
o Word: "cats" → "cat"
3. Step 3: Remove more suffixes based on vowel/consonant patterns
o Rule: Remove suffixes like "ed", "ly", "es" when appropriate. The algorithm checks
whether the removal leaves a valid root word (after considering the surrounding
letters).
o Example: "happiness" becomes "happi" or "hopeful" becomes "hope".
Key Rules Applied by Porter Stemmer:
1. Generalization rule: Remove "ing", "ly", "es", "ed" when appropriate.
o For example: "running" → "run", "happily" → "happi", "cats" → "cat".
2. Special rules: Some words are handled through special rules based on the structure of the
word:
o Example: "caresses" → "caress" (it removes "es").
o Example: "dies" → "die" (removes the "es").

Part of Speech (POS) in NLP

Part of Speech (POS) refers to the grammatical category of words in a language, such as nouns, verbs,
adjectives, etc. Identifying the POS of words in a sentence is crucial for understanding the syntactic
structure and meaning of the text.
In NLP, POS tagging is the process of assigning the correct part of speech to each word in a given
sentence.
Common Parts of Speech:
1. Noun (NOUN): Names of people, places, things, or ideas.
 o Example: dog, happiness, city
2. Pronoun (PRON): Replaces a noun.
o Example: he, she, it, they
3. Verb (VERB): Denotes action or state.
o Example: run, is, write
4. Adjective (ADJ): Describes or modifies a noun.
o Example: beautiful, large, red
5. Adverb (ADV): Modifies verbs, adjectives, or other adverbs.
o Example: quickly, very, silently
Example of POS Tagging:
Sentence: "The quick brown fox jumps over the lazy dog."
POS Tags:
• The → Determiner (DET)
• quick → Adjective (ADJ)
• brown → Adjective (ADJ)
• fox → Noun (NOUN)
• jumps → Verb (VERB)

Challenges in part of speech tagging (2024)

Part of Speech (POS) Tagging is the process of identifying the type of word in a sentence, such as
noun, verb, adjective, etc. While it’s crucial for many language tasks, it comes with several
challenges:
1. Ambiguity
A word can have different meanings depending on the context. For example:
• "run" can be a verb (to move quickly) or a noun (an event). The tagger must figure out which
one to choose based on how the word is used in the sentence.
2. Homophones and Homographs
• Homophones are words that sound the same but have different meanings (e.g., "bare" vs.
"bear").
• Homographs are words that are spelled the same but have different meanings (e.g., "lead"
the metal vs. "lead" to guide). The tagger needs to understand the meaning based on the
context to assign the correct part of speech.
3. Contextual Variability
The same word can change its part of speech depending on the sentence. For example:
• "set" can be a verb ("I will set the table") or a noun ("This is my set of tools"). The tagger has
to recognize the surrounding words to decide which tag is correct.
4. New Words and Slang
New words or slang terms may not be in the tagger’s dictionary. Words like "LOL" or "selfie" may not
have a clear part of speech, making them hard to tag.
5. Multilingual and Domain-Specific Texts
Different languages or specific topics (like medical or legal terms) may use the same word in different
ways. For example, a word like "prescription" might be a noun in regular language but a verb in
medical contexts.
6. Sentence Structure
The way a sentence is arranged can affect POS tagging. For example, “The dog bit the man” and “The
man bit the dog” use the same words, but the meaning changes based on the word order. The tagger
needs to understand sentence structure to assign the right tags.
7. Unseen Words
Sometimes, the tagger encounters words it hasn't seen before, making it hard to assign a POS tag to
those words. For example, a new slang word might confuse the system.
8. Functional Words
Words like "to," "is," and "the" may seem simple but can be tricky to tag correctly depending on how
they are used in a sentence.

Rule-Based POS Tagging

Rule-based POS tagging assigns part-of-speech tags to words using a set of manually crafted linguistic
rules. These rules rely on the syntactic and morphological characteristics of words, such as their
suffixes, prefixes, or context within a sentence.
How Rule-Based Tagging Works:
1. Lexicon Lookup:
o A lexicon (or dictionary) is used to assign a default POS tag to each word.
o Example: "run" might be tagged as a verb (VB) in the lexicon.
2. Rules Application:
o Linguistic rules are applied to refine the tags based on the word's context.
o Rules take into account the relationships between neighboring words in a sentence.
o Example Rules:
▪ If a word ends in "-ly", it is likely an adverb (RB).
▪ If a word follows a determiner (e.g., "the," "a"), it is likely a noun (NN).
Key Components:
1. Lexicon:
o A predefined list of words with their possible tags.
o Example: book → {Noun (NN), Verb (VB)}.
2. Rules:
o Context-sensitive rules are used to resolve ambiguity.
o Examples:
▪ If a word is preceded by an article (a, an, the), tag it as a noun.
▪ If a word follows an auxiliary verb (is, are, was), tag it as a verb.
Advantages:
• Simple to implement and interpret.
• Does not require a large annotated dataset (unlike statistical or neural methods).
• Effective for morphologically rich languages with well-defined rules.

Stochastic POS Tagging

Stochastic POS tagging uses probabilities to decide the correct part-of-speech (POS) tag for a word in
a sentence. It learns patterns from a large dataset of tagged sentences and uses these patterns to tag
new sentences.
How It Works:
1. Learn from Data:
o The model looks at a big collection of sentences where words are already tagged
(e.g., "book/NN" for noun and "run/VB" for verb).
o It calculates:
▪ How often a word is used with each tag.
▪ What tags are likely to follow each other.
2. Apply Probabilities:
 o When tagging a new sentence, it chooses the most likely tag for each word based
on:
▪ The word itself (How is this word usually tagged?).
▪ The surrounding words (What tags come before or after this word?).
Example:
Imagine the word "bank":
• In the sentence "He went to the bank", it’s a noun (NN).
• In "They bank on his support", it’s a verb (VB).
A stochastic tagger looks at the context (other words in the sentence) to decide the correct tag.
Techniques:
1. Unigram Tagging:
o Uses only the word itself.
o Example: "book" is tagged as a noun because it’s usually used as a noun.
2. Bigram/Trigram Tagging:
o Looks at the previous word (bigram) or the two previous words (trigram).
o Example: If "the" comes before "book", it’s likely a noun.
3. Hidden Markov Model (HMM):
o Uses both:
▪ How likely a tag is to follow another tag.
▪ How likely a word is to belong to a specific tag.

How N-gram Helps in Spelling Correction:

1. Context-Based Suggestions: When you type a misspelled word, the N-gram model looks at
the surrounding words to guess what the correct word should be.
o Example:
▪ If you type "teh" instead of "the," the N-gram model might guess that "the"
is the correct word because "the" is often seen in similar contexts.
2. More Frequent Words: The model uses a lot of text data to learn which words or word
combinations are most common. The more common a word or sequence is, the more likely it
is to be the correct choice.
o Example:
▪ The phrase "to the store" is much more common than "to teh store," so the
model will correct "teh" to "the."
3. Common Typos: The model also learns common typing mistakes, like switching letters or
adding extra ones, and can correct them based on what usually happens in normal writing.
o Example:
▪ If you type "accomodation," the N-gram model will notice that
"accommodation" is much more common, so it will suggest the correct
spelling.
4. Word Boundaries: If you accidentally combine words (like "ineedhelp"), the model can help
split them correctly into "I need help."
Example:
• Input: "I went to teh store."
• Bigram Analysis: The model looks at the phrase "went to" and "teh store" and sees that
"teh" is less common than "the."
• Output: "I went to the store."
Transformation-Based Tagging (TBL)
Transformation-Based Tagging, also known as Brill Tagging, is a hybrid approach to Part-of-Speech
(POS) tagging that combines rule-based and statistical methods. It applies a set of transformation
rules to iteratively refine the initial POS tags, improving accuracy.
How It Works:
1. Initialization (Baseline Tagging):
o Start with a simple method to assign an initial tag to each word.
▪ Example: Assign the most frequent tag for each word from a training corpus
(Unigram tagging).
2. Transformation Rules:
o Apply rules to fix tagging errors based on patterns in the data.
o Rules are learned automatically from training data.
▪ Example: If a word is tagged as NN (noun) but follows a determiner (DT),
change the tag to JJ (adjective).
3. Iterative Refinement:
o Rules are applied one by one to refine the tags.
o Each rule improves accuracy by fixing a specific type of error.
4. Stopping Condition:
o Stop when no more rules improve the tagging accuracy or when a predefined
number of iterations is reached.
Key Components:
1. Initial Tagging:
o Tags are assigned using a baseline tagger (like Unigram or default rules).
2. Rules Format:
o Rules follow the format: "Change tag X to tag Y in context Z."
▪ Example: Change "NN" to "VB" if the previous word is "to".
3. Rule Learning:
o Rules are learned by comparing incorrect tags in the baseline output with the correct
tags in the training data.
Example Workflow:
Step 1: Initial Tagging
• Sentence: "The old man walks."
• Initial Tags: The/DT old/NN man/NN walks/VB.
Step 2: Apply Transformation Rules
• Rule 1: If a word is tagged as NN but follows a determiner (DT), change it to JJ.
o Updated Tags: The/DT old/JJ man/NN walks/VB.
• Rule 2: If a word is tagged as VB but ends with "s" and follows a noun, change it to VBZ (verb,
3rd person singular).
o Final Tags: The/DT old/JJ man/NN walks/VBZ.

Hidden Markov Model (HMM) for POS Tagging (Simplified)

A Hidden Markov Model (HMM) is a statistical model used in Part-of-Speech (POS) tagging, where
the goal is to figure out the correct POS tag (such as noun, verb, adjective) for each word in a
sentence. Here's a simplified explanation of how it works:
Key Concepts:
1. States: These are the possible POS tags. For example: NOUN, VERB, ADJ, etc.
 2. Observations: These are the actual words in the sentence. For example: "cat", "runs",
"quickly", etc.
3. Transition Probabilities: This is the probability of one POS tag following another. For
example, how likely is it that a noun is followed by a verb?
4. Emission Probabilities: This is the probability of a word being associated with a certain POS
tag. For example, the word "cat" is most likely a noun.
5. Initial Probabilities: This is the probability that a sentence starts with a certain POS tag. For
example, how likely is it that the first word in a sentence is a noun?
Steps in Using HMM for POS Tagging:
1. Training:
o HMM learns from a labeled dataset (a dataset where each word already has a tag) to
calculate the transition probabilities (e.g., how often a noun is followed by a verb)
and emission probabilities (e.g., how likely it is that the word "runs" is a verb).
2. Tagging a Sentence:
o Given a sentence (like "She runs fast."), the goal is to figure out the most likely POS
tags for each word.
o Viterbi Algorithm: This is the main algorithm used in HMM. It helps find the best
possible sequence of POS tags by calculating the probabilities of each tag for each
word in the sentence.
The algorithm works by:
o Starting with the first word and figuring out which tag is most likely.
o Moving to the next word and calculating the best tag based on the previous word's
tag.
o Repeating this process for every word in the sentence.
Example (Simplified):
• Sentence: "She runs fast."
• Step 1: Calculate probabilities:
o Transition probabilities: How likely is it that a noun will be followed by a verb?
o Emission probabilities: How likely is it that the word "runs" is a verb?
• Step 2: Use Viterbi Algorithm to figure out the best tags for each word:
o "She" → likely a pronoun (or noun).
o "runs" → likely a verb.
o "fast" → likely an adjective or adverb.
Result:
The algorithm gives you the most probable tags for each word:
• "She" → PRONOUN
• "runs" → VERB
• "fast" → ADVERB
Why Use HMM for POS Tagging?
• Context-based: HMM looks at the entire sentence to figure out which tag fits best for each
word, not just the word itself.
• Probabilistic: It uses probabilities to make predictions, meaning it's often quite accurate
when trained with enough data.
Issues in HMM for POS Tagging
Hidden Markov Models (HMMs) are good for POS tagging, but they have some problems that make
them less effective in certain situations. Here's a simpler explanation of these issues:
1. Limited Context Understanding
• HMM assumes that a word’s tag depends only on the tag of the previous word. It doesn’t
consider the entire sentence or distant words for better context.
o Example: In the sentence "The book that I read was interesting," the relationship
between "book" and "read" isn't captured well by HMM.
2. Struggles with New Words
• If a word wasn’t seen during training (like new or rare words), HMM has no idea how to tag
it.
o Example: If the word "blockchain" doesn’t exist in the training data, HMM might tag
it incorrectly.
3. Ambiguity
• Some words can have more than one tag depending on their use in a sentence (this is called
ambiguity). HMM can struggle to pick the correct tag without enough context.
o Example: The word "play" can be a verb ("I play football") or a noun ("The play was
great").
4. Relies Too Much on Training Data
• HMM relies heavily on probabilities learned from the training data. If the training data is
small or biased, it may not work well for new or different sentences.
5. Can't Use Extra Features
• HMM can’t use useful clues like:
o Whether a word starts with a capital letter.
o Whether it ends with "-ing" (often verbs).
o Word meanings or modern embeddings (like word2vec).
6. Computational Challenges
• When there are many possible tags, HMM becomes slower because it has to calculate
probabilities for more combinations.
7. Poor Handling of Long Dependencies
• HMM can’t understand long-range relationships in sentences.
o Example: In "The boy who likes football is here," HMM won’t realize "boy" matches
with "is."
8. Older Compared to New Methods
• Newer methods like neural networks (LSTMs or Transformers) are better at understanding
context and handling complex sentences.

Discriminative Models
A discriminative model is a type of machine learning model that directly learns the boundary
between different classes (e.g., tags, categories, etc.) rather than modeling the distribution of each
class separately. In other words, it focuses on what makes one class different from another.
How Discriminative Models Work:
Discriminative models learn the relationship between the input (e.g., words in a sentence) and the
output (e.g., part-of-speech tags) directly. They try to predict the boundary between different classes
rather than the overall distribution of the data.
• Focus: Given an input, the model learns to classify it into one of the possible categories
based on the input features.
 • Formula: The model learns the conditional probability P(y∣x)P(y | x)P(y∣x), which represents
the probability of the output (label) yyy given the input xxx.
Discriminative vs. Generative Models
• Discriminative Models: Learn the boundary between classes.
o Example: Logistic Regression, Support Vector Machines (SVM), Conditional Random
Fields (CRF).
• Generative Models: Learn how the data is generated (both input and output) and model the
entire data distribution.
o Example: Naive Bayes, Hidden Markov Models (HMM).
Key Difference:
• Discriminative models try to predict directly: “What is the most likely output for this input?”
• Generative models try to model the whole data: “How did this input-output pair come to
be?”

Maximum Entropy Model

The Maximum Entropy Model is a machine learning approach that makes predictions by choosing
the least biased solution, given the available data. In simple terms, it tries to make the least number
of assumptions about the data except for what it knows.
How It Works:
1. Features:
The model looks at features (or characteristics) of the data that can help predict the
outcome. For example, in Part-of-Speech (POS) tagging, the features could be the word
itself, the word's position in the sentence, or neighboring words.
2. Maximum Entropy:
The model tries to maximize the entropy (uncertainty) when making its predictions. This
means it doesn’t assume any unnecessary information about the input data that it doesn’t
know. It only uses what the data provides to make the best prediction.
Example: POS Tagging with Maximum Entropy
Let’s take the sentence:
"The cat sat."
The task is to figure out the correct Part-of-Speech (POS) tag for the word "cat." The tags we are
interested in could be:
• Noun (NN) for "cat"
• Verb (VB) for "cat"
• Adjective (JJ) for "cat"
Now, the model looks at features that might help decide whether "cat" is a noun, verb, or adjective:
• The word itself: "cat"
• The previous word: "The" (this is useful because "The" is often followed by a noun)
• The next word: "sat" (this helps because "sat" is a verb, suggesting "cat" could be a noun)
Step-by-Step Process:
1. Identify Features:
o Word "cat"
o Previous word: "The"
o Next word: "sat"
o Whether the word "cat" is capitalized (could help identify if it's a proper noun)
2. Assign Weights: The model assigns weights to these features, based on how important they
are for predicting the tag. For example:
 o The previous word "The" might increase the probability of "cat" being a noun.
o The next word "sat" might also increase the probability of "cat" being a noun, since
"sat" is often associated with a subject.
3. Maximize Entropy: The model then calculates the probabilities of each tag (noun, verb,
adjective) based on the features. It aims to choose the tag that is most likely given the
features but doesn't make any extra assumptions.
4. Prediction: Finally, the model selects the tag with the highest probability. For example, it
might predict that "cat" is a noun (NN) because the features support this (it’s preceded by
"The" and followed by a verb "sat").

Conditional Random Field (CRF)

A Conditional Random Field (CRF) is a discriminative probabilistic model used for labeling and
segmenting sequences of data, like Part-of-Speech (POS) tagging, named entity recognition, or
image segmentation. CRFs are useful for tasks where the output labels are interdependent, meaning
the prediction for one label depends on the neighboring labels in the sequence.
• CRFs are sequential models that predict the best sequence of labels based on the input
data, such as words in a sentence.
• The main strength of CRFs is that they consider the relationships between neighboring
labels. For example, in a sentence, if one word is tagged as a noun, the word next to it is
more likely to be tagged as a verb (based on context).
How Does It Work?
1. Sequence of Data:
CRFs work on sequences of data. For example, in POS tagging, the input is a sequence of
words, and the output is a sequence of tags (e.g., noun, verb, adjective).
2. Labels Depend on Each Other:
The label for one word (e.g., "cat") is not decided in isolation. It depends on the labels of
neighboring words. For example, "cat" is likely to be tagged as a noun if it follows "The" (a
determiner).
3. Features:
CRFs use features to help make predictions. Features can be:
o The current word itself (e.g., "cat").
o The previous word (e.g., "The" before "cat").
o The next word (e.g., "sat" after "cat").
o The previous tag (e.g., if the previous word is tagged as a determiner, "cat" is likely to
be a noun).
4. Conditional Probability:
CRFs calculate the conditional probability of the label sequence, given the input sequence
(e.g., the sentence "The cat sat"). They aim to maximize the likelihood of the correct labels
based on the features.

What is a Parser in NLP?

A parser in Natural Language Processing (NLP) is a tool that takes sentences as input and generates
a structure that represents the grammatical structure of the sentence. This structure is called a parse
tree or syntax tree.
A parser helps break down a sentence into its components (like subject, verb, object) and shows how
they relate to each other. This is crucial for tasks like machine translation, question answering, and
information extraction.
Top-Down Parser: Simple Explanation
A Top-Down Parser is a type of syntax parser that tries to generate a sentence starting from the
highest-level structure (the root of the sentence, often called the "start symbol") and works its way
down to the actual words (the leaves of the tree). It begins with the goal of producing the whole
sentence and then breaks that goal into smaller sub-goals, following the rules of a grammar.
How Does a Top-Down Parser Work?
1. Start with the Start Symbol:
It begins with the start symbol of the grammar (often S for sentence). The start symbol
represents the entire sentence.
2. Break the Sentence into Subparts:
The parser uses grammar rules (also called production rules) to break the start symbol into
smaller parts (e.g., a sentence can be broken down into a noun phrase (NP) and a verb
phrase (VP)).
3. Continue Expanding:
It continues expanding these parts recursively until the leaves of the tree match the actual
words in the sentence.
4. Check if the Sentence Matches:
The parser checks whether the parts it has expanded match the words in the sentence. If it
does, the parse succeeds; if not, it tries other rules or backtracks.

What is a Bottom-Up Parser?

A bottom-up parser works in the opposite direction compared to a top-down parser. Instead of
starting with the start symbol (like "S" for Sentence) and breaking it down, a bottom-up parser starts
with the input words (the leaves of the tree) and works its way up, gradually combining them into
higher-level structures (like noun phrases, verb phrases) until it reaches the start symbol (the
sentence).
In simple terms, a bottom-up parser tries to build the parse tree from the bottom (the words) to the
top (the sentence).
How It Works:
1. Start with the Words:
The process starts with the words in the sentence, treating each word as its own entity.
2. Combine Words into Phrases:
The parser tries to combine adjacent words into phrases according to the grammar rules. For
example, a noun phrase (NP) can be created by combining a determiner (Det) and a noun
(N).
3. Build the Parse Tree:
The parser continues combining words into phrases (like verb phrases, prepositional phrases)
until it has a complete parse tree that represents the entire sentence.

CYK Parser (Cocke-Younger-Kasami Algorithm)

The CYK Parser is a method used to check if a given sentence (or string) can be generated using a set
of grammar rules. It works like solving a puzzle, starting with small pieces (words) and combining
them step by step to form the whole sentence.
Key Points to Understand
1. Grammar Rules in Special Format:
 o The grammar must be in Chomsky Normal Form (CNF).
Example of CNF rules:
▪ S → AB
▪ A→a
▪ B→b
This means:
o Each rule produces either two non-terminals (like AB) or one terminal (like a).
2. Dynamic Table:
o Imagine a triangle-shaped table where:
▪ Rows represent substrings of increasing lengths.
▪ Columns represent the starting position of substrings in the sentence.
3. Goal:
o Build up from single words to check if the entire sentence can be formed using the
grammar.
How CYK Works (Step-by-Step)
Example Problem:
Check if the string "ba" can be derived using this grammar:
S → AB | BC
A→b
B→a
C→b
Steps:
1. Break the String into Words:
o The string is "ba". Split it into individual letters: b and a.
2. Fill the Table for Single Words:
o For "b", check which rules produce b:
▪ A→b → Add A.
▪ C→b → Add C.
o For "a", check which rules produce a:
▪ B→a → Add B.
Table after this step:
[A, C] [B]

3. Combine Words to Form Bigger Substrings:

o Look at the substring "ba". Combine the first part ("b") and the second part ("a"):
▪ "b" is A or C.
▪ "a" is B.
▪ Check grammar rules to see what can combine A+B or C+B:
▪ S→AB: Add S (since A→b and B→a).
Final Table:
[S]
4. Check the Top of the Table:
o If the top cell contains the start symbol (S), the string is valid.
Result: The string "ba" is valid.
Probabilistic Context-Free Grammar (PCFG)
A Probabilistic Context-Free Grammar (PCFG) is an extension of a Context-Free Grammar (CFG) that
assigns probabilities to its production rules. This makes it useful for tasks where multiple parse trees
are possible for a given input, such as natural language parsing. The goal of a PCFG is to determine
the most probable parse tree for a sentence.
Key Concepts:
1. Context-Free Grammar (CFG):
o A CFG consists of:
▪ A set of non-terminals (e.g., S,NP,VP).
▪ A set of terminals (e.g., words like "cat", "sat").
▪ A set of production rules (e.g., S→NP VP).
▪ A start symbol (e.g., S).
o Example CFG:
S → NP VP
NP → Det N
VP → V NP
Det → "the"
N → "cat" | "dog"
V → "chased"
2. Adding Probabilities:
o In PCFGs, each production rule has a probability. The sum of the probabilities of all
rules expanding the same non-terminal must equal 1.
o Example PCFG:
S → NP VP [1.0]
NP → Det N [0.6]
NP → N [0.4]
VP → V NP [0.7]
VP → V [0.3]
Det → "the" [1.0]
N → "cat" [0.5]
N → "dog" [0.5]
V → "chased" [1.0]
o Here, probabilities represent how often a rule is used in generating sentences.
3. Parse Tree Probability:
o The probability of a parse tree is the product of the probabilities of all the rules used
to generate the tree.
o Example: For the sentence "the cat chased":

Parse tree:
S
├── NP
│ ├── Det ("the")
│ └── N ("cat")
└── VP
└── V ("chased")
 4. Most Probable Parse Tree:
o In PCFG parsing, we choose the parse tree with the highest probability.

Shift-Reduce Parser
A Shift-Reduce Parser is a type of bottom-up parser used for syntax analysis in compilers and natural
language processing. It processes the input string from left to right and tries to construct a parse tree
by repeatedly applying shift and reduce operations.
Key Operations
1. Shift:
o Move the next input symbol to the stack.
o This means you are not yet ready to apply any grammar rule, so you move forward in
the input.
2. Reduce:
o Replace a sequence of symbols (from the stack) that matches the right-hand side of
a grammar rule with its left-hand side.
o For example, if the stack contains NP VP, and there is a rule S→NP VPS → NP \,
VPS→NPVP, replace NP VP with S.
3. Accept:
o The parser stops when the stack contains the start symbol (SSS), and the input is fully
processed.
4. Error:
o If no valid action (shift or reduce) can be performed, the parser reports an error.
Example of a Shift-Reduce Parse
Grammar:
1. S → NP VP
2. NP → Det N
3. VP → V NP
4. Det → the
5. N → dog
6. V → sees
Input:
the dog sees
Steps:
Step Action Stack Input
1 Shift [the] dog sees
2 Reduce (Det→the) [Det] dog sees
3 Shift [Det, dog] sees
4 Reduce (N→dog) [Det, N] sees
5 Reduce (NP→Det N) [NP] sees
6 Shift [NP, sees]
 Step Action Stack Input
7 Reduce (V→sees) [NP, V]
8 Reduce (VP→V NP) [S]
9 Accept [S]
How it Works
1. Start with an empty stack and the input string.
2. Decide between shift or reduce based on:
o Whether you can match a grammar rule to reduce symbols in the stack.
o If no rule applies, perform a shift to move the next input symbol to the stack.
3. Keep processing until:
o The stack contains the start symbol SSS (parse completed).
o Or no valid action can be performed (syntax error).

Semantic Analysis in NLP

Semantic analysis is the process of understanding the meaning of a sentence or text by analyzing the
relationships between words, phrases, and their context. It aims to bridge the gap between human
language and machine understanding, making systems capable of interpreting the intended meaning
of words or sentences.
Key Goals of Semantic Analysis:
1. Identify Meaning: Understand the meaning of individual words and their relationships
within a sentence.
2. Disambiguation: Resolve ambiguities in language (e.g., "bank" as a financial institution vs.
riverbank).
3. Context Understanding: Derive meaning based on the context (e.g., "He saw a bat" could
refer to an animal or sports equipment).

Describe the semantic analysis in Natural Language processing (2024)

Semantic Analysis in Natural Language Processing (NLP) is about understanding the meaning behind
words, sentences, or entire texts. While syntactic analysis looks at the structure (how words are put
together), semantic analysis focuses on what the words actually mean. Here are the main parts of
semantic analysis in simple terms:
1. Word Sense Disambiguation (WSD)
Some words have multiple meanings. Word Sense Disambiguation helps determine the correct
meaning of a word based on the context it’s used in.
• Example:
o "I went to the bank to fish." (Here, "bank" means the side of a river.)
o "I deposited money in the bank." (Here, "bank" means a financial institution.)
2. Named Entity Recognition (NER)
NER is about identifying important things in the text, like people, places, organizations, and dates.
This helps to understand the context.
• Example:
o "Apple was founded by Steve Jobs in California."
o "Apple" is a company, "Steve Jobs" is a person, and "California" is a place.
3. Sentiment Analysis
Sentiment analysis looks at the feelings or emotions in a text—whether it's positive, negative, or
neutral.
• Example:
o "I love this movie!" → Positive sentiment.
o "I hate waiting in long lines." → Negative sentiment.
4. Coreference Resolution
Coreference resolution helps find out which words refer to the same thing in a sentence. For
example:
• "John went to the store. He bought some milk."
o Here, "He" refers to "John."
5. Semantic Role Labeling (SRL)
This is about understanding who is doing what in a sentence. It identifies roles like the person doing
the action, the receiver of the action, and the object involved.
• Example:
o "John gave Mary a gift."
o "John" is the giver, "Mary" is the receiver, and "gift" is the object.
6. Word Embeddings
Word embeddings are a way to represent words as numbers in a computer. Words with similar
meanings are closer to each other in this number space.
• Example:
o "King" and "Queen" are represented as similar in meaning, so they are close in the
number space.
7. Conceptualization and Ontologies
This is about organizing knowledge. For example, in a healthcare system, we might define
relationships between words like "doctor," "nurse," and "patient."
Challenges in Semantic Analysis:
1. Ambiguity: Words can have different meanings, making it hard to figure out the right one.
2. Sarcasm: Machines struggle to understand when someone is being sarcastic because the
meaning is the opposite of the words.
3. Cultural Knowledge: Understanding text often requires knowing about specific cultures or
current events.

Lexical Semantics
Lexical semantics is about understanding the meaning of words and how they relate to each other in
a language. It helps us figure out what individual words mean and how different words connect or
interact to form the meaning of a sentence.
Key Points:
1. Word Meaning:
o Words have meanings, and lexical semantics helps us understand what those
meanings are.
o For example, the word "dog" refers to a type of animal.
2. Word Relationships:
o Words can be related to other words in different ways:
▪ Synonyms (similar meanings): happy ↔ joyful.
▪ Antonyms (opposite meanings): hot ↔ cold.
▪ Homonyms (same word, different meanings): bank (river side) ↔ bank
(financial institution).
 3. Ambiguity:
o Some words have multiple meanings, and we need to figure out which meaning is
correct based on the context.
o Example: "I went to the bank" could mean a financial institution or the side of a
river.
4. Context Matters:
o The meaning of a word can change depending on where and how it is used in a
sentence.
o Example: "He is a cool guy" means something different than "It’s cool outside."
Example to Understand It:
Step 1: Input Sentence
• Sentence: "He went to the bank to fish."
The word "bank" can mean two things:
1. A financial institution (place where you keep money).
2. The edge of a river or lake (where you might fish).
We need to figure out which meaning of "bank" is being used in this sentence.
Step 2: Context Clues
• The word "fish" in the sentence suggests that the bank here refers to the edge of a river,
because people fish near rivers.
Step 3: Checking Word Relationships
• Words like "fish" and "river" are related to "bank" when it means the edge of a river. This
helps us decide which meaning of "bank" to choose.

Corpus Study
Corpus Study refers to the analysis of a large collection of texts (called a corpus) to study language
patterns, structures, or trends. It is widely used in Natural Language Processing (NLP), linguistics,
and data analysis.
What is a Corpus?
A corpus is a structured collection of texts that are used for linguistic research. The texts can be
anything:
• Books
• News articles
• Conversations
• Social media posts
• Scientific papers
Corpus studies help us:
1. Understand how language is used in the real world.
2. Find patterns, such as the most common words or phrases.
3. Analyze trends over time, like how the use of certain words changes.
4. Train AI models for tasks like translation, chatbots, and sentiment analysis.
What Do Corpus Studies Analyze?
1. Word Frequency:
o What are the most common words?
Example: In English, words like "the", "and", and "is" are very frequent.
2. Collocations:
 o What words often appear together?
Example: "Fast food" appears together more often than "quick food."
3. Grammatical Patterns:
o How is grammar used?
Example: Finding how often people use passive voice.
4. Semantic Analysis:
o What do words mean in context?
Example: The word "bank" can mean a financial institution or the side of a river.
5. Trends Over Time:
o How has language changed?
Example: New words like "selfie" becoming common in the past decade.

Study of Language Dictionaries (Like WordNet)

The study of resources like WordNet helps us understand the meanings of words and how they
relate to each other. These are like smart dictionaries for computers, making it easier to analyze and
process languages.

What Is WordNet?
• WordNet is a big collection of words grouped by their meanings.
• It doesn’t just give definitions; it shows how words are connected:
o Synonyms: Words with similar meanings (happy ↔ joyful).
o Antonyms: Words with opposite meanings (hot ↔ cold).
o Categories: Specific words under broader terms (dog is an animal).
o Parts of something: wheel is a part of a car.
How It Helps
1. Find Word Meanings and Connections:
o Example: The word bank can mean a place for money or the side of a river. WordNet
helps figure out the right meaning based on context.
2. Synonyms for Writing:
o Need another word for big? WordNet suggests large, huge, or gigantic.
3. Understand Relationships:
o It shows how words are linked. Example:
▪ Rose is a type of flower.
▪ Flower is a type of plant.

What is BabelNet?
BabelNet is like a super dictionary that works for many languages. It not only tells you the meanings
of words but also shows how they are connected to other words and concepts across the world. It
combines dictionaries (like WordNet), encyclopedias (like Wikipedia), and other sources to create
one big knowledge network.
Key Features of BabelNet
1. Multilingual:
o Works in many languages (e.g., English, Hindi, Spanish, etc.).
o Example: The word dog in English links to perro in Spanish, kutte in Hindi, and chien
in French.
 2. Multiple Meanings:
o Helps you understand all possible meanings of a word.
o Example: Apple could mean the fruit or the company.
3. Connections Between Words:
o Shows relationships like:
▪ Synonyms (big ↔ large).
▪ Opposites (hot ↔ cold).
▪ Broader terms (car ↔ vehicle).
4. Encyclopedia Information:
o Includes facts and details about famous people, places, and things.
o Example: Apple Inc. includes info about its founders, products like iPhone, etc.

Homonymy
Homonymy refers to a situation where two or more words have the same spelling or pronunciation,
but they have different meanings. These words are called homonyms.
Types of Homonymy
There are two main types of homonymy:
1. Homophones:
o Words that sound the same but have different meanings and may also have different
spellings.
o Example:
▪ Flour (used for baking) and Flower (a plant).
▪ Both are pronounced the same but mean different things.
2. Homographs:
o Words that are spelled the same but have different meanings. They may or may not
be pronounced the same.
o Example:
▪ Lead (to guide) and Lead (a type of metal). They are spelled the same but
have different meanings. Lead (guide) is pronounced with a long "e" sound,
while Lead (metal) is pronounced with a short "e."
3. True Homonyms:
o Words that sound the same and are spelled the same, but have entirely different
meanings.
o Example:
▪ Bat:
▪ A flying mammal.
▪ A piece of sports equipment used in baseball.
Examples of Homonymy
1. Homophones:
o Right (correct) and Write (to form words).
o Sea (large body of water) and See (to look at something).
2. Homographs:
o Bow:
▪ A curved weapon for shooting arrows (bow).
▪ To bend forward as a gesture of respect (bow).
o Tear:
▪ To rip something (tear).
 ▪ A drop of water from the eye (tear).

Polysemy
Polysemy refers to a situation where a single word has multiple meanings that are related to each
other. These different meanings are all connected by some core idea or concept, making them
different from homonymy, where words have unrelated meanings.
Key Features of Polysemy
1. One Word, Multiple Related Meanings:
o A polysemous word has different meanings, but all of these meanings are linked by a
common concept.
o Example:
▪ Head:
▪ The top part of the body (e.g., "He hurt his head").
▪ The leader of a group or organization (e.g., "She is the head of the
company").
▪ The front part of something (e.g., "He is at the head of the line").
All these meanings of head are related to the idea of being the top or front of something.
2. Meaning Changes Based on Context:
o The exact meaning of a polysemous word depends on the context in which it's used.
Examples of Polysemy
1. Bank:
o Financial institution: "I need to go to the bank to withdraw money."
o Side of a river: "The fish swam near the riverbank."
o Both meanings are related by the concept of a location or place.
2. Mouth:
o The opening of the body: "He opened his mouth to speak."
o The opening of a river: "The river's mouth is wide."
o Both meanings are related by the idea of an opening.
3. Light:
o Visible illumination: "The room was full of light."
o Not heavy: "This box is light to carry."
o Both meanings are connected by the idea of being less dense or easy to perceive.

Synonymy
Synonymy refers to the relationship between two or more words that have the same or very similar
meanings. These words can be used interchangeably in many contexts, although sometimes one
word might be preferred over another depending on the situation.
Key Features of Synonymy
1. Same or Similar Meaning:
o Synonyms are words that convey similar meanings, but they might not be exactly
the same in every context.
o Example: Happy and Joyful are synonyms because they both describe a positive
emotional state.
2. Context Matters:
o Even though synonyms have similar meanings, context matters in choosing the right
word.
o Example:
 ▪ "He is wealthy" vs. "He is rich."
▪ Both words mean having a lot of money, but wealthy sounds a bit more
formal than rich.
3. Subtle Differences:
o Some synonyms might have slightly different connotations or be used in different
contexts.
o Example:
▪ Big and huge both mean large, but huge usually emphasizes an even larger
size than big.
Examples of Synonymy
1. Happy ↔ Joyful
o Both words describe a state of positive emotion.
2. Fast ↔ Quick
o Both describe speed, but quick is often used for actions, and fast can describe
objects (e.g., a fast car or a fast runner).
3. Smart ↔ Intelligent
o Both refer to someone who is mentally capable.
4. House ↔ Home
o Both words refer to a place where you live, but home has a warmer, more personal
connotation than house.

Hyponymy
Hyponymy refers to a relationship between words where one word (the hyponym) represents a
more specific concept that is included under a broader category (the hypernym). In simple terms,
hyponymy shows how one word is a subcategory or kind of another, more general word.
Key Features of Hyponymy
1. Specific vs. General:
o The hyponym is a more specific term, while the hypernym is the general category
under which the hyponym falls.
o Example:
▪ Dog (hyponym) is a type of Animal (hypernym).
▪ Carrot (hyponym) is a type of Vegetable (hypernym).
2. Hierarchical Relationship:
o Hyponymy creates a hierarchy between words, where the hyponym is a subclass or
type of the broader hypernym.
3. Multiple Hyponyms for a Single Hypernym:
o A hypernym can have many hyponyms, each representing a different kind within
that broader category.
o Example:
▪ Fruit (hypernym) can have hyponyms like Apple, Banana, Orange, etc.

Semantic Ambiguity
Semantic ambiguity occurs when a word or phrase has multiple meanings and its meaning is unclear
without additional context. In other words, a word or sentence can be interpreted in different ways
because it has more than one possible meaning.
Key Features of Semantic Ambiguity
1. Multiple Meanings of a Word:
 o A word with semantic ambiguity can have different meanings depending on how it's
used in a sentence.
o Example: The word "bat" can refer to:
▪ A flying mammal.
▪ A sports equipment used in baseball.
2. Context Determines Meaning:
o To resolve semantic ambiguity, we need to look at the context in which the word is
used.
o Example:
▪ "She saw the bat in the sky." (Here, bat refers to a flying mammal.)
▪ "He hit the ball with a bat." (Here, bat refers to the sports equipment.)
3. Different Senses:
o The different meanings of a word in semantic ambiguity are often called senses.
o The same word may have distinct senses based on how it's used in different
contexts.
Examples of Semantic Ambiguity
1. Bank:
o Bank can mean:
▪ A financial institution where you store money.
▪ The side of a river.
o Without context, it's unclear which meaning is intended.
2. Light:
o Light can mean:
▪ Not heavy (e.g., "The box is light").
▪ Visible illumination (e.g., "The room is full of light").
o The meaning depends on the context of the sentence.

Word Sense Disambiguation (WSD)

Word Sense Disambiguation (WSD) is the process of identifying the correct meaning (sense) of a
word in a given context. Many words have multiple meanings, and WSD helps determine which
meaning is intended.
Why Do We Need WSD?
Some words can have multiple meanings depending on the context. For example:
• "Bank":
o A financial institution (I deposited money in the bank).
o The side of a river (We sat on the river bank).
Without WSD, machines might misunderstand the meaning, which can lead to errors in tasks like
translation, search engines, or chatbots.
How Does WSD Work?
To determine the correct meaning, WSD looks at the context around the word. Here's how it works:
1. Input: A word with multiple possible meanings in a sentence.
2. Analyse Context: Examine nearby words to understand the word’s role.
3. Output: The correct meaning of the word in this situation.
Q Techniques for Word Sense Disambiguation
1. Knowledge-based methods:
o These methods use dictionaries, thesauruses, or lexical databases (like WordNet) to
determine possible senses of a word.
o The system looks at the meanings listed in a database and matches the context to
the correct sense.
2. Supervised learning:
o This approach uses machine learning and labeled data (where the senses of words
are manually tagged) to train models. These models learn from examples to predict
the correct sense of a word.
o Example: If you have a dataset where the word "bark" is labeled with its meanings
(e.g., the sound a dog makes or the outer layer of a tree), a machine learning model
can learn to choose the correct sense in future sentences.
3. Unsupervised learning:
o This method doesn’t use labeled data. Instead, it looks at patterns in large amounts
of text to group words with similar meanings and then uses these patterns to
disambiguate words.
4. Corpus-based methods:
o These methods analyze large collections of texts (called corpora) to observe how
words are used in different contexts and to identify the most likely sense based on
frequency of usage.
Example of Word Sense Disambiguation
• Sentence 1: "She went to the bank to cash her check."
o Here, "bank" refers to a financial institution.
• Sentence 2: "The boat is floating near the bank of the river."
o Here, "bank" refers to the side of the river.

Q Explain the Lesk algorithm for Word Sense Disambiguation

Lesk Algorithm for Word Sense Disambiguation (WSD)
The Lesk Algorithm is a classic technique used in Word Sense Disambiguation (WSD), which is the
task of determining the correct meaning (sense) of a word based on its context. This is important
because many words have multiple meanings, and understanding the correct one is crucial for tasks
like machine translation, information retrieval, and question answering.
The basic idea behind the Lesk Algorithm is simple: it tries to determine the sense of a word by
looking at the overlap between the definitions (or glosses) of different senses of the word and the
surrounding words in the context.
Steps in the Lesk Algorithm:
How Does It Work?
1. Input:
o A target word (ambiguous word).
o The context (sentence or nearby words).
2. Retrieve Senses:
o Fetch all possible senses (meanings) of the target word from a lexical resource like
WordNet.
3. Overlap Comparison:
o For each sense, compare the definition (gloss) of the sense with the context.
o Count how many words overlap (are the same) between the gloss and the context.
 4. Choose the Best Sense:
o The sense with the highest overlap score is selected as the correct meaning.
Example:
Let’s go through an example using the word "bank", which can have different meanings:
• Sense 1: A financial institution (e.g., "She went to the bank to deposit money").
• Sense 2: The side of a river (e.g., "They sat on the river bank to relax").
Input:
• Target word: "bank"
• Context: "They sat on the river bank to relax."
Step 1: Retrieve Definitions
• Sense 1: A financial institution, where people keep money.
• Sense 2: The land alongside a river or lake.
Step 2: Calculate Overlap
• Sense 1 (Financial institution): Gloss = "A financial institution, where people keep money."
o Overlap with the context ("river bank to relax"): 0 common words (there’s no
mention of money, financial terms, etc. in the context).
• Sense 2 (River bank): Gloss = "The land alongside a river or lake."
o Overlap with the context: "river" and "bank" are both present in the context.
There’s a strong overlap.
Step 3: Choose the Best Sense
• Since Sense 2 (the riverbank) shares more words with the context ("river" and "bank"), Sense
2 is chosen as the correct sense for the word "bank" in this context.

Q Yarowsky Algorithm for Word Sense Disambiguation (Semi-Supervised

Method)
The Yarowsky algorithm is a semi-supervised approach to Word Sense Disambiguation (WSD). It is
based on two key principles:
1. One sense per collocation: A word tends to have the same meaning in similar contexts.
2. One sense per discourse: A word usually has the same meaning throughout a single
document or context.
This algorithm is widely used because it can achieve good results even with limited labelled data by
leveraging unlabelled data.

How the Yarowsky Algorithm Works

The algorithm follows these steps:
Step 1: Seed Initialization
• Start with a small set of labeled examples (seed data).
Example: For the word "bank":
o Label examples for riverbank: "bank near the water", "sitting on the river bank".
o Label examples for financial institution: "bank account", "money in the bank".
Step 2: Unlabeled Data
• Use a large amount of unlabeled data where the meanings of the target word are unknown.
Step 3: Training
• Iteratively label the unlabeled data using the following ideas:
1. One Sense per Collocation:
▪ If "bank" often appears with "river" or "water", assign it the "riverbank"
sense.
 ▪ If "bank" appears with "money" or "loan", assign it the "financial institution"
sense.
2. One Sense per Discourse:
▪ Within the same document, assume the word has the same meaning
throughout.
Step 4: Refinement
• Use the newly labeled data to improve the model and repeat the process.
• Over time, the algorithm improves its understanding of the correct sense by leveraging the
patterns in the data.

Example
Input:
• Seed data for "bank":
o Riverbank: "river bank", "beside the water".
o Financial: "bank account", "loan from the bank".
• Unlabeled sentences:
1. "He sat on the bank near the river."
2. "She opened a bank account last week."
3. "The bank is flooded after the heavy rain."
Process:
1. Step 1: Start with the labeled examples and define patterns:
o Riverbank: Words like "river", "water", "flood".
o Financial: Words like "account", "loan", "money".
2. Step 2: Analyze the unlabeled data:
o Sentence 1: Contains "river" → Assign "riverbank" sense.
o Sentence 2: Contains "account" → Assign "financial institution" sense.
o Sentence 3: Contains "flood" → Assign "riverbank" sense.
3. Step 3: Use the newly labeled data to refine the patterns.
4. Step 4: Repeat the process with updated patterns until no significant changes occur.
Output:
• Labeled Data:
o Sentence 1: "He sat on the bank near the river." → Riverbank.
o Sentence 2: "She opened a bank account last week." → Financial.
o Sentence 3: "The bank is flooded after the heavy rain." → Riverbank.
Why is Yarowsky Semi-Supervised?
• It starts with a small set of labeled data (supervised).
• It then uses unlabeled data and self-training to expand the labeled dataset, making it semi-
supervised.

HyperLex Algorithm for Word Sense Disambiguation (Unsupervised Method)

HyperLex is an unsupervised approach to Word Sense Disambiguation (WSD) that uses semantic
similarity and word clustering to group occurrences of ambiguous words into distinct senses. Unlike
supervised or semi-supervised methods, it does not rely on labeled data and instead uses
unstructured, unlabeled text.
Key Idea
HyperLex identifies the different meanings (senses) of a word by clustering its occurrences based on
the similarity of their contexts.
 • Clustering: Words with similar contexts are grouped together, assuming that similar contexts
imply the same sense of the word.
• Context Representation: Each occurrence of the ambiguous word is represented as a feature
vector, built from the surrounding words.
Example
Input: Sentences with the word "bank"
1. The fisherman sat on the bank of the river.
2. He deposited money in the bank.
3. The bank is near the water.
4. She took out a loan from the bank.
Step-by-Step:
1. Extract Contexts:
o Sentence 1: {"fisherman", "sat", "river"}.
o Sentence 2: {"deposited", "money"}.
o Sentence 3: {"bank", "water"}.
o Sentence 4: {"loan", "money"}.
2. Feature Representation:
o Convert each context into vectors using BoW or embeddings.
3. Similarity Calculation:
o Compare contexts based on similarity (e.g., cosine similarity).
4. Clustering:
o Cluster 1: {"fisherman", "river", "water"} → Riverbank.
o Cluster 2: {"money", "loan", "deposit"} → Financial institution.
5. Output:
o Sentence 1: Sense = Riverbank.
o Sentence 2: Sense = Financial institution.

Dictionary-Based Approach to WSD

The Dictionary-Based Approach to Word Sense Disambiguation uses external resources like
dictionaries or lexical databases (e.g., WordNet) to figure out which sense of a word is correct. Here's
how it works:
Steps in Dictionary-Based WSD:
1. Retrieve Possible Word Senses:
o For a given word, look up all possible meanings (senses) from a dictionary or lexical
resource. For example, if the word is "bank," the dictionary will provide several
meanings like a financial institution or the side of a river.
2. Contextual Information:
o Look at the surrounding words in the sentence or document (the context) to
understand how the word is being used. For example, in the sentence "He went to
the bank to fish," the word "fish" provides a strong clue that bank refers to the side
of a river, not a financial institution.
3. Choose the Best Sense:
o The dictionary-based approach compares the different senses of the word and
chooses the one that best matches the context. This can be done using overlap
between the context words and the word senses' definitions.
4. Word Sense Selection:
 o Once the right sense is chosen based on context, the word is disambiguated. In the
example above, the bank related to the side of a river is selected.
Example:
Let's consider the word "bat" in the sentence:
• Sentence: "The bat flew out of the cave."
1. Retrieve Possible Senses:
o Sense 1: A flying mammal (the animal).
o Sense 2: A piece of equipment used in sports (like baseball).
2. Analyze Context:
o The context word here is "flew," which indicates that the word bat refers to a flying
mammal, not the equipment.
3. Choose the Correct Sense:
o Based on the context word "flew," the dictionary-based approach selects the first
sense (the flying mammal) for bat.

Pragmatic and Discourse Processing (Simplified)

1. Pragmatic Processing
• Pragmatics focuses on the hidden meaning behind words and sentences, considering
context and speaker intentions.
• It answers: "What does the speaker really mean?"
Examples:
1. Implied Meaning:
o Sentence: "It’s cold here."
o Pragmatic Meaning: A request to close the window.
2. Deixis (Context Words):
o Sentence: "Take this over there."
o Meaning depends on knowing:
▪ This = what object?
▪ There = which place?
3. Tone/Sarcasm:
o Sentence: "Great job!" (after a mistake)
o Pragmatic Meaning: Sarcasm, not praise.

2. Discourse Processing
• Discourse looks at how sentences connect to make a meaningful conversation or text.
• It answers: "How do sentences work together?"
Examples:
1. Coreference:
o Text: "John bought a car. He loves it."
o "He" refers to John, "it" refers to the car.
2. Coherence:
o Good: "I was hungry, so I ate."
o Bad: "I was hungry. The sky is blue."
3. Ellipsis:
o Text: "I’ll go to the market, and she will too."
o Full meaning: "She will go to the market."
Reference Resolution (Simplified)
Reference Resolution is the process of figuring out what a word or phrase refers to in a sentence or
conversation. It is crucial in understanding the meaning of text, especially when pronouns (like "he,"
"she," "it") or other vague words are used.
Types of References
1. Anaphora:
o Refers back to something mentioned earlier.
o Example:
▪ "John ate an apple. He enjoyed it."
▪ "He" = John, "it" = apple.
2. Cataphora:
o Refers to something mentioned later.
o Example:
▪ "When he arrived, John greeted everyone."
▪ "He" = John.
3. Coreference:
o Multiple expressions refer to the same entity.
o Example:
▪ "Mary loves her dog. She plays with it every day."
▪ "Mary" and "She" refer to the same person.
4. Exophora:
o Refers to something outside the text, relying on context.
o Example:
▪ "Look at that!"
▪ "That" depends on what’s being pointed at in the environment.

Steps in Reference Resolution

1. Identify References:
o Find pronouns, definite nouns (like "the dog"), or vague phrases that need
resolution.
2. Find Possible Antecedents:
o Look for earlier nouns or entities in the text or conversation.
3. Choose the Best Match:
o Use grammar, context, or logic to match the reference to its antecedent.
Applications
• Chatbots/Virtual Assistants:
o Resolving "it" in queries like "What is the weather? Will it rain today?"
• Machine Translation:
o Maintaining references across languages.
• Question Answering:
o Resolving "Who" or "What" in questions based on prior context.
Q Explain three types of referents that complicate the reference resolution
problem (5 mark)
In language processing, reference resolution is about figuring out what words (like pronouns) refer
to in a sentence. This can be tricky because of different types of referents. Here are three types that
make it difficult:
1. Pronouns
Pronouns like "he," "she," "it," and "they" are used to refer to someone or something mentioned
earlier. The problem is knowing exactly who or what the pronoun is talking about.
• Example:
o "John and Peter went to the store. He bought some milk."
o Who does "He" refer to? It could be John or Peter.
2. Ambiguous Nouns
Some nouns can mean more than one thing, so it’s not always clear which meaning is being used.
• Example:
o "The bank was closed."
o Does "bank" mean a financial institution or the side of a river?
3. Anaphoric Expressions
Anaphora refers to words like "this," "that," "these," or "those" that refer back to something
mentioned earlier in the sentence. It can be unclear what exactly these words are referring to.
• Example:
o "The teacher gave a lecture. This was very informative."
o What does "This" refer to? Is it the teacher or the lecture?

Summary:
1. Pronouns (like "he" or "she") might refer to different people.
2. Ambiguous nouns (like "bank") can have multiple meanings.
3. Anaphora (like "this" or "those") can refer to different things in the sentence.

Reference Phenomena
Reference phenomena deal with how words or phrases in a sentence refer to people, objects, or
events in the context of a conversation or text. This helps link ideas and maintain coherence.
Types of Reference Phenomena
1. Anaphora:
o Refers to something mentioned earlier in the text.
o Example:
▪ "Lisa bought a car. She loves it."
▪ "She" = Lisa, "it" = the car.
2. Cataphora:
o Refers to something mentioned later in the text.
o Example:
▪ "Before he arrived, John called to say he was late."
▪ "He" = John (appears later).
3. Coreference:
o When two or more words refer to the same entity.
o Example:
▪ "Mark said Mark’s car broke down. He called a mechanic."
 ▪ "Mark" and "He" refer to the same person.
4. Exophora:
o Refers to something outside the text, relying on external context.
o Example:
▪ "Pass me that."
▪ "That" refers to an object in the environment.
5. Endophora:
o Refers to something inside the text (includes anaphora and cataphora).
o Example:
▪ "The cake was delicious. Everyone loved it."
▪ "It" = the cake.
6. Ellipsis Resolution:
o Resolves missing information implied by the context.
o Example:
▪ "John likes pizza, and Mary does too."
▪ Full meaning: "Mary likes pizza too."
7. Bridging:
o Refers to something indirectly connected to what’s mentioned.
o Example:
▪ "I saw a car. The wheels were shiny."
▪ "The wheels" are part of the car but not explicitly mentioned earlier.

Syntactic and Semantic Constraints on Coherence

Coherence in language means how well sentences and ideas connect to make the text or
conversation easy to understand. To achieve coherence, sentences must follow rules of grammar
(syntax) and rules of meaning (semantics). These rules help sentences fit together logically.
1. Syntactic Constraints (Grammar Rules)
Syntactic constraints ensure that sentences are grammatically correct so they make sense.
Key Points:
1. Grammatical Consistency:
o Sentences must follow grammar rules.
o Example (Correct): "The cat sat on the mat."
o Example (Incorrect): "The cat sit on the mat."
2. Referencing Consistency:
o Pronouns (like "he," "it") must clearly point to the right thing.
o Example (Correct): "John found his book. He was happy."
o Example (Incorrect): "John found his book. They were happy." (Who is "they"?)
3. Word Order:
o Words must be in the correct order.
o Example (Correct): "I went to the park."
o Example (Incorrect): "Went I park to the."
2. Semantic Constraints (Meaning Rules)
Semantic constraints make sure the meaning of the sentences fits together and makes sense in
context.
Key Points:
1. Logical Consistency:
o Sentences must follow a logical pattern.
 o Example (Correct): "John went to the store to buy milk."
o Example (Incorrect): "John went to the store to buy milk. The sun is shining." (Not
related)
2. Thematic Consistency:
o All sentences in a paragraph should talk about the same topic.
o Example (Correct): "The movie was great. The actors did an excellent job."
o Example (Incorrect): "The movie was great. My dog loves playing outside."
3. World Knowledge:
o Sentences must make sense in the real world.
o Example (Correct): "The car drove on the road."
o Example (Incorrect): "The car flew across the sky." (Cars don't fly)
How Syntax and Semantics Work Together
1. Together for Meaning:
o Syntax (grammar) and semantics (meaning) help each other. A sentence can be
grammatically correct but still not make sense if the meaning is off.
o Example: "The table eats quickly." (Grammatically correct but makes no sense.)
2. Ambiguity:
o Sometimes, a sentence can be unclear, but grammar and meaning help figure it out.
o Example: "I saw the man with a telescope."
▪ Did I have the telescope, or the man had it?

Q Explain Anaphora Resolution using Hobbs and Cantering Algorithm

Anaphora Resolution (Hobbs and Cantering Algorithm)
Anaphora resolution is a linguistic mechanism that helps speakers and writers avoid repetition by
using a word or phrase to refer to a previously mentioned entity.
Anaphora Resolution is the process of determining what a pronoun (or other anaphoric expression
like "it," "he," "they," etc.) refers to in a sentence or text. It is important for understanding text and
conversation, as it helps link pronouns to their antecedents (the nouns they refer to).
Hobbs Algorithm (1978)
The Hobbs Algorithm is a well-known approach for anaphora resolution. It uses syntactic and
semantic rules to resolve anaphora based on the surface structure of the text. In simple terms, it
focuses on the order of the words in the sentence to resolve references.
Steps in Hobbs Algorithm:
1. Identify the Pronoun: Find the anaphor (the word like "he," "she," "it," etc.).
2. Search Backwards: Look at the preceding noun phrases (e.g., "John," "the book," etc.) for
potential antecedents. The pronoun usually refers to something mentioned earlier in the
sentence.
3. Check for Syntactic Agreement: The antecedent must match the pronoun in terms of
number (singular/plural), gender (masculine/feminine), and person (first, second, third).
4. Pick the Most Likely Antecedent: If multiple candidates are found, choose the one closest to
the pronoun in the sentence. This is based on the left-to-right order of the text.
Example:
• Text: "John went to the park. He played soccer."
o Pronoun: "He"
o Antecedent: "John"
o The Hobbs algorithm would identify "John" as the antecedent because it is the
closest noun phrase before the pronoun "He."
Cantering Algorithm (1993)
The Cantering Algorithm is an extension of the Hobbs algorithm. It also focuses on syntactic rules,
but with an emphasis on discourse structure. The idea is that the meaning and reference of
anaphoric expressions can be better understood by considering larger discourse contexts rather than
just sentence boundaries.
Steps in Cantering Algorithm:
1. Identify the Anaphor: Like Hobbs, first, identify the anaphoric expression (e.g., "he," "she,"
"it," etc.).
2. Parse the Discourse Structure: Cantering looks at the larger discourse (beyond just individual
sentences). It considers how the sentences are connected in the text.
3. Search for the Antecedent in Discourse Context: Instead of just looking backward in the
sentence, Cantering considers the entire discourse to find the most likely antecedent. For
example, if an earlier sentence provides additional context, it will be included in the search.
4. Syntactic and Semantic Matching: The algorithm matches the pronoun to a potential
antecedent based on both syntax (grammatical structure) and semantics (meaning).
Example:
• Text: "John entered the room. He smiled."
o Pronoun: "He"
o Antecedent: "John"
o The Cantering algorithm, like Hobbs, would identify "John" as the antecedent, but it
might also take into account that the discourse (i.e., the flow of the conversation or
narrative) suggests that "John" is the most likely candidate.

Key Differences:
• Hobbs Algorithm: Focuses only on the closest noun in the sentence (a simple approach).
• Cantering Algorithm: Looks at larger context (more sentences) to decide what the pronoun
refers to.

Machine Translation (MT) in NLP

Machine Translation (MT) is the process of using computers to automatically translate text from one
language to another. The goal is to help people communicate across language barriers without
needing a human translator.
How Machine Translation Works (in simple terms)
Machine Translation uses NLP techniques to understand the meaning of text in one language and
generate the same meaning in another language. It breaks down sentences, analyzes the structure,
and then tries to find the closest equivalent words or phrases in the target language.
Here’s a basic breakdown of how MT works:
1. Input Text: You provide a sentence in one language (e.g., English).
2. Analyze the Sentence: The system understands the grammar and meaning of the sentence in
the source language (e.g., English).
3. Translate: The system generates a sentence in the target language (e.g., French) that has the
same meaning.
4. Output Text: The final translation is given to the user.
Types of Machine Translation
1. Rule-Based Translation:
o Uses grammar rules to translate words one by one.
o Example: "I am happy" becomes "Je suis content" by following a set of rules.
 o Limitations: Sometimes it doesn't sound natural and can make mistakes in complex
sentences.
2. Statistical Translation:
o Learns from lots of examples (like bilingual books) and finds the most likely
translation.
o Limitations: Can sometimes create awkward translations.
3. Neural Translation (the best one):
o Uses smart computer programs (called AI) to understand whole sentences, not just
words.
o Advantage: Produces more natural and accurate translations.
o Example: Google Translate uses this method.

Q Explain Machine Translation Approaches used in NLP (5mark)

1. Rule-Based Machine Translation (RBMT)
• How it works: This method uses detailed grammar rules and dictionaries for both languages
to translate text.
• Example: If you're translating from English to French, it uses rules to turn "I am going" into
"Je vais."
2. Statistical Machine Translation (SMT)
• How it works: This method learns from large amounts of translated text (called parallel
data). It looks for patterns of word pairs (like "hello" and "hola") to guess the correct
translation.
• Example: It sees many examples of "I love you" translating to "Te quiero" and learns this
pattern.
3. Neural Machine Translation (NMT)
• How it works: This method uses deep learning (artificial intelligence) to translate by
considering the whole sentence and its meaning, not just word-by-word.
• Example: It looks at the sentence "I am learning Spanish" and understands the meaning to
translate it as "Estoy aprendiendo español."
4. Example-Based Machine Translation (EBMT)
• How it works: This method uses past translation examples. It finds similar phrases in its
database and reuses those translations.
• Example: If it has seen "I am happy" translate to "Estoy feliz," it will use this example to
translate the same sentence.
5. Hybrid Machine Translation
• How it works: This method combines multiple approaches (like RBMT, SMT, and NMT) to get
the best results.
• Example: It might use rules for formal sentences and neural networks for casual speech.

Question Answering System (QAS)

A Question Answering System (QAS) is a type of computer system designed to answer questions
posed by users in natural language. The goal is to give an accurate, direct answer to a question, just
like a human would, based on the available information.

How It Works (in Simple Steps)

1. Input Question: You ask the system a question in natural language (e.g., "What is the capital
of France?").
 2. Understand the Question: The system processes the question to figure out what you are
asking. It uses NLP techniques to understand the meaning behind the words.
3. Search for the Answer: The system looks for relevant information in a database, website,
document, or other sources. It might search through stored knowledge or access real-time
information from the internet.
4. Provide the Answer: The system then gives a direct answer to your question (e.g., "Paris").

Types of Question Answering Systems

1. Closed-Domain QAS:
o These systems are specialized and focus on answering questions within a specific
topic or field.
o Example: A medical QAS might only answer questions about health and diseases.
2. Open-Domain QAS:
o These systems are more general and can answer questions on a wide range of topics,
from history to science, to current events.
o Example: Google’s search engine or virtual assistants like Siri and Alexa.
Example Interaction with a QAS
• Question: "Who is the president of the United States?"
• QAS Answer: "Joe Biden."
• Question: "What is the capital of Japan?"
• QAS Answer: "Tokyo."
Where We Use QAS
1. Google Search: You ask, "What's the weather today?" and it shows the answer right away.
2. Virtual Assistants: Ask questions like "Who won the last World Cup?" and get an instant
answer.
3. Customer Support Chatbots: They answer questions on websites, like "How do I reset my
password?"
Challenges
1. Unclear Questions: Sometimes, questions can be unclear, like "What’s the apple?" (Is it the
fruit or the tech company?).
2. Understanding Context: If your question refers to something you said earlier, the system has
to remember the context to give the right answer.
3. Complex Questions: Some questions are tricky, like "What’s the weather and who is playing
today?" – these need the system to handle multiple parts at once.

Information Retrieval (IR) in NLP

In the context of Natural Language Processing (NLP), Information Retrieval (IR) refers to the process
of finding relevant pieces of information (like documents, texts, or web pages) from a large collection
based on a query written in natural language. It's about helping computers understand and retrieve
useful information that matches what a user is asking for.
How It Works (Simple Steps)
1. You ask a question or type a search term: For example, "What is the capital of France?"
2. The system understands your question: It looks at the key parts of your query, like "capital"
and "France".
3. The system searches for relevant documents: It looks through lots of information to find the
ones that best answer your question.
4. It ranks the results: The system organizes the results by how well they match your query.
 5. You get the results: You see the list of websites, articles, or documents that provide the best
answer to your question.
Where It's Used
• Search Engines: When you type something in Google, it's IR in action.
• Virtual Assistants: When you ask Siri or Alexa a question, they use IR to find an answer.
• E-commerce: When you search for products online, IR helps find the most relevant ones.
• Chatbots: In customer service, chatbots use IR to answer your questions.

Challenges
• Ambiguity: Some words have multiple meanings, which can confuse the system.
o Example: "Apple" could mean the fruit or the tech company.
• Relevance: The system needs to understand what is most relevant to your question, not just
similar words.

Steps in Text Processing for Information Retrieval (IR)

Text processing in IR is about preparing the text so that a computer can easily search and find
relevant information. Here are the main steps, explained simply:
1. Tokenization
• What It Is: Splitting the text into smaller pieces like words or phrases.
• Why It's Needed: So the computer can understand and work with each word or phrase
separately.
Example:
• "I love programming" becomes ["I", "love", "programming"]
2. Lowercasing
• What It Is: Making all words lowercase.
• Why It's Needed: To avoid treating the same word differently based on capitalization.
Example:
• "Apple" and "apple" are treated the same.
3. Removing Stop Words
• What It Is: Removing common words like "the," "is," "in" that don't add much meaning.
• Why It's Needed: To focus on the important words that help in searching.
Example:
• "The quick brown fox" becomes ["quick", "brown", "fox"]
4. Stemming
• What It Is: Reducing words to their root form.
• Why It's Needed: So words like "running," "runner," and "ran" are treated the same.
Example:
• "Running" becomes "run"
5. Lemmatization
• What It Is: Similar to stemming, but it considers the correct dictionary form of a word.
• Why It's Needed: To make sure the word is in the correct form.
Example:
• "Better" becomes "good"
6. Part-of-Speech Tagging
• What It Is: Identifying whether a word is a noun, verb, adjective, etc.
• Why It's Needed: Helps understand the role of each word in the sentence.
Example:
 • "The dog runs fast" → "dog" is a noun, "runs" is a verb.
7. Named Entity Recognition (NER)
• What It Is: Identifying important names (like people, places, or dates).
• Why It's Needed: Helps the computer focus on key information, like finding the name of a
person or place.
Example:
• "Apple is located in Cupertino" → "Apple" (Company), "Cupertino" (Place)
8. Indexing
• What It Is: Creating a list of words and where they appear in the documents.
• Why It's Needed: So the computer can quickly find relevant documents when you search.
Example:
• Word "apple" might appear in documents 1, 3, and 5.
9. Vectorization (Feature Extraction)
• What It Is: Turning words into numbers so the computer can process them.
• Why It's Needed: Computers understand numbers, not text, so we need to convert text into
numerical data.
Example:
• "Information Retrieval" becomes a set of numbers like [0.2, 0.8, 0.5, ...]
10. Query Processing
• What It Is: When you search for something, the system processes your query the same way
it processed the documents.
• Why It's Needed: So the search can match the query to the documents.
Example:
• Searching "What is IR?" → System processes "IR" to find related documents.

Text Summarization
Text summarization is the process of creating a shorter version of a text while keeping its important
information. It's like reading a long article and then writing a shorter version that includes the key
points. There are two main types: extractive and abstractive.
1. Extractive Summarization
• What It Is: It picks important sentences or phrases directly from the text and combines them
to make a summary.
• How It Works: The system looks at the text, scores sentences based on importance, and then
selects the best ones to form the summary.
Example:
• Original Text: "The dog ran in the park. It was a sunny day. The dog played with a ball."
• Extractive Summary: "The dog ran in the park. The dog played with a ball."
Advantages:
• Simple and easy to do.
• Often accurate because it uses original sentences.
Disadvantages:
• May not sound smooth or flow well since it just selects sentences.

2. Abstractive Summarization
• What It Is: It creates new sentences that explain the main ideas of the text, often using
simpler words or rephrasing.
 • How It Works: The system understands the meaning of the text and then generates a short
version using its own words.
Example:
• Original Text: "The dog ran in the park. It was a sunny day. The dog played with a ball."
• Abstractive Summary: "The dog had fun playing outside."
Advantages:
• The summary sounds more natural and easier to read.
• It can shorten the text more effectively by rephrasing.
Disadvantages:
• It's harder to do and may sometimes make mistakes in meaning.

Q Explain the ambiguities associated at each level with example for Natural
Language processing
1. Lexical Ambiguity (Word Level)
A word can have different meanings depending on the context.
• Example:
o "Bank" can mean a financial institution ("I went to the bank.") or the side of a river
("We sat by the river bank.").
2. Syntactic Ambiguity (Sentence Level)
A sentence can be unclear because its structure allows multiple meanings.
• Example:
o "I saw the man with the telescope."
▪ Does it mean I saw a man who had a telescope?
▪ Or does it mean I used a telescope to see the man?
3. Semantic Ambiguity (Meaning Level)
A sentence can be unclear because a word can mean different things in that context.
• Example:
o "He gave her a ring."
▪ Did he give her a piece of jewelry (a ring)?
▪ Or did he call her on the phone (ring)?
4. Pragmatic Ambiguity (Context Level)
The meaning of a sentence can depend on the situation it's used in.
• Example:
o "Can you pass the salt?"
▪ Does it mean "Are you able to pass the salt?"
▪ Or is it a polite request for someone to pass the salt?
5. Discourse Ambiguity (Conversation Level)
A sentence can be unclear because of what came before or after it in a conversation.
• Example:
o "John went to the store. He bought some bread."
▪ Who is "he"? Is it John, or someone else?
6. Quantifier Ambiguity (Amount or Number)
Words like "all," "some," or "many" can be unclear about what they are referring to.
• Example:
o "Some students passed the exam."
▪ Does it mean only a few students passed, or just any group of students?
7. Anaphoric Ambiguity (Pronoun Reference)
A pronoun like "he," "she," or "it" can be unclear about who or what it refers to.
• Example:
o "Maria gave Sara her book."
▪ Whose book is it? Maria’s or Sara’s?
8. Temporal Ambiguity (Time Reference)
A sentence can be unclear about when something happens.
• Example:
o "She will meet him tomorrow."
▪ Does it mean she will meet him tomorrow or she already planned it but is
confirming the time?

Q Represent output of morphological analysis for Regular verb, Irregular

verb, singular noun, plural noun Also Explain Role of FST in Morphological
Parsing with an example
ChatGPT said:
ChatGPT
Morphological Analysis Output for Different Word Types:
Morphological analysis breaks down words into their smallest meaningful units, called morphemes.
Let's look at the analysis for different word types:
1. Regular Verb
A regular verb follows a standard pattern when forming different tenses (e.g., adding "-ed" for past
tense).
• Example: "Talk"
o Root: talk
o Inflection (for past tense): -ed
o Output: talk + ed = talked
2. Irregular Verb
An irregular verb doesn't follow standard patterns and often changes the entire form of the word.
• Example: "Go"
o Root: go
o Past Tense: went (irregular form)
o Output: go → went
3. Singular Noun
A singular noun refers to one person, place, or thing.
• Example: "Dog"
o Root: dog
o Number: singular
o Output: dog (no additional morphemes)
4. Plural Noun
A plural noun refers to more than one person, place, or thing, and usually ends in "-s" or "-es" in
English.
• Example: "Dogs"
o Root: dog
o Inflection: -s
o Output: dog + s = dogs
Role of Finite State Transducers (FST) in Morphological Parsing
A Finite State Transducer (FST) is a tool used in language processing to analyze and generate word
forms. It helps in breaking down and building words by applying rules.
How FST Works in Morphological Parsing:
• Morphological Parsing means figuring out how a word changes (like from present to past
tense, singular to plural, etc.).
• An FST looks at the word, applies rules (like adding "-ed" for past tense), and gives you the
correct form of the word.
Example with FST:
Take the verb "talk":
1. Start: The word is "talk."
2. Rule: Add "-ed" for past tense.
3. Result: The word becomes "talked".
For an irregular verb like "go" → "went":
1. Start: The word is "go."
2. Rule: The FST knows that "go" changes to "went" (this is an irregular rule).
3. Result: The word becomes "went".

Q Explain FSA for nouns and verbs. Also Design a Finite State Automata (FSA)
for the words of English numbers 1-99.
Finite State Automaton (FSA) for Nouns and Verbs in Simple Terms
A Finite State Automaton (FSA) is like a simple machine or system that reads words and follows
specific rules to decide whether a word belongs to a certain category, like nouns or verbs.

FSA for Nouns (like "dog," "book")

For nouns, we need to check if the word is a singular noun (like "dog") or plural noun (like "dogs"),
or if it has a possessive form (like "John's dog").
• Start: The automaton starts by looking at the word.
• Noun: If it sees a noun (like "dog" or "book"), it moves to the next state.
• Possessive or Plural: If the noun is possessive (like "John's") or plural (like "dogs"), the
automaton adjusts the state accordingly.
• Accept: If the word fits one of these patterns, the automaton accepts it as a valid noun.

FSA for Verbs (like "run," "ran")

For verbs, we need to check if the word is in present tense (like "run") or past tense (like "ran").
• Start: The automaton starts by looking at the word.
• Verb: If it sees a verb (like "run"), it moves to the next state.
• Tense: It checks if the verb is in past tense (like "ran") or present tense (like "run").
• Accept: If the word is a valid verb form, the automaton accepts it.
FSA for Numbers 1-99 in English
Now, let’s design an FSA for English numbers between 1 and 99. Numbers in English follow specific
patterns:
• Single-digit numbers: 1 to 9 (like "one," "two").
• Tens numbers: 10, 20, 30, etc. (like "ten," "twenty").
• Teens numbers: 11-19 (like "eleven," "twelve").
• Combination of tens and units: Numbers like "twenty-one," "thirty-five."

Steps in the FSA for Numbers 1-99:

 1. Start: The automaton begins by reading the number.
2. Single-Digit Numbers: If the number is a single digit (1-9), the FSA accepts it (like "one,"
"two").
3. Tens: If the number is a multiple of 10 (like "ten," "twenty"), it moves to the next state.
4. Teens: If the number is between 11 and 19, it accepts that too (like "eleven," "twelve").
5. Tens + Units: For numbers like "twenty-one," the FSA looks at the tens (like "twenty") and
then checks if there is a unit number (like "one").
Accept: Once the FSA matches a valid number, it accepts it as correct

Q Explain Good Turing Discounting? (2023)

Good-Turing Discounting is a method used to adjust probabilities when we don't see certain events
(like rare words or things that haven't appeared in the data). In simple terms, it's a way of giving a
small chance to things that haven't been seen before, based on how often other things appear in the
data.
Here’s how it works in simple steps:
1. Count how often things happen: First, you look at how often each thing (like a word)
appears in your data. Some things might appear a lot, some a little, and some maybe not at
all.
2. Adjust probabilities: For things that appeared only once or twice, Good-Turing gives them a
slightly higher chance of happening again, even if we haven’t seen them much. It shifts
probability from the things that appeared a lot to the things that appeared less often.
3. Handle unseen things: It also gives a small chance to things that haven't been seen at all by
looking at how often other things appeared.

Q What are five types of referring expressions? Explain with the help of
example
In Natural Language Processing (NLP), referring expressions are used to refer to entities, people,
objects, or concepts within a conversation or text. These expressions can vary depending on the
context and role they play in the sentence. The five main types of referring expressions are:
1. Pronouns
Pronouns are words that stand in for a noun or noun phrase. They are commonly used to avoid
repetition and make sentences more concise.
• Example:
o "John went to the store. He bought some groceries."
Here, "He" is a pronoun referring to John.
2. Proper Nouns
Proper nouns refer to specific names of people, places, or things. They are usually capitalized and do
not require additional context to identify the entity.
• Example:
o "Alice is going to the party."
Here, "Alice" is a proper noun referring to a specific person.
3. Definite Descriptions
A definite description refers to a particular entity known to both the speaker and listener, often using
"the" or "this."
• Example:
 o "I saw the man who lives next door."
Here, "the man" refers to a specific man who is known to both the speaker and
listener, likely because of prior context.
4. Indefinite Descriptions
Indefinite descriptions refer to non-specific entities, often using words like "a" or "an." These
expressions introduce new information to the conversation.
• Example:
o "I saw a cat in the garden."
Here, "a cat" refers to any cat, not a specific one, introducing a new entity to the
conversation.
5. Demonstratives
Demonstratives are words that point to specific things, often based on proximity or context.
Common demonstratives include "this," "that," "these," and "those."
• Example:
o "Can you pass me this pen?"
Here, "this" points to a specific pen near the speaker.

Define affixes. Explain the types of affixes.

Affixes are morphemes (the smallest units of meaning) that are added to a base or root word to
change its meaning or function. Affixes do not stand alone and must be attached to a word. They
play a key role in word formation and are commonly used in many languages, including English.
Types of Affixes:
There are three main types of affixes:
1. Prefix
A prefix is an affix that is added to the beginning of a word. It changes the meaning of the
root word.
o Example:
▪ Un- in "unhappy" (means not happy)
▪ Re- in "replay" (means to play again)
2. Suffix
A suffix is an affix that is added to the end of a word. It can change the grammatical category
of the word (e.g., from noun to adjective or verb to noun).
o Example:
▪ -ly in "happily" (changes adjective "happy" to adverb "happily")
▪ -ness in "happiness" (changes adjective "happy" to noun "happiness")
3. Infix
An infix is an affix that is inserted within the root word, rather than at the beginning or end.
Infixes are less common in English, but they do exist in some other languages.
o Example (in English, often used informally or for emphasis):
▪ "Un-freaking-believable!" (The word "freaking" is inserted into
"unbelievable.")
4. Circumfix
A circumfix is a pair of affixes that are added to both the beginning and the end of a word.
This is rare in English but is used in other languages.
o Example (in German):
▪ Ge- and -t in "gespielt" (played)
Q Describe open class words and closed class words in English with examples.
(5 mark)
1. Open Class Words
These are words that carry important meaning and can easily change or grow over time. New words
are often added to this group.
• Examples:
o Nouns (things or people): dog, car, city
o Verbs (actions): run, eat, think
o Adjectives (describes nouns): happy, big, slow
o Adverbs (describes verbs/adjectives): quickly, very, here
• Key Point: Open class words are flexible, and new words can be added as language changes.

2. Closed Class Words

These words are used for grammar. They help connect ideas or show relationships but don't change
much, and new words aren't easily added to this group.
• Examples:
o Pronouns (replace nouns): he, she, it
o Prepositions (show relationship between words): in, on, at
o Conjunctions (link words or sentences): and, but, because
o Determiners (introduce nouns): the, a, this
o Auxiliary Verbs (help main verbs): is, have, will
• Key Point: Closed class words stay the same and don't change much over time.

Summary:
• Open class words are the "meaning" words like nouns, verbs, and adjectives. They can grow
and change.
• Closed class words are the "grammar" words like pronouns, prepositions, and conjunctions.
They don't change much or get new words added.