0% found this document useful (0 votes)

21 views

EUC1502 Module6 TextualAnalysis

This document provides an overview of textual analysis, including common tasks like text classification, clustering, and association analysis. It discusses preprocessing techniques like stemming, removing stopwords, and feature selection. Representation models covered include the vector space model, bag-of-words, TF-IDF, n-grams, and statistical language models. Linguistic resources like lexicons, gazetteers, and ontologies are also mentioned.

Uploaded by

Радомир Мутабџија

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

21 views

EUC1502 Module6 TextualAnalysis

Uploaded by

Радомир Мутабџија

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 99

Analysis of Textual Data

1
Outline

1. Introduction to textual analysis

2. Tasks
3. Preprocessing and feature selection
4. Linguistic resources
5. Representation models and weighting schemes
6. Learning and evaluating models
7. Avoiding overfitting
8. Practice with R: Language Variety Identification

2
Outline

1. Introduction to textual analysis

3
Introduction to textual analysis

Textual analysis aims at extracting knowledge from collections of

unstructured text documents.

Unstructured text documents means:

● Natural language texts

● Source code

● Any other kind of textual information

4
Outline

1. Introduction to textual analysis

5
Tasks

● Terms extraction

● Text classification / categorisation

● Clustering

● Association (of concepts)

6
Tasks
- Terms extraction

7
Tasks
- Classification &
categorisation

Is the following sentence positive, negative, neutral or none?

Risk premium is 120

It depends on the subjectivity of the receiver, e.g.
- President of the government
- Leader of the opposition
- Director of the European bank
- Investors
- People with mortgages

8
Tasks
- Classification &
categorisation

9
Tasks
- Clustering

10
Tasks
- Association (of concepts)

11
Outline

1. Introduction to textual analysis

12
Preprocessing and feature selection
- Stemming

consign

consign ed
STEMMING
consign
consign ing

consign ment

13
Preprocessing and feature selection
- Stopwords

● List of words without

meaning

● Also named function words

○ The have syntactical

function

14
Preprocessing and feature selection
- Information Gain

IG measures the expected reduction in entropy when selecting

the examples according to a given attribute.

● T: training examples
● a: attribute
● val(a): value of attribute a
● x: example
● y: class 15
Preprocessing and feature selection
- Principal Components
Analysis

16
Outline

1. Introduction to textual analysis

17
Linguistic resources
- Lexicons

WORD EMOTION PFA

abundance joy 0.830

harass anger 0.364

terrorise fear 0.966

18
Linguistic resources
- Gazetteers

19
Linguistic resources
- Ontologies

20
Linguistic resources
- Ontologies

21
Outline

1. Introduction to textual analysis

22
Representation models & weighting schemes
- Vector Space Model

23
Representation models & weighting schemes
- Vector Space Model

24
Representation models & weighting schemes
- Bag of Words

● Simplest way to represent text

● A text is represented as a set of its words
● Does not consider the order of the words
● Create a vector with the size of the vocabulary seen in train
● Each sentence is represented counting the number of times
each word appears(frequency)
● Or simply counting if the word appears or not (One-hot
representation)
● Frequencies can be normalized
25
Representation models & weighting schemes
- Bag of Words

S1 = “The cat and the dog are friends”

S2 = “The dog hate the cat because they are not friends”

Vocabulary = “the, cat, and, dog, are, friends, hate, because, they, not”

BoW(S1) = “2, 1, 1, 1, 1, 1, 0, 0, 0, 0”

BoW(S2) = “2, 1, 0, 1, 1, 1, 1, 1, 1, 1”

26
Representation models & weighting schemes
- Binary weighting

S1 = “The cat and the dog are friends”

S2 = “The dog hate the cat because they are not friends”

Vocabulary = “the, cat, and, dog, are, friends, hate, because, they, not”

BoW(S1) = “1, 1, 1, 1, 1, 1, 0, 0, 0, 0”

BoW(S2) = “1, 1, 0, 1, 1, 1, 1, 1, 1, 1”

27
Representation models & weighting schemes
- Frequency weighting

S1 = “The cat and the dog are friends”

S2 = “The dog hate the cat because they are not friends”

Vocabulary = “the, cat, and, dog, are, friends, hate, because, they, not”

BoW(S1) = “2/7, 1/7, 1/7, 1/7, 1/7, 1/7, 0, 0, 0, 0”

BoW(S2) = “2/10, 1/10, 0, 1/10, 1/10, 1/10, 1/10, 1/10, 1/10, 1/10”

28
Representation models & weighting schemes
- TF-IDF

● Common technique from Information Retrieval (IR)

● This word representation tries to model how important a

word is to a document in a corpus

● Two terms are combined (or weighted): term frequency and

inverse document frequency

29
Representation models & weighting schemes
- TF-IDF

● Term Frequency

○ Count the number of times that term t occurs in a corpus c

● Inverse Document Frequency

○ To avoid common terms with low relevance, the idf

measures the relative importance in all the documents
number of documents where
the term t appears
30
Representation models & weighting schemes
- Statistical Language
Model

● A probability distribution over sequences of words

● Able to handle some word ordering

● Several assumptions are taken into account:

○ The probability of a word w depends on certain history

(previous words) seen

○ The Markov assumption: the probability of a word only

depends on a fixed number of the previous words (1 for
unigrams, 2 for bigrams, . . . ) 31
Representation models & weighting schemes
- Statistical Language
Model

● Given a sequence of length n, the probability that W random

variables took the values of the sequence w 1 n can be
described using the chain rule:

32
Representation models & weighting schemes
- Statistical Language
Model

● Applying the Markov assumption:

● Statistical language models can deal with out-of-vocabulary

words applying different smoothing techniques such as:

○ Backoff
○ Linear interpolation
○ Good-Turing
○ ...
33
Representation models & weighting schemes
- n-grams

● A BoW approach can be atomized to use smaller elements

● A n-gram is a continuous sequence of n elements: words or

characters

● Most common n-grams:

○ 1-gram (unigrams)

○ 2-grams (bigrams)

○ 3-grams (trigrams)
34
Representation models & weighting schemes
- n-grams

S1 = “The cat and the dog are friends”

S2 = “The dog hate the cat because they are not friends”

2-grams = “the cat, cat and, and the, the dog, dog are, are friends, dog
hate, hate the, cat because, because they, they are, are not, not friends”

2-grams(S1) = “1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0”

2-grams(S2) = “1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1”

35
Representation models & weighting schemes
- EmoGraph

The objective of EmoGraph is to capture the use of emotions and

their rol in the textual structure. Rangel et al. IP&M 2016

36
Representation models & weighting schemes
- EmoGraph

37
Representation models & weighting schemes
- EmoGraph

38
Representation models & weighting schemes
- EmoGraph

39
Representation models & weighting schemes
- EmoGraph

40
Representation models & weighting schemes
- EmoGraph

41
Representation models & weighting schemes
- EmoGraph

42
Representation models & weighting schemes
- EmoGraph