Lecture 1: Introduction

CS312 - Natural language processing

Spring 2024

1
Course information
➢ Lectures
○ Time: 9:45am - 11:15am Mon/Wed

○ Venue: CR2

➢ Materials
○ Textbook: Speech and Language Processing (3rd ed. draft), Dan Jurafsky and James H. Martin

■ https://web.stanford.edu/~jurafsky/slp3/

○ Lectures/Homework: Canvas

➢ Instructor
○ Linh Huynh ([email protected])

○ Office hours:

■ 15 minutes after the lectures

■ Tue/Thu (1pm-2pm, appointments by email) 2

Grading
➢ Homework: 8 sets = 35%
○ No late homework

➢ Mini-exam (4 sets) = 28%

○ 30-minute writing exam

➢ Project = 37%
○ Question-answer tasks

100%

3
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Any improvements?

❖ Is GitHub Copilot useful?

➢ Any improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

4
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Any improvements?

❖ Is GitHub Copilot useful?

➢ Any improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

5
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Any improvements?

❖ Is GitHub Copilot useful?

➢ Any improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

6
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Any improvements?

❖ Is GitHub Copilot useful?

➢ Any improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

7
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Ideas for improvements?

❖ Is GitHub Copilot useful?

➢ Any improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

8
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Ideas for improvements?

❖ Is GitHub Copilot useful?

➢ Any improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

9
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Ideas for improvements?

❖ Is GitHub Copilot useful?

➢ Ideas for improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

10
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Ideas for improvements?

❖ Is GitHub Copilot useful?

➢ Ideas for improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

11
Why CS312?
❖ What does “ChatGPT” stand for?
❖ Is ChatGPT perfect?
➢ Is ChatGPT good enough?
➢ Ideas for improvements?

❖ Is GitHub Copilot useful?

➢ Ideas for improvements?

❖ Programming languages will be replaced by natural languages soon?

❖ …

12
What is natural language processing (NLP)?
➢ Automating the analysis and generation of human “natural” language
○ Analysis

■ Inputs are languages

○ Generation

■ Outputs are languages

13
Why NLP?
❖ Data scientists
➢ Process text data

❖ Software engineers
➢ Develop programming tools

❖ Even biologists …
➢ Utilize medical text data
➢ Process DNA sequences

14
A brief history
➢ Symbolic NLP (1950s – early 1990s)

➢ Statistical NLP (1990s - 2010s)

➢ Neural NLP (2010s - )

15
Levels of linguistic representation
Input text

Morphology

Lexemes

Syntax

Semantics

Pragmatics

16
Discourse
Why NLP is not easy?
➢ List all flights on Monday

➢ Retrieve all the local patient files

➢ I saw her duck with a telescope

17
Why NLP is not easy?
➢ List all flights on Monday

➢ Retrieve all the local patient files Ambiguity is ubiquitous!

➢ I saw her duck with a telescope

18
NLP applications
❖ Common tasks
➢ Named entity recognition
➢ Sentiment analysis
➢ Semantic parsing
➢ Coreference resolution
➢ Discourse analysis
❖ High-level tasks
➢ Automatic summarization
➢ Machine translation
➢ Dialogue management
➢ Question answering
➢ Text-to-image generation
➢ … 19
 In
tro
du
ct
io
Ba n
s ic
te
xt
La pr
ng oc
ua es
ge si
ng
Ba m
y es od
ia el
n in
g

Exam 1
Lo c la
gi ss
Tentative schedule

st ifi
ic ca
W r eg tio
or re n
d ss
em io
be n
dd
Exam 2 in
gs

N
eu
ra
ll
an
gu
ag
e
m
od
el
Exam 3

Ap
p lic
at
io
ns
Exam 4

St
u de
n tp
re
se
nt
a
20

tio
n
Summary
➢ Natural language processing NLP
○ Analysis

○ Generation

➢ CS302 only covers some key topics

“We can only see a short distance ahead,

but we can see plenty there that needs to be done”

Alan Turing

21