CIS 607         Seminar on Deep Learning for Natural Language Processing

Course Description

Deep learning has recently proved itself as a powerful branch of machine learning that has applications in a variety of domains (i.e., computer vision, robotics, etc.). In natural language processing (NLP), deep learning has been transformative and led to a new generation of methods with better performance, portability and robustness (e.g., machine translation, text generation/dialog systems). Especially, with the recent breakthrough in pretrained language models (e.g., BERT, GPT), building state-of-the-art NLP models can be done efficiently for different domains and languages. All of those advances are very recent and the demand for data scientists with deep learning expertise is growing very quickly. At the beginning of this seminar, we will cover the basic concepts of deep learning and NLP, and possibly provide some hand-on experience to implement the models. Afterward, we will review and discuss a collection of research papers on NLP with deep learning, including but not limited to the typical tasks of language modeling, question answering, information extraction, machine translation, natural language inference, dialog, summarization, domain adaptation, transfer learning, and multilingual learning. NLP is growing fast these days and we expect to read many exciting recent papers in this field.

Instructor

Thien Huu Nguyen, [email protected]

Lectures

Prerequisites

• Some background in algebra and machine learning and NLP (e.g., CIS 472/572) is helpful. Please talk with the instructor if you are not sure if you have the necessary background.

Textbooks and Readings

• DL:

  Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, 2016 (Online!)

• Goldberg: Neural Network Methods for Natural Language Processing, 2017 (free download when on UO campus)

• Kyunghyun Cho's note

• ACL Anthology (free and covering the papers in most of major NLP venues)

Major Topics

• Introduction to natural language processing: 1 lecture
• Research paper presentations: 9 lectures

Expected Learning Outcomes

In this seminar, we will review and discuss a collection of research papers on NLP with deep learning, including but not limited to the typical tasks of language modeling, question answering, information extraction, natural language inference, dialog and summarization.

Upon successful completion of the course, students will be able to:

• demonstrate facility with research topics and applications in natural language processing;
• demonstrate facility with recent advances and models in natural language processing, i.e., transformer, pre-trained language models, BERT, GPT.

Acquired Skills

Upon successful completion of the course, students will have acquired the following skills:

• build skills on technical presentation and discussion;
• build skills to read research papers, develop ideas, think critically and write technical papers to summarize research results.

Class Organization

IMPORTANT:

Tentative Schedule

Course Requirements and Grading

This course will be taught in-person. We might also stream and record the lectures over Zoom upon request. We use Canvas and Piazza for communication and discussion.

Grading will be based on P/NP.