CS 561: Artificial Intelligence

Instructor: Prof Hadi Moradi, Prof. Moradi [email protected] Lectures: M-Th 09:00-10:40, OHE136 Office hours: MW 2:30 4:00 pm, SAL310,
Or b O by appointment i TAs: Jeong-Yoon Lee
SAL 112 Office hours: TTH 1:00-2:30 Email: [email protected]

CS 561: Artificial Intelligence

Course web page:
http://www-scf.usc.edu/~csci561a Up to date information, lecture notes Relevant dates, links, etc. Also you may check http://den.usc.edu

Class format: two sections of 45 minutes Course material:

[AIMA] Artificial Intelligence: A Modern Approach, by Stuart Russell and Peter Norvig. 2nd edition

CS 561: Artificial Intelligence

Course overview: foundations of symbolic intelligent systems. Agents, search, problem solving, logic, representation, reasoning, symbolic programming, probabilistic reasoning, and robotics. Prerequisites: CS 455x, i.e.,
programming principles, discrete mathematics for computing, software design and software engineering concepts. Some knowledge of C/C++ for some programming assignments.

CS 561: Artificial Intelligence

Grading: 25% for midterm 25% for final 40% for homeworks and projects 10% for Quizzes f Q i

Practical issues
Class list: use learn usc edu learn.usc.edu
Login with your USC username and password If CSCI561A is not listed as your courses, notify the TA. ot y t e

Submissions: See class web page under Assignments submit -user csci561 -tag HW3 HW3.tar.gz

Administrative Issues
Midterm 1: 7/26/10 9:00 - 10:40pm Midterm 2: 8/10/10 9:00 - 10:40pm See also the class web page: http://den.usc.edu/ http://den usc edu/

Why study AI?

Search engines Science Medicine/ Diagnosis Labor Appliances What else?

Humanoid Robots: From Honda to Sony

Walk

Turn
http://world.honda.com/robot/

Stairs

Sony AIBO

movie1

http://www.aibo.com

Natural Language Question Answering

http://aimovie.warnerbros.com

http://www.ai.mit.edu/projects/infolab/

Robot Teams

USC robotics Lab

Modular robots self re-assembly.

What is AI?
The exciting new effort to make The study of mental faculties The computers thinks machine through the use of computational with minds, in the full and models (Charniak et al. 1985) literal sense (Haugeland 1985) The art of creating machines that perform functions that require intelligence when performed by people (Kurzweil, 1990) A field of study that seeks to explain and emulate intelligent behavior in terms of computational processes (Schalkol, 1990)

AI The Bigger Picture

?

Computer Science

Philosophy p y

Artificial Intelligence Cognitive Science (Psychology)

Robotics (Engineering)

Neuroscience (Biology)

Acting Humanly: The Turing Test

Alan Turing s 1950 article Computing Machinery Turing's and Intelligence discussed conditions for considering a machine to be intelligent

Acting Humanly: The Turing Test

What tasks require AI?

AI is the science and engineering of AI making intelligent machines which can perform tasks that require intelligence when performed by humans

What tasks require AI?

What tasks require AI?

Tasks that require AI: q
Solving a differential equation Brain surgery Inventing stuff Playing Jeopardy Playing Wheel of Fortune What about walking? What about grabbing stuff? What about pulling your hand away from fire? What about watching TV? What about day dreaming?

Acting Humanly: The Full Turing Test

Problem:

What would a computer need to pass the Turing test?

Communication: Memory: Reasoning: Learning:

What would a computer need to pass the Turing test?

Sensing:

Motor M t control (t t l t t) t l (total test):

10

Thinking Humanly: Cognitive Science

1960 Cognitive Revolution : Cognitive Revolution: information-processing psychology replaced behaviorism

Thinking Humanly: Cognitive Science

Cognitive science and modeling the activities of the brain
What level of abstraction? Knowledge or Circuits? How to validate models?

11

Thinking Rationally: Laws of Thought

Aristotle (~ 450 B.C.) attempted to codify ( right thinking
What are correct arguments/thought processes?

Thinking Rationally: Laws of Thought

Problems:

12

Acting Rationally: The Rational Agent

Rational behavior: Doing the right thing! Provides the most general view of AI because it includes:

Acting Rationally: The Rational Agent

Advantages:

13

How to achieve AI?

How is AI research done?
Theoretical Experimental

How to achieve AI?

There are two main lines of research:
Biological, study humans and imitate their psychology or physiology. phenomenal, study and formalize common sense facts about the world and the problems that the world presents to the achievement of goals.

The two approaches interact to some extent, and both should eventually succeed. It is a race, but both racers seem to be walking. [John McCarthy]

14

Branches of AI
Logical AI Search Natural language processing pattern recognition Knowledge representation Inference From some facts, others can be inferred. , Automated reasoning Learning from experience Planning To generate a strategy for achieving some goal

AI Prehistory

15

Brief History of AI
Ancient Times 384 B.C.

Thinking Rat tionally:Laws of Thought T

M iddle Age

1200

- Aristotle - Logic: The science of knowing. Ramon Lull Ars Magnus: a rule-based device to model man's behavior and nature - Empiricism Explanation of processes - Gottfried Leibniz - 1st system of formal logic Rene Descartes Dualism

Next time implement links

Renaissance 17 th Century

18 th Century

19 th Century

1845

- Charles Babbage - Analytical Engine - George Boole - Formalization of the Laws of Logic -

1879-1903

Early 20th Century

1910-1912

Gottlob Frege First-order predicate calculus Russel-Whitehead Principia Mathematica

Bertrand Russel

1931

- Kurt Godel - Incompleteness Theorem of Logic -

Roots of AI in Science:
Aristotle(b.384-): syllogism formal reasoning Ramon Lull (b.1235): Ars Magna a machine capable of answering all questions Rene Descartes (1596): mind / body separation (dualism); "cogito ergo sum Wilhelm Liebniz (1646-1716): a mechanical concept g generator; "materialism" ; Charles Babbage(1792-1871), Ada Lovelace (1815-1860): Analytical Engine a general-purpose calculator George Boole(1815-1864): logic algebras - logical encoding and calculation of thoughts Gottlob Frege(1848-1925): predicate calculus

16

Birth of Artificial Intelligence

1940-1956 1942 1943 ENIAC :First digital computer Mc Culloch and Pitts Artificial neural network J. Von Neumman Modern computer architecture

1945

1949

Claude Shannon Use of heuristics to solve complex problems

Grea Expectations at

1950

- Alan M.Turing - Computing Machinery and - Intelligence: Turing Test - Herbert Simon,Alan Newell - 1st AI program:Logic Theorist Herbert Simon

1955

1956

- Dartmouth Conference -

The Beginning of AI
McCulloch & Pitts developed theory of artificial neurons (precursor to ANN's) 1943 Alan Turing "Can Machines Think?" the turing test (1950) the turing machine Marvin Minsky & Dean Edmonds first ANN constructed, 1951 John McCarthy convened the Dartmouth conference that coined the term artificial intelligence (AI) (1956) and set the research agenda symbolic AI connectionism LISP (list processing) 1958 1st AI language

17

The Rise of AI
1957- 1960s 1958 - John McCarthy. - LISP Marvin Minsky Theory of Frames Herbert Simon,Alan Newell GPS:General Problem Solver
Herbert Simon

1960

1961

Growing Disenchantment g
Eliza - NLP

1962

Frank Rosenblatt Perceptron: Learning in Neural Networks

1965

- L tfi A. Zadeh Lotfi A Z d h Fuzyy Logic Fuzzy Sets Joseph Weizenbaum ELIZA: simulates diagnosis by a psychiatrist. - Marvin Minsky,Seymour Papert - Limitations of Perceptrons S. Papert

1968

1969

An Optimistic Start
In the 50's, 60's and early 70's, much exciting progress was being made in AI: Chess
Claude Shannon, 1950

The Logic Theorist

Alan Newell, Cliff Shaw, Herb Simon, 1957

Checkers (Machine Learning)

Arthur Samuels, 1959 Joseph Weizenbaum, 1966

DENDRAL Knowledge-Based System

Feigenbaum, Buchanan, Lederberg, 1969

SHRDLU NLP (Blocks World)

Terry Winnograd, 1972

GPS (General Problem Solver)

Alan Newell & Herb Simon, 1972

18

The 70s
Birth d Ri Bi th and Rise of E f Expert S t t Systems
1970-mid 1980s 1973 Alain Colmerauer PROLOG Paul Werbos Neural Networks Back Propagation Law E. Feigenbaum, R. Lindsay. Dendral
E.FeigenBaum

1974

1975

Edward Shortliffe MYCIN 19761980 1982 R. Duda, P.Hart, P. Barnett PROSPECTOR: The first commercial Expert System
P.Hart

John McDermott XCON "Expert Configurer

In the 70's, AI researchers began to discover that the problem wasn't as easy as it looked! The Frame Problem Lack f C L k of Common Sense Reasoning S R i Combinatorial Explosion The Gap "Toy" vs. "Real" worlds Perceptrons, by Minsky & Papert (1969) proved limitations of perceptron networks and acted to limit significant research in the 70's 70 s Lighthill Report 1973: curtailed research funding in British Universities AI developed a reputation as "over-hyped" and unrealistic

The Plateau

19

1982

The 80s
Rebirth of Art tificial Neural Netw works Commercializa ation of Expert Syst tems

- John Hopfield - Hopfield Networks Teuvo Kohonen self-organising feature maps for speech recognitizion

1982

1986

Terrence Sejnowski S j ki - T - NETTalk Rumerhalt,McMelland

Neural Networks Rediscovering of Back-Propagation Learning 1987 - Marvin Minsky - The Society of Minds -

Fuzzy Appliances

1989

- Dean Pomerleau - ALVINN -

Commercial Success
Despite it's reputation as "over hyped", certain it s over-hyped AI applications became very successful during the 70's 80's:

Expert Systems Industrial Robotics Planning & Scheduling Applications

AI became a $2,000,000,000 industry by 1988

20

Nowadays
Early 90s - Major advances in all areas of AI, with - significant demonstrations 1995 Birth of Intelligent Systems

1997

The Deep Blue chess program beats Garry Kasparov

Late 90s

- Web crawlers - AI-based information extraction - programs Intelligent Room and Emotional Agents at MIT's AI Lab

2000-

Interactive robot pets The Nomad robot

The Gartner Hype Curve

typical of the hype surrounding new technologies

Interest in AI followed this pattern pattern,

21

AI State of the art

Have the following been achieved by AI?
World-class chess playing Playing table tennis Cross-country driving Solving mathematical problems Discover and prove mathematical theories Engage in a meaningful conversation Understand spoken language Observe and understand human emotions

Types of expertise
Deep cognitive skills Highly creative Analytical
Musician Senior manager

(with examples)

Judgmental High-level skills social skills Author, poet

Mathematician i i

Economist, Social programmer scientist i ti t Driver Social worker

Typist Strictly procedural

22

A driving example: Grand Challenge

Goal:

Artificial Intelligence Applications

Artificial Intelligence

Cognitive Science Applications

Expert Systems Fuzzy Logic Genetic Algorithms Neural Networks

Robotics Applications

Natural Interface Applications

Natural Language Speech Recognition Multisensory Interface Virtual Reality

Visual Perceptions Locomotion Navigation Tactility

23

AI Application Areas in Business

Neural Networks Fuzzy Logic Systems Genetic Algorithms Virtual Reality y AI Application Areas in Business Intelligent Agents Expert Systems

Components of Expert Systems

The Expert System
Expert Advice

User Interface I t f Programs

Inference Engine E i Program

Knowledge K l d Base

User

Workstation

Expert System Development

Knowledge Engineering

Knowledge Acquisition Program Workstation

Expert and/or Knowledge Engineer

24

Expert System Applications

Decision Management Diagnostic/Troubleshooting

Maintenance/Scheduling

Design/Configuration

Major Application Categories of Expert Systems

Selection/Classification

Process Monitoring/Control

Course Overview
General Introduction
Introduction. [AIMA Ch 1] Course Schedule.
Homeworks, exams and grading. Course material, TAs and office hours. Why study AI? What is AI? The Turing test. Rationality. Branches of AI. Research disciplines connected to and at the foundation of AI. Brief history of AI. Challenges for the future. Overview of class syllabus.

25

Agent

effectors

sensors

Course Overview

General Introduction
an intelligent agent? Examples. Doing the right thing (rational action). Performance measure. Autonomy. Environment and agent design. Structure of agents Agent types Reflex agents agents. types. agents. Reactive agents. Reflex agents with state. Goal-based agents. Utility-based agents. Mobile agents. Information agents.

Intelligent Agents. [AIMA Ch 2] What is

Course Overview (cont.)

Problem solving and search.
[AIMA Ch 3]
measuring problem. Types of problems. More examples. Basic idea behind search algorithms. Complexity. Combinatorial explosion and NP completeness. Polynomial hierarchy. 3l 5l 9l
Using these 3 buckets, measure 7 liters of water.

Traveling salesperson problem

26

Course Overview (cont.)

How can we solve complex problems?

Uninformed search. [AIMA Ch

3]
Depth-first. Breadth-first. Uniform-cost. Depth-limited. Iterative deepening. Examples. Properties.

3l

5l

9l

Using these 3 buckets, measure 7 liters of water.

Traveling salesperson problem

Course Overview (cont.)

How can we solve complex p p problems?

Informed search. [AIMA Ch 4]

Best-first. A* search. Heuristics. Hill climbing. Problem of local extrema. Simulated annealing.

Traveling salesperson problem

27

Course Overview (cont.)

Practical applications of search
Constraint Satisfaction
[AIMA Ch 5]
Backtracking g Local search

Course Overview (cont.)

Practical applications of search
Game playing
[AIMA Ch 6]
The minimax algorithm. g Resource limitations. Aplha-beta pruning. Elements of chance and non-deterministic games.
tic-tac-toe

28

Course Overview (cont.)

Towards intelligent agents

Agents that reason logically 1

[AIMA Ch 7]
Knowledge-based agents. Logic and representation. Propositional (boolean) logic.

wumpus world

Course Overview (cont.)

Towards intelligent agents

Agents that reason logically 2.

[AIMA Ch 7]
Inference in propositional logic. ii ll i Syntax. Semantics. Examples.

wumpus world

29

Course Overview (cont.)

Building knowledge-based u d g o edge ased agents: 1st Order Logic
First-order logic 1. [AIMA Ch 8]
Syntax. Semantics. Atomic sentences. sentences Complex sentences. Quantifiers. FOL knowledge base. Situation calculus.

Course Overview (cont.)

Building knowledge knowledgebased agents: 1st Order Logic
[AIMA Ch 9]

First-order logic 2.
Describing actions. Planning. Action sequences.

30

Course Overview (cont.)

Reasoning Logically
Inference in first-order logic.
[AIMA Ch 9]
Proofs. Unification. Unification Generalized modus ponens. Forward and backward chaining.
Example of backward chaining

Course Overview (cont.)

Representing and Organizing Knowledge
Building a knowledge base.
[AIMA Ch 10]
Knowledge bases. Vocabulary and rules. Ontologies Organizing knowledge.
An ontology for the sports domain

31

Course Overview (cont.)

Systems that can Plan y Future Behavior
Planning.[AIMA Ch 11]
Definition and goals. Basic representations for planning. l i Situation space and plan space. Examples.

Course Overview (cont.)

Learning from Observation g
Decision Trees
[AIMA 18]
Introduction to decision trees. Information theory. Constructing DT. Examples.

32

Course Overview (cont.)

Expert Systems p y
Probabilities + Bayesian Networks
[AIMA 13 + 14]
Basics of probability theory Bayesian rule. Conditional reasoning. d l Bayesian Networks. Reasoning under uncertainty

Course Overview (cont.)

Statistical Learning Methods g
Neural Networks.
[AIMA 20]
Human brain structure Neuron and activation function. Forward and backward propagations. Examples.

33

Course Overview (cont.)

Logical Reasoning in the g g Presence of Uncertainty
Fuzzy logic
[Handout]
Center of gravity

Introduction to fuzzy logic. y g Linguistic Hedges. Fuzzy inference. Examples.

Center of largest area

Course Overview (cont.)

Machine Learning g
Genetic Algorithms
[Handout + AIMA 4]
Genetic algorithm approach. Mutation, Crossover, Fitness function. Examples.

34

Course Overview (cont.)

What challenges remain? g
Towards intelligent machines.
[AIMA Ch 25]
The challenge of robots:
with what we have learned, what hard problems remain to be solved? Different types of robots. Tasks that robots are for. Parts of robots. Architectures. Configuration spaces.

robotics@USC

Course Overview (cont.)

What challenges remain?
Overview and summary. [all of the
above]
What have we learned. learned Where do we go from here?

robotics@USC

35

Outlook
AI is a very exciting area right now. now This course will teach you the foundations.

36