CS294-137: Theory and

Applications of Virtual
Reality and Immersive
Computing
Bjoern Hartmann, Allen Y. Yang

Fall, 2019
Instructors

Bjoern Hartmann Allen Y. Yang

Email: Email:
[email protected] [email protected]
GSI

Joe Menke

PhD student

Email:
[email protected]
Course Resources: Jacobs Institute
hAp://jacobsinsDtute.berkeley.edu/
Course Resources: FHL Vive Center

hAps://vivecenter.berkeley.edu/
Course Resources: Berkeley VR Club

l  Founded in Spring 2015

l  >200 members in each
semester enrollment

vr.berkeley.edu

VR DeCal (CS198-080)
Prerequisites
1.  Mandatory:
1.  CS Lower division or equivalent (CS61a, CS61b)
2.  Ability to program in C, C++, C#

2.  OpEonal (but you should know about at least one of these topics):
1.  User Interfaces (CS160)
2.  Computer Graphics (CS184)
3.  OpEcal Engineering (EE218A)
4.  Image Processing (EE225B)
5.  Computer Vision (CS280)
6.  InteracEve Device Design (CS294-84)
Enrollment
Graduate students can enroll directly.

Undergraduate students:
To request enrollment, please complete the following form:
hZp://bit.ly/enroll-arvr-fall2019

We have a small number of slots available. We will email out enrollment
codes on a first come, first served basis for those who meet the prerequisites.
Grading Policy
Your final grades will be determined by three factors:

1.  AZendance by you (10%)

2.  Homework assignments (40%)
•  Spotlight presentaEon of an AR/VR experience
•  Follow ARCore/ARKit tutorial to develop an AR board game
•  Develop an AR landmark tracking algorithm using OpenCV
•  (Team) Develop a Tilt Brush app on Vive Focus Plus

3.  Course/capstone project (50%)

•  Intermediate deliverables (20%)
•  Final presentaEon and demo (15%)
•  A research paper + video discussing your project and its relevant
literature and commercial applicaEons (15%)
Course Project Requirements

•  Goal: demonstrate either a novel, compelling applicaEon of AR/VR
technologies or make a research contribuEon to the field (could be
graphics, vision, interacEon)

•  Each team must consist of 3-4 students

(EECS M.Eng students will form their own teams and projects)

•  Team formaEon will happen by end of September.

•  Encourage working closely with our industrial partners in AR/VR

Course Project Schedule
•  Encourage students to pitch and form their own projects

•  Pitch Day: Sep 24. Aeer that, all students must commit to one project

•  Mid-term presentaEon: 10 min presentaEon plus 2-page research

summary paper

•  Final Exam: 20-minute presentaEon, research paper, video
Example Projects

Human Avatar – Bill Zhou, Winnie Lee

12
Example Projects

hAps://www.youtube.com/watch?v=LuqFP2Vx358
Sibo Chen, Junce Wang, SanDago Guerra, Neha MiAal, Soravis Prakkamakul
13
Example Projects

14
Administrivia

Course website on bCourses
hZps://bcourses.berkeley.edu/courses/1486180

CommunicaEon with Instructors:
Use Piazza! Do not send individual email – you can elect to only message
instructors for personal maZers.
Office Hours

•  Bjoern: Thu 5-6pm, 220A Jacobs

•  Allen:
•  General: Tue 10-12, 337 Cory Hall or by appointment
•  M.Eng students: weekly meeEngs previously assigned.

•  Joe: TBD
Recommended Reading Material
•  PercepEon: SensaDon and PercepDon
by Bruce Goldstein
•  Virtual Reality: Virtual Reality
By Steven LaValle (and checkout his YouTube lectures)
•  Computer Graphics: Fundamentals of CG
by Peter Shirley
•  Computer Vision: An InvitaDon to 3-D Vision
by Yi Ma, et al.
•  Display: Mobile Displays
by Achin Bhowmik, et al.
•  AR/VR Market Research: Virtual & Augmented Reality,
understanding the race for the next compuDng plaeorm
by Goldman Sachs
•  3D User Interfaces: Theory and PracDce
LaViola et al.
Goals of the Course
•  Understand the fundamental theories that enable VR/AR and Immersive
CompuDng (IC)

•  Understand the main technology drivers of VR/AR and IC markets

•  Become an expert in criDquing the current sogware and hardware

soluDons

•  Be able to design, prototype and implement new soluDons that address

exisDng needs / pain points of VR/AR and IC applicaDons

•  (OpDonal) Being moDvated to pursue a career in relevant research or

entrepreneurial fields
Ready Player One: A Future in VR
Leia’s Hologram: Probably the first
AR experience from Hollywood
More in Hollywood Movies
A Real Market since 2016
CES 2017: Samsung 4D VR Experience
What we have learned about the challenges and
opportunities?
Sensorama: The First VR Experience

Morton Heilig, 1958

The First VR Head-Mounted Display

Telesphere Mask Patent

Morton Heilig, 1960
The First AR Prototype

Ivan Sutherland, 1968

The First AR Prototype

Ivan Sutherland, 1968

First VR/AR Applications in 1980s

Military Training NASA VR Lab

First Consumer Products

MaAel’s Power Glove as Virtuality

Nintendo controller

Sony 3D Viewer
SEGA VR Nintendo Virtual Boy
1958 1977 1980s 1990s-2010s 2012 2018
What is Reality?
The Senses of Reality

Reasoning Taste
Speech Touch
AcEvity Planning OrientaEon

Vision

Hearing
Smell Fina Muscle
Movement
Anatomy of an AR Device: HoloLens
Summary of Corresponding Techs

Graphics Photography Stereoscopy

3D Audio InteracDon
 Early Forms of Photography

Camera Obscura, circa 400BC First photo on paper, 1800s Kodachrome, 1935
3D Illusion with Perspective

Ames Room, by Adelbert Ames, Jr., 1946

Seeing 3D from Stereo

Wheatstone mirror stereoscope, 1838 Holmes stereoscope, 1861

Audio: From Mono to Spatial 3D

Edison cylinder phonograph, 1899 InvenDon of headphones, 1910s

Dolby Stereo, 1977 Dolby Atmos for VR

 Interaction Modalities

Typewriter keyboard, Christopher Mouse, Douglas Engelbart, 1963

Sholes 1870s

Dataglove, Thomas Zimmerman, 1982 iOS and Siri, Apple

Outside-In Localization 

(Vive Lighthouse)
Inside-Out Localization 

(Google Tango)
Tilt Brush in Virtual 3D
AR on Smart Phones
Other Forms of Space Augmentation
Definition: Virtual Reality
•  VR is a computer technology that uses head mounted
displays, sometimes in combination with other
sensory devices, to generate realistic images, sounds,
and other sensations (touch, smell, motion, etc.) that
simulate a user’s physical presence in a virtual
environment.

46
Definition: Augmented Reality
•  AR is a computer technology that augments a
physical, real-world environment directly or through
its indirect view computer-generated sensory
information, including graphics, video, and sound. AR
may alter a user’s view of reality, and may also
enhance one’s perception of reality.

47
Assignments
•  Undergraduates / waitlist:

Complete enrollment request form 

due Sunday Sep 1, 11:59pm


•  Install Unity and follow intro tutorials (if you haven’t

used it before) by Thu Sep 5, 3:29pm

48