Virtual Reality A. Fundamentals: Content
Virtual Reality A. Fundamentals: Content
Content
1. Background
1.1 Literature and fundamentals
1 2 From flight simulator to flight simulator
1.2
A. Fundamentals 3. Applications
3.1 Transport
Daniel Thalmann – Stéphane Gobron 3 2 Civil
3.2 Ci il engineering
i i
3.3 Psychotherapy
3.4 Bio-medical
3.5 Entertainment
4. Short debate
1
06/03/2010
• Potential applications
~ medical imaging: training & diagnostic, future surgery?
~ interior design & architectural mock-up, civil engineering
~ videoconferencing
~ exploration of future worlds
~ ethics, philosophy, psychology, who am I & what are we?
~ entertainment
• VR often thought of as new technology
~ but development dates back almost 50 years to flight simulators built by
aircraft industry and U.S. Air Force during and after World War II
2
06/03/2010
1.2 From flight simulator to flight simulator 1.2 From flight simulator to flight simulator
History of Virtual Reality Cinerama
• VR's future influenced by film techniques!
• Stereoscopic 3-D cinema and wide-screen systems • Cinerama, best-known of these technologies, sought to expand movie-
• Hollywood
H ll d starting
t ti ffrom early
l 1950's
1950' going experience by filling a larger portion of audience's
audience s visual field
AVATAR, 2009
1.2 From flight simulator to flight simulator 1.2 From flight simulator to flight simulator
Cinerama Sensorama
• Film then synchronized and • in 1962, Sensorama: simulation
projected onto 3 large screens of sensory experiences of a
that curved inward, wrapping motorcycle ride by combining
around audience's peripheral 3D movies, stereo sound,
visual field wind, and aromas
• Technology proved too costly to • Special seat and binocular
be embraced by most showing California landscapes
commercial theatres and Brooklyn streets
• Theoryy of visual immersion
becomes an important VR
element ?
3
06/03/2010
... ...
Ivan Sutherland
1.2 From flight simulator to flight simulator 1.2 From flight simulator to flight simulator
Ivan Sutherland (2) The Aspen Movie Map
• 1968: described HMD that tracked viewer and
updated graphics display to correctly reflect new • Developed by group of researchers including Scott Fisher at MIT
viewing position • Showed video images of Aspen,
Aspen Colorado,
Colorado USA
• Used of two displays visible from a pair of • Visitors could navigate by indicating their choices on touch-
half_silvered mirrors => stereoscopic computer sensitive display screen
graphics images overlaid onto the real world
• In the late 1960's and 1970's, research on a
number of fronts formed basis of today VR
Scott Fisher
4
06/03/2010
1.2 From flight simulator to flight simulator 1.2 From flight simulator to flight simulator
Videoplace NASA
• Videoplace: 1970 • Mid-1980's: different technologies converged to
• One of several experimental
p artistic environments y
create first true VR system
designed by arts scholar Myron Krueger • Researchers at NASA' s Ames Research Centre charged
• Computer responds to gestures of audience by Myron Krueger with creating affordable pilot training system for
interpreting, and even anticipating actions manned space missions
• Audience members could "touch" each other's video- • People involved: Scott Fisher, Stephen Ellis, Michael
generated silhouettes, and animated organisms McGreevy, and Warren Robinett
computers used to create what Krueger called "artificial • => development of Virtual Interface Environment
reality" Workstation Scott Fisher @ NASA
• First system combined such standard VR elements as
computer graphics and video imaging, 3-D sound, voice
recognition and synthesis, and HMD
1.2 From flight simulator to flight simulator 1.2 From flight simulator to flight simulator
NASA (2) Visual Programming Language (VPL)
• Jaron Lanier's company, VPL Research, first company to
• Data glove, based on invention designed to play air
focus efforts on developing products for infant VR
guitar, completed the system
industry, and provided headgear and gloves used in many
• 1984: Mike McGreevy and Jim Humphries early VR applications
=> VIVED (VIrtual Visual Environment Display ) • Head- tracking helmets and data gloves, wired to a
• They evaluated potential of monochrome HMD system VIVED specially programmed computer system, provide
for future astronauts traditional entry into virtual worlds
VIEW
Jaron Lanier
5
06/03/2010
1.2 From flight simulator to flight simulator 1.2 From flight simulator to flight simulator
Virtual worlds without using immersion devices Flight simulators
• PHANTOM
• developed at MIT A.I. Lab • Development of flight simulators has
• creates illusion of touching virtual objects made very significant contributions to
• Projected systems development of VR
• often used in museums and for medical displays • Much of technology needed for VR
• take image of user's motions developed for US Air Force
• display it with other images on large screen • Graphics included
• Simulation VR • friend /foe identification
• widely found in VR game arcades
• use combination of video monitors and movable platforms • targeting information
PHANTOM
• create
t virtual
i t l experiences
i • threat information,
information e.g.
e g based missile sites
• optimal flight path information
1.2 From flight simulator to flight simulator 1.2 From flight simulator to flight simulator
Flight simulators (2) Flight simulators (3)
• Magic Edge • Fighter pilot
• Restaurant/bar in Mountain View, California, offers VR flight simulators • operating under extremely high stress levels
• Fighter
g pilots battle each other
p
• Fully rendered virtual landscape • both
b h cognitive and
d physical
h l
• yet has to assimilate and process masses of data
Magic Edge
6
06/03/2010
2. Keywords 2. Keywords
Triangle of Virtual Reality Triangle of Virtual Reality (2)
• Interaction
Possibilty of moving in the 3D space and manipulate objects
VR • Real Time
Actions can immediatelyy modify
fy the state off the space
p
Immersion Interaction
7
06/03/2010
2. Keywords 2. Keywords
Virtual Environments (VEs) VEs (2)
• VE: technology capable of shifting a subject into a different • Acceptance of VEs in industrial application, i.e.:
environment without physically moving him/her • product development
• Subject's
S bj t' sensory organs manipulated
i l t d to
t perceived
i dd desired
i d VE • product presentation
2. Keywords 2. Keywords
Immersion Immersion (2)
• Paradigm: a world representation, a way to see the world
According to Slater, participant "immersed" in VE in two ways
• Immersion is a key issue in VR, central to paradigm where
1. Through VE displaying sensory data depicting his surroundings
• user becomes part of simulated world
• Part of immediate surroundings consist of representation of participant's body and
• rather than simulated world being feature of user's own world environment displayed from unique position and orientation defined by place of
participant's viewpoint within environment
• First “immersive VR systems”: flight simulators where immersion
• Body tracking devices, such as electromagnetic sensors enable movements of person's
achieved by subtle mixture of real hardware and virtual imagery whole body and limbs to become part of dynamic changes to objects in VE under his
immediate control
2 Proprioceptive signals about disposition and dynamic behavior of
2.
human body and its parts become overlaid with consistent sensory
data about representation of human body: "Virtual Body" (VB)
8
06/03/2010
2. Keywords 2. Keywords
Immersion (3) Immersion (4) – e.g. turning the head around
• Usual conditions
A very simple example
• Displays should be stereoscopic, e.g. HMDs.
• Resolution should be significant
g
• Continual stream of sensory data: 1. A perceiver experiences a change in ambient optic array corresponding to turn
• Visual 2. Side objects become occluded by head, go out of view, others come into view
• Auditory 3. There is translation of whole, and
• Tactile re-arrangement of object occlusion
• Kinesthetic relationships
• Olfactory sensory data 4 Objects and parts of body,
4. body now occlude
different parts of other surfaces, and
become occluded in different way
• External sensory data not enough to create immersion themselves
• human body must itself be tracked
• displays driven by movements of human body
2. Keywords 2. Keywords
Immersion (5) – e.g. turning the head around (2) Presence
• The person cause changes in the scene • According to Slater:
• e.g. movement of arm will cause change in occlusion structure “An Immersive Virtual Environment (IVE) may lead to sense of presence for participant
takingg part
p in such an experience”
p
• Presence: psychological sense of "being there“
• Immersion should requires overall body tracked in environment based on technologically founded immersive base
=> changes transmitted to display systems => “great immersion” • Any given immersive system => not always lead presence
=> depends of person
• Head tracking essential: users point of view for proper rendering • Presence so fundamental to our everyday existence
• graphical projections => very difficult to define
• eventually 3D sound, etc.
9
06/03/2010
2. Keywords 2. Keywords
Presence (2) Presence (3): competing signals
• Consider negation sense of presence: • Presence especially interesting when competing signals from
• loss of locality at least two ‘environments’
• “no
no presence
presence” => “no
no locality”
locality • How
H will
ill you act?
t?
• Participant “forget”
forget physical world =>
> treat virtual world as real
• E.g. flight simulators => very high degree of presence for users
• Only provide presence in one relatively fixed environment => airplane cockpit
• VR systems should provide presence
• limited only by imaginations of environment designers
2. Keywords 2. Keywords
Presence (4): as a selector Presence vs. immersion
• Given competing signals • A book
Choose action based on selection amongst hypotheses • A TV program
• I am in this world • as game where the public could interact
• I am in that world
• A Film
• Am I mixed up?
• A video game at home:
• Hypotheses relating to the fundamental question:
• Small screen, mouse & keyboard
~ Where am I?
• Large video screen and 3D sound
• Eye toys© Sony
• A video game in game center
• A simulation VR center
• A dream
10
06/03/2010
2. Keywords
Domain responses
• Reading a novel => psychological
• Movie => + some body movement
• 3D movie => + more body movement
• Game => + interactive body movement
• Immersive VR => + total body movement
• Specific emotional topic: e.g. “Fire!”
• In a movie or computer game, no one would rush out of the room
• In immersive VR – they might do so!
2. Keywords 2. Keywords
Effect: implications for measurement Measurement
• Questionnaires deliver an integration over time of • BIPs ‘Breaks In Presence’ – possible to build a measure
• conscious/voluntary/supported responses based on when these occur
• Behavioural
B h i l measures require
i • Sampling
S li
• imposition of events • Behavioural observation
• may not be part of the environment
• Unification of BIPs + physiological?
• E.g. standing on top of a chair in virtual or real worlds
• Deliberate introduction of conflicting signals
• E.g. shadows
• Physiological measures => specific types of event
• E.g. anxiety provoking: social Phobia
• Biofeedback measures
• hearth rhythm, blood pressure, skin conductivity, etc.
VR Treadmill
“makes running indoors feel like running outdoors”
11
06/03/2010
2. Keywords
Cave
• Surround-screen projection-based VR system
• Illusion of immersion created by projecting 3D CG into
• E.g.
E cube
b composed
d off display
di l screens surround
d the
h viewer
i 3 Applications
3. A li ti
• Coupled with head tracking systems 3.1 Transport
• Other tracking systems e.g. Hand 3.2 Civil engineering
• Usually surround audio feedback 3.3 Psychotherapy
3.4 Bio-medical
• Viewer explores virtual world by moving and interacting in the VE 3.5 Entertainment
12
06/03/2010
Major issues
• Extremely difficult to conduct => special training
• Expensive and dangerous load => security
e.g. Augmented “virtuality”?!
13
06/03/2010
• Archit. Michel Marot (inaug. 1972) • an e.g. of complexity: how solving the slope issue and mixing nature?
14
06/03/2010
15
06/03/2010
16
06/03/2010
Helsinki
NY
Berlin
Venezia
LaDefence
• D
Do you have
h some
free time to improve
virtual Lausanne? ☺
Lausanne
17
06/03/2010
18
06/03/2010
19
06/03/2010
3.4 Bio-medical
Simulation (1/3): testing and experiment
3. Applications
3.4 VR applied to bio-medical • Microorganism spreading simulation
- Simulation:
Si l ti di
diagnostic,
ti training,
t i i f
forecast
t
- Dealing with pain • Phytoplankton:
population density
- Surgery
and health situation
- Future
• Food supply
density
20
06/03/2010
21
06/03/2010
(Burdea et al.)
3.4 Bio-medical
Future?
3. Applications
3.5 VR applied to emergency situations
Psychotherapy, phobia, treatment of PTSDT
real-time
full body
scanning
22
06/03/2010
23
06/03/2010
24
06/03/2010
3.6 Entertainment
Video-game
• area which starts to drive
development of VR technology
3. Applications
pp
• biggest limiting factor in VR
3.6 VR applied to Entertainment research today: sheer expense of
Video game, exploring, sport training technology because low volumes
• For entertainment, mass
production required
• Another alternative: development
of "Virtual
Virtual Worlds"
Worlds for
Lunaparks/casinos
Atlantis Cyberspace
25
06/03/2010
3.6 Entertainment
Sport training: e.g. Football
• Faculty of sport science of Marseille, FR
• Full immersive environment: four projection
screens (of 3x4 m)
• Virtual human development with motion
capture of professional soccer player
-Olympic de Marseille
4. Short discussion
26