1.

Introduction to multimedia
• What is Multimedia?
• Components of Multimedia
• Multimedia Research Topics and Projects
• Current Multimedia Projects:
• Hypermedia and Multimedia
• HyperText Transfer Protocol (HTTP)
• HyperText Markup Language (HTML)
• Extensible Markup Language (XML)
• Overview of Multimedia Software Tools
• Music Sequencing and Notation
• Digital Audio
• Graphics and Image Editing
• Video Editing
• Animation
• Multimedia Authoring
• What is Multimedia?
Definition of Multimedia:
Computer-based techniques of text, images, audio, video, graphics, animation,
and any other medium where every type of information can be represented,
processed, stored, transmitted, produced and presented digitally.

• Components of Multimedia
• Video teleconferencing
• Telemedicine
• Cooperative work environments that allow business people to edit a shared
document
• "Augmented" reality: placing real-appearing computer graphics and video
objects into scenes so as to take the physics of objects and lights (e.g.,
shadows) into account
• Building "inverse-Hollywood" applications that can re-create the process by
which a video was made, allowing storyboard pruning and concise video
summarization
• Multimedia Research Topics and Projects
• Multimedia processing and coding. This includes multimedia content analysis,
retrieval, security, audio/image/video processing, compression, and so on.
• Multimedia system support and networking. Network protocols, Internet,
operating systems, servers and clients, quality of service (QoS), and databases.
• Multimedia tools, end systems, and applications. These include hypermedia
systems, user interfaces, authoring systems, web-everywhere devices, multimedia
education, collaborative learning and design, and applications of virtual
environments
• Current Multimedia Projects:
• 3D motion capture can also be used for multiple actor capture, so that multiple
real actors in a virtual studio can be used to automatically produce realistic
animated models with natural movement.
• Multimedia applications aimed at handicapped persons, particularly those with
poor vision and the elderly, are a rich field of endeavor in current research.
• "Digital fashion" aims to develop smart clothing that can communicate with other
such enhanced clothing using wireless communication, so as to artificially
enhance human interaction in a social setting. The vision here is to use technology
to allow individuals to allow certain thoughts and feelings to be broadcast
automatically, for exchange with others equipped with similar technology.
• Hypermedia and Multimedia
• Hypermedia - It can include other media, such as graphics, images, and
especially the continuous media - sound and video. The World Wide Web
(www)is the best example of a hypermedia application.
• Multimedia fundamentally means that computer information can be
represented through audio, graphics, images, video. and animation in
addition to traditional media (text and graphics). Hypermedia can be
considered one particular multimedia application.
Hyper Text is Nonlinear
• HyperText Transfer Protocol (HTTP)
• HTTP is a protocol designed for transmitting hypermedia and supports
transmission of any file type. HTTP is a "stateless" request/response protocol, in
the sense that a client typically opens a connection to the HTTP server, requests
information, the server responds, and the connection is terminated - no
information is carried over for the next request.
• The basic request format is
Method URI Version
Additional-Headers
Message-body
• The Uniform Resource Identifier (URI identifies the resource accessed, such as
the host name, always preceded by the token ''http://''. Method is a way of
exchanging information or performing tasks on the URI. Two popular methods
are GET and POST. GET specifies that the information requested is in the request
string itself, while the POST method specifies that the resource pointed to in the
URI should consider the message body. POST is generally used for submitting
HTML forms.
• Additional Headers specifies additional parameters about the client. For
example, to request access to this textbook's web site, the following HTTP
message might be generated:
• GET http://vrolW.cs.sfu.ca/mmbook/ HTTP/1.1
• HyperText Markup Language (HTML)
• HTML is a language for publishing hypermedia on the World Wide Web. It is
defined using SGML and derives elements that describe generic document
structure and formatting.
• The next generation of HTML is XHTML, a reformulation of HTML using XML.
• HTML uses tags to describe document elements. The tags are in the format
<token params> to define the start point of a document element and </ token>
to define the end of the element.
• Some elements have only inline parameters and don't require ending tags.
• HTML divides the document into a HEAD and a BODY part as follows:
<HTML>
<HEAD>
</HEAD>
<BODY>
</BODY>
</HTML>
• The HEAD describes document definitions, which are parsed before any
document rendering is done. These include page title, resource links, and meta-
information the author decides to specify.
• The BODY part describes the document structure and content. Common
structure elements are paragraphs, tables, forms, links, item lists, and buttons.
• Extensible Markup Language (XML)
• Using XML, you would use a global Document Type Definition (DTD) you have already defined for
stock data.
• You will send users your XML Style Sheet (XSL), depending on the type of device they use to display
the information, so that your document looks best both on a computer with a 2l-inch CRT monitor
and on a cellphone.
• Overview of Multimedia Software Tools
The categories of software tools we examine here are
• Music sequencing and notation
• Digital audio
• Graphics and image editing
• Video editing
• Animation
• Multimedia authoring
• Music Sequencing and Notation
• Cakewalk Cakewalk is a well known older name for what is now called Pro Audio.
• Macromedia Soundedit Soundedit is a mature program for creating audio for multimedia projects.
• Digital Audio
• Sound Forge Sound Forge is a sophisticated PC-based program for editing WAY files.
• Sound can be captured from a CD-ROM drive or from tape or microphone through the sound card,
then mixed and edited.
• Pro Tools Pro Tools is a high-end integrated audio production and editing environment that runs on
Macintosh computers as well as Windows.
• Graphics and Image Editing
• Adobe Illustrator illustrator is a powerful publishing tool for creating and
editing vector graphics easily exported on the web.
• Adobe Photoshop Photoshop is the standard in a tool for graphics, image
processing, and image manipulation. Layers of images, graphics, and text can
be separately manipulated.
• Video Editing
• Adobe Premiere Premiere is a simple, intuitive video editing tool for
nonlinear editing - putting video clips into any order.
• Animation
• Multimedia APIs
• Java3D is an API used by Java to construct and render 3D graphics, similar to the way
Java Media Frameworkhandles media files. top of OpenGL or DirectX
• Maya, a competing product to Softimage, is a complete modeling package. It features a
wide variety of modeling and animation tools, such as to create realistic clothes and fur.
• Multimedia Authoring
• Macromedia Flash Flash allows users to create interactive movies by using
the score metaphor - a timeline arranged in parallel event sequences, much
like a musical score consisting of musical notes.
Synchronized Multimedia Integration
Language (SMIL)
• The SMIL language structure is similar to XHTML. The root element is smil,
which contains the two elements head and body.
• Head contains information not used for synchronization - metainformation,
layout information, and content control, such as media bitrate.
• Body contains all the information relating to which resources to present, and
when.
• seq specifies that the elements grouped are to be presented in the specified
order (sequentially).
• Alternatively, par specifies that all the elements grouped are to be presented
at the same time (in parallel).
An example of SMIL code:
Video Transitions
• Video transitions can be an effective way to indicate a change to the
next section.
• Video transitions are syntactic means to signal "scene changes" and
often cany semantic meaning.
• Many different types of transitions exist; the main types are cuts,
wipes, dissolves, fade-ins and fade-outs.
Cuts/Wipes
• A cut, as the name suggests, carries out an abrupt change of image
contents in two consecutive video frames from their respective clips.
It is the simplest and most frequently used video transition.
• A wipe is a replacement of the pixels in a region of the viewport with
those from another video.
• If the boundary line between the two videos moves slowly across the
screen, the second video gradually replaces the first.
• Wipes can be left-to-right, right-to-Ieft, vertical, horizontal, like an iris
opening, swept out like the hands of a clock, and so on
Dissolves/fade-ins and fade-outs
• A dissolve replaces every pixel with a mixture over time of the two
videos, gradually changing the first to the second.
• A fade-out is the replacement of a video by black (or white), and fade-
in is its reverse.
• Most dissolves can be classified into two types, corresponding, for
example, to cross dissolve and dither dissolve in Adobe Premiere
video editing software.
• In type I (cross dissolve), every pixel is affected gradually. It can be
defined by

• where A and B are the color 3-vectors for video A and video B. Here,
α(t) is a transition function, which is often linear with time t:
• Type II (dither dissolve) is entirely different.
• Determined by α(t), increasingly more and more pixels in video A will
abruptly (instead of gradually, as in Type 1) change to video B.
• The positions of the pixels subjected to the change can be random or
sometimes follow a particular pattern.
• Obviously, fade-in and fade-out are special types of a Type I dissolve,
in which video A or B is black (or white).
• Wipes are special forms of a Type II dissolve, in which changing pixels
follow a particular geometric pattern.
• Despite the fact that many digital video editors include a preset
number of video transitions, we may also be interested in building our
own.
• For example, suppose we wish to build a special type of wipe that
slides one video out while another video slides in to replace it.
• The usual type of wipe does not do this.
• Instead, each video stays in place, and the transition line moves
across each "stationary" video, so that the left part of the viewport
shows pixels from the left video, and the right part shows pixels from
the right video (for a wipe moving horizontally from left to right).
• Suppose we would like to have each video frame not held in place,
but instead move progressively farther into (out of) the viewport: we
wish to slide VideoL in from the left and push out VideoR.
• Figure shows this process.
• Each of VideoL and VideoR has its own values of R, G, and B.
• Note that R is a function of position in the frame, (x, y), as well as of
time t.