Chapter 3

Multimedia Basics and

Representation
(Audio & Image)

1
 Audio Format: Digital Audio
Application of Digital Audio - Selected
Examples
– Music Production
» Hard Disk Recording
» Sound Synthesis
» Samplers
» Effects Processing
– Video
- Audio Important Element: Music and Effects
– Web
- Many uses on Web
» Spice up Web Pages
» Listen to Cds
» Listen to Web Radio
– Many More Uses
- try and think of some?

2
 Digitization of Sound
 Sound is a continuous wave that
travels through the air
 The wave is made up of pressure
differences.
 Sound is detected by measuring the
pressure level at a particular
location.
 Sound waves have normal wave
properties (reflection, refraction,
diffraction, etc.).
3
 Digitizing Audio
• That is the basic idea of digitizing a sound
unfortunately things are (practically
speaking) not so simple.

• Questions for producing digital audio

(Analogue-to-Digital Conversion):
1. How often do you need to sample the signal?
2. How good is the signal?
3. How is audio data formatted?

4
 Computer Manipulation of Sound
 Once Digitised processing the digital sound is
essentially straightforward although it depends
on the processing you wish to do (e.g. volume is
easier to code than accurate reverb)
Sample Rates and Bit Size
How do we store each sample value
(Quantisation)?
• 8 Bit Value
– (0-255)
• 16 Bit Value
– (Integer) (0-65535)
How many Samples to take?
– 11.025 KHz Speech (Telephone 8KHz)
– 22.05 KHz Low Grade Audio (WWW Audio, AM Radio)
– 44.1 KHz CD Quality

5
 Implications of Sample Rate and
Bit Size
Affects Quality of Audio
 Ears do not respond to sound in a linear
fashion
 Decibel (dB) a logarithmic measurement
of sound
 16-Bit has a signal-to-noise ratio of 98
dB - virtually inaudible
 8-bit has a signal-to-noise ratio of 50 dB
 Therefore, 8-bit is roughly 8 times as
noisy
6 dB increment is twice as loud
6
 Typical Audio Formats
 Popular audio file formats include .au
(Unix workstations), .aiff (MAC,
SGI), .wav (PC, DEC workstations)

 A simple and widely used audio

compression method is Adaptive Delta
Pulse Code Modulation (ADPCM).
 Based on past samples, it predicts the
next sample and encodes the
difference between the actual value
and the predicted value.
7
 Computer Music - MIDI
• Sound waves, whether occurred natural or
man-made, are often very complex, i.e.,
they consist of many frequencies.
• Digital sound is relatively straight forward
to record complex sound. However, it is
quite difficult to generate (or synthesize)
complex sound.
• There is a better way to generate high
quality music.
– This is known as MIDI — Musical Instrument
Digital Interface.

8
 MIDI
 It is a communication standard developed
in the early 1980s for electronic
instruments and computers.
 It specifies the hardware connection
between equipments as well as the format
in which the data are transferred between
the equipments.
 Common MIDI devices include electronic
music synthesisers, modules, and MIDI
devices in common sound cards.
 General MIDI is a standard specified by
MIDI Manufacturers Association. To be GM
compatible, a sound generating device
must meet the General MIDI system level 1
performance requirement. 9
 MIDI
• Musical Instrument Digital Interface
a protocol that enables computer, synthesizers, keyboards,
and other musical devices to communicate with each
other.  Setup:
 MIDI OUT of synthesizer is
connected to MIDI IN of
sequencer.
 MIDI OUT of sequencer is
THRU IN OUT IN OUT
connected to MIDI IN of
Synthesizer/Keyboard MIDI Interface/Sound Card synthesizer and "through" to
(Sequencer) each of the additional sound
IN THRU
MIDI Module A
modules.
 Working:
IN THRU
 During recording, the keyboard-
MIDI Module B equipped synthesizer is used to
Etc. send MIDI message to the
sequencer, which records them.
Typical Sequencer setup
 During play back, messages are
10
sent out from the sequencer to
 MIDI: Data Format
 Information traveling through the hardware is
encoded in MIDI data format.
 The encoding includes note information like
beginning of note, frequency and sound volume;
upto 128 notes
 The MIDI data format is digital
 The data are grouped into MIDI messages
 Each MIDI message communicates one musical
event between machines. An event might be
pressing keys, moving slider controls, setting
switches and adjusting foot pedals.
 10 mins of music encoded in MIDI data format is
about 200 Kbytes of data. (compare against CD-
audio!)
11
 MIDI files
 When using computers to play MIDI music,
the MIDI data are often stored in MIDI files.
 Each MIDI files contains a number of
chunks.
 There are two types of chunks:
1) Header chunk — contains information
about the entire file: the type of MIDI
file, number of tracks and the timing.
2) Track chunk — the actual data of MIDI
track.

There three types of MIDI file:

– 0 single multi-channel track
– 1 one or more simultaneous track of a
sequence 12
 Tracks, channels and
patches
• Multiple tracks can be played at the
same time.
• Each track can be assigned to a
different channel.
• Each channel can accept more than
one track.
• Each channel is assigned a patch,
therefore generates sound of a
particular instrument.

13
How MIDI Sounds Are Synthesized
A simplistic view is that:
 the MIDI device stores the characteristics of
sounds produced by different sound sources;
 the MIDI messages tell the device which kind
of sound, at which pitch is to be generated,
how long the sound is played and other
attributes the note should have.

14
There are two ways of synthesizing sounds:
 FM Synthesis (Frequency Modulation)—Using one sine
wave to modulate another sine wave, thus generating
a new wave which is rich in timbre.
 It consists of the two original waves, their sum and
difference and harmonics.
 The drawbacks of FM synthesis are: the generated
sound is not real; there is no exact formula for
generating a particular sound.
 Wave-table synthesis— It stores representative digital
sound samples.
– It manipulates these samples, e.g., by changing the
15
 MIDI Sound Attributes
• The shape of the amplitude envelop
has great influence on the resulting
character of sound.
• There are two different types of
envelop:
– Diminishing sound — gradually die out;
– Continuing sound — sustain until turned
off. 16
IMAGE

17
 Creation of multimedia images
 Images obviously play a very important role in
multimedia products
o Images may be photograph-like bitmaps, vector-based
drawings, or 3D renderings
 The type of still images created depends on the display
resolution, and hardware and software capabilities.

 Access to the right tools and right hardware for image

development is important!
o E.g., graphic designers like to have large, high-
resolution monitors or multiple monitors

18
 Types of Still Images
 Still images are generated in two
ways:
1) Bitmaps (or raster-
based) .
2) Vector-drawn
graphics.

19
 Bitmaps
 Bitmap is derived from the words ‘bit’,
which means the simplest element in
which only two digits are used, and ‘map’,
which is a two-dimensional matrix of
these bits.
• A bitmap is a data matrix describing the
individual dots of an image that are the
smallest elements (pixels) of resolution on
a computer screen or printer.

20
 Bitmaps
• Bitmaps are an image format suited for
creation of:
– Photo-realistic images.
– Complex drawings.
– Images that require
fine detail.
• Bitmapped images are known as paint graphics.
• Bitmapped images can have varying bit and color
depths.

21
 Bitmaps
Bitmaps can be inserted by:
– Using clip art galleries. Clip Art
– Using bitmap software.
– Capturing and editing images.
– Scanning images.
Drawn

Scan Capture

22
 Vector-Drawn Images
 Vector-drawn images - created from
geometric objects such as lines,
rectangles, ovals, polygons using
mathematical formulas
 Vector-drawn images are used in the following areas:
o Computer-aided design (CAD) programs.
o Graphic artists designing for the print media.
o 3-D animation programs.
o Applications requiring drawing of graphic shapes.

23
 Vector-Drawn Images v/s Bitmaps

 Vector images use less memory space and

have a smaller file size (.svg) as compared to
bitmaps.
 Bitmaps are not easily scalable and resizable.
 Bitmaps can be converted to vector images
using auto-tracing.

 Vector images cannot be used for

photorealistic images.

 Vector images require a plug-in for Web-

based display. 24
 3-D Drawing and Rendering
• 3D graphics tools, such as
Macromedia Extreme3D, or Form-Z,
typically extend vector-drawn
graphics in 3 dimensions (x, y and z)

Z
y
25
3-D Drawing and Rendering
• Objects as a whole have properties such as
shape, color, texture, shading & location.
• A 3D application lets you model an object’s
shape, then render it completely.

26
 Features of a 3-D Application
1. Modeling involves drawing a shape, such as a 2D letter,
then extruding it or lathing it into a third dimension.
o extruding : extending its shape along a defined path
o lathing : rotating a profile of the shape around a
defined axis
 Modeling also deals with lighting, setting a camera
view to project shadows

27
 Features of a 3-D Application
2.Rendering : produces a final output of a
scene and is more compute-intensive.

28
 Computer Color Models
 Models used to specify color in
computer terms are:
o RGB model - A 24-bit methodology
where color is specified in terms of red,
green, and blue values ranging from 0 to
255.
o HSB and HSL models – Color is
specified as an angle from 0 to 360
degrees on a color wheel.
o Other model include CMYK

29
 RGB Model
■ Add red, green and blue to create colors, so it is an
additive model.
■ Assigns an intensity value to each pixel ranging from 0
(black) to 255 (white)
o A bright red color might have R 246, G 20, B 50

30
 HSB and HSL Color Models
 Based on human perception of color, three
fundamental properties of color:
 Hue – angle of 0 -360 degrees
 Saturation – intensity of color (%)
 Brightness / Lightness - relative lightness or
darkness of color (%)

31
 CMYK Model
 Based on light-absorbing quality of ink printed on
paper
 As light is absorbed, part of the spectrum is
absorbed and part is reflected back to eyes
 Associated with printing; called a modsubtractive
model
 Four channels: Cyan (C ), magenta (M), yellow (Y) and
black (K) ■ In theory, pure colors should produce black,
but printing inks contain impurities, so this
combination produces muddy brown
■ K is needed to produce pure black, hence
CMYK is four-color process printing

32
 Image File Types used in Multimedia
 Macintosh formats.
 Windows formats.
 Cross-platform formats.

33
 Macintosh Formats
 On the Macintosh, the most commonly
used format is PICT(Personalized Image Capture
Technology ).

 PICT is a complicated and versatile

format developed by Apple.
 Almost every image application on
the Macintosh can import or export
PICT files.

34
 Windows Formats
 The most commonly used image file format on Windows is DIB
or known as BMP.
 DIB stands for Device-independent bitmaps.
 Bitmap formats used most often by Windows
developers are
– BMP - A Windows bitmap file.
• Native bitmap file format of the Microsoft Windows
environment
– TIFF – Tagged Image File Format.
• Used to exchange documents between different
applications and platforms
– PCX - Used by MS-DOS paint software.
• One of the oldest bitmapped formats popularized by MS-
DOS paint programs that first appeared in the early 1980's
35
 Cross-Platform Formats
 The image file formats that are compatible
across platforms are:
o DXF, IGS or IGES - Used by CAD applications.
o CDR – CorelDraw, PSD – Photoshop n AI - Illustrator
o JPEG, PNG and GIF - Most commonly used formats on
the Web.

36