EEE51502 MULTIMEDIA

TECHNOLOGY AND APPLICATIONS

&
EEE52214 NETWORKED MULTIMEDIA

Lecture 2
Dr. Hiba Hassan Sayed
Department of Electrical and Electronic Engineering
University of Khartoum
11/9/2024 Dr. Hiba HAssan : U of K 2

Multimedia Presentation
• To make readable colors on a screen it is
customary to use the principal complementary
color as the background for text.
• For color values in the range 0–1 (or 0–255), if
the text color is R, G, B, the background color is
usually given by,
(R, G, B) ⇒ (1 − R, 1 − G, 1 − B)
• That is, the color is “opposite” in terms of
contrast, brightness…etc.
11/9/2024 Dr. Hiba HAssan : U of K 3

Color Wheel
• The colors are represented in a color wheel.
• The color wheel used for Graphics design is
different from the color wheel used by artists.
• In the color wheel for graphics design, the
opposite colors are calculated by the equation
mentioned earlier,
(𝑅𝑅, 𝐺𝐺, 𝐵𝐵) ⇒ (1 − 𝑅𝑅, 1 − 𝐺𝐺, 1 − 𝐵𝐵)
• While artist’s color wheel is based on feel rather
than on an algorithm.
• The following diagrams show the 2 color wheels.
11/9/2024 Dr. Hiba HAssan : U of K 4

Artists color wheel Graphics Design Color

Wheel
 11/9/2024 Dr. Hiba HAssan : U of K 5

Color Models
• These components are considered in color models.
• Hence, an RGB model is further represented by:
• The Hue, Saturation and Value (HSV).
• The hue (‫ )درﺟﺔ اﻟﻠون‬saturation (‫ﺻﻔﺎء اﻟﻠون‬/‫)ﺗﺷﺑﻊ‬
resembles various shades of brightly colored
paint, and the value resembles the mixture of
those paints with varying amounts of black
(closer to 0)or white (approaching 255) paint.
• And the Hue, Saturation and Lightness (HSL)
• places fully saturated colors around a circle at a
lightness value of 1⁄2
11/9/2024 Dr. Hiba HAssan : U of K 6

Human Visual System (HVS)

• The theory of opponent colors states that the
human visual system de-correlates its input into
white-black, red-green and blue-yellow difference
signals, which are processed in separate visual
channels.
• For video transmission, this color difference
(chroma) signals are computed instead of direct
red, blue or green information.
• Furthermore, the analog video systems made use
of that by representing the luminance (perception
of brightness) and color difference (chroma)
separately.
11/9/2024 Dr. Hiba HAssan : U of K 7

Image/Video Processing
• In component video, for example, the resulting
color space is referred to as YUV or YCBCR,
where Y encodes luminance, U or CB the
difference between the blue primary and
luminance, and V or CR the difference between
the red primary and luminance .
• Computing the color differences instead of direct
colors has reduced the amount of data processed
considerably.
11/9/2024 Dr. Hiba HAssan : U of K 8

Image/Video Processing (cont.)

• Current high-efficiency digital color image data
compression schemes such as JPEG and MPEG
store RGB color internally in YCBCR format.
• The human visual system is more sensitive to the
luminance information than to the color information.
• Hence, the luminance components, Y, to
chrominance components, U and V, are usually 4:1:1.
• To illustrate; the number of samples in each CIF
video frame is: 352 × 288 Y samples, 176 × 144 U
samples, and 176 × 144 V samples.
11/9/2024 Dr. Hiba HAssan : U of K 9

Graphics/Image Data types

• Some common file formats:
11/9/2024 Dr. Hiba HAssan : U of K 10

1-bit images
• Images consist of pixels.
• A 1-bit image consists of on and off bits only and
thus is the simplest type of image.
• Each pixel is stored as a single bit (0 or 1).
Hence, such an image is also referred to as a
binary image.
• It is also called a 1-bit monochrome image,
since it contains no color.
 11/9/2024 Dr. Hiba HAssan : U of K 11

1-bit images (cont.)

• A 640×480 monochrome
image requires 38.4
kilobytes (kB) of storage
(= 640×480/8).
Monochrome 1-bit images
can be satisfactory for
pictures containing only
simple graphics and text.
Moreover, fax machines
use 1-bit data.
11/9/2024 Dr. Hiba HAssan : U of K 12

8-bit (Gray-Scale) Images

• Each pixel is represented by a byte.
• Each pixel has a value between 0 and 255.
• A dark pixel may have a value of 10, and a bright
one value may be 230.
• These images are two-dimensional arrays of pixel
values, also known as bitmaps.
• An Image resolution refers to the number of
pixels in a digital image. The higher the number
of pixels, the higher the resolution.
• Storage Example: A 640 x 480 greyscale image
requires 307.2 KB of storage.
 11/9/2024 Dr. Hiba HAssan : U of K 13

8-bit - Bit Planes

• An 8-bit image can be thought of as a set of bit-
planes
• Each plane consists of a 1-bit representation of the
image at higher levels of elevation.
• A bit is turned on if the image pixel >= bit level
11/9/2024 Dr. Hiba HAssan : U of K 14

8-bit (cont.)
• The pixel value is calculated as follows:
 11/9/2024 Dr. Hiba HAssan : U of K 15

8-bit (cont.)
• The first plane gives the most contribution to the
image.
 11/9/2024 Dr. Hiba HAssan : U of K 16

24-Bit Color Images

• Each pixel is represented by three bytes (e.g., RGB)
• It supports 256 x 256 x 256 (16,777,216) possible
colors.
• A 640 x 480 24-bit color image is 921.6 KB large
• Some color images are 32-bit images,
• the extra byte of data for each pixel is used to
store an alpha value representing special effect
information.
 11/9/2024 Dr. Hiba HAssan : U of K 17

Color Lookup table

• Each image is assigned an index, or a code value.
• Those indices are stored in a table and it is called
color lookup table (LUT).
• Hence, if a pixel stores, e.g., the value 25, the RGB
color is retrieved by looking up row 25 in a LUT.
• The LUT is often called a palette.
11/9/2024 Dr. Hiba HAssan : U of K 18
 11/9/2024 Dr. Hiba HAssan : U of K 19

Palette Animation
• A simple animation process can be achieved by
changing the color table, this process is called color
cycling or palette animation.
• The color cycling technique was used in early
computer games.
• Storing one image and changing its palette required
less memory and processor power than storing the
animation as several frames (images).
11/9/2024 Dr. Hiba HAssan : U of K 20

Palette Animation Example

• http://www.effectgames.com/demos/canvascycle/
?sound=1
11/9/2024 Dr. Hiba HAssan : U of K 21

Image Resolution
• There are 3 forms of resolutions in images:
1. Intensity resolution
Each pixel has “Depth” bits for colors/intensities.
2. Spatial resolution
Image has “Width” x “Height” pixels.
3. Temporal resolution
Monitor refreshes images at specific rates
(measured in Hz).
11/9/2024 Dr. Hiba HAssan : U of K 22

Error Sources
• There are 3 main error sources in digital image
processing:
• Intensity quantization;
Not enough intensity resolution
• Spatial aliasing;
Not enough spatial resolution
• Temporal aliasing;
Not enough temporal resolution
11/9/2024 Dr. Hiba HAssan : U of K 23

Quantization
• Quantization corresponds to a discretization of the
intensity values.
• Hence, for an image of intensity i(x,y), different
intensities are represented by LEVELS.
• For uniform quantization, p(x,y), is defined by
p(x,y) = trunc(LEVELS * i(x,y))
• And the error is given by
e(x,y) = p(x,y)/(LEVELS-1) - i(x,y).
• The image total error is the mean square error.
11/9/2024 Dr. Hiba HAssan : U of K 24

Uniform Quantization
Example
11/9/2024 Dr. Hiba HAssan : U of K 25

Applying Quantization
• Notice the color jumps, known as contouring.
 11/9/2024 Dr. Hiba HAssan : U of K 26

Dithering
• A possible solution for the contouring effect is
Dithering.
• Dithering distributes errors amongst pixels
11/9/2024 Dr. Hiba HAssan : U of K 27

Dithering Techniques
• There are several dithering techniques.
• The following will be exploited in this course:
1) Random dither
2) Ordered dither
3) Error diffusion dither
 11/9/2024 Dr. Hiba HAssan : U of K 28

Random Dither
• The quantization errors are randomized, i.e. errors
are diffused.
• The pixels are scanned in order and errors in
realizing a pixel's intensity are distributed (ie diffused)
to keep the overall intensity of the image closer to the
input intensity.
P(x,y)=trunc(I(x,y) + noise(x,y))
11/9/2024 Dr. Hiba HAssan : U of K 29
 11/9/2024 Dr. Hiba HAssan : U of K 30

Order Dithering
• Dithering is often used when converting greyscale
images to bit-mapped ones e.g. for printing.
• The pixel value is represented by a larger pattern,
(2×2 or 4×4 matrix).
• This results in number of printed dots that
approximates the varying-sized disks of ink used in
halftone printing.
• Halftone printing is an analog process used in
newspaper printing. It uses smaller or larger filled
circles of black ink to represent shading.
 11/9/2024 Dr. Hiba HAssan : U of K 31

Halftone Patterns
• The intensities in a 3× 3 cluster are given by:
11/9/2024 Dr. Hiba HAssan : U of K 32

Order Dithering (cont.)

• We use an n × n matrix of on–off 1-bit dots to
represent n2 + 1levels of intensity resolution
• Hence, a dithered image size is much larger than
the original one since replacing
• Each pixel by a 4 × 4 array of dots, say, makes an
image 16 times as large.
11/9/2024 Dr. Hiba HAssan : U of K 33

Order Dithering (cont.)

• If a pixel is replaced by a 4 × 4 array of dots, the
intensities it can approximate from 0 (no dots) to
16 (full dots). Given a 4 × 4 dither matrix e.g.

• We can re-map pixel values from 0 - 255 to a new

range 0-16 by dividing the value by (256/17) (and
rounding down).
11/9/2024 Dr. Hiba HAssan : U of K 34

Dithering (cont.)
• Replace each pixel by a 4× 4 dots (binary pixels).
If the remapped intensity is > the dither matrix
entry, put a dot at the position (set to 1) otherwise
set to 0.
• To keep the image size: an ordered dither
produces an output pixel with value 1 iff the
remapped intensity level just at the pixel position
is greater than the corresponding matrix entry.
11/9/2024 Dr. Hiba HAssan : U of K 35

Order Dithering Example

• Dithering of grayscale images. a 8-bit gray image
lenagray.bmp; b dithered version of the image; c
detail of dithered version for the right eye.
11/9/2024 Dr. Hiba HAssan : U of K 36
11/9/2024 Dr. Hiba HAssan : U of K 37

Comparing Original, Random and

Ordered Dither Images
11/9/2024 Dr. Hiba HAssan : U of K 38

Error Diffusion Dither

• Spread quantization error over to neighboring pixels

• Where 𝛼𝛼 + 𝛽𝛽 + 𝛾𝛾 + 𝛿𝛿 = 1
• The most famous such technique is Floyd–Steinberg
dithering.
11/9/2024 Dr. Hiba HAssan : U of K 39

Floyd–Steinberg Dithering Algorithm

11/9/2024 Dr. Hiba HAssan : U of K 40
11/9/2024 Dr. Hiba HAssan : U of K 41

Color Quantization
• Reasonably accurate color images can be
obtained by quantizing the color information to
collapse it.
• Many systems can make use of only 8 bits of color
information (constitutes 256 colors) in producing a
screen image.
• Even if a system has the electronics to use 24-bit
information, backward compatibility demands that it
can understand 8-bit color image files.
11/9/2024 Dr. Hiba HAssan : U of K 42

Some Common Image Formats

• GIF: Graphic Interchange Format
• JPEG: Joint Photographic Experts Group
• PNG: Portable Network Graphic
• TIFF: Tag Image File Format
• EXIF: Exchange Image File
• BMP: Microsoft Bitmap
• WMF: Windows Metafile
 11/9/2024 Dr. Hiba HAssan : U of K 43

Common System Independent Formats

• Graphics Interchange Format (GIF87a, GIF89a):
• Devised by the UNISYS Corp. and Compuserve, initially
for transmitting graphical images over phone lines via
modems.
• Uses the Lempel-Ziv Welch algorithm (a form of Huffman
Coding).
• It is lossless format that is limited to 8-bit (256) color
images making it appropriate for images with few
distinctive colors (e.g., graphics drawing).
• It supports interlacing; interlacing allows a quick sketch
to appear when a web browser displays the image,
followed by more detailed fill-ins. .
• GIF89a: supports simple animation, transparency index
etc.
11/9/2024 Dr. Hiba HAssan : U of K 44

GIF File Format

 11/9/2024 Dr. Hiba HAssan : U of K 45

JPEG
• It is a standard for photographic image
compression created by the Joint Photographic
Experts Group.
• It takes advantage of limitations in the human
vision system to achieve high rates of
compression.
• The color information in JPEG is decimated
(partially dropped, or averaged).
• JPEG is a lossy compression which allows user to
set the desired level of quality/compression.
11/9/2024 Dr. Hiba HAssan : U of K 46

Cont.
• In JPEG image compression standard, the
amount of compression is controlled by a value Q
in the range 0–100 (and see Sect. 9.1 for details).
The “quality” of the resulting image is best for Q =
100 and worst for Q = 0.
11/9/2024 Dr. Hiba HAssan : U of K 47

Fundamentals of Multimedia , by: Ze-Nian Li, Mark S. Drew,

Jiangchuan Liu, 2e – Chapter 2

Fig. 2.9 JPEG compression: a original uncompressed image; b JPEG

compression with Quality Factor Q = 75 (the typical default); c, d Factors Q = 25
and Q = 5
11/9/2024 Dr. Hiba HAssan : U of K 48

Fundamentals of Multimedia , by: Ze-Nian Li, Mark S.

Drew, Jiangchuan Liu, 2e – Chapter 2
11/9/2024 Dr. Hiba HAssan : U of K 49

PNG
• Portable Network Graphics (PNG) is meant as
system independent image format.
• PNG intended to surpass GIF standard
• Some features of PNG
1. Support up to 48 bits per pixel {more accurate
colors}
2. Support description of gamma-correction and
alpha-channel for controls such as transparency
3. It supports both lossless and lossy compression
with performance better than GIF.
• PNG is widely supported by various web browsers
and imaging software.
 11/9/2024 Dr. Hiba HAssan : U of K 50

Tagged Image File Format (TIFF)

• TIFF, stores many different types of images (e.g., bit-
map, greyscale, 8-bit & 24-bit RGB, etc.)
• tagged Developed by the Aldus Corp. in the 1980's and
later supported by the Microsoft
• It supports attachment of additional information “tags”.
• Its most important tag is a format signifier; image type,
compression type, …etc.
• TIFF is a typically lossless format but a tag may be
utilized to opt for lossy compressions such as JPEG
2000.
• TIFF files are divided into sections, each of which can
store different image type.
• Each section’s data type is specified in its tag.
11/9/2024 Dr. Hiba HAssan : U of K 51

Postscript (PS)
• A typesetting language which includes text as well as
vector/structured graphics and bit-mapped images.
• Output in several popular graphics programs (E.g.
Illustrator, FreeHand)
• PostScript page description language does not provide
compression, they are stored as ASCII.
• Consequently, files are often large.
• It is common for PS files to be made available only
after compression by some Unix utility, such as
compress or gzip.
• Many high-end printers have a PostScript interpreter
built into them.
 11/9/2024 Dr. Hiba HAssan : U of K 52

Portable Document Format (PDF)

• It utilizes LZW compression.
• Therefore , “PDF files that do not include images
have about the same compression ratio, 2:1 or 3:1,
as do files compressed with other LZW-based
compression tools, such as the Unix compress or
gzip, or the PC-based winzip (a variety of pkzip) or
WinRAR.
• For files containing images, PDF may achieve
higher compression ratios by using separate JPEG
compression for the image content.”
 11/9/2024 Dr. Hiba HAssan : U of K 53

Bitmap (BMP)
• It is a major system standard image file format for
Microsoft Windows.
• It uses raster graphics (grid or dot matrix data
structure).
• BMP supports many pixel formats, including
indexed color (up to 8 bits per pixel), and 16, 24,
and 32-bit color images.
• It makes use of Run-Length Encoding (RLE)
compression.
• BMP images can also be stored uncompressed.
 11/9/2024 Dr. Hiba HAssan : U of K 54

Video Standardization Organizations

• There are two main video coding standardization
organizations:
1. The International Telecommunications Union —
Telecommunication Standardization Sector (ITU-
T), emerged from a United Nations Organization,
formerly known as International Telegraph and
Telephone Consultative Committee (CCITT).
2. The Moving Picture Experts Group (MPEG)
emerged from the International Organization for
Standardization (ISO), and the International
Electrotechnical Commission (lEC).
 11/9/2024 Dr. Hiba HAssan : U of K 55

Video Standards
• Experts from these two standardization organizations
had collaborated several times to produce joint
standards. The most significant standards produced as
such, so far, are:
1. The H.262/ MPEG-2 Part 2 developed as a result of
joint partnership in 1996.
2. The advanced video coding (AVC) H.264/ MPEG-4
Part 10 developed by the Joint Video Team (JVT) in
2003.
3. The high efficiency video coding (HEVC);
H.265/MPEG-H Part 2 developed by Joint
Collaborative Team on Video Coding (JCT-VC) in
2013.
 11/9/2024 Dr. Hiba HAssan : U of K 56

Latest Standards
• HEVC is restricted by patents, and the licensing
fees are far higher than those for H.264/AVC, this
has discouraged most major tech companies from
deploying it.
• Hence, some of the largest companies; Google,
Mozilla, Intel, Microsoft, Netflix, Amazon, Cisco,
have founded a group called Alliance for Open
Media (AOMedia) in 2015.
• The AOMedia is working on a standard called
AOMedia Video 1 (AV1). First working version
released on January 2019. It targets real-time
applications.
11/9/2024 Dr. Hiba HAssan : U of K 57

Compression
• There are 2 types of compression: Lossy &
Lossless :
• Lossless : Ideal (e.g. zip, unix compress) not
good enough for MM data.
• Lossy :Throw away nonessential (perceptually
less relevant) parts of the data stream.
• FILTER the data.
• Examples: MP3, JPEG, MPEG Video/Audio
11/9/2024 Dr. Hiba HAssan : U of K 58

Compression (cont.)
• Compression via Synthesis :
• Encode how to make (synthesize) the data can
be done in many less bits in certain cases.
• Examples: Vector Graphics (Flash), MPEG Video,
MP4 (Audio), MIDI