Scribd
Upload
46 views10 pages

Compression: Background

The document discusses Shannon-Fano encoding, which is a method for constructing efficient binary codes for information sources without memory. It works by recursively splitting the set of messages into equiprobable subsets at each step and assigning code bits until each message has a unique code. This ensures the average code length does not exceed the entropy of the message set. An example is provided to illustrate the encoding process. The properties are that the Shannon-Fano code is not necessarily unique and may not always be optimal if equiprobable partitioning is not possible.

Uploaded by

tsegay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
46 views10 pages

Compression: Background

The document discusses Shannon-Fano encoding, which is a method for constructing efficient binary codes for information sources without memory. It works by recursively splitting the set of messages into equiprobable subsets at each step and assigning code bits until each message has a unique code. This ensures the average code length does not exceed the entropy of the message set. An example is provided to illustrate the encoding process. The properties are that the Shannon-Fano code is not necessarily unique and may not always be optimal if equiprobable partitioning is not possible.

Uploaded by

tsegay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

Compression:

Background
 The main idea behind the
compression is to create such a
code, for which the average length of
the encoding vector (word) will not
exceed the entropy of the original
ensemble of messages.
 This means that in general those
codes that are used for compression
are not uniform.
Shannon-Fano
Encoding
 Sources without memory are such sources
of information, where the probability of
the next transmitted symbol (message)
does not depend on the probability of the
previous transmitted symbol (message).
 Separable codes are those codes for which
the unique decipherability holds.
 Shannon-Fano encoding constructs
reasonably efficient separable binary
codes for sources without memory.
Shannon-Fano
Encoding
 Shannon-Fano encoding is the first
established and widely used encoding
method. This method and the
corresponding code were invented
simultaneously and independently of
each other by C. Shannon and R.
Fano in 1948.
Shannon-Fano
Encoding
 Let us have the ensemble of the
original messages to be transmitted
with their corresponding
probabilities:

 Our task is to associate a sequence C k


of binary numbers of unspecified
length n k to each message x k such
that:
Shannon-Fano
Encoding
 No sequences of employed binary
numbers C k can be obtained from each
other by adding more binary digits to the
shorter sequence (prefix property).
 The transmission of the encoded message
is “reasonably” efficient, that is, 1 and 0
appear independently and with “almost”
equal probabilities. This ensures
transmission of “almost” 1 bit of
information per digit of the encoded
messages.
Shannon-Fano
Encoding
 Another important general
consideration, which was taken into
account by C. Shannon and R.
Fano, is that (as we have already
considered) a more frequent message
has to be encoded by a shorter
encoding vector (word) and a less
frequent message has to be encoded
by a longer encoding vector (word).
Shannon-Fano
Encoding: Algorithm
 The letters (messages) of (over) the input alphabet
must be arranged in order from most probable to
least probable.
 Then the initial set of messages must be divided into
two subsets whose total probabilities are as close as
possible to being equal. All symbols then have the
first digits of their codes assigned; symbols in the
first set receive "0" and symbols in the second set
receive "1".
 The same process is repeated on those subsets, to
determine successive digits of their codes, as long as
any sets with more than one member remain.
 When a subset has been reduced to one symbol, this
means the symbol's code is complete .
Shannon-Fano Encoding:
Example
Message X1 X2 X3 X4 X5 X6 X7 X8
Probability 0.25 0.25 0.125 0.125 0.0625 0.0625 0.0625 0.0625

x1,x2,x3,x4,x5,x6,x7,x8
0 1
x1,x2 x3,x4,x5,x6,x7,x8
00 01 10 11
x1 x2 x3,x4 x5,x6,x7,x
8
100 101 110 111
x3 x4 x5,x6 x7,x8

1100 1101 1110


x5 x6 x7 x8
Shannon-Fano Encoding:
Example
Message X1 X2 X3 X4 X5 X6 X7 X8
Probabilit 0.25 0.25 0.125 0.125 0.0625 0.0625 0.062 0.062
y 5 5
Encoding 00 01 100 101 1100 1101 1110 1111
vector

Entropy 1 1 1 1 1 1
H 2 log 2 log 4 log 2.75
4 4 8 8 16 16

Average length of the encoding vector

1 1 1
L P xi ni 2 2 2 3 4 4 2.75
4 8 16

The Shannon-Fano code gives 100% efficiency


Shannon-Fano Encoding:
Properties
 It should be taken into account that the Shannon -
Fano code is not unique because it depends on the
partitioning of the input set of messages, which, in
turn, is not unique.
 If the successive equiprobable partitioning is not
possible at all, the Shannon-Fano code may not be an
optimum code, that is, a code that leads to the lowest
possible average length of the encoding vector for a
given D.

You might also like