Scribd
Upload
76 views

Assignment - Digital Communication

- Block coding of symbols taken together is more efficient than symbol-by-symbol coding. Variable length coding is more efficient than fixed length coding. The most efficient approach is block symbol coding with variable length coding. - The efficiency of source coding is measured as the ratio of entropy to average code length. It should be less than or equal to 1, with 1 indicating perfect reliability without errors. - Variable length coding maps source symbols to a variable number of bits. Demarcation is needed to recover data at the receiver, which is handled by inserting synchronizing sequences.

Uploaded by

netsanet
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
76 views

Assignment - Digital Communication

- Block coding of symbols taken together is more efficient than symbol-by-symbol coding. Variable length coding is more efficient than fixed length coding. The most efficient approach is block symbol coding with variable length coding. - The efficiency of source coding is measured as the ratio of entropy to average code length. It should be less than or equal to 1, with 1 indicating perfect reliability without errors. - Variable length coding maps source symbols to a variable number of bits. Demarcation is needed to recover data at the receiver, which is handled by inserting synchronizing sequences.

Uploaded by

netsanet
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 6

Addis Ababa university

Addis Ababa institute of technology


School of electrical and computer engineering

Digital communication system

Assignment 2 on discrete source

Name: Netsanet Sinor


ID:7944/11

Submitted to Dr. Dereje H. Mariam


May 5/2020
1. Consider a discrete memoryless source (DMS) that generates symbols from a fine alphabet.
1.1. From the input side encoding the symbols can be implemented by taking each symbol at a
time (called symbol-by-symbol encoding) or by taking block of symbols at a time. Which of
these two options do you think is efficient? Explain by taking a simple example.
Answer: -

e.g. consider we have letters s1 and s2 with probability of 0.9 and 0.1 respectively. And the self-
information and codeword will be shown below
symbol Probability Codeword
S1 0.9 0
S2 0.1 1

 Let us calculate the efficiency of symbol by symbol and block of symbol encoding
For symbol by symbol

H(x)=−∑ P(s i) log2 P(si)


=0.47
R=∑ P( X )nk = 1 bits/symbol

H (X ) 0.476 bits/ symbol


η= X100%= x100%=47.6%
R 1 bits/symbol

For block of symbol

 The probability and codeword of block symbol encoding are listed below in the table

symbol probability I(s) Codeword


s1s1 0.81 0.243 0
s1s2 0.09 0.312 10
s2s1 0.09 0.312 110
s2s2 0.01 0.0664 111

H(x)=−∑ P( s i) log2 P(si)


= 0.94 bits/symbol

R =∑ P(si)nk
=0.243x1+0.312x2+0.312x3+0.0664x3
R =1.29 bits/symbol

H (X ) 0.94 bits /symbol


η= X100%= 1.29 bits/ symbol x100%
R

η = 72.8%
 Therefore, we can say that block symbol is more efficient than symbol by symbol
encoding.

1.2. From the output side, the generated codeword (i.e., group of bits that are assigned to a given
symbol) can be of fixed length (in terms of the number of bits per codeword) or variable length.
Which of these two options do you think is efficient? Explain by taking a simple example.

Answer: variable length code


e.g. Let us say we have the following source symbol

Source Probability Fixed code Codeword Variable code Codeword


symbol word length length
S0 1/2 00 2 0 1
S1 1/4 01 2 10 2
S2 1/8 10 2 110 3
S3 1/8 11 2 1114 4

For fixed length code

R=∑ P( X )n k =2x(1/2+1/4+1/8+1/8)=2
H(x)=−∑ P( x i)log2 P(xi)
=1.75bits/symbol

H (X ) 1.75bits /symbol
η= X100%= x100%=0.875%
R 2bits /symbol

For variable code

R=∑ P( X )n k =1.875
H(x)=−∑ P( x i)log2 P(xi)
=1.75bits/symbol

H (X ) 1.75bits /symbol
η= X100%= x100%=93.3%
R 1.875bits /symbol

 Therefore, we can conclude that variable length encoding is more efficient than fixed
length encoding.
1.3. If we combine the options in 1.1 and 1.2, we have four ways of implementing source coding.
Mention these four ways and which way do you think is more efficient? Explain by using a
simple example.
Answer: -
 block symbol coding
 Symbol by symbol coding
 Fixed length coding
 Variable length coding
 And Block of symbol coding is more efficient
e.g. Let us say we have symbols x1, x2 and x3 with probabilities 0.45, 0.35, 0.2, respectively.
And the code word will be
Note: - I take this example directly from the slide

R= ∑ P( X )nk=3.0675 bits/symbol

H(X)=∑ P ( X ) I ( X )=3.036bits/symbol

η =H(x)/R
3.0675bits /symbol
= *100%=99%
3.036 bits/symbol
 And if we continuous the process to three or four symbol at a time we can get 100%
efficiency.

2. Write the mathematical expression that measures efficiency of source coding.

entropy
Answer: - η = X 100 %=¿H(x)/Rx100%
average code length

2.1. Using your own terms, explain what this efficiency measures.
Answer: - It measures the performance and reliability of the code
2.2. Should this efficiency be greater than one? Or it is less or equal to one? Both? Explain the
implication of having efficiency in the three indicated ranges.
Answer: - Efficiency should be less than equal to one. Efficiency equal to one results a good
reliable of transmission without error. if its less than one, there will be some error in the decoder
side and if its greater than one, the transmitted information is incorrect or full of error because
efficiency cannot be greater than one.

3. Answer the following questions in relation to variable length encoding.


3.1. What is variable length encoding?
Answer: -is an encoding method in which the codeword length is not fixed and it’s a code which
maps source symbols to a variable number of bits. i,e (0,10,010,101)
3.2. One challenge of this encoding is the need for demarcation or synchronization. What is
demarcation or synchronization? Why is it needed? How is it handled?

Answer: -
 demarcation or synchronization is the process of recovering data from receiver side. In
variable length code one problem is that if burst or random error causes a decoding
failure all the subsequent data may be incorrectly decoded, therefore a method of
demarcation or synchronization is needed to ensure the decoding of subsequent data is
correct after the loss of data due to error. once resynchronization occurs data can be again
decoded correctly.
 by inserting a number of distinct keywords, each consisting of a synchronizing sequence
and an explicit or implicit cyclic count into the data at intervals.

3.3. What is a “prefix free” code?


Answer: -is a code in which no code word can be formed by adding code symbols to another
code word, in which no code word is prefix of another.
4. A discrete memoryless source has an alphabet of seven symbols whose probabilities of
occurrence are as described here:

4.1. Using Huffman algorithm, generate the codewords corresponding to the symbols.
Answer: -

By using binary tree, the codeword of the above alphabet is

Letter probability I(x) codeword Code length


S0 0.25 2 10 2
S1 0.25 2 11 2
S2 0.125 3 001 3
S3 0.125 3 010 3
S4 0.125 3 011 3
S5 0.0625 4 0000 4
S6 0.0625 4 0001 4
4.2. Compute the average codeword length of the Huffman code.
R= ∑ P( X )nk =0.25x2+0.25x2+(0.125x3)x3+(0.0625x4)x2
=2.625bits/symbol
4.3. Compute the efficiency of the Huffman code.
Answer
Efficiency= H(X)/R
H(X)=∑ P ( X ) I ( X )
H(X) =∑ P ( X ) log 2 P ( x i)
= (0.25x2) x2+(0.125x3) x3+(0.0625x4) x2
=2.625bits/symbol
Therefore, η =H(x)/R
2.625 bits/symbol
= *100=100%
2.625 bits/symbol

4.4. If fixed-length encoding is used instead, compute the average codeword length and
efficiency of this fixed-length coding?
Answer: -The average codeword length of fixed length code will be
R=log27=2.8 bits/symbol
Therefore, we take approximate value of 2.8 =3 bits/symbol

2.625 bits/ symbol


η = H(x)/R = *100=87.5%
3 bits/ symbol

4.5. Compare the efficiencies of Huffman code and Fixed-length encoding. What do you
conclude from the results?
Answer: - Efficiency of Huffman code is 100% and fixed length ending has 87.5% efficiency.
Therefore, Huffman coding is more efficient than fixed length encoding.

You might also like