Scribd
Upload
35 views

EE 6340: Information Theory Quiz II - Apr/10/2014: I TH I TH 1 2 3 4

This document contains a 3 question quiz on information theory. Question 1 involves analyzing a Huffman code constructed for a source with 4 outcomes and determining codeword lengths. Question 2 involves determining if the number of outcomes K is odd for a source encoded with a ternary code. Question 3 analyzes the capacity of a 3-input, 3-output discrete memoryless channel for various values of a parameter p.

Uploaded by

Dominic Mendoza
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
35 views

EE 6340: Information Theory Quiz II - Apr/10/2014: I TH I TH 1 2 3 4

This document contains a 3 question quiz on information theory. Question 1 involves analyzing a Huffman code constructed for a source with 4 outcomes and determining codeword lengths. Question 2 involves determining if the number of outcomes K is odd for a source encoded with a ternary code. Question 3 analyzes the capacity of a 3-input, 3-output discrete memoryless channel for various values of a parameter p.

Uploaded by

Dominic Mendoza
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

EE 6340: Information Theory

Quiz II - Apr/10/2014

Remarks

• You are allowed to bring one formula sheet (hand-written).

• If anything is not clear, make/state your assumptions and proceed.

Problems

1. (5 pts) Let source X has four outcomes {1, 2, 3, 4} with pi being probability of ith
outcome. Let a binary Huffman code is constructed for X and `i denote the length of
codeword for ith outcome. It is given that p1 > p2 = p3 = p4 .

(a) Show that if p1 > 0.4 then `1 = 1.


(b) Show (by example) that if p1 = 0.4 then a Huffman code exists with `1 > 1.

2. (5 pts) Let X be a random variable with K outcomes. Let H3 (X) denote the entropy
of X in ternary units. An instantaneous ternary code is found for this source with
expected code length L = H3 (X). Show that K is odd.

3. (10 pts) Consider the 3-input 3-output discrete memoryless channel illustrated below.

(a) What is the channel capacity for p = 1?


(b) What is the channel capacity for p = 0.5?
(c) Show that, for any value of p, channel capacity C ≥ 1 (bit per channel use).
(d) Find the channel capacity for a general value of p.

X Y
1−p
0 0
p

p
1 1
1−p

2 1
2

You might also like