0% found this document useful (0 votes)
47 views14 pages2210 Sol
Uploaded by
Grace LeeCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
0% found this document useful (0 votes)
47 views14 pages2210 Sol
Uploaded by
Grace LeeCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
/ 14
STUDENT ID NO
MULTIMEDIA O | UNIVERSITY
MULTIMEDIA UNIVERSITY
FINAL EXAMINATION
‘TRIMESTER 1, 2022/2023 (TERM ID 2210)
TIT3131 - INFORMATION THEORY
(Alll sections / Groups)
11 FEBRUARY 2023
9.00 am ~ 11.00 am
(2 Hours)
INSTRUCTIONS TO STUDENTS
1. This question paper consists of 7 pages with Four questions only.
2. Attempt AN questions. All questions carry equal marks and the distribution of the marks
for each question is given,
3. Please print all your answers in the Answer Booklet provided. RY:
QUESTION 1.
Question 1 contains 10 MCQ questions, please choose the best answer and write it clearly
in the answer booklet.
@ Which of the following statement is FALSE? [1 mark] & G
[A] Entropy isa measure of uncertainty. lecture ol-02- part | 4)
[B] The amount of information, I(S)) is related to the inverse of the
probability of occurrence.
Huffman coding represents high order relationships. V
‘None of above.
® From the encoding Table 1 below, decode the following codeword,
"y100101101111". [1 mark}
Tablet
Symbol Code.
A 0
B 10
a io
L 110
E nit
“LABEL” CABCE
“CABCE”
{Cl “CABLE”
{D] None of above.
© Which of the following statement is TRUE? [1 mark]
[A] Block codes in which message bits are transmitted in altered form
are called systematic codes.
[B] Block codes in which message bits are transmitted in unaltered form
are called non-systematic code.
A cyclic code is net a linear block code.
‘None of above.
Property 1 restates that
a cyclic code is a linear block code
(ie. it can be described as a parity-check code)
Continued...
CCTEO wa TnL
INEORMATION THEOR} HL FEBRUARY 2023
@ Given an DMS, S = {A, B, C, D} and the probability of occurrence for these
symbols are showed in the Table 2. Caleulate the probability of symbol “B”
and information gained for symbol B. [1 mark]
Table2
Se A B Cc D
Bk. O1 2 oa| or 2e
1S) sy [7 sah
[A] pa=0.01 and44Sx)—3:3219 bivsymbol
[B] po=0.1 and I(Ss) = 2.3219 bit/symbol
IC] pa =0.2 and (Sp) = 1.3219 bit/symbol
‘None of above,
© is more when uncertainty is more. (0 mark]
Information.
Entropy.
[C] Source coding.
[D] Channel coding.
© Given a code C which is defined as C= {1 = 10000, c2= 11101, e3= 00101,
ca= 11001}. Then C is not linear. [1 mark]
TRUE.
[B]_ FALSE.
© Entropy is directly proportional to the uncertainty. [mark]
tainty +
arue, MPH tT martenty
] FALSE. ontvopy b wnuvtainty
) From the Table 3 below, given that the Code I satisfies Kraft inequality
and it is a prefix code, illustrate the meaning of “Unique decodable” by
estimating a suitable code x. [1 mark]
Table3
‘Symbol Code i
A 00.
B OL
C x
D i
{A} x=000.
(B] x=011.
(C] x=10.
[D] None of above.
Continued...
CCTEO 37 mmmiaa___Infomarion THEORY _1 FEBRUARY 2023
o
Consider that a 100 kbps data stream is to be transmitted on a voice-grade
telephone line with a bandwidth of 3 KHz. with signal-to-noise of 10 dB. The
Channel capacity C = 10378.2949 bit/sec. {1 mark}
IA] TRUE
[B] FALSE
Consider a generator matrix (6, 3) linear block code below. The generator
‘matrix is expressed in systematic form. Ui mark]
011100
G=111010
110101
TRUE
FALSE,
aT IARY
QUESTION 2
PF conic the following source with discrete memoryless source with source
Ww
©
CCTEO
alphabet $ = {So, Si, S2} with po = 0.6, pi = 0.2, pa = 0.2. Prove that HIS*) = 3
HOS). Stop AH iL [3 marks]
Construct the encoder circuit for the generator polynomial for a (6, 3) cyclic code
generated by g(D)=1+D+D°. jk Afi \ecture wete [4 marks]
Given convolutional code of path 1, path 2 and path 3 in Figure 1. Find gl, g2
and g3.
[3 marks]
Path 1
Output
@ Path3.
Figure 1
Poth 1: my a)
GE ec
Path + m, £0,0,19
gla) = 0?
Path 2: Who + m, C1,1, 0)
g@)= 14D
7 Ry 200
QUESTION 3
(@) A source X produces letters from a two symbol alphabet with the probability
assignment P(Xi) = 0.3 and P (%2) = 0.7. Each source letter X is transmitted through
achannel from a source X'to the destination Y as shown in Figure 2. The transition
probabilities are indicated as below,
Paw)
Figure 2
i From Figure 2, what is the value of & and h? Prepare the conditional
probability P(Y1X) and joint probability tables P(Y,X). (3 marks}
ii, Write all the steps to show that I(Y,X) is equal to I(&,1). [3 marks}
(©) Complete the Table 4 and construct a state diagram given a convolutional path of
gi(l,1,1] and g2{1,1,0). [4 marks}
Table 4
me | om] om | g: | g | Current | Next
Input State | State
Continued...
CCTEO 67 QUESTION 3
(@) A souree X produces letters from a two symbol alphabet with the probability
assignment P(%i) = 0.3 and P (X4) = 0.7. Each source letter X is transmitted through
achannel from a source X to the destination Yas shown in Figure 2. The transition
probabilities are indicated as below.
Pax)
Figure 2
From Figure 2, what is the value of & and A? Prepare the conditional
probability P(Y1X) and joint probability tables P(Y,X).
[3 marks]
ii, Write all the steps to show that I(¥,X) is equal to 1%,Y). [3 marks}
) pcx x)
Ys, We ds POX)
a [keo9 | ONG | O-\ 0.3
Mn kote y =o. W =0-14 0.4
conditional probability , PLY IX)
k tort 0.4 =]
ke j-0.98
= 64D
ke ohoet
ghz )- 0-7?
204
n= 024 [2
OY
" PLY, e) = PLYT A) POX)
My Yo U3 PoxX)
x 0.42 x03 O18 «0-3 0.10 «0-3
\ = O.2Ib = 0.054 = 0.03 0.3
m2 | oe ono o yoae 0+
PLY) 049 0.152 oe \
Toint Probability Tabu. PLY, x)
Ci) caudlitiona! — probability fable PLXIY)
PCA) = PLX,Y) / PeYD
“, Hy PCY)
0.316 /0-72 | 0-504 J0.72
Yr |e og = 0-4 692
0.054 70-1 | 5 ggg /o.1se
Ya |e oasss [2o.0tt7 | OASL
0.03 /0.124 | 0.098 / 0-126
Ya bepssuy - oress | OVK
POS | oF JI
HO = 63 log, (08) + OF Ing, Cv0-D
6.8813
HOY) = 0-42 log, (1/092) + 0-18% log, C 0-52) + 0-128 og, C1 /o-12¥)
2 LBE HOXIY) = Gate log, C1/0-32 +
ObDY log. C10. +
0.054 log, C022) + g gge log, C 70. 6447) +
0-03 Jog, C1/e2s) + 0.098 Ing, C1) o-test)
= 0.8448
HIYTX) = ole logs (i/o-7a) + 0-05% Ia, COI) + 0-03 log, C1 0.1) +
O-5D fog, Cif 0-79) + 0.048 bog, 61) 1K) x2
= 1130h
LC x, Y) = LlY,*)
Hu) - HOXIN) = HOY - HOYT x)
0 813 - 0.8FR =
1
20000 Wo
0.0035 = 0.0086
HABY = |. 20% (b) Complete the Table 4 and construct a state diagram given a convolutional path of
gif[1,1,1] and g2[1,1,0]. {4 marks]
Table 4 vy My HO
mo | m | m a @ | Current | Next
Input State_| State
0 0 0 0 00 00)
1 0 0 v 1 00, 0
0 1 0 1 1 10 | or
1 1 o o 0 10 NV
0 0 1 v 0 OL 00
1 0 1 0 T
0
1 u W
4 dashed
ol T
Ree ere Tien 0 avd
1 1 q 0 TB. INFORMATION THEORY LLEBBRUARY 2023
QUESTION 4
(a) Consider the generator polynomial for a (6,3) cyclic code generated by
g polyni
g(D)=1+ D+ D? . Construct the generator matrix, in "systematic" form.
[3 marks]
(6) Consider the generator polynomial for a (6,3) cyclic code generated by
g(D)=1+D-+ D>. Construct all the valid code words for the following messages
in "non-systematic” form. Show your steps.
[3 marks]
‘Message m(D) (D) ‘Code word
i
110
100
(© Consider the generator polynomial for a (6,3) cyclic code generated by
8(D)=1+D+ D* . Construct the valid code words for the following messages in
"systematic" form. Show your steps.
[4 marks]
Message m(D) @) Code word
001
010
O11
101
End of Page
creo 7 QUESTION 4
(a) Consider the generator polynomial for a (6,3) cyclic code generated by
g(D)=1+D+D*. Construct the generator matrix, in "systematic" form.
{3 marks]
go) = 1tb+0?
Ox gt) = De Citd +O) = Dt 0 +P"
Dixglo)= Dry cit0tD) = o + or +P
110100
GQ: fovioro | odd RI t PS
oo\lo\
’ Ta
10 100 .
Gi: fon ovo | gystrmatic fom
iy o01 (b) Consider the generator polynomial for a (6,3) cyclic code generated by
g(D)=1+D+ D? , Construct all the valid code words for the following messages
in "non-systematic” form. Show your steps.
[3 marks]
Message m(D) <(D) Code word
ii \+bd + D7 140 +08 +04 +5 torn
110 [+D 1+ Dt + Dt 0% youll o
100 Tt [+ +0* Ho1eo
lot mp) = 1+D+D*
let mcp)= 1
CLD) = med) g (>> 1) cLp) = mt) g)
= (it0+p) CItD . | CED 4D»
2B 4p) ec Baw t oD) + COE DEED) = pet
2 Lor + Dt Ot +P
let mco) = 1+D
Cp) = mCp) glo)
2(1 4p) (14D +DD
= (14h +0?) + C eo + ot)
= 1+ vp +p® +t (©) Consider the generator polynomial for a (6,3) cyclic code generated by
g(D)=1+D-+D? . Construct the valid code words for the following messages in
"systematic" form. Show your steps.
[4 marks]
Message mD)
