Convolutional Codes

In block coding, the encoder accepts a k-bit message block

and generates an n-bit code word. Thus, codewords are
produced on a block-by-block basis.
– Buffering is needed.

m1 m2 c1 c2
Block code
Encoder
m2 m1 c2 c1

In some applications, message bits come in serially rather

than in large blocks.

WY Tam - EIE POLYU Conv.1

Convolutional Codes
A convolutional encoder generates redundant bits by using
modulo-2 convolutions.

In a convolutional code, the block of n code digits generated

by the encoder in a time unit depends not only the block of k
message digit with that time unit, but also the preceding N-1
block of message digits. Usually the values of k and n are
small. (the redundant bits are generated by using modulo-2
convolutions.)
m1 m2 c1
Convolutional c1≠c2
Encoder c2
m2 m1
WY Tam - EIE POLYU Conv.2

1
 Code rate
For a convolutional encoder with n modulo-adder and M
Flip-flop, an L-bit message produces
n( L + M ) bits
L Flip-flop M
r= Input Flip-flop 1
n( L + M )
1
≈ Q L >> M
n
Adder 1 Adder n

Output

WY Tam - EIE POLYU Conv.3

Constraint length
Constraint length K is the number of shifts over which a
single message can influence the encoder output

K=M+1
Flip-flop M
Input Flip-flop 1

Adder 1 Adder n

Output
WY Tam - EIE POLYU Conv.4

2
 Example
For n=2, K=3
An 1000 bit message sequence produces a coded output
sequence of 2(1000+2) bits. The code rate is

1000 1000
r= ≈ = 1/ 2
Input 2(1000 + 2) 2000

Output
WY Tam - EIE POLYU Conv.5

Example
A convolutional code with rate k/n can be implemented by
using k separate shift register and n adders

Example: Input

An L-bit message produces

3(M/2+L/2) bits.

Therefore,
L 2
r= ≈
3(2 / 2 + L / 2) 3 Output
WY Tam - EIE POLYU Conv.6

3
 Impulse Response and Generator polynomial
Impulse response
– response of a path to a symbol 1 applied to its input
Example: Impulse response of path 1 is (1,1,1)
Impulse response of path 2 is (1,0,1)

Input

Path 2
Path 1

Output
WY Tam - EIE POLYU Conv.7

Impulse Response and Generator polynomial

The impulse response can be represented by a polynomial
Example: (1,0,1,1) can be written as
1 + 0 ⋅ D + 1⋅ D 2 + 1⋅ D 3 = 1 + D 2 + D 3

Example: (1,1,1) can be written as g(1)(D)=1+D+D2

(1,0,1) can be written as g(2)(D)=1+D2

Input

Path 2
Path 1
Output
WY Tam - EIE POLYU Conv.8

4
 Impulse Response and Generator polynomial
For the message sequence (10011), we have
m(D)=1+D3 +D4

As the convolution in the time domain is transformed into

multiplication in D-domain

The output of path 1 is

c(1)(D)= g(1)(D) m(D)= (1+D+D2)(1+D3 +D4)
= 1+D+D2+D3 +D6
= 1111001

WY Tam - EIE POLYU Conv.9

Impulse Response and Generator polynomial

The output of path 2 is
c(2)(D)= g(2)(D) m(D)= (1+D2)(1+D3 +D4)
= 1+D2+D3 +D4 +D5 +D6
= 1011111

Finally, multiplexing the two output sequence of path 1

(1111001) and path 2 (1011111) gives
c = (11,10,11,11,01,01,11)

Note: for the shift register to be restored to its zero initial

state, a terminating sequence of K-1=2 zeros is appended to
the last input bit of the message. These zeros is called the
tail of the message. WY Tam - EIE POLYU Conv.10

5
 State diagram
A Convolutional encoder can be specified by the state
diagram.
Example: (n=2,k=1,M=2) convolutional encoder
The state of this encoder is the content of the 2-stage shift
1 1 0
register.
Input
State: S0: 00 S1: 10
Path 2
S2: 01 S3: 11 Path 1
Output
10
Example: Suppose input is 1 and the original state is S1,
the output would be 10 and the new state become S3
WY Tam - EIE POLYU Conv.11

State diagram
Original Input Output new 00
State state
S0 00 0 00 00
S0
S1 10 0 01 01 01 11
S2 01 0 01 00
01 S1
S3 11 0 00 01 S2
00 1 11 10
10 10
10 1 10 11 00
01 1 10 10 S3
Input=1
11 1 11 11
Input=0
11
WY Tam - EIE POLYU State diagram Conv.12

6
 Trellis diagram
To show the time evolution of the coded sequence, a trellis
diagram is used.
– Trellis diagram is obtained directly from the state
diagram by including, from left-to-right, the dimension of
time 00
00 00
S0 S0 S0 S0 S0
11 11 01 11
01 S1
S1 01 S1 S1 S2
10
10 10 10
S2 S2 00
Input=101 S3
Input=1
Output=110110 S3 S3 Input=0 11
WY Tam - EIE POLYU Conv.13

Decoding

m Channel c r c' m'

Encoder
noise
Channel Decoder
Since there is a one-to-one corresponding between m and c,
m' = m if and only if c' = c

To reducing the probability of decoding error, the

maximum-likelihood principle is used to decode the
received sequence r into a code sequence c which maximizes
the likelihood function p(r|c) or log[p(r|c)].
Unknown
known
WY Tam - EIE POLYU Conv.14

7
 Maximum-likelihood Decoding
Assume that a codeword c=(c0,c1,…,cN) 1− p
is transmitted through a binary symmetric 1 p 1
p
channel and the received sequence is
0 1− p 0
r = (r0,r1,…,rN)
N
P (r | c) = Π p(ri | ci )
i −0
N
⇒ log P (r | c) = ∑ log p (ri | ci ) Q log( AB ) = log A + log B
i −0

Maximum-likelihood decoding is obtained by choosing the

nearest codeword to the received sequence in Hamming
distance which maximum the likelihood function.
WY Tam - EIE POLYU Conv.15

Maximum-likelihood Decoding
Assume d(r,c)=d, the likelihood function 1− p
becomes 1 p 1
N
log P(r | c) = ∑ log p (ri | ci ) p
i −0 0 1− p 0
= d log p + ( N − d ) log(1 − p )
 p   p 
= d log  + N log(1 − p) log  < 1
 1 − p  1− p 
As Nlog(1-p) is a constant for all c and p<1/2 in practice, the
maximum-likelihood decding rule become
choose the estimate c’ that minimizes the Hamming
distance between the r and c
WY Tam - EIE POLYU Conv.16

8
 Viterbi algorithm
To decode the received sequence r, it is compared with each
possible transmitted code vector c, and the particular one
closest to r is chosen as the correct transmitted code vector.
Min{d(r,c)} for all possible c

Viterbi decoder
The Viterbi decoder chooses that path through the trellis of
the code which has a minimum Hamming distance from the
received sequence.

d (r , c) = ∑ d (ri , ci )
i
WY Tam - EIE POLYU Conv.17

Example
Suppose m=10111
Input
then c = (11,01,10,10,11,00,01)
r = (11,01,11,10,11,00,01) Path 2
Path 1
Output
Time Received Trellis diagram ds m’
word
00 Estimated message
0 11 S0 2 0
11
S1 0 1
1

∑ d (r , c ) i i
Initial state must be 00 i =0
Two possible paths from S0
WY Tam - EIE POLYU Conv.18

9
 Example r = (11,01,11,10,11,00,01)
00
Time Received Trellis diagram ds m’
word S0
01 11
1 10 S0 00 00 3 00
11 11 01 S1
S1 3 01 S2
01
S2 0 10 10 10
10 00
S3 2 11 S3
11
00 00 00 ds=5
2 11 S0 1 100
11 11 01 ds=1
S1 01 1 101
01
S2 10 11 4 010
S3 2 111
WY Tam - EIE POLYU Conv.19

Example r = (11,01,11,10,11,00,01)
00
Finally, we have
S0 S0
11 01 11
01 S1
S1 S2
01 00
10 10
10 00
S2 10
S3
00 11
11
S3

13
c’=(10,01,10,10,11,00,01)
m’ =(1011100)
∑ d (r , c ) = 1
i =0
i i

WY Tam - EIE POLYU Conv.20

10
 Error Probability

m Channel c r Error c' m'

Decoder
Encoder Correction
noise

PBlock
– probability of a transmitted codeword does not
correspond to the decoded codeword
PBlock = P (c' ≠ c)
PBit
– probability of a transmitted information bit is incorrect
PBit = P (m' ≠ m)

WY Tam - EIE POLYU Conv.21

Error Probability
Consider a Binary Symmetric Channel (BSC)
1− p
1 p 1
p
0 1− p 0
Probability of t errors occur during transmission of a binary
codeword consisting of n bits is
 n  n n!
p (t ) =   p t (1 − p ) n −t where   =
t  t  (n-t)!t!
Example: n = 3, t = 1, the patterns for sending 111 are
011, 101, 110.
 3 3!
  = =3
1 1! (3 - 1)!
WY Tam - EIE POLYU Conv.22

11
 Error Probability
If e = (d min − 1) / 2 ,
n
 n j
PBlock = ∑   p (1 − p ) n − j
j = e +1  j 

e
 n
= 1 − ∑   p j (1 − p ) n − j
j =0  j 

WY Tam - EIE POLYU Conv.23

Example
Repetition code of length n = 3.
– a linear block code with k = 1 and n = 3.
– Codewords are 000 and 111.
• dmin = 3
• Correction capability: (d min − 1) / 2 = 1
1
PBlock = 1 − ∑ C nj p j (1 − p ) n − j
j =0

= 1 − p 0 (1 − p ) 3 − 3 p (1 − p ) 2
If p = 0.01, PBlock= 0.000298. (PBit = Pblock)
• 97% of error are corrected.
• Error rate has been reduced by 34 times.
WY Tam - EIE POLYU Conv.24

12
 Applications of Channel coding: LBC
Linear Block Code
Hamming code have been used in error control of computer
memories and long-distance telephony.

Example: In ECC memory, 64 bits data are stored in 72 bit

physical memory.

Can correct 1 error bit and detect 2 error bits

WY Tam - EIE POLYU Conv.25

Applications of Channel coding: CRC

16 bit CRC-ITU
CRC codes are used to perform error detection in almost all
modern communication systems.
Example: detecting errors at the data-link layer and the
transport layer of the Open Systems
Interconnection (OSI) reference model

16 bit CRC-ITU was recommended by ITU (International

Telecommunication Union) for many applications.
Example: Communication protocols: XYModem, Zmodem
Kermit file-transfer protocols (FTPs)
X.25 for packet communication
WY Tam - EIE POLYU Conv.26

13
 Applications of Channel coding: CRC

In these applications, data-resending approach (ARQ -

automatic repeat request) is the simplest way to correct
transmission errors.
• Error detection only

Long CRCs are more effective than short ones at

detecting errors, but also introduce more overhead
redundancy and implementation complexity.

WY Tam - EIE POLYU Conv.27

Applications of Channel coding: CRC

8 bit CRC-ATM
ATM (Asynchronous Transfer Mode) is a high-bandwidth,
low-delay switching and multiplexing technology for both
public and private communication networks
– Aimed at users of high-speed data, local-area network
interconnections, imaging and multimedia applications.

Five layers: service layer

adaptation layer
ATM layer
convergence layer
physical layer
WY Tam - EIE POLYU Conv.28

14
 Applications of Channel coding: CRC
Employed in the adaptation layer of the ATM
(Asynchronous Transfer Mode) protocol
g(X ) = X 8 + X 2 + X + 1
CRC-12
Used when the character length is 6 bits
g ( X ) = X 12 + X 11 + X 3 + X 2 + X + 1

CRC-16
Used for 8-bit characters
g ( X ) = X 16 + X 15 + X 2 + 1

WY Tam - EIE POLYU Conv.29

Applications of Channel coding: CRC

CRC-32
IBM Token-ring
IBM PC Network
CSMA/CD Ethernet LANs

g ( X ) = X 32 + X 26 + X 23 + X 22 + X 16
+ X 12 + X 11 + X 10 + X 8 + X 7
+ X 5 + X 4 + X 2 + X +1

WY Tam - EIE POLYU Conv.30

15
 Application of Channel Coding: BCH codes
(511,493) BCH code
ITU-T Recommendation H.261
– Video codec for audiovisual services at p x 64 kbit/s
– video coding standard used for video conferencing and
videophone
g ( X ) = X 18 + X 15 + X 12 + X 10
+ X 8 + X 7 + X 6 + X 3 +1

(40,32) BCH code

– used in the ATM (Asynchronous Transfer Mode ) layer of
ATM
WY Tam - EIE POLYU Conv.31

Application of Channel Coding: RS codes

RS (255,233)
– recommended by Consultative Committee for Space Data
Systems (CCSDS) for communication channel coding
standard for spacecraft
– Examples
• Mars Observer
launched in 1992 and arrived Mars in 1993
• Pathfinder lander
launched in 1996 and arrived Mars in 1997
• Cassini
launched in 1997 and to arrive Saturn in 2004
WY Tam - EIE POLYU Conv.32

16
 Application of Channel Coding: RS codes
Compact Disc (CD)
Source of errors
– dust, fingerprints and scratches on the disc
– surface roughness
The CD error-control system is designed to handle bursts by
means of Interleaving and the RS codes (Cross Interleaved
RS code)

RS1 Interleaver RS2

1.41Mbps 1.88Mbps
RS1: RS (28,24) RS2: RS (32,28)
Also used for the CD-ROM, CD-I, and the MiniDisc
WY Tam - EIE POLYU Conv.33

Application of Channel Coding: RS codes

Digital-Video-Disk (DVD)
– Increased physical density implies that physical
imperfections affect proportionally more bits
– DVD is a true multimedia disc
Therefore, a more powerful error-correction code is needed.

RS1: RS (208,192) RS2: RS (182,172)

Maximum correctable-burst length:

CD: 500 bytes (2.4mm)
DVD: 2200 bytes (4.6mm)
WY Tam - EIE POLYU Conv.34

17
 Application of Channel Coding: Convolutional codes
Global System for Mobile communications (GSM)
Speech coding in GSM

Bandpass Speech Channel

Microphone A/D Encoder
filter Encoder Modulator
0.3-3.4kHz fs=8kHz
13 bits 13kbps 22.8kbps
104kbps

Three classes of speech data bits: Class I - Class III

WY Tam - EIE POLYU Conv.35

Application of Channel Coding: Convolutional codes

Step 1:
Class I: block coded with a CRC code
Class I Class II Class III 260bits 200ms

Class I Class II Class III

CRC bits (error detection)

Class III 456bits

Step 2:
Class I and Class II: convolutional coded (r=1/2, K=5)

WY Tam - EIE POLYU Conv.36

18
 Application of Channel Coding: Convolutional codes
Data Channel in the GSM system
Five different data channels in the GSM system
Example: 9.6 kbps data transmission
9.6 kbps
Coding in the terminal equipment (base station)
240 bits (=12 kbps) 4 0s to reset the encoder
Convolutional code with r=1/2, K=5

488 bits

WY Tam - EIE POLYU Conv.37

Application of Channel Coding: Convolutional codes

Singaling Channel in the GSM system
Most important data in the network
184 bits
Block code (Fire code) for detection and correction of burst error
184 bits 40 bits 4 0s to reset the encoder
Check bits
Convolutional code: r=1/2, K=5
456 bits

WY Tam - EIE POLYU Conv.38

19
 Application of Channel Coding: Convolutional codes
CDMA cellular systems
– CDMA: Code-division multiple access
– users in the CDMA-based cellular system are
distinguished from each other by a code
– IS-95 (standard proposed by North America)

PN code
– spread the spectrum of a radio which shares access in a
system by the use of a particular high speed pseduo-
random (PN) code.

WY Tam - EIE POLYU Conv.39

Application of Channel Coding: Convolutional codes

– If each radio has its own radio-specific PN code, then all
radios, except the one it is desired to receive, look like
noise to all other receivers in the system.

Code 1 Desired receiver

Base
Station
noise

noise
noise

WY Tam - EIE POLYU Conv.40

20
 Application of Channel Coding: Convolutional codes
Reverse Channel (mobile unit to base station)
r=1/3, K=9
Add 8 bit Convolutional Symbol
4.4kbps encoder tail 4.8kbps Encoder 14.4kbps Repetition

28.8kbps

4.8kbps
64-ary
Interleaver
Modulator
28.8kbps
PN chip: 1.2288 Mcps

WY Tam - EIE POLYU Conv.41

21