LDPC Constr
LDPC Constr
LDPC CODES
OLGICA MILENKOVIC
UNIVERSITY OF COLORADO, BOULDER
1. Linear block codes with sparse (small fraction of ones) parity-check matrix
2. Have natural representation in terms of bipartite graphs
3. Simple and efficient iterative decoding in the form of belief propagation
(Pearl, 1980-1990)
THE CODE GRAPH AND ITERATIVE DECODING
Message passing:
Variable nodes: communicate to check nodes their
reliability (log-likelihoods)
Check nodes: decide which variables are not reliable
Variable nodes Check nodes and “suppress” their inputs
(Irregular degrees/codes)
Small number of edges in graph = low complexity
1 1 1 0 1 0 0
Nodes on left/right with constant degree: regular code
H 0 1 1 1 0 1 0
1 0 1 1 0 0 1 Otherwise, codes termed irregular
Can adjust “degree distribution” of variables/checks
Best performance over standard channels: long, irregular, random-like LDPC codes
Have dmin proportional to length of code, but correct many more errors
DECODING OF LDPC CODES
Iterative decoding optimal only if the code graph has no cycles
Vardy et.al. 1997: All “good codes” must have cycles
• What are desirable code properties?
• Large girth (smallest cycle length): for sharp transition to waterfall
region; large minimum distance;
• Number of cycles of short length as small as possible;
• Very low error-floors (girth-optimized graphs are not a very good choice
Richardson 2003);
• Make the performance capacity approaching: irregular degree
distribution (depends on the channel, “safe” gap to capacity about 1dB);
(Density Evolution, Richardson and Urbanke, 2001)
• Practical applications: mathematical formula for positions of ones in H
HOW CAN ONE SATISFY THESE
CONSTRAINTS?
Cycle-related constraints
Encyclopedia of
Sparse Codes on
Graphs:
by D. MacKay
http://www.inference.phy.cam.ac.uk/mackay/codes/data.html
Tanner 2000; Vasic and Milenkovic 2001; Kim, Prepelitsa, Pless 2002; Fossorier 2004
P1,1 P1, 2 ... P1,n 0 1i i
0 ... 0 i i
P1 P2 P3 ... P s
is
P P2, 2 ... P2,n 0 0 1 0 ...
P P i1 P i2 ... P i s 1
H
2 ,1 P H
... ... ... ... ... ... ... ... ...
0 0 ... 0 1
i s m 2
Pn k ,1 Pn k , 2 ... Pn k ,n 1
0 0 ... 0 P P i s m 3
P i s m 4
... P i s m1
Pi , j P ( i 1)( j 1) P
0
P P 0 0 0 P 0
0 0 P P P 0 P
P P 0 P P 0 0
0 Irregular
I I I Masking…
0 0 P 0 0 P P
P Codes
I P P q 1 0 P P P 0 P 0 0
P 0 0 0 P 0 P P
P i1 0 0 0 0 P i6 P7 P8
i i
0
I P
r 1
P q 1r 1 i
P 9
P i1
i
0 0 0 P5
i
i5
i
P i6 P 7
i6
P i8
i
Codes
0 P9 P i1 0 0 0 P P P7
H 0 P i8 P9
i
P i1 0 0 0 P5
i
P i6 with large
i
girth
i
( ri 1)( c j 1) 0 0 P i8 P9 P i1 0 0 0 P 5
Pi , j P 0 0 0 P i8
P9
i
P i1 0 0 0
i8 i9 i1
0 0 0 0 P P P 0 0
I I I I 0 0 P2 P2 0 0
P P P
I P 2
0 0 I I I 0 0 P2
H 0 I 0 P 2
I P 2
0 I I 0
P 2 0 I 0 0 I 0 I P2 I
0 0 P2 I P2 0 I 0 I I
P P P
MATHEMATICAL CONSTRUCTIONS OF
EXPONENTS
P P P P r1 LOW TO MODERATE RATE CODES:
Fan, 2000: Cycles exist if sum of exponents in
P P P P r2 closed path is zero modulo size (P)=q
( r1 r2 )ci1 ( r2 r3 )ci2 ... ( rl r1 )cil 0 mod q
P P P P r3
Proper Array Codes (PAC): Row-Labels {ri}
P P P P r4 form an Arithmetic Progression with common
mean ri+1-ri=a;
Improper Array Codes (IAC): Row-Labels {ri}
ci cj ck cl do not form an Arithmetic Progression;
2i+j-k-2l=0
i+j-2k=0 mod q (CW3) i+j-k-l=0
i+j-2k=0 and i+2j-3k=0 mod q (CW4)
2k+i-3j=0
2j-i-k=0
ANALYTICAL RESULTS ON ACHIEVABLE
RATES
Number theoretic bounds for contructions:
For PACs with girth g=8: C q log log q / log q
For IACs with girth g=10: 2 q
Results based on work of Bosznay, Erdos,Turan, Szemeredi, Bourgain
1/ 3
3q 2
q 1 (1 /(2i 1)) , or
1 i 2 ( q 1)
2( q 1)
( d 1) k 2 1 log q
, k
k log(dD 1)
s=1 0,1,5,14,25,57,88,122,198,257,280,...
s=1 0,1,3,4,9,10,12,13,27,28,30,38,...
s=2 0,2,7,18,37,65,99,151,220,233,545,...
s=2 0,2,3,5,9,11,12,14,27,29,30,39,...
s=3 0,3,7,18,31,50,105,145,186,230,289,...
s=3 0,3,4,7,9,12,13,16,27,30,35,36,...
P i1 P i2 P i3 ... P i s
is
P P i1 P i2 ... P i s1
H
... ... ... ... ...
i s m 2 i s m 1
P P i s m 3 P i s m 4 ... P
GF ( q m ) : {a : 0 i q m 1, a GF ( q)}
I I I I 0 0 P2 P2 0 0
I P2 0 0 I I I 0 0 P2
H 0 I 0 P2 I P2 0 I I 0
2
P 0 I 0 0 I 0 I P2 I
0 0 P2 I P2 0 I 0 I I
ERROR FLOOR ANALYSIS
Near codewords (Neal/MacKay, 2003)
Trapping sets (Richardson, 2003)
Considered to be the most outstanding problem in the area!
Margulis code: based on Cayley graphs (algebraic construction), has good girth
Frame error rate curve has floor at 10E-7, SNR=2.4dB, AWGN
Iteration number:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ….
182 142 108 73 66 55 38 29 20 16 15 14 14 14 14 14 14 14 14 ….
Number of erroneous bits:
At “freezing” point, all values of variable messages are +1/-1 and significant
oscillations happen +1 → -1
Frozen activity lasts for 13 iterations: then “bad messages” propagate through
the whole graph; can have jumps from -0.33 to 49.67
ALGORITHM FOR ELIMINATING THE ERROR FLOOR
“Oscillations” reminiscent to problems encountered in theory of
divergent series (G.H. Hardy, Divergent Series)
Trick: use a low-complexity “extra-step” in sum-product algorithm
based on result by Hardy
Parameters tunable: when do you “start” using “extra-step”, what
“numerical biases” should one use
Can do density evolution analysis for optimization purposes – no loss
expected in waterfall region (nor one observed)
(Paper with S. Laendner, in preparation)
1000 good frames: Standard and modified message passing both take 7.4
iterations on average for correct decoding
Modified message passing never takes more than 16 iterations to converge
20 Bad frames (in MATLAB): Corrected 2/3 fraction of errors after 35, 41, 26, 119,
38, 98… iterations (standard failed even after 10000 iterations)
Can use slightly more complicated result from G.H.H to reduce number of iteration
to 20-30
VARIABLE AND CHECK NODE ARCHITECTURE
(JOINT WORK WITH N. JAYAKUMAR, S. KHATRI)
Globecom 2004, TCAS 2005
Adder
PLA From
Array Adder
channel Array To checks
To variable
nodes
Adder
Adder PLA
PLA
- From check 2
-
Log(dv+1)+1 Stages of Two-Input
log(tanh) and arctanh PLAs Manchester Adders, Rabaey 2000
S2 S3
Check Node
Variable Nodes
S4 Bank
Ring of C/V Node Clusters
P i1 P i2 P i3 P i4 P i5 P i6 0 0 0 0 0 0
i6 i1 i2 i3 i4 i5
P P P P P P 0 0 0 0 0 0
0 0 0 0 0 0 P i1
P i2
P i3
P i4
P i5 P i6
0 0 0 0 0 0 P i6 P i1 P i2 P i3 P i4 P i5
H
0 0 0 P i1 P i2 P i3 P i4 P i5 P i6 0 0 0
0 0 0 P i6 P i1 P i2 P i1 P i4 P i5 0 0 0
P i4 P i5 P i6 0 0 0 0 0 0 P i1 P i2 P i3
P P i2
i3
P i4 P i5 0 0 0 0 0 0 P i6 P i1
S1 , S2 , S3 , S4
1-p(G)
FINITE MEMORY POLYA URN CHANNEL
Binary additive channel with memory
Occurrence of error increases probability of future errors
Output sample at time i (Yi) is modulo-two sum of input Xi and noise sequence Zi ,
where {Xi} and {Zi} are independent sequences
The noise process is formed based on the following scheme:
i
(1)(1 a)1 q( ) (2) b1 q( )
Backward message vector (2) P {S (i ) S1 ;Y1 ; }
( (1), (2))
(1) 1 a 2 1 b
LLR ( 1)Yi log
(2) 1 1 a 2 b
CODE DESIGN CONSTRAINTS: ESTIMATOR
AND DECODER
Completion of codeword
“Interior bits”
BCJR Equations for trellis
Bit
Time
Symbol Time
ITERATIVE DECODING OF
CLASSICAL ALGEBRAIC CODES
APPLICATIONS FOR UNEQUAL
ERROR PROTECTION
GRAPHICAL REPRESENTATIONS OF
ALGEBEBRAIC CODES
Consider a [7,4,3] Hamming code
0 0 0 1 1 1 1
H 0 1 1 0 0 1 1
1 0 1 0 1 0 1
0 0 0 1 1 0 0 1
0 1 1 0 0 0 0 1
H1
1 0 1 0 1 0 1 0
0 0 0 0 0 1 1 1
MAXIMUM CYCLE NUMBER (MC)
ALGORITHM
w 1 Hw H
nw n mw m
Approach by Sundararajan
and Vasic, 2004 (SVS): use
Determine the coordinates i, j of the (non -diagonal)
4 2 2 largest element in H w H wT
HH 2 4 2
T
Output Hw
Kumar, Milenkovic: CISS’2005, IEEE CTL 2005
CYCLE OVERLAP STRATEGY (COS)
i0 1, j 0 2,
COS is a variant of MCS D0 0
algorithm
Determine B. Construct submatrix HB of H w
Consider two rows i, j with with columns indexed by B.
intersection set of their
support sets Compute s the cardinality of the set
{(l , m) | l m and ( H B H BT ) lm ( H w H wT ) ij }
Let s be number of rows with
a support set S l such that
Sl Compute D= ( HH ) ij s
T
.
2 2
Choose indices that
maximize If D D0 then, D0 D, i i0 , j j 0 .
Update the values of j o . i0 sequentially.
( HH T ) ij s
2 2
Dimensions of four-cycle free parity-check
representations
If {v1 , v 2 , v3 } GF (q) are the values taken by the three variable nodes in R CF then,
v3 v 2
Type 2 cycle elimination introduces six-cycle while eliminating four-cycle
Number of Type-1 cycles: (q 1) 3
Number of Type-2 cycles: (q 2)(q 1) 3
ALGEBRAIC CODES AND UNEQUAL
ERROR PROTECTION BASED ON
ITERATIVE DECODING
Codes which provide UEP: practical applications for
Non-uniform channels
Joint coding for parallel channels
Image transmission over wireless channels
Page oriented optical memories
As in other UEP schemes involving LDPC codes, one can vaguely associate
the UEP obtained here to the different degree distributions
|u|u+v|:
Have “two copies” of u
Only one copy of v, which is implicit
Reduced-complexity encoding
Low-complexity soft-output decoders
Flexibility of UEP code construction, levels of UEP error
protection and adaptability to channel conditions
Excellent performance for best protected components
MS Multi-Stage
TMS Threshold Multi-Stage
MR-MS Multi-Round Multi-Stage
Kumar, Milenkovic
Globecom 2004, TCOMM 2004
FLOWCHART OF MS, MS-MR DECODER
Channel
Information Channel
Information
Liy ' Liy ''
Liy ' Liy '' comp
2
,1 ch /( j 1)
2
for a1 0,
comp
2
,1 ch / j
4
for a1 1
v
L i
Lvi
Decode v to comp
2
,1 ch /( j 1)
2
for a1 0,
v̂ Decode v to
v̂ comp
2
,1 ch
2
for a1 1
Decode u to
Output û
| uˆ | uˆ vˆ |
THANK YOU!