EXIT Chart Based Design of LDPC Codes for

Higher Order Constellations

Bhargav Joshi Andrew Thangaraj
Department of Electrical Engineering Department of Electrical Engineering
Indian Institute of Technology, Madras Indian Institute of Technology, Madras
Chennai, India 600036 Chennai, India 600036
Email: [email protected] Email: [email protected]

Abstract—EXtrinsic Information Transfer (EXIT) chart is a known to provide good performance with manageable delays.
graphical tool for analysis of low density parity check (LDPC) In contrast, BICM is a much simpler system using only a
and other codes with iterative decoding. In this paper, we propose single binary code and an interleaver before mapping to the
the design of LDPC codes for coded modulation schemes by
application of linear programming with conditions derived from constellation symbols. It has been shown that when employed
EXIT charts. Points on the EXIT curve for bit and check with Gray constellation mapping, the performance loss of
node operations are first computed either using analysis or by BICM compared to MLC/PID is small [3]. Several other
simulation. The entire EXIT curve is then obtained by linear interesting modifications of the basic BICM and MLC idea
interpolation. Finally, the EXIT curves are optimized to obtain have been proposed in the literature. In [4], the authors have
the maximum possible rate for the code under a given threshold
condition. To demonstrate the design procedure, we consider the considered a system that combines the good characteristics of
cases of M-PAM and 8-PSK over the additive white Gaussian BICM and MLC. See [4] for references to other works in this
noise (AWGN) channel. The optimization method presented direction.
is versatile, and can be extended to other constellations and Designing codes for BICM or MLC is not straight forward
situations easily.
because the equivalent binary channels at each level are
not symmetric. Authors of [3] introduced iid adapters to
I. I NTRODUCTION
impose symmetry on the overall channel, and used differential
Low-density parity-check (LDPC) codes are being widely evolution for obtaining optimal degree distributions for both
used in many existing communication systems. Several proper- MLC and BICM. In [5], optimization based on Gaussian
ties of LDPC codes are crucial for this wide usage. Firstly, the approximation and EXIT chart [6] was proposed for 8-PSK.
analysis of LDPC codes is on established theoretical grounds In [4], Gaussian approximation and linear programming were
and design follows from the analysis for the class of symmetric used in a combined MLC/BICM code design.
channels, which occur in many communication scenarios. In this paper, our focus will be on optimizing LDPC codes
Secondly, the implementation of LDPC encoders and decoders for BICM. We use linear programming for optimization, which
in VLSI and software has proven to be simple and flexible. is simpler and more flexible than differential evolution particu-
Finally, even in asymmetric channel scenarios, coding systems larly for larger constellations. We use EXIT charts for deriving
using LDPC codes as components have been designed and constraints and avoid an explicit Gaussian approximation. We
accepted in some communication standards. However, the demonstrate the design with some examples, simulation results
optimization of LDPC codes in asymmetric channels continues and comparisons.
to have some room for improvement. In this article, we present
results of one such effort in the design of LDPC codes for II. LDPC C ODES : BACKGROUND AND N OTATION
higher order constellations such as M -PAM for M = 4, 8 and Analysis and optimization of Low-Density Parity-Check
8-PSK. (LDPC) codes has been studied in detail for the last two
Two schemes typically proposed for use in higher order decades and is now considered standard. We will briefly
constellations are MultiLevel Coding (MLC) [1] and Bit- describe the salient background and introduce notation. See
Interleaved Coded Modulation (BICM) [2]. MLC uses mul- [7] for details.
tiple binary codes and offers different protection to bits at An LDPC code has a sparse parity-check matrix conve-
different levels. MLC is known to be a capacity-achieving niently represented by a bipartite Tanner graph with left nodes
strategy, and designing optimal codes for MLC involves joint standing for the bits or columns and right nodes standing for
optimization of component code rates and threshold [3]. The the rows or checks. The (λ, ρ) ensemble of Tanner graphs is
optimal decoder for MLC is the multistage decoder that suffers the set of all bipartite graphs with a given pair of edgewise
from high latency, but Parallel Independent Decoding (PID) is degree distributions for the left (bit) and right (check) nodes
Pdvmax i−1
978-1-4673-5952-8/13/$31.00
c 2013 IEEE given by polynomials λ(x) = i=2 λi x and ρ(x) =
Pdcmax
j=2 ρj xj−1 , respectively. Here, λi (resp. ρj ) is the fraction along the edges, the weighing factors are λi ’s. So, we get
of edges incident on degree-i bit (resp.
P degree-jP check) nodes.
X (i)
X (i)
So, we have 0 ≤ λi , ρj ≤ 1, and i λi = j ρj = 1. The IE,V = λi IE,V , IA,V = λi IA,V , (1)
P ρj P λi  i i
design rate of the ensemble is R(λ, ρ) = 1− j j / i i .
X (j)
X (j)
IE,C = ρj IE,C , IA,C = ρj IA,C . (2)
Since most codes in the ensemble perform similarly, it is
j j
common to say (λ, ρ) LDPC code instead of ensemble.
We will stick to AWGN channels in this article with σ An EXIT chart for the decoder consists of two curves - one
denoting the variance of the additive Gaussian noise. LDPC for the bit-node update and one for the check-node update.
codes are decoded by soft-decision message-passing decoders Each curve is a plot that maps extrinsic information coming
with the channel symbol-wise Log Likelihood Ratio (LLR) out of the node versus a priori information going into the node.
as input for each bit. For a transmitted bit b and corre- The bit-node curve is a plot of IE,V (y-axis) as function of
sponding received value y, the channel LLR is defined as IA,V (x-axis), and the check-node curve is a plot of IE,C (x-
l = log Pr{b=0|y} axis) as a function of IA,C (y-axis). Note that the check-node
Pr{b=1|y} . The decoder proceeds in iterations with
each iteration consisting of a bit-node update and a check- curve is on a switched axis. On the EXIT chart, a decoding
node update. The analysis is by a method known as Density trajectory starts at IE,V (IA,V = 0) and then hops between
Evolution (DE), which assumes that the all-zero codeword is the two curves to attempt to reach the (IE,V = 1, IA,V = 1)
transmitted. In DE, the PDF of the channel LLR is the input, point. So for decoding to converge at a given noise level σ,
which is computed using the channel model and denoted p0 . the bit-node curve should always lie above check-node curve
The basic step in DE is the approximate computation of the in the EXIT chart. This is a useful condition that is exploited
PDF of the LLR after iteration i, which we denote pi . The in the optimization of LDPC degree distributions.
R
probability of error after iteration i is ei = t<0 pi (t), and For more details on EXIT charts, see [6], [8].
the DE threshold is defined as σ ∗ (λ, ρ) = sup{σ : e∞ = 0}, III. O PTIMIZATION OF LDPC C ODES FOR BICM
which is roughly the highest noise variance that results in zero
error probability. The optimization of LDPC codes for Bit-Interleaved Coded
Modulation (BICM) systems is complicated by the fact that the
The design of LDPC codes is essentially an optimization equivalent binary-input channels are not symmetric. A binary-
problem that typically maximizes the rate R(λ, ρ) of the code input channel is called symmetric if its output y and transition
under the constraint that σ ∗ (λ, ρ) > σth , where σth is a desired probability satisfy p(y|0) = p(−y|1). Without symmetry,
threshold. The optimization problem is highly non-convex and application of DE [9] becomes infeasible because the all-zero
non-linear, and direct solutions are typically by powerful but codeword is not sufficient as input. To overcome this problem,
complex methods such as simulated annealing or differential we adopt the method of iid adapters proposed in [3].
evolution. A popular simplification is by the Gaussian approx-
imation, which assumes that pi is Gaussian and DE simply A. BICM with channel adapters
tracks the mean of pi . Another useful approximation to DE
Fig. 1 shows the channel model used in this article. The
is the EXtrinsic Information Transfer (EXIT) chart described
next.
Encoder Interleaver Mapper

A. EXIT Charts iid source

Channel

EXIT chart is a graphical tool that provides an intuitive (−1)()

picture of the message passing decoder [6]. The idea is to
track the mutual information between the extrinsic LLR and Decoder De-interleaver Demodulator
the corresponding codeword bit at each iteration, instead of
the entire PDF. The bit-node and check-node update steps of
Fig. 1. BICM model with iid adapters [3].
an iteration are approximated as processes that transfer their
input extrinsic mutual information from the previous step into
codeword out of the LDPC encoder and an equiprobable iid
improved extrinsic mutual information output.
(i) binary source are XOR-ed with each other and fed to an
We follow standard notation for the metrics involved: IE,V interleaver. The interleaver is assumed to have considerable
denotes the mutual information between the extrinsic LLR out randomness and depth of interleaving operation (as is usual
(i)
of a degree-i bit node and the codeword bit, IA,V is the mutual in BICM analysis). The mapper picks m bits at a time and
information between a priori LLR going into a degree-i bit maps them to a symbol from a 2m -sized alphabet A. This
(j)
node and the codeword bit. For check-nodes, we define IE,C symbol is transmitted over an Additive White Gaussian Noise
(j)
and IA,C along similar lines. The overall metrics are weighted (AWGN) channel. The demodulator receives the corrupted
averages of degree components [8], and since LLRs are passed symbol and outputs channel LLRs for each of the m bits.
Before passing the LLRs to the message passing decoder, 1) EXIT curves: For a bit-node degree i ∈ L, we use den-
the effects of interleaver and XOR-ing are undone. The latter sity evolution of an (i, r)-regular ensemble with r = 2, 3, . . .
is accomplished by reversing the signs of the LLR of bits and input PDF set as p0 in (3) computed using simulations.
that were flipped on the transmitter side. The requirement of During each iteration, we compute the PDF of LLRs going
having the same sequence bits at the receiver is addressed by into and coming out of the bit node, and calculate the mutual
using identical random number generators on both sides and information between the incoming LLR and the codeword
initializing them with same seeds [3]. Thus, the overall channel bit. Similarly, we calculate the mutual information between
as seen across encoder and decoder is now symmetric. Hence, the outgoing LLR from a bit node and the codeword bit.
(i) (i)
the all-zero codeword can be used to study decoder behavior. This calculation gives us one point on the (IA,V , IE,V ) curve
To perform density evolution, we need to compute the PDF corresponding to a particular left degree i. Multiple iterations
of the channel LLR input to the message-passing decoder. for different values of r provides many such points.
For this, we first ignore the channel adapters, and let Ui Similarly, for a check node degree j ∈ ∇, we repeat the
be the random variable denoting the channel LLR of the i- above process using density evolution of an (l, j)-regular en-
th bit among the m bits of a symbol. Note that Ui will be (j) (j)
semble with l = 2, 3, . . . and obtain points on the (IA,C , IE,C ).
some function of the received value, and is computed using We repeat the procedure for all left and right degrees in the
only the alphabet A and the bit mapping. Using the standard ranges of interest. This provides us with the required data set
assumptions of BICM and iid channel adapters in Fig. 1, the for deriving constraints for the optimization.
input channel LLR will have the PDF [3]
   Fig. 3 shows the EXIT√curves for left node degrees 2 to 30
X 1 1 X X for A = {−3, −1, 1, 3}/ 5 at σ = 0.53. From the computed
p0 = 
m
 p(Ui |a) + p(−Ui |a) , (3)
i
m 2 i i
a∈A0 a∈A1

where Aib
is the set of symbols in A having their i-th bit equal 1

to b. In this work, we estimate p0 using simulations for a given 0.9

dv=20

noise variance σ, modulation alphabet A and a bit mapping.

0.8
Fig. 2 shows typical estimated PDFs of p0 for 8-PAM and
4-PAM. From the plot, the asymmetry of the PDF is apparent. 0.7
dv=2
0.6
0.02
8−PAM
IE,V

4−PAM 0.5
0.018
0.4
0.016
0.3
0.014
0.2
0.012
0.1
p0(x)

0.01
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0.008 IA,V

0.006

0.004 Fig. 3. Bit-node EXIT curves obtained through interpolation.

0.002
points, we interpolate to find a smooth EXIT curve. A similar
0
−10 −8 −6 −4 −2 0
x
2 4 6 8 10 plot for right nodes degrees 2 to 11 is shown in Fig. 4.
2) Linear Program: We follow the standard method [7] of
Fig. 2. Estimated PDF of input channel LLR for 8-PAM and 4-PAM. fixing the right degree distribution to have only two terms
with consecutive degrees, i.e. ρ(x) = ρj x(j−1) + (1 − ρj )xj
Also, the spikes in the PDF make accuracy an important issue
with a fixed ρj . For a fixed check-node distribution, the task
in density evolution calculations.
of maximizing rate reduces to maximizing i λii , which is
P
B. Optimization Procedure linear in the unknown λi ’s.
We fix a modulation alphabet A, a bit mapping and a The constraint of operating at the desired noise threshold
desired threshold σth . We fix the maximum bit-node degree σth is enforced through the EXIT chart. The interval [0, 1] is
dvmax , which fixes the range of left degrees as the set L = quantized, and the conditions (1) are enforced at a large but
{2, 3, . . . , dvmax }. From capacity calculations using A and σth , discrete set of points. We see that the EXIT chart conditions
we obtain a rough range of possible rates, and this provides are linear in λi . However, to ensure successful convergence,
a range of right degrees R. These ranges are initialized as we enforce that the separation between the two EXIT curves
possible bit and check degrees. should be at least a small value δ.
 1 first column of Table I. The threshold of the optimized code
is 2.6197dB.
0.9
dc=11 Optimized EXIT chart in Fig. 5 shows the final EXIT curves
0.8 for the bit nodes and check nodes. We see that the two curves
0.7 1
Bit
0.6 Check
0.9
IA,C

0.5
dc=2 0.8

0.4
0.7

0.3
0.6

IE,V/IA,C
0.2
0.5

0.1
0.4

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.3
IE,C
0.2

Fig. 4. Check-node EXIT curves obtained through interpolation. 0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
I /I
Finally, the linear program will be as follows: A,V E,C

X λi Fig. 5. Optimized EXIT chart for rate(1/2) 4-PAM code.

maximize (4)
i
i
are almost on top of each other, and the bit node curve is
subject to marginally above the check node curve.
X Figs. 6 and 7 show the bit-error and frame-error rate of
0 ≤ λi ≤ 1, λi = 1, (5) the code obtained by simulation. The BER and FER curves
i
 
−1
X (i)
X (j) 10 Hou et al
λi IE,V (x) ≥ min  ρj IA,C (x) + δ, 1 , (6) Our 25k
Our 20K
i j PID Capacity
−2 Threshold
10
where x runs through all the quantized points in the interval
[0, 1].
To fix the best possible ρ(x), we vary j from 2 to (dcmax −1) 10
−3
BER

and for each j, we vary ρj from 0 to 1 in steps of desired

accuracy. For each such ρ(x), we repeat the linear program to −4
10
find the best λ(x) and ρ(x) that have yielded maximum rate.
For right-regular codes, the procedure simplifies even fur-
ther. We take the check-node curve of a particular degree one 10
−5

at a time and apply linear programming to find maximum

achievable rate for that degree. Finally, we select the degree
−6
distribution corresponding to the check-node degree that max- 10
2 2.5 3 3.5 4 4.5
SNR (Es/N0) (dB)
imizes rate.
Fig. 6. Bit error rate for rate(1/2) 4-PAM code.
IV. S IMULATIONS AND N UMERICAL R ESULTS
We applied the method discussed above to get optimized show a threshold behavior close to the calculated threshold.
degree distribution pairs for M-PAM and 8-PSK constellations. Moreover, the gap from the PID capacity for the calculated
For illustration, we describe the optimization procedure for 4- threshold is 0.35 dB, while the gap from the 10−6 -BER point
PAM modulation. at block length of 25000 is about 0.9 dB. The code in [3]
For a target spectral efficiency of 1 bit/symbol, we choose has better BER performance and calculated threshold (about
an operating SNR (Es /N0 ) to be 2.62 dB which is 0.35 dB 0.11 dB from PID capacity) than our design. This is because
away from the PID capacity. We fix dvmax and dcmax as 30 of possible errors in our interpolation of the EXIT curve.
and 15, respectively. By running density evolution, we get the Improving this design is part of future work.
EXIT curves shown in Figs. 3 and 4. Application of linear The table I also lists optimized code profiles obtained for
programming with δ = 0.001 yielded the code shown in the irregular and right-regular 8-PAM as well as 8-PSK constella-
 0 0
10 10
n = 25000 FER
n = 20000 BER
PID Capacity 8−PAM Capacity
Threshold Threshold
−1
10
−1
10

FER
−2
FER

10

−2
10
−3
10

−4
−3 10
10 6 6.2 6.4 6.6 6.8 7 7.2 7.4 7.6 7.8 8
2 2.5 3 3.5 SNR (Es/N0) (dB)
SNR (Es/N0) (dB)

Fig. 7. Frame error rate for rate(1/2) 4-PAM code. Fig. 8. Bit-error and frame-error rate for 8-PAM code (second column in
Table I).

tions. For the 8-PAM code, we did not find other comparisons
constructed code performs fairly well in comparions with
4-PAM 8-PAM 8-PAM 8-PSK similar work in the literature.
λ2 0.1306 0.2093 0.1926 0.1000 In future work, we plan to improve our EXIT curve
λ3 0.3773 0.2784 0.2260 0.4493 construction procedure by relying on more accurate density
λ4 0.0006 evolution and avoiding interpolation.
λ6 0.2122
λ9 0.2749 0.0365 R EFERENCES
λ10 0.2028 0.0196 0.0068
λ11 [1] H. Imai and S. Hirakawa, “A new multilevel coding method using error-
λ12 0.0473 0.2621 correcting codes,” Information Theory, IEEE Transactions on, vol. 23,
λ13 0.1812 no. 3, pp. 371–377, 1977.
λ27 [2] G. Caire, G. Taricco, and E. Biglieri, “Bit-interleaved coded modulation,”
λ28 0.0057 IEEE Transactions on Information Theory, vol. 44, no. 3, pp. 927–946,
λ30 0.2420 0.2121 0.3328 1998.
[3] J. Hou, P. H. Siegel, L. B. Milstein, and H. D. Pfister, “Capacity-
ρ8 0.03 0.5200 approaching bandwidth-efficient coded modulation schemes based on
ρ9 0.97 0.4800 1.0000 low-density parity-check codes,” IEEE Transactions on Information The-
ρ19 0.69 ory, vol. 49, no. 9, pp. 2141–2155, 2003.
ρ20 0.31 [4] H. Sankar, N. Sindhushayana, and K. R. Narayanan, “Design of low-
Rate 0.5003 0.5012 0.4998 0.7808 density parity-check (LDPC) codes for high order constellations,” in
σ∗ 0.523 0.330 0.327 0.282 GLOBECOM, 2004, pp. 3113–3117.
[5] Y. Li and W. Ryan, “Design of LDPC-coded modulation schemes,” in
TABLE I Proc. 3rd Int. Turbo Symp, 2003, pp. 551–554.
O PTIMIZED DEGREE DISTRIBUTIONS FOR DIFFERENT CONSTELLATIONS . [6] S. ten Brink, G. Kramer, and A. E. Ashikhmin, “Design of low-density
parity-check codes for modulation and detection,” IEEE Transactions on
Communications, vol. 52, no. 4, pp. 670–678, 2004.
[7] T. Richardson and R. Urbanke, Modern Coding Theory. Cambridge
in the literature for BICM thresholds. For 8-PSK, the work University Press, 2008.
in [5] provides a code of rate 2/3 at the same threshold of [8] A. Ashikhmin, G. Kramer, and S. ten Brink, “Extrinsic information
σ ∗ = 0.282, while our design provides a code of rate 0.78. transfer functions: model and erasure channel properties,” Information
Theory, IEEE Transactions on, vol. 50, no. 11, pp. 2657–2673, 2004.
Fig. 8 shows the bit-error and frame-error rate of the code [9] T. Richardson, M. Shokrollahi, and R. Urbanke, “Design of capacity-
in the second column of Table I over 8-PAM obtained by approaching irregular low-density parity-check codes,” Information The-
simulation. We see that the 10−4 BER point is almost 0.6 dB ory, IEEE Transactions on, vol. 47, no. 2, pp. 619–637, 2001.
away from the calculated threshold. This is a minor inaccuracy
that needs to be rectified in future work. However, the overall
performance is within 1.2 dB of the PID capacity for 8-PAM.
V. C ONCLUSION
We have used a simple scheme for the optimization of
degree-distribution pairs of LDPC codes for higher-order
modulations. The optimization procedure is demonstrated for
M -PAM and 8-PSK, and simulation results show that the