Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007)

Lecture 14: List Decoding Capacity

October 2, 2007
Lecturer: Atri Rudra Scribe: Thanh-Nhan Nguyen

In the last lecture, we stated a theorem for list decoding capacity, which we restate here:
1
Theorem 0.1 (List-Decoding Capacity). Let q ≥ 2 be an integer, and 0 < ρ < 1 − q
be a real.
(i) Let L ≥ 1 be an integer, there exists an (ρ, L)-list decodable code with rate
1
R ≤ 1 − Hq (ρ) −
L
(ii) For every (ρ, L) code of rate 1 − Hq (ρ) + ε, L needs to be exponential in block length of the
code.

In this lecture, we will prove this theorem.

1 Proof of Theorem 0.1

Proof. We start with the proof of (i). Pick a code C at random where
1
|C| = q k , k ≤ (1 − Hq (ρ) − )n.
L
That is, as in Shannon’s proof, for every message m, pick C(m) uniformly at random from [q]n .
Definition 1.1. Given y ∈ [q]n , and m0 , · · · , mL ∈ [q]k , tuple (y, m0 , · · · , mL ) defines a “bad
event” if
C(mi ) ∈ B(y, ρn), 0 ≤ i ≤ L
where recall that B(x, e) = {z|∆(x, z) ≤ e}
Fix y ∈ [q]n , m0 , · · · , mL ∈ [q]k .
Note that for fixed i, by the choice of C, we have:
V olq (y, ρn)
P r[C(mi ) ∈ B(y, ρn)] = n
≤ q −n(1−Hq (ρ)) , (1)
q
where the inequality follows from the upper bound on the volume of a Hamming ball that we have
already seen. Now the probability of a bad event given (y, m0 , · · · , mL ) is
"L # L
^ Y
Pr C(mi ) ∈ B(y, ρn) = P r[C(mi ) ∈ B(y, ρn)] ≤ q −n(L+1)(1−Hq (ρ)) ,
i=0 0

1
where the equality follows from the fact that the random choice of codewords for distinct messages
are independent and the inequality follows from (1). Then,
 k 
q
P r[ any bad event] ≤ q n
q −n(L+1)(1−Hq (ρ)) (2)
L+1
≤ q n q Rn(L+1) q −n(L+1)(1−Hq (ρ)) (3)
1
−n(L+1)[1−Hq (ρ)− L+1 −R]
= q
1 1
≤ q −n(L+1)[1−Hq (ρ)− L+1 −1+Hq (ρ)+ L ] (4)
n
= q− L
< 1

In the above, (2) follows by
counting the number of y’s , and the number of L + 1 tuples. (3)
follows from the fact that ab ≤ ab , and k = Rn. (4) follows by assumption R ≤ 1 − Hq (ρ) − L1 .
Rest of the steps follow from rearranging and canceling the terms. Therefore, by probabilistic
method, there exists C such that it is (ρ, L)-list decodable.
Now we turn to the proof of part (ii). For this part, we need to show the existence of a y ∈ [q]n
such that |C ∩ B(y, ρn)| is super-polynomially large for every C of R ≥ 1 − Hq (ρ) + ε.
Pick y ∈ [q]n at random. Fix c ∈ C. Then

P r[c ∈ B(y, ρn)] = P r[y ∈ B(c, ρn)]

V ol(y, ρn)
= (5)
qn
≥ q −n(1−Hq (ρ))−o(n) , (6)

where (5) follows from the fact that y is chosen uniformly at random from [q]n and (6) follows by
the lower bound on the volume of the Hamming ball that we have seen earlier. We define

1 if c ∈ B(y, ρn)
Xc =
0 otherwise

We have
X
E[|B(y, ρn)|] = E[Xc ] (7)
c∈C
X
= P r[Xc = 1]
c∈C
X
≥ q −n(1−Hq (ρ)+o(n)) (8)
c∈C
n[R−1+Hq (ρ)−o(1)]
= q
≥ q Ω(n)
(9)

2
In the above, (7) follows by the linearity of expectation, (8) follows from (6), and (9) follows by
choice of R. Hence, by probabilistic method, there exists y such that |B(y, ρn) ∩ C| is q Ω(n) , as
desired.

Remark 1.2. The proof above can be modified to work for random linear codes. In particular, one
can show that with high probability, a random linear code is (ρ, L)-list decodable code as long as
1
R ≤ 1 − Hq (ρ) − .
dlogq (L + 1)e

The details are left as
an exercise. This means that there exists linear codes with rate 1 − Hq (ρ) −
ε that are ρ, q O(1/ε) -list decodable. However, just for q = 2, one can show the existence of
(ρ, O(1/ε))-list decodable codes [2] (though it is not a high probability result).

The following questions are still open:

1. Is a random linear binary code of rate 1 − H(ρ) − ε with high probability (ρ, O(1/ε))-list
decodable?


2. Does there exist a q-ary linear code (for q > 2) of rate 1 − Hq (ρ) − ε that is ρ, q o(1/ε)
list-decodable?

It has been conjectured that the answer to both of these questions is positive [1].
Update: Jan 2010 Guruswami, Håstad and Kopparty have solved the open questions above by
showing that random linear codes of rate 1 − Hq (ρ) − ε are (ρ, O(1/ε))-list decodable.

References
[1] Venkatesan Guruswami. List decoding of error-correcting codes. Number 3282 in Lecture
Notes in Computer Science. Springer, 2004. (Winning Thesis of the 2002 ACM Doctoral
Dissertation Competition).

[2] Venkatesan Guruswami, Johan Håstad, Madhu Sudan, and David Zuckerman. Combinatorial
bounds for list decoding. IEEE Transactions on Information Theory, 48(5):1021–1035, 2002.