Week 7

Hamming codes
Version 2023-11-04. To accessible online version of this chapter
Synopsis. Hamming codes are essentially the first non-trivial family of codes that we shall
meet. We give a construction of a q-ary Hamming code and prove that it is perfect with
minimum distance 3. We show that syndrome decoding works for Hamming codes in an
especially simple way.

Finding a check matrix

Before we can construct Hamming codes, we need to discuss check matrices further and
prove a result (the Distance Theorem) which will allow us to find the minimum distance of
a linear code from its check matrix.
The following result allows us to find a generator matrix of C ⊥ , assuming that C has a
generator matrix in standard form.

Theorem 7.1: a check matrix construction

Assume that C has a k × n generator matrix G = [ Ik | A ] in standard form. Then
the dual code C ⊥ has generator matrix

H = [ −AT | In−k ].

Proof. H has n − k rows which are linearly independent (due to In−k present). It is enough
to show that each row r of H is a codevector of C ⊥ : indeed, we have n − k linearly
independent vectors in C ⊥ , and n − k is the dimension of C ⊥ by Theorem 5.1, so a linearly
independent set of n − k vectors must be a basis of C ⊥ .
By Theorem 5.1, it is enough to show that rGT = 0. We will show this at once for all rows

73
Hamming codes 74

of H, by proving that HGT is the zero matrix. Indeed,

 
Ik
[ −A | In−k ] −−− = −AT Ik + In−k AT = −AT + AT = 0.
T
AT

How can one find a check matrix of C if C has no generator matrix in standard form? We
address this question below.

Linearly equivalent codes

Definition: linearly equivalent codes

Two linear codes C, C ′ ⊆ Fnq are linearly equivalent, if C ′ can be obtained from C
by a sequence of linear transformations of the following types:

(C1) choose indices i, j; in every codeword, swap symbols xi and xj ;

(C2) choose index i and non-zero λ ∈ Fq ; in every codeword, multiply xi by λ.

Exercise. Linearly equivalent codes have the same length, dimension and weight. They
have the same weight enumerator. (Reason: (C1) and (C2) do not change the weight of
any vector.)

Fact: known from linear algebra

Every generator matrix can be brought into the standard form by using row operations
(R1), (R2), (R3) considered above and column operations (C1).

Reason: any matrix can be brought to reduced row echelon form, RREF, by (R1)–(R3); a
generator matrix has linearly independent rows so the RREF won’t have zero rows and will
have a leading entry 1 in each of the k rows; the k columns which contain the leading entries
are columns of the identity matrix of size k; use (C1) to move all these columns to the left.
Conclusion: we can always find a generator matrix in standard form for a linearly equivalent
code.

The Distance Theorem

We already know how to read the length and the dimension of a linear code C off a check
matrix H of C:

ˆ the number of columns of H is the length of C;

ˆ the number of columns minus the number of rows of H is the dimension of C.
Hamming codes 75

The following theorem tells us how to determine the minimum distance of C using H.

Theorem 7.2: Distance Theorem for linear codes

Let C ⊆ Fnq be a linear code with check matrix H. Then d(C) = d if and only if
every set of d − 1 columns of H is linearly independent and some set of d columns
of H is linearly dependent.

Proof. Let e be the size of a smallest linearly dependent subset of the set {h1 , . . . , hn } of
columns of H. The theorem claims that e = d(C). Note that e is the minimum positive
number of non-zero coefficients xi in the linear combination

x1 h1 + x2 h2 + . . . + xn hn = 0,

i.e., the minimum weight of non-zero x = (x1 , . . . , xn ) such that xH T = 0. By Theorem 5.1,
such vectors x are exactly the codevectors of C, so e = w(C) = d(C) as claimed.

Example: calculate d(C) using the Distance Theorem

Use the Distance Theorem

" to find the
# minimum distance of the ternary linear code
0 1 2 1
with check matrix H = .
2 0 1 1

Solution. Step 1. H has no zero columns. Hence every set of 1 column is linearly indepen-
dent (a one-element set is linearly dependent iff that element is zero). So d ≥ 2.
Step 2. Any two columns of H are linearly independent, because no two columns are
proportional to each other. So d ≥ 3.
Step 3. There are three linearly
" # "dependent
# " # columns in H: for example, columns 1, 2 and 3
0 1 2
form linear combination + + = 0. Therefore, d = 3.
2 0 1

Hamming codes: the construction

Definition: line, representative vector, projective space

A line is a 1-dimensional subspace of the vector space Fnq .

A representative vector of a line is a non-zero vector u from that line. The line is
then given by {λu | λ ∈ Fq }.
The projective space Pn−1 (Fq ) is the set of all lines in Fnq .
Hamming codes 76

Remark: the terminology comes from euclidean geometry — in the euclidean plane, the set
of all vectors proportional to a given non-zero vector is a straight line through the origin.
Projective spaces over the field R of real numbers are well-studied geometric objects.
For example, P1 (R) — the set of all lines through the origin in the euclidean plane — can
be thought of as the unit circle with antipodes identified. We are working over a finite field
Fq where these notions are less intuitive.

Definition: Hamming codes

Let r ≥ 2 be given. We let Ham(r, q) denote an Fq -linear code whose check

matrix has columns which are representatives of the lines in Pr−1 (Fq ), exactly one
representative vector from each line.

Remark: Ham(r, q) is not one code but a class of linearly equivalent codes

Ham(r, q) is defined up to a linear equivalence. Indeed, we can:

ˆ multiply a column by non-zero λ to get another representative of the same line;

ˆ put columns in any order.

This means that Ham(r, q) is not just one code but a class of linearly equivalent codes.
We will therefore say “a Ham(r, q) code” to mean any of the linearly equivalent codes.

Let us see how the construction works in historically the first example of a Hamming code.

Example: Ham(3, 2)

Construct a parity check matrix for a binary Hamming code Ham(3, 2). Then find a
generator matrix in standard form for Ham(3, 2).

Solution: we need to take one non-zero column from each line in F32 . For binary vectors, a
line {λu | λ ∈ F2 } consists only of two points, 0 and u. This means that a check matrix for
a binary Hamming code consists of all non-zero binary columns or the required size.
Start filling in the check matrix by putting the identity columns at the end (this is convenient
for finding a generator matrix). In total, there are 7 non-zero binary vectors of size 3:
 
1 1 1 0 1 0 0
H= 1 1 0 1 0 1 0 
 
1 0 1 1 0 0 1
Hamming codes 77

From this H, we can reconstruct the generator matrix G = [Ik | A] by Theorem 7.1:
 
1 0 0 0 1 1 1
 0 1 0 0 1 1 0 
G=
 

 0 0 1 0 1 0 1 
0 0 0 1 0 1 1
This is, up to linear equivalence, the generator matrix of the original code of R. Hamming.
Historical remark. Despite their name, the q-ary Hamming codes for q > 2 were not
invented by Hamming. Richard Hamming told Claude Shannon (who he shared an office
with at Bell Labs) about his binary [7, 4, 3]-code, and Shannon mentioned it in his paper of
1948. That paper was read by Marcel J. E. Golay (1902–1989), a Swiss-born American
mathematician and electronics engineer, who then suggested the Ham(r, q) construction in
his paper published in 1949. Golay went further and constructed two perfect codes which
are not Hamming codes. He asked whether there are any more perfect codes.
We will see the Golay codes, and will learn about an answer to Golay’s question about perfect
codes, later in the course.

Parameters of a Hamming code

We considered an example of a Ham(3, 2) code, which — by looking at its generator matrix

— turns out to be a [7, 4, d]2 code. It is not difficult to see directly that d = 3. By explicitly
computing the Hamming bound, one can show that all [7, 4, 3]2 -codes are perfect.
We will now generalise this and show that all Hamming codes are perfect.

Theorem 7.3: properties of Hamming codes

qr − 1
Ham(r, q) is a perfect [n, k, d]q code where n = , k = n − r, d = 3.
q−1

Proof. The length n of the code is equal to the number of columns in the check matrix,
which is #Pr−1 (Fq ), the number of lines in Frq .
Observe that two lines intersect only at one point, namely 0. The set Frq \ {0} is therefore
a disjoint union of lines. Each line {λu : λ ∈ F } contains q − 1 non-zero points.
#(Frq \ {0}) qr − 1
So the number of lines in Frq can be found as = .
q−1 q−1
We have k = dim Ham(r, q) = n − r since, by construction, the check matrix H has r rows.
To find d, we use the Distance Theorem for linear codes. Any two columns of H are linearly
independent because they are from different lines in Frq . (Two vectors are linearly dependent
only if they are proportional to each other, i.e., belong to the same line.) Therefore, d ≥ 3.
Hamming codes 78

On the other hand, H has columns (a, 0, 0, . . . , 0)T , (0, b, 0, . . . , 0)T and (c, c, 0, . . . , 0)T ,
from three different lines (where a, b, c ∈ Fq \ {0}). These columns are linearly dependent:
     
a 0 c
0 b c
     
a−1  −1   −1  
 ..  + b  ..  − c  ..  = 0.

. . .
0 0 0

So d = 3 by the Distance Theorem.

It remains to show that Ham(r, q) is perfect. We calculate t = [(d − 1)/2] = [2/2] = 1.
The Hamming bound (in logarithmic form) then says
n n 
k ≤ n − logq + (q − 1) = n − logq (1 + n(q − 1)) .
0 1

By the already proved formulae for n and k we have n(q − 1) = q r − 1 and k = n − r. Hence
the bound is n − r ≤ n − logq (q r ) = n − r — attained. Thus, Ham(r, q) is perfect.

Remark: (q r − 1)/(q − 1) is an integer

qr − 1
The proof shows that the fraction is an integer. In fact, this can be seen for
q−1
qr − 1
all integers q, r > 1 by a formula for summing a geometric progression, =
q−1
q r−1 + q r−2 + · · · + q + 1; the right-hand side is obviously an integer.

Decoding a Hamming code

Algorithm 7.4: decoding algorithm for a Hamming code

Let a Hamming code be given by its check matrix H. Suppose a vector y is received.

ˆ Calculate S(y) = yH T . If S(y) = 0, DECODE(y) = y.

ˆ Otherwise, S(y) = λ × some column of H. Let this be the ith column of H.
ˆ Subtract λ from the ith position in y. The result is the codevector DECODE(y).

Proof of validity of the algorithm. We prove that the algorithm outputs the nearest neigh-
bour of y in the code C. This is clear if S(y) = yH T = 0: by Proposition 5.2 y is a
codevector, and so its own nearest neighbour in C. Hence it is correct to decode y to itself.
If yH T ̸= 0, the line in Frq which contains yH T has a representative column in H — say,
hi . As yH T lies on the line spanned by hi , we must have yH T = λhi for some λ ∈ Fq .
Hamming codes 79

Note that λhi equals (λei )H T where ei is the unit vector with symbol 1 in position i and
zeros elsewhere. It follows that

(y − λei )H T = λhi − λhi = 0,

hence y − λei is a codevector. Finally, since d(y, y − λei ) = 1, and no codevector can be at
distance less than 1 from y, we conclude that y − λei is the nearest neighbour of y in C.

Remark: properties of a Hamming decoder

The Algorithm and the proof above imply:

ˆ Every coset leader of Ham(r, q) is 0 or λei , i.e., a vector of weight 0 or 1.

ˆ The decoder changes at most one symbol in the received vector.

Note that the fact that every coset leader is of weight ≤ 1 also follows, in a different way,
from Exercise 4.3.
For Ham(3, 2), a clever ordering of columns in the parity check matrix can make the decoding
algorithm especially elegant:

Example: special check matrix for Ham(3, 2)

Construct a decoder for the Ham(3, 2) code with parity check matrix
 
0 0 0 1 1 1 1
H = 0 1 1 0 0 1 1 .
 
1 0 1 0 1 0 1

Solution. If y ∈ F72 is received, yH T is either 0 or one of the columns of H. Now note

that, by Algorithm 7.4,

ˆ if yH T = 001, the decoder must subtract 1 from the first bit in y, because 001 is the
first column of H;

ˆ if yH T = 010, the decoder must subtract 1 from the second bit in y, because 010 is
the second column of H;

and so on. Subtracting 1 from a bit in F2 is the same as “flipping” the bit, i.e., replacing 0
by 1 and 1 by 0.
Thus, to decode the received vector y, we calculate the syndrome yH T . If this is 000,
output y, otherwise read the syndrome yH T as the binary representation of a number
i ∈ {1, 2, . . . , 7} and decode by flipping the ith bit in y.