Single Error Correcting Codes (SECC)

Basic idea:
– Use multiple parity bits, each covering a subset of the
data bits.
– No two message bits belong to exactly the same subsets,
so a single error will generate a unique set of parity check
errors.

Suppose we check the

6.02 Spring 2011 Modulo-2
parity and discover that P1
and P2 indicate an error?
Lecture #9 addition,
aka XOR
P0 B0 P1 bit B2 must have flipped

B3 What if only P2 indicates

• How many parity bits? an error?
B1 B2
• Dealing with burst errors P0 = B0!B1!B3 P2 itself had the error!
• Reed-Solomon codes P1 = B0!B2!B3
P2 = B1!B2!B3 P2
6.02 Spring 2011 Lecture 9, Slide #1 6.02 Spring 2011 Lecture 9, Slide #2

Checking the parity Using the Syndrome to Correct Errors

• Transmit: Compute the parity bits and send them along with
Continuing example from previous slides: there are
the message bits
three syndrome bits, giving us a total of 8 encodings.

• Receive: After receiving the (possibly corrupted) message, E 2E 1E 0 Single Error Correction
compute a syndrome bit (Ei) for each parity bit. For the code
000 No errors
on previous slide: What happens if
001 P0 has an error, flip to correct
there is more than
E0 = B0 ! B1 ! B3 ! P0 010 P1 has an error, flip to correct one error?
E1 = B0 ! B2 ! B3 ! P1 011 B0 has an error, flip to correct
E2 = B1 ! B2 ! B3 ! P2 100 P2 has an error, flip to correct
101 B1 has an error, flip to correct
• If all the Ei are zero: no errors!
110 B2 has an error, flip to correct
111 B3 has an error, flip to correct
• Otherwise the particular combination of the Ei can be used to
figure out which bit to correct.
The 8 encodings indicate the 8 possible correction actions: no
errors, error in one of 4 data bits, error in one of 3 parity bits
6.02 Spring 2011 Lecture 9, Slide #3 6.02 Spring 2011 Lecture 9, Slide #4
 (n,k,d) Systematic Block Codes A simple (8,4,3) code P0 is parity bit
for row #1
Idea: start with rectangular
• Split message into k-bit blocks
array of data bits, add parity
• Add (n-k) parity bits to each block, making each block n bits B0 B1 P0
checks for each row and
long. column. Single-bit error in
k n-k
data will show up as parity B2 B3 P1
errors in a particular row
Message bits Parity bits and column, pinpointing the P2 P3 P3 is parity bit
The entire block is called bit that has the error. for column #2
a code word and this
n
is an (n,k) code.
011 011 011
110 100 111
10 10 10
• Often we’ll use the notation (n,k,d) where d is the minimum
Hamming distance between code words. Parity for each row Parity check fails for Parity check only fails
• The ratio k/n is called the code rate and is a measure of the and column is row #2 and column #2 for row #2
correct " no errors " bit B3 is incorrect " bit P1 is incorrect
code’s overhead (always ! 1, larger is better).
Can you verify this code has a Hamming distance of 3?
6.02 Spring 2011 Lecture 9, Slide #5 6.02 Spring 2011 Lecture 9, Slide #6

How many parity bits are needed? Error-Correcting Codes

• Suppose we want to do single-bit error correction • Parity is a (n+1,n,2) code
– Need unique combination of syndrome bits for each possible – Good code rate, but only 1-bit error detection
single bit error + no errors • Replicating each bit r times is a (r,1,r) code
– n-bit blocks → n possible single bit errors – Simple way to get great error correction; poor code rate
– Syndrome bits all zero → no errors – Handy for solving quiz problems!
• Assume n-k parity bits (out of n total bits) – Number of parity bits grows linearly with size of message
– Hence there are n-k syndrome bits • Rectangular codes with row/column parity
– 2n-k – 1 non-zero combinations of n-k syndrome bits – Easy to visualize how multiple parity bits can be used to
• So, at a minimum, we need n ! 2n-k – 1 triangulate location of 1-bit error
– Given k, use constraint to determine minimum n needed to – Number of parity bits grows as square root of message size
ensure single error correction is possible • Hamming single error correcting codes (SECC) are (n,n-p,3)
– (n,k) Hamming SECC codes: (7,4) (15,11) (31,26) where n = 2p-1 for p > 1
The (7,4) Hamming SECC code is shown on slide 19, see the – See Wikipedia article for details
Notes for details on constructing the Hamming codes. The – Number of parity bits grows as log2 of message size
clever construction makes the syndrome bits into the index
needing correction.
6.02 Spring 2011 Lecture 9, Slide #7 6.02 Spring 2011 Lecture 9, Slide #8
 Correcting single-bit errors is nice, but
Noise models in many situations errors come in
bursts many bits long (e.g., damage to
Dealing with
• Gaussian noise
storage media, burst of interference on
wireless channel, …). How does
Burst Errors
– Equal chance of noise at each sample single-bit error correction help with
that?
– Gaussian PDF: low probability of large amplitude
– Good for modeling total effect of many small, random noise Well, can we think of a way to turn a B-bit error burst into B single-bit
sources errors?

• Impulse noise
– Infrequent bursts of high-amplitude noise, e.g., on a wireless
Row-by-row Col-by-col
channel
B transmission B transmission
– Some number of consecutive bits lost, bounded by some burst order order
length B
(parity bits are
– Single-bit error correction seems like it s useless for dealing with shown shaded)
impulse noise…
or is it???
Problem: Bits from a particular Solution: interleave bits from B
code word are transmitted different code words. Now a B-bit
sequentially, so a B-bit burst burst produces 1-bit errors in B
produces multi-bit errors. different code words.
6.02 Spring 2011 Lecture 9, Slide #9 6.02 Spring 2011 Lecture 9, Slide #10

Interleaving Framing
• The receiver needs to know
– the beginning of the B-way interleaved block in order to do
message errors deinterleaving
Receive
Compute CRC – the beginning of each ECC block in order to do error correction.
B-way interleaved block – Since the interleaved block is made up of B ECC blocks, knowing
message crc
Deinterleave where the interleaved block begins automatically supplies the
Partition necessary start info for the ECC blocks
k+p k+p k+p k+p
k k k k • 8b10b encoding provides what we need! Here’s what gets
Correct errors
Apply ECC transmitted
k k k k
k+p k+p k+p k+p – Prefix to help train clock recovery (alternating 0s/1s, …)
Check CRC – 8b10b sync symbol
Interleave
message crc – Packet data: B ECC blocks recoded as 8b10b symbols
B-way interleaved block
Deliver or discard (after 8b10b decoding and error correction we get {#,data,chk})
Transmit – Suffix to ensure transmitter doesn’t cutoff prematurely, receiver
has time to process last packet before starting search for
beginning of next packet
– On some channels: idle time (no transmission)
6.02 Spring 2011 Lecture 9, Slide #11 6.02 Spring 2011 Lecture 9, Slide #12
 Our Recipe (so far) Remaining agenda items
• With B ECC blocks per message, we can correct somewhere
• Transmit • Receive between 1 and B errors depending on where in the message
– Packetize: split message into – Perform clock recovery using they occur.
fixed-size blocks, add sequence transitions, derive bit stream
– Can we make an ECC that corrects up to B errors without any
numbers, checksum from voltage samples constraints where errors occur?
– SECC: split {#,data,chk} into k- – 8b10b decoding: locate sync, – Yes! Reed-Solomon codes
bit blocks, add parity bits to decode
create n-bit code words with – SECC: deinterleave to spread
min Hamming distance of 3, B- • Framing is necessary, but the sync itself can’t be protected by
out burst errors, perform error
way interleaving correction on n-bit blocks
an ECC scheme that requires framing.
– 8b10b encoding: provide producing k-bit blocks – This makes life hard for channels with higher BERs
synchronization info to locate – Packetize: verify checksum and – Is there an error correction scheme that works on un-framed bit
start of packet and sufficient discard faulty packets. Keep streams?
transitions for clock recovery track of received sequence – Yes! Convolutional codes: encoding and the clever decoding
– Convert each bit into numbers, ask for retransmit of scheme will be discussed next week.
samples_per_bit voltage missing packets. Reassemble
samples packets into original message.

6.02 Spring 2011 Lecture 9, Slide #13 6.02 Spring 2011 Lecture 9, Slide #14

In search of a better code Thought experiment…

• Problem: information about a particular message unit (bit, • Suppose you had two 8-bit values to communicate: A, B
byte, ..) is captured in just a few locations, i.e., the message
unit and some number of parity units. So a small but • We’d like an encoding scheme where each transmitted value
unfortunate set of errors might wipe out all the locations included information about both A and B
where that info resides, causing us to lose the original – How about sending y = Ax + B for various values of x?
message unit. – Standardize on a particular sequence for x, known to both the
transmitter and receiver. That way, we don t have to actually
• Potential Solution: figure out a way to spread the info in each send the x s – the receiver will know what they are. For
message unit throughout all the code words in a block. example, x = 1, 2, 3, 4, ...
Require only some fraction good code words to recover the – How many values do you need to solve for A and B?
original message. – We’ll send extra to provide for recovery from errors…

6.02 Spring 2011 Lecture 9, Slide #15 6.02 Spring 2011 Lecture 9, Slide #16
 Example Spreading the wealth…
• Suppose you received four values from the transmitter y = 73, • Generalize this idea: oversampled polynomials. Let
249, 321, 393, corresponding to x = 1, 2, 3 and 4
– 4 Eqns: A·1+B=73, A·2+B=249, A·3+B=321, A·4+B=393
P(x) = m0 + m1x + m2x2 + … + mk-1xk-1
• We need two of these equations to solve for A and B; there are where m0, m1, ..., mk-1 are the k message units to be encoded.
six possible choices for which two to use Transmit value of polynomial at n different predetermined
• Take each pair and solve for A and B points v0, v1, ..., vn-1 :
A "1+ B = 73 A "1+ B = 73 A "1+ B = 73
P(v0), P(v1), P(v2), ..., P(vn-1)
A " 2 + B = 249 A " 3 + B = 321 A " 4 + B = 393
A=175, B=-102 A=124, B=-51 A=106.6, B=-33.6 Use any k of the received values to construct a linear system
of k equations which can then be solved for k unknowns m0,
A " 2 + B = 249 A " 2 + B = 249 A " 3 + B = 321
m1, ..., mk-1. Each transmitted value contains info about all
! A " 3 + B = 321 ! A " 4 + B = 393 ! A " 4 + B = 393 mi.
A=72, B=105 A=72, B=105 A=72, B=105 • Note that using integer arithmetic, the P(v) values are
• Majority rules: A=72, B=105 numerically greater than the mi and so require more bits to
! represent than the mi. In general the encoded message
– The received!value 73 had an error !
would require a lot more bits to send than the original
– If no errors: all six solutions for A and B would have matched message!
6.02 Spring 2011 Lecture 9, Slide #17 6.02 Spring 2011 Lecture 9, Slide #18

Solving for the mi Finite Fields to the Rescue

• Solving k linearly independent equations for the k unknowns • Reed’s & Solomon’s idea: do all the arithmetic using a finite
(i.e., the mi): field (also called a Galois field). If the mi have B bits, then
use a finite field with order 2B so that there will be a field
#1 v v0
2
! v0
k"1&# m0 & # P(v 0 ) & element corresponding to each possible value for mi.
0
% 2 k"1(% ( % (
%1 v1 v1 ! v1 (% m1 ( = % P(v1 ) ( • For example with B = 2, here are the tables for the various
arithmetic operations for a finite field with 4 elements. Note
%" " " # " (% " ( % " ( that every operation yields an element in the field, i.e., the
% 2 k"1 (% ( % ( result is the same size as the operands.
$1 v k"1 v k"1 ! v k"1 '$ mk"1 ' $ P(v k"1 )'
+ 0 1 2 3 * 0 1 2 3 A -A A-1
• Solving a set of linear equations using Gaussian Elimination
0 0 1 2 3 0 0 0 0 0 0 0 0
(multiplying rows, switching rows, adding multiples of rows to
other rows) requires add, subtract, multiply and divide 1 1 0 3 2 1 0 1 2 3 1 1 1
! operations. 2 2 3 0 1 2 0 2 3 1 2 2 3
3 3 2 1 0 3 0 3 1 2 3 3 2
• These operations (in particular division) are only well defined
over fields, e.g., rational numbers, real numbers, complex
numbers -- not at all convenient to implement in hardware. A + (-A) = 0 A * (A-1) = 1
6.02 Spring 2011 Lecture 9, Slide #19 6.02 Spring 2011 Lecture 9, Slide #20
 How many values to send? Use for error correction
• Note that in a Galois field of order 2B there are at most 2B • If one of the P(vi) is received incorrectly, if it s used to solve
unique values v we can use to generate the P(v) for the mi, we’ll get the wrong result.
– if we send more than 2B values, some of the equations we might
use when solving for the mi will not be linearly independent and • So try all possible (n choose k) subsets of values and use
we won t have enough information to find a unique solution for
each subset to solve for mi. Choose solution set that gets the
the mi.
– Sending P(0) isn’t very interesting (only involves m0)
majority of votes.
– No winner? Uncorrectable error… throw away block.
• Reed-Solomon codes use n = 2B-1 (n is the number of P(v)
values we generate and send). • (n,k) code can correct up to (n-k)/2 errors since we need
– For many applications B = 8, so n = 255 enough good values to ensure that the correct solution set
– A popular R-S code is (255,223), i.e., a code block consisting of gets a majority of the votes.
223 8-bit data bytes + 32 check bytes – R-S (255,223) code can correct up to 16 symbol errors; good for
error bursts: 16 consecutive symbols = 128 bits!

6.02 Spring 2011 Lecture 9, Slide #21 6.02 Spring 2011 Lecture 9, Slide #22

Erasures are special Example: CD error correction

• If a particular received value is known to be erroneous (an
erasure ), don’t use it all! • On a CD: two concatenated R-S codes
– How to tell when received value is erroneous? Sometimes there’s
channel information, e.g., carrier disappears. errors
– See next slide for clever idea based on concatenated R-S codes 32-byte block 32-byte block ! 32-byte block
De-interleave

• (n,k) R-S code can correct n-k erasures since we only need k 32-byte block 32-byte block ! 32-byte block (32,28) code
equations to solve for the k unknowns. Uncorrectable error Handles up to
28-byte block 28 erasures 28-byte block 2 byte errors
erasures
• Any combination of E errors and S erasures can be corrected De-interleave
28-byte block 28-byte block 28-byte block
so long as 2E + S ! n-k. (28,24) code
Handles up to
24-byte block 24-byte block 24-byte block 4 byte erasures
De-interleave
24-byte block 24-byte block 24-byte block

Result: correct up to 3500-bit error bursts (2.4mm on CD surface)

6.02 Spring 2011 Lecture 9, Slide #23 6.02 Spring 2011 Lecture 9, Slide #24