ENSC 861 Source Coding in

Digital Communications
Arithmetic Coding 1
1/25/2009 J. Liang SFU ENSC861 1
Arithmetic Coding 1
Jie Liang
Engineering Science
Simon Fraser University
[email protected]
Announcement
HW 1 due today.
HW 2 announced. Due: Feb 9
1/25/2009
J. Liang SFU ENSC861 2
Outline
Introduction
Basic Encoding and Decoding
Uniqueness and Efficiency
Scaling and Incremental Coding
E1 scaling
E2 scaling
1/25/2009
J. Liang SFU ENSC861 3
E3 scaling
Integer Implementation
Next class:
Adaptive Arithmetic Coding
Binary Arithmetic Coding
Applications
JBIG, H.264, JPEG 2000
Huffman Coding
Replacing an input symbol with a codeword
Need a probability distribution
Hard to adapt to changing statistics
Need to store the codeword table
Minimum codeword length is 1 bit
Arithmetic Coding
1/25/2009
J. Liang SFU ENSC861 4
Replace the entire input with a single floating-point
number
Does not need the probability distribution
Adaptive coding is very easy
No need to keep and send codeword table
Fractional codeword length
Arithmetic Coding
Introduction
1
00
010 011
Recall table look-up decoding of Huffman code
N: alphabet size
L: Max codeword length
Divide [0, 2^L] into N intervals
One interval for one symbol
Interval size is roughly
1/25/2009
J. Liang SFU ENSC861 5
000 010 011 100
proportional to symbol prob.
Arithmetic coding applies this idea recursively
Normalizes the range [0, 2
L
] to [0, 1].
Map an input sequence to a unique tag in [0, 1).
0 1
abcd..
dcba..
Arithmetic Coding
Disjoint and complete partition of the range [0, 1)
[0, 0.8), [0.8, 0.82), [0.82, 1)
Each interval corresponds to one symbol
Interval size is proportional to symbol probability
The first symbol restricts the tag
0 1
a b c
1/25/2009
J. Liang SFU ENSC861 6
Observation: once the tag falls into an interval, it
never gets out of it
0 1
The first symbol restricts the tag
position to be in one of the intervals
The reduced interval is partitioned
recursively as more symbols are
processed.
0 1
0 1
Some Questions to think about:
Why compression is achieved this way?
How to implement it efficiently?
How to decode the sequence?
Why is it better than Huffman code?
1/25/2009
J. Liang SFU ENSC861 7
Possible Ways to Terminate Encoding
1. Define an end of file (EOF) symbol in the
alphabet. Assign a probability for it.
2. Encode the lower end or middle point of the
0
1
a b c EOF
1/25/2009
J. Liang SFU ENSC861 8
2. Encode the lower end or middle point of the
final range (How many bits are needed?)
3. If number of symbols is known to the
decoder, encode any nice number in the
final range.
Example:
Symbol Prob.
1 0.8
2 0.02
3 0.18
1 2 3
0 0.8 0.82 1.0
Map to real line range [0, 1)
Order does not matter
Decoder needs to use the same order
1/25/2009
J. Liang SFU ENSC861 9
Decoder needs to use the same order
Disjoint but complete partition:
1: [0, 0.8): 0, 0.7999999
2: [0.8, 0.82): 0.8, 0.8199999
3: [0.82, 1): 0.82, 0.9999999
1 2 3
Concept of the encoding (not practical)
Input sequence: 1321
Range 1
1 2 3
0 0.8 0.82 1.0
Range 0.8
1 2 3
0 0.64 0.656 0.8
1/25/2009
J. Liang SFU ENSC861 10
Range 0.00288
1 2 3
0.7712 0.773504 0.7735616 0.77408
Range 0.144
1 2 3
0.656 0.7712 0.77408 0.8
Termination: Encode the lower end or midpoint to signal the end.
Difficulties: 1. Shrinking of interval requires very high precision for long sequence.
2. No output is generated until the entire sequence has been processed.
Encoder Pseudo Code

= =
x
X
dx x p x X P x F ) ( ) ( ) (
Probability Mass Function
X
1 2 3 4
0.2 0.2
0.4
0.2
Cumulative Density Function (CDF)
For continuous distribution:

i
For discrete distribution:
1/25/2009
J. Liang SFU ENSC861 11
X
CDF
1 2 3 4
0.2
0.4
0.8
1.0
Properties:
Non-decreasing
Piece-wise constant
Each segment is closed at the lower end.

=
= = =
i
k
X
k X P i X P i F ) ( ) ( ) (
). 1 ( ) ( ) ( = = i F i F i X P
X X
Encoder Pseudo Code
LOW=0.0, HIGH=1.0;
while (not EOF) {
n = ReadSymbol();
RANGE = HIGH - LOW;
HIGH = LOW + RANGE * CDF(n);
LOW = LOW + RANGE * CDF(n-1);
}
Keep track of
LOW, HIGH, RANGE
Any two are
sufficient, e.g.,
LOW and RANGE.
1/25/2009
J. Liang SFU ENSC861 12
output LOW;
Input HIGH LOW RANGE
Initial 1.0 0.0 1.0
1 0.0+1.0*0.8=0.8 0.0+1.0*0 = 0.0 0.8
3 0.0 + 0.8*1=0.8 0.0 + 0.8*0.82=0.656 0.144
2 0.656+0.144*0.82=0.77408 0.656+0.144*0.8=0.7712 0.00288
1 0.7712+0.00288*0=0.7712 0.7712+0.00288*0.8=0.773504 0.002304
Concept of the Decoding (not practical)
Decode 1
1 2 3
0 0.8 0.82 1.0
Decode 3
1 2 3
0 0.64 0.656 0.8
Suppose encoder encodes the lower end: 0.7712
1/25/2009
J. Liang SFU ENSC861 13
Decode 1
1 2 3
0.7712 0.773504 0.7735616 0.77408
Decode 2
1 2 3
0.656 0.7712 0.77408 0.8
Drawback: need to recalculate all thresholds each time.
1 2 3
0 0.8 0.82 1.0
1 2 3
Receive 0.7712
Decode 1
x =(0.7712-0) / 0.8
= 0.964
Decode 3
Simplified Decoding (Still needs floating-point )
range
low x
x

Normalize RANGE to [0, 1) each time

No need to recalculate the thresholds.
1/25/2009
J. Liang SFU ENSC861 14
1 2 3
0 0.8 0.82 1.0
1 2 3
0 0.8 0.82 1.0
0 0.8 0.82 1.0
Decode 3
x =(0.964-0.82) / 0.18
= 0.8
Decode 2
x =(0.8-0.8) / 0.02
= 0
Decode 1.
Stop.
Decoder Pseudo Code
Low = 0; high = 1;
x = GetEncodedNumber();
While (x low) {
n = DecodeOneSymbol(x);
output symbol n;
1/25/2009
J. Liang SFU ENSC861 15
x = (x - CDF(n-1)) / (CDF(n) - CDF(n-1));
};
But this method still needs high-precision floating point operations
Outline
Introduction
Basic Encoding and Decoding
Uniqueness and Efficiency
Scaling and Incremental Coding
1/25/2009
J. Liang SFU ENSC861 16
Integer Implementation
Uniqueness and Efficiency
How to represent the final tag uniquely and efficiently?
Answer: Take the binary value of the tag T(X) and truncate to
(X is the sequence {x1, , xm})
Range 0.00288
1 2 3
0.7712 0.773504 0.7735616 0.77408
Termination: Encode the lower end or midpoint to signal the end.
1/25/2009
J. Liang SFU ENSC861 17
bits. 1
) (
1
log ) ( +
(
(
(

=
X p
X l
0. p(X) ), (
2
1
1) - F(X T(X) > + = X p Proof: Assuming midpoint tag:

) ( ) ( T(X)
) (
X F X T
X l
<
First show the truncated code is unique, that is, code is within [F(X-1), F(X)).
1).
So

) (
T(X)
X l
is below the high end of the interval.
1 bit longer than
Shannon code
Uniqueness and Efficiency
) (
2
1
2 2 2
1
) (
1
log
1
) (
1
log
) (
X p
X p
X p
X l
= =
|
|

\
|
+
|
|

\
|
+
(
(
(

0. p(X) ), (
2
1
1) - F(X T(X) > + = X p
l(X)
X p
-
2 ) (
2
1
1) - F(X - T(X) =
F(X)
2).
By def,
1/25/2009
J. Liang SFU ENSC861 18
( )
( )
T(X)- T(X) 2
l X
l X

(


). 1 ( T(X)
) (
X F
X l

) ( T(X) ) 1 (
) (
X F X F
X l
<
So the truncated code is still in the interval. This proves the uniqueness.
Together with
Thus
F(X-1)
T(X)

) (
T(X)
X l
) (
2
X l

) (
2
X l

Uniqueness and Efficiency

Prove the code is uniquely decodable (prefix free):
Any code with prefix

|

+
) ( ) ( ) (
2
1
T(X) , T(X)
X l X l X l

) (
T(X)
X l
is in
Need to show that this is in [F(X-1), F(X) ):
F(X)

) (
T(X)
X l

) (
) (
2
1
T(X)
X l
X l
+

). 1 ( T(X) X F We already show
1/25/2009
J. Liang SFU ENSC861 19
F(X-1)

). 1 ( T(X)
) (
X F
X l
We already show
Only need to show

) ( ) (
2
1
T(X) ) (
X l X l
X F >

) ( ) (
2
1
2
) (
) ( ) ( T(X) ) (
X l X l
X p
X T X F X F > = >

) (
T(X)
X l
is prefix free if
bits. 1
) (
1
log ) ( +
(
(
(

=
X p
X l
Uniqueness and Efficiency
Efficiency of arithmetic code:
bits. 1
) (
1
log ) (
1
1
+
(
(
(

=
m
m
X p
X l
{ } 1
) (
1
log ) ( ) ( ) (
1
1 1 1
|
|

\
|
+
(
(
(

= =

m
m m m
X p
X P X l X p E L
1 1
: { ,..., }
m
m
X x x
1/25/2009
J. Liang SFU ENSC861 20
2 ) ( 1 1
) (
1
log ) (
) (
1
1
1
1
+ =
|
|

\
|
+ +
\
(

m
m
m
X H
X p
X P
X p
l(X) is the bits to code a sequence {x1, , xm}.
Assume iid sequence,
) ( ) (
1
X mH X H
m
=
m
X H
m
L
X H
2
) ( ) ( + L/m H(X) for large m.
Uniqueness and Efficiency
Comparison with Huffman code:
Expected length of Huffman code:
1 ) ( ) (
*
+ X H L X H
Huffman code can reduce the overhead by
1/25/2009
J. Liang SFU ENSC861 21
jointly encoding more symbols, but needs
much larger alphabet size.
Arithmetic coding is more efficient for longer
sequences.
Outline
Introduction
Basic Encoding and Decoding
Uniqueness and Efficiency
Scaling and Incremental Coding
1/25/2009
J. Liang SFU ENSC861 22
Integer Implementation
Scaling and Incremental Coding
Problems of Previous examples:
Need high precision
No output is generated until the entire sequence is
encoded
Key Observation:
As the RANGE reduces, many MSBs of LOWand HIGH become
identical:
1/25/2009
J. Liang SFU ENSC861 23
identical:
Example: Binary form of 0.7712 and 0.773504:
0.1100010.., 0.1100011..,
We can output identical MSBs and re-scale the rest:
Incremental encoding
Can achieve infinite precision with finite-precision integers.
Three scaling scenarios: E1, E2, E3
Important Rules: Apply as many scalings as possible before
further encoding and decoding.
E1 and E2 Scaling
E1: [LOW HIGH) in [0, 0.5)
LOW: 0.0xxxxxxx (binary),
HIGH: 0.0xxxxxxx.
0 0.5 1.0
E2: [LOW HIGH) in [0.5, 1)
Output 0, then shift left by 1 bit
[0, 0.5) [0, 1): E1(x) = 2 x
0 0.5 1.0
1/25/2009
J. Liang SFU ENSC861 24
0 0.5 1.0
E2: [LOW HIGH) in [0.5, 1)
LOW: 0.1xxxxxxx,
HIGH: 0.1xxxxxxx.
Output 1, subtract 0.5,
shift left by 1 bit
[0.5, 1) [0, 1): E2(x) = 2(x - 0.5)
0 0.5 1.0
The 3
rd
scaling, E3, will be studied later.
Encoding with E1 and E2
0 0.8 1.0
Input 1
0 0.656 0.8
Input 3
E2: Output 1
2(x 0.5)
0.312 0.5424 0.54816 0.6
Input 2
0.0848 0.09632
E2: Output 1
E1: 2x, Output 0
Symbol Prob.
1 0.8
2 0.02
3 0.18
1/25/2009
J. Liang SFU ENSC861 25
0.0848 0.09632
0.1696 0.19264
E1: 2x, Output 0
E1: Output 0
0.3392 0.38528
0.6784 0.77056
E1: Output 0
0.3568 0.54112
E2: Output 1
Input 1
0.3568 0.504256
Encode any value
in the tag, e.g., 0.5
Output 1
All: 1100011 (0.7734)
To verify
LOW = 0.5424 (0.10001010... in binary),
HIGH = 0.54816 (0.10001100... in binary).
So we can send out 10001 (0.53125)
Equivalent to E2E1E1E1E2
After left shift by 5 bits:
LOW = (0.5424 0.53125) x 32 = 0.3568
1/25/2009
J. Liang SFU ENSC861 26
LOW = (0.5424 0.53125) x 32 = 0.3568
HIGH = (0.54816 0.53125) x 32 = 0.54112
Same as the result in the last page.
(In this example, suppose 7 bits are enough for the
decoding)
Comparison with Huffman
Input Symbol 1 does not cause any output
Input Symbol 3 generates 1 bit
Input Symbol 2 generates 5 bits
Symbol Prob.
1 0.8
2 0.02
3 0.18
Rule: Complete all possible scaling before
encoding the next symbol
1/25/2009
J. Liang SFU ENSC861 27
Symbols with larger probabilities generates less
number of bits.
Sometimes no bit is generated at all
Advantage over Huffman coding
Large probabilities are desired in arithmetic coding
Can use context-adaptive method to create larger probability
and to improve compression ratio.
Incremental Decoding
0 0.8 1.0
Read 5 bits: Decode 1.
After reading 6 bits:
Tag: 110001, 0.765625
No scaling.
If input is 1100011, the 1
st
symbol can be decoded without ambiguity after 5 bits
are read. When 6 bits are read, the status is:
0 0.656 0.8
Decode 3, E2 scaling:
Shift out 1 bit, read 1 bit
Tag: 100011 (0.546875)
0.312 0.5424 0.54816 0.6
1/25/2009
J. Liang SFU ENSC861 28
Tag: 100011 (0.546875)
0.312 0.5424 0.54816 0.6
0.0848 0.09632
Decode 2, E2 scaling
Tag: 000110 (0.09375)
0.1696 0.19264
E1: Tag: 001100 (0.1875)
E1: Tag: 011000 (0.375)
0.3392 0.38528
0.6784 0.77056
E1: Tag: 110000 (0.75)
Decode 1
0.3568 0.54112
E2: Tag: 100000 (0.5)
Encoding Pseudo Code with E1, E2
EncodeSymbol(n) {
//Update variables
RANGE = HIGH - LOW;
Symbol Prob. CDF
1 0.8 0.8
2 0.02 0.82
3 0.18 1
(For floating-point implementation)
Rule: Complete all possible scalings
before further decoding. Adjust LOW,
HIGH and Tag together.
1/25/2009
J. Liang SFU ENSC861 29
RANGE = HIGH - LOW;
HIGH = LOW + RANGE * CDF(n);
LOW = LOW + RANGE * CDF(n-1);
//Apply all possible scalings before encoding the
//next symbol
while LOW, HIGH in [0, 0.5) or [0.5, 1) {
Output 0 for E1 and 1 for E2
scale LOW, HIGH by E1 or E2 rule
}
}
Decoding Pseudo Code with E1, E2
DecodeSymbol(Tag) {
RANGE = HIGH - LOW;
n = 1;
While ( (tag - LOW) / RANGE >= CDF(n) ) {
n++;
}
Symbol Prob. CDF
1 0.8 0.8
(For floating-point implementation)
1/25/2009
J. Liang SFU ENSC861 30
HIGH = LOW + RANGE * CDF(n);
LOW = LOW + RANGE * CDF(n-1);
//keep scaling before decoding next symbol
while LOW, HIGH in [0, 0.5) or [0.5, 1) {
scale LOW, HIGH by E1 or E2 rule
read one more bit, update Tag
}
return n;
}
2 0.02 0.82
3 0.18 1
Outline
Introduction
Basic Encoding and Decoding
Uniqueness and Efficiency
Scaling and Incremental Coding
1/25/2009
J. Liang SFU ENSC861 31
E1 scaling
E2 scaling
Integer Implementation
Integer Implementation
Old formulas:
HIGH LOW + RANGE * CDF(n);
LOW LOW + RANGE * CDF(n-1);
Finite-Precision Integer approximation of CDF ( ):
The number of occurrence of each symbol is usually
collected by a counter.
Allow adaptive arithmetic coding
1/25/2009
J. Liang SFU ENSC861 32
k nk P(k) Sum(k)
0 - - 0
1 40 0.8 40
2 1 0.02 41
3 9 0.18 50
1 2 3
0 40 41 50
Allow adaptive arithmetic coding

i Sum

n
i F CDF
i
k
k
X
) (
) (
1
= =

=
Integer Implementation
HIGH LOW + RANGE * CDF(n);
LOW LOW + RANGE * CDF(n-1);


; / ) 1 (
; 1 / ) (
; 1
n Sum RAGE LOW LOW
n Sum RAGE LOW HIGH
LOW HIGH RAGE
+
+
+ =
Why + 1 in RANGE and 1 in HIGH?
Semi-open sub-interval: [0.8, 0.82)
1/25/2009
J. Liang SFU ENSC861 33
Semi-open sub-interval: [0.8, 0.82)
HIGH should be less than the LOW of the next interval
The best integer value for HIGH is HIGH = (next LOW) 1
[HIGH, HIGH + 1) still belongs to the current interval,
although we could not represent it explicitly.
n-1 n
LOW2
LOW1
0
HIGH1
Examples:
8-bit precision
LOW = 0
HIGH = 255
RANGE = 256
Example
k P(k) Nk Sum(k)
0 - - 0
1 0.8 40 40
2 0.02 1 41
3 0.18 9 50
8-bit integers, initial LOW = 0, HIGH = 255 RANGE=256


n Sum RAGE LOW LOW
n Sum RAGE LOW HIGH
LOW HIGH RAGE
/ ) 1 (
1 / ) (
1
+
+
+ =
0
50
0 256
0 =
(

+ = LOW
If n = 1:
203 1
50
40 256
0 =
(

+ = HIGH
1/25/2009
J. Liang SFU ENSC861 34
1 2 3
0 203 208 255
50
(

50
(

204
50
40 256
0 =
(

+ = LOW
If n = 2:
208 1
50
41 256
0 =
(

+ = HIGH
209
50
41 256
0 =
(

+ = LOW
If n = 3:
255 1
50
50 256
0 =
(

+ = HIGH
Overflow without -1
E1 Scaling in Integer Implementation
E1 Scaling: [0, 0.5) [0, 1), E1(x) = 2 x.
LOW = 0xxxxxxx, HIGH =0xxxxxxx
Output the MSB value 0, then shift left by 1 bit
To achieve high precision: Shift in 1 to HIGH and 0 to LOW
HIGH: 0xxxxxxx xxxxxxx1 HIGH = (HIGH << 1) + 1;
LOW: 0xxxxxxx xxxxxxx0 LOW = LOW << 1;
1/25/2009
J. Liang SFU ENSC861 35
LOW: 0xxxxxxx xxxxxxx0 LOW = LOW << 1;
HIGH 0 . 0 x x x x x x x 1 1 1 ...
LOW 0 . 0 x x x x x x x 0 0 0 ...
Always assume HIGH ends with infinite number of 1s:
So that it approximates the LOW of the next interval.
This also ensures the RANGE is doubled after scaling:
HIGH LOW + 1 (2 x HIGH + 1 2 x LOW + 1) = 2(HIGH LOW + 1)
E2 Scaling in Integer Implementation
E2 Scaling: [0.5, 1) [0, 1), E2(x) = 2 (x - 0.5)
LOW = 1xxxxxxx, HIGH =1xxxxxxx
Output the MSB, then shift left by 1 bit (mul by 2)
Same trick: Shift in 1 to HIGH and 0 to LOW
HIGH: 1xxxxxxx xxxxxxx1 HIGH = (HIGH << 1) + 1;
LOW: 1xxxxxxx xxxxxxx0 LOW = LOW << 1;
1/25/2009
J. Liang SFU ENSC861 36
LOW: 1xxxxxxx xxxxxxx0 LOW = LOW << 1;
0 . 1 x x x x x x x 1 1 1 ...
0 . 1 x x x x x x x 0 0 0 ...
Outline
Introduction
Basic Encoding and Decoding
Uniqueness and Efficiency
Scaling and Incremental Coding
1/25/2009
J. Liang SFU ENSC861 37
E1 scaling
E2 scaling
E3 scaling
E3 Scaling: [0.25, 0.75)[0, 1)
If RANGE straddles 1/2, E1 and E2 cannot be applied,
but the range can be quite small
Example: LOW=0.4999, HIGH=0.5001
Binary: LOW=0.01111., HIGH=0.1000
We may not have enough bits to represent the interval.
1/25/2009
J. Liang SFU ENSC861 38
0 1
0.5
0.25
0.75
E3 Scaling:
[0.25, 0.75) [0, 1):
E3(x) = 2(x - 0.25)
0 1
0.5
Integer Implementation of E3
01xxxxxx
- 01000000
LOW:
10xxxxxx
- 01000000
HIGH:
Same trick: Shift in 1 to HIGH and 0 to LOW
HIGH = ((HIGH QUARTER) << 1) + 1;
LOW = (LOW - QUARTER) << 1;
QUARTER = 2^(m - 2) for m-bit integer. (64 for m = 8 bits)
1/25/2009
J. Liang SFU ENSC861 39
- 01000000
00xxxxxx
2
0xxxxxx0
- 01000000
01xxxxxx
2
1xxxxxx0
+ 1
1xxxxxx1
Another way to implement E3 (Sayood book pp. 99):
Left shift old LOW and HIGH, complement the new MSB.
LOW 01xxxxxx 0xxxxxx0
HIGH 10xxxxxx 1xxxxxx1
Signaling of E3
Recall: we generate 1 if E2 is used, and 0 if E1 is used
But what should we generate when E3 is used?
Important relationships: (www.copro.org/download/ac_en.pdf)
( ) ( )
( ) ( )
1 2 3 1
E E E E
E E E E
n n
n n
o o
o o
=
=
1/25/2009
J. Liang SFU ENSC861 40
: n Ej scalings, followed by an Ei scaling.
A series of E3 followed by an E1 is equivalent to an E1
followed by a series of E2.
A series of E3 followed by an E2 is equivalent to an E2
followed by a series of E1.
( ) ( )
2 1 3 2
E E E E o o =
( )
n
j i
E E o
Notation:
Example
0 0.8 1.0
Input 1
0 0.656 0.8
Input 3
0.312 0.5424 0.54816 0.6
Input 2
0.0848 0.09632
E2: Output 1
E1: Output 0
Previous example without E3:
(E2: Output 1)
1/25/2009
J. Liang SFU ENSC861 41
0.1696 0.19264
E1: Output 0
With E3:
0.312 0.6
0.124 0.7
E3: (x-0.25)x2
0.1696 0.19264
E2: Output 1
Input 2
0.5848 0.59632
state after E2E3 = state after E1E2, but outputs are different
Encoding Operation with E3
0.312 0.5424 0.54816 0.6
Input 2
0.0848 0.09632
E2: Output 1
0.1696 0.19264
E1: Output 0
With E3:
Without E3:
1/25/2009
J. Liang SFU ENSC861 42
0.312 0.6
0.124 0.7
E3 (no output)
0.1696 0.19264
E2: Output 1
Input 2
0.5848 0.59632
Dont send anything when E3 is used, but send a 0 after E2:
Same output, same final state Equivalent operations
Output 0 here!
General Solutions
If there is an E2 after n E3 operations:
Send 1, 0, 0, , 0 after E2
( ) ( )
( ) ( )
2 1 3 2
1 2 3 1
E E E E
E E E E
n n
n n
o o
o o
=
=
1/25/2009
J. Liang SFU ENSC861 43
If there is an E1 after n E3:
Send 0, 1, 1, , 1 after E1
n
n
Decoding for E3
0 Without E3: 0.8 1.0
Input 1100011
Read 6 bits:
Tag: 110001 (0.765625)
Decode 1
0 0.656 0.8
Decode 3, E2 scaling
Tag: 100011 (0.546875)
0.312 0.5424 0.54816 0.6
0.0848 0.09632
Decode 2, E2 scaling
Tag: 000110 (0.09375)
E1:
Tag: 001100 (0.1875)
1/25/2009
J. Liang SFU ENSC861 44
0.1696 0.19264
Tag: 001100 (0.1875)
With E3:
0.312 0.6
Tag: 100011 (0.546875)
0.5848 0.59632
0.124 0.7
E3:
Tag: 100110 (0.59375)
0.1696 0.19264
Decode 2, E2 scaling
Tag: 001100 (0.1875)
Apply E3 whenever it is possible, everything else is same.
A Simple Proof of E2E3 = E1E2
Given the same interval r:
Given an original interval r = [a, b):
After applying E2: [0.5, 1) [0, 1), the interval becomes
r1 = (r 0.5) x 2
After E1: [0, 0.5) [0, 1):
r2 = r1 x 2 = ((r 0.5) x 2) x 2
1/25/2009
J. Liang SFU ENSC861 45
Given the same interval r:
After E3: [0.25, 0.75) [0, 1):
r3 = (r 0.25) x 2
After E2:
r4 = (r3 0.5) x 2 = ((r 0.25) x 2 0.5) x 2
= (r 0.5) x 2 x 2
= r2
For formal proof: www.copro.org/download/ac_en.pdf
Summary of Different Scalings
Need E1 scaling
Need E2 scaling
Need E3 scaling
0 0.25 0.5 0.75 1.0
1/25/2009
J. Liang SFU ENSC861 46
No scaling is required.
Continue to
encode/decode the next
symbol.
Outline
Introduction
Basic Encoding and Decoding
Uniqueness and Efficiency
Scaling and Incremental Coding
1/25/2009
J. Liang SFU ENSC861 47
E1 scaling
E2 scaling
E3 scaling
Integer Implementation
Complete integer algorithm with E1, E2, E3
Encoding Pseudo Code with E1, E2, E3
EncodeSymbol(n) {
//Update variables
RANGE = HIGH - LOW + 1;
HIGH = HIGH + RANGE * Sum( n ) / N - 1;
LOW = LOW + RANGE * Sum(n-1) / N;
(For integer implementation)
Round off to integer
1/25/2009
J. Liang SFU ENSC861 48
LOW = LOW + RANGE * Sum(n-1) / N;
//Scaling before encoding next symbol
EncoderScaling(); //see next slide
}
Encoding Pseudo Code with E1, E2, E3
EncoderScaling( ) {
while (E1, E2 or E3 is possible) {
if (E3 is possible) {
HIGH = ((HIGH - QUARTER) << 1) + 1;
LOW = (LOW - QUARTER) << 1;
Scale3++; //Save number of E3, no output here
}
if (E1 or E2 is possible) {
Let b=0 for E1 and b=1 for E2
1/25/2009
J. Liang SFU ENSC861 49
Let b=0 for E1 and b=1 for E2
send b
HIGH = (HIGH << 1) + 1;
LOW = (LOW << 1);
while (Scale3 > 0) { //Compensate for E3 here:
send 1-b; //E2 (E3)^n = (E1)^n E2
Scale3--; //Send one bit for each E3
}
}
}
}
DecodeSymbol(Tag) {
RANGE = HIGH - LOW + 1;
n = 1;
While (Tag > LOW + RANGE * Sum(n) / N - 1) {
n++;
}
Decoding Pseudo Code with E1, E2, E3
(For integer implementation)
Intervals: [0, 203], [204, 208], [209, 255]
Round off to integer: HIGH of each interval
1/25/2009
J. Liang SFU ENSC861 50
HIGH = LOW + RANGE * Sum(n) / N - 1;
LOW = LOW + RANGE * Sum(n-1) / N;
//keep scaling before decoding next symbol
DecoderScaling(Tag); //next slide
return n;
}
Round off to integer: HIGH of each interval
Decoding Pseudo Code with E1, E2, E3
DecoderScaling(Tag) {
while (E1, E2 or E3 is possible) {
if (E1 or E2 is possible) {
LOW = LOW << 1;
HIGH = (HIGH << 1) + 1;
Tag = Tag << 1;
Tag = Tag | ReadBits(1);
}
1/25/2009
J. Liang SFU ENSC861 51
if (E3 is possible) {
LOW = (LOW - QUARTER) << 1;
HIGH = ((HIGH - QUARTER) << 1) + 1;
Tag = (Tag - QUARTER) << 1;
Tag = Tag | ReadBits(1);
}
}
}
0 203 255
Input 1
0 167 203
Input 3
E2: Output 1
78 151
E3: Scale3=1
Input 2
28 146 148 175
36 41
E2: Output 1
Output 0
Scale3 = 0
Integer Encoding with E1, E2, E3
8-bit precision
1/25/2009
J. Liang SFU ENSC861 52
72 83
E1: Output 0
32 79
E2: Output 1
144 167
E1: Output 0
36 41 Scale3 = 0
64 159
E1: Output 0
E3: Scale3=1
0 152 191
Input 1
Final output: 11000100 10000000.
Final LOW=0: send 8 zeros, but Scale3=1, so insert an 1 after the first zero.
0 203 255
Integer Decoding with E1, E2, E3
Read 8 bits:
11000100 (196)
Decode 1
0 167 203
Decode 3
Input: 11000100 10000000
E2: Tag=10001001 (137)
1/25/2009
J. Liang SFU ENSC861 53
E3: Tag = 10010010 (146)
Decode 2
28 146 148 175
78 151
E2: Tag=10001001 (137)
36 41
E2: Tag=00100100 (36)
Decode 1
Tag = LOW: stop.
Next Class
Binary Arithmetic Coding
Adaptive Arithmetic Coding
Applications
JBIG, H.264, JPEG 2000
1/25/2009
J. Liang SFU ENSC861 54