A HIGH-PERFORMANCE HARDWARE IMPLEMENTATION OF THE H.

264
SIMPLIFIED 8X8 TRANSFORMATION AND QUANTIZATION

Ihab Amer, Wael Badawy, and Graham Jullien

Advanced Technology Information Processing Systems (ATIPS)

Calgary, Alberta, Canada, T2N 1N4
{amer, badawy, jullien}@atips.ca

ABSTRACT and state-of-the-art standard [5]. Network friendliness and

good video quality at high and low bit rates are two
The recently approved digital video standard known as important features that distinguish H.264 from other
H.264 promises to be an excellent video format for use standards [6]-[10].
with a large range of applications. Real-time Unlike current standards, the usual floating-point 8x8
encoding/decoding is a main requirement for adoption of DCT is not the basic transformation in H.264. Instead, a
the standard to take place in the consumer marketplace. new transformation hierarchy is introduced that can be
Transformation and quantization in H.264 are relatively computed exactly in integer arithmetic. This eliminates
less complex than their correspondences in other video any mismatch issues between the encoder and the decoder
standards. Nevertheless, for real-time operation, a speedup in the inverse transform [7], [11]. In the initial H.264
is required for such processes. Especially after the recent standard, which was completed in May 2003, the
proposal to use an 8x8 integer approximation of Discrete transformation is primarily 4x4 in shape, which helps
Cosine Transform (DCT) to give significant compression reduce blocking and ringing artifacts.
performance at Standard Definition (SD) and High In July 2004, a new amendment called the Fidelity
Definition (HD) resolutions. This paper discusses a high- Range Extensions (FRExt, Amendment I) was added to
performance hardware implementation of the H.264 the H.264 standard. This amendment is currently receiving
simplified 8x8 transformation and quantization. The wide attention in the industry. It actually demonstrates
results show that the architecture satisfies the real-time further coding efficiency against current video coding
constraints required by different digital video applications. standards, potentially by as much as 3:1 for some key
applications. The FRExt project produced a suite of some
1. INTRODUCTION new profiles collectively called High profiles. Beside
supporting all features of the prior Main profile, all the
Due to the remarkable progress in the development of High profiles support an adaptive transform-block size
products and services offering full-motion digital video, and perceptual quantization scaling matrices [5]. In fact,
digital video coding currently has a significant economic the concept of adaptive transform-block size has proven to
impact on the computer, telecommunications, and imaging be an efficient coding tool within H.264 video coding
industry [1]. This raises the need for an industry standard layer design [12]. This led to the proposal of a seamless
for compressed video representation with extremely integration of a new 8x8 integer approximation of DCT
increased coding efficiency and enhanced robustness to (and prediction modes) into the specification with the least
network environments [2]. possible amount of technical and syntactical changes [13]-
Since the early phases of the technology, international [15].
video coding standards have been the engines behind the So far, most of the work in H.264 is software
commercial success of digital video compression. ITU-T oriented. However, a hardware implementation is
H.264/MPEG-4 (Part 10) Advanced Video Coding desirable for consumer products to provide compactness,
(commonly referred as H.264/AVC) is the newest entry in low power, robustness, cheap cost, and most importantly,
the series of international video coding standards. It was real-time operation. In our previous work [16]-[26], we
developed by the Joint Video Team (JVT), which was proposed hardware implementations for various blocks in
formed to represent the cooperation between the ITU-T the initial H.264 transformation hierarchy model and
Video Coding Experts Group (VCEG) and the ISO/IEC entropy coding. In this paper, we propose a high-
Moving Picture Experts Group (MPEG) [3]-[5]. performance hardware implementation of the H.264
Compared to the currently existing standards, H.264 newly-proposed simplified 8x8 transform and
has many new features that makes it the most powerful quantization.

0-7803-8874-7/05/$20.00 ©2005 IEEE II - 1137 ICASSP 2005

 The rest of this paper is organized as follows: Section Stage 2:
2 overviews the H.264 simplified 8x8 transform and b[0] = a[0] + a[3];
b[1] = a[1] + a[2];
quantization. In section 3, a description of the proposed b[2] = a[0] - a[3];
hardware prototype is introduced. Section 4 presents the b[3] = a[1] - a[2];
simulations and results achieved. Finally, section 5 b[4] = a[5] + a[6] + ((a[4]>>1) + a[4]);
concludes the paper. b[5] = a[4] – a[7] – ((a[6]>>1) + a[6]);
b[6] = a[4] + a[7] – ((a[5]>>1) + a[5]);
b[7] = a[5] – a[6] + ((a[7]>>1) + a[7]);
2. H.264 SIMPLIFIED 8X8 TRANSFORM AND
QUANTIZATION Stage 3:
w[0] = b[0] + b[1];
An integer approximation of 8x8 DCT was proposed in w[1] = b[2] + (b[3]>>1);
FRExt to be added to the JVT specification based on the w[2] = b[0] - b[1];
w[3] = (b[2]>>1) - b[3];
fact that at SD resolutions and above, the use of block w[4] = b[4] + (b[7]>>2);
sizes smaller than 8x8 is limited [15]. This transform is w[5] = b[5] + (b[6]>>2);
applied to each block in the luminance component of the w[6] = b[6] – (b[5]>>2);
input video stream. It allows for bit-exact implementation w[7] = -b[7] + (b[4]>>2);
for all encoders and decoders. In spite of being more
complex compared to the 4x4 DCT-like transform that is Hence, the 2-D transform operation can be
adopted by the initial H.264 specification, the proposed implemented using signed additions and right-shifts only,
transform gives excellent compression performance when avoiding expensive multiplications. The post-scaling and
used for high-resolution video streams using a number of quantization formulas are shown in Equations (3)-(5).
operations comparable to the number of operations qbits 15  (QP DIV 6) (3)
required for the corresponding four 4x4 blocks using the Z ij SHR ( Wij .MF  f , qbits  1) (4)
fast butterfly implementation of the existing 4x4 transform
[13], [14]. Sign ( Z ij ) Sign (W ij ) (5)
The 2-D forward 8x8 transform is computed in a where QP is a quantization parameter that enables the
separable way as a 1-D horizontal (row) transform encoder to accurately and flexibly control the trade-off
followed by a 1-D vertical (column) transform as shown between bit rate and quality. It can take any integer value
in Equation (1). from 0 up to 51. Zij is an element in the quantized
T
W Cf XCf (1) transform coefficients matrix. MF is a multiplication
factor that depends on (m = QP mod 6) and the position (i,
where the Matrix C f is given by Equation (2).
j) of the element in the matrix as shown in Table 1. SHR()
is a procedure that right-shifts the result of its first
ª8 8 8 8 8 8 8 8 º argument a number of bits equal to its second argument. f
«12 is defined in the reference model software as 2qbits/3 for
10 6 3 3  6  10  12 »
« » Intra blocks and 2qbits/6 for Inter blocks [3], [4].
«8 4 4 8 8 4 4 8 »
«10  3  12  6 6 12 3  10 » Table 1. Multiplication Factor (MF)
Cf « » .1 / 8 (2) m (i, j) (i, j) (i, j) (i, j) (i, j) (i, j)
8 8 8 8
«8 8 8 8
» 0
 G0
13107
 G1
11428
 G2
20972
 G3
12222
 G4
16777
 G5
15481
«6  12 3 10  10  3 12  6» 1 11916 10826 19174 11058 14980 14290
«4 8 8 4 4 8 8 4
» 2 10082 8943 15978 9675 12710 11985
« » 3 9362 8228 14913 8931 11984 11295
«¬3 6 10  12 12  10 6  3»¼ 4 8192 7346 13159 7740 10486 9777
5 7282 6428 11570 6830 9118 8640
Each of the 1-D transforms is computed using 3- *G0: i = [0, 4], j = [0, 4]
stages fast butterfly operations as follows [14]: G1: i = [1, 3, 5, 7], j = [1, 3, 5, 7]
G2: i = [2, 6], j = [2, 6]
Stage 1: G3: (i = [0, 4], j = [1, 3, 5, 7]) ˆ (i = [1, 3, 5, 7], j = [0, 4])
a[0] = x[0] + x[7]; G4: (i = [0, 4], j = [2, 6]) ˆ (i = [2, 6], j = [0, 4])
a[1] = x[1] + x[6]; G5: (i = [2, 6], j = [1, 3, 5, 7]) ˆ (i = [1, 3, 5, 7], j = [2, 6])
a[2] = x[2] + x[5];
a[3] = x[3] + x[4];
a[5] = x[0] - x[7];
a[6] = x[1] - x[6];
a[7] = x[2] - x[5];
a[8] = x[3] - x[4];

II - 1138
 3. HARDWARE PROTOTYPE The architecture is composed of two main stages. The
first one contains two blocks; the Transform block, which
A block diagram of the proposed architecture is shown in is composed of the three stages of the fast butterfly
Figure 1. The architecture uses 8x8 parallel blocks, QP, a operations mentioned in Section 2, and the QP-Processing
synchronizing clock, and an enabling signal (Input Valid) block, which is responsible for calculating the
as inputs. It outputs the quantized transform coefficients intermediate variables needed for quantization, such as f,
and the signal Output Valid. A block diagram of the qbits, and (P0 – P5), which are the values of the
architecture is shown in Figure 1. multiplication factors at the six different groups of
positions in the matrix as shown in Table 1. Finally, the
Quantization process takes place in the second main stage
Input Block of the design. This is done by performing the addition and
Quant.
(X00-X77) 8x8 Trans. multiplication operations in the Arithmetic block, and
Forward Coefficients finally the shifting operations in the Shifter block.
Trans. &
Quant.
QP
4. SIMULATIONS AND RESULTS

CLK Output Valid The architecture of the H.264 simplified 8x8

transformation is prototyped using VHDL language. It is
Input Valid
simulated using the Mentor Graphics© ModelSim 5.4®
Figure 1. A block diagram of the proposed hardware simulation tool, and synthesized using Synplify Pro 7.1®
architecture from Synplicity©. The target technology is the FPGA
device XC2V4000 (BF957 package) from the Virtex-II
The architecture is designed to perform pipelined family of Xilinx©.
operations, which drastically reduces the required memory Table 2 summarizes the performance of the
resources and accesses, avoids any stall states, and prototyped architecture.
dramatically improves the throughput of the architecture.
Figure 2 gives a detailed block diagram of the proposed Table 2. Performance of the prototyped architecture
architecture showing the flow of signals between the main
Critical CLK Freq. # of i/p # of o/p
stages of the design.
Path (ns) (MHz) Buffers Buffers
14.598 68.5 583 1217
Quantization # of I/O #of Reg. Bits Total # of # of clock
Transform Reg. Bits not inc. (I/O) LUT buffers
1219 16893 29018 1
Stage 1

Stage 2

Stage 3

(X00- (W00-
X77) W77)
A 14.598 ns critical path is estimated by the synthesis
tool. Since at steady state, the architecture outputs a whole
8x8 encoded block with each clock pulse, therefore the
Arithmetic

(Z00- time required to encode a whole SD frame of 704 u 480

Quant. Enable
Z77) pixels can be calculated as follows:
Shifter

P0- Time required per CIF frame =

P5 Time required per block u Number of
QP blocks per frame
QP-Processing Number of pixels per frame
Output Valid = 14.598 ns u
f Number of pixels per block
Input Valid
(704 u 480) pixels per frame
CLK = 14.598 ns u
qbits (8 u 8) pixels per block
 77.1 Ps
This value is about 216 times less than the 16.67 ms
Figure 2. A detailed block diagram of the proposed time required for continuous motion (assuming a refresh
hardware architecture rate of 60 frames/sec). Similarly, it can be shown that the
time required to encode a whole High Definition

II - 1139
Television (HDTV) frame of a 720 u 1280 pixels Transactions on Circuits and Systems For Video Technology, Vol. 13,
No. 7, pp. 560-576, July 2003.
resolution, and a 60 frames/sec frame rate is 0.21 ms, [7] “Emerging H.264 Standard: Overview and TMS320DM642-Based
which is about 79 times less than the 16.6 ms time Solutions for Real-Time Video Applications,” A white paper. [Online].
required for continuous motion. Hence, the introduced Available:
architecture satisfies the real-time constraints for SD, HD, http://www.ubvideo.com, December 2002.
[8] K. Denolf, C. Blanch, G. Lafruit, and A. Bormans, “Initial memory
and even higher resolution video formats. complexity analysis of the AVC codec,” IEEE Workshop on Signal
Processing Systems, 2002 (SIPS’02), pp. 222-227, October 2002.
5. CONCLUSION [9] T. Stockhammer, M. M. Hannuksela, T. Wiegand, “H.264/AVC in
wireless environments,” IEEE Transactions on Circuits and Systems For
Video Technology, Vol. 13, No. 7, pp. 657-673, July 2003.
A high-performance architecture for prototyping the [10] M. Horowitz, A. Joch, F. Kossentini, A. Hallapuro, “H.264/AVC
simplified 8x8 transformation and quantization, which Baseline Profile Decoder Complexity Analysis,” IEEE Transactions on
was recently adopted by the H.264 standard is developed. Circuits and Systems For Video Technology, Vol. 13, No. 7, pp. 704-
The architecture is designed to perform pipelined 716, July 2003.
[11] H. S. Malvar, A. Hallapuro, M. Karczewicz, and L. Kerofsky,
operations, which drastically reduces the required memory “Low-Complexity Transform and Quantization in H.264/AVC,” IEEE
resources and accesses, avoids any stall states, and Transactions on Circuits and Systems For Video Technology, Vol. 13,
dramatically improves the throughput of the architecture. No. 7, pp. 598-603, July 2003.
It satisfies the real-time constraints required by different [12] M. Wien, “Clean-up and improved design consistency for ABT,”
Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, doc. JVT
high-resolution digital video applications. The system is –E025.
simulated using ModelSim 5.4®, and synthesized using [13] S. Gordon, D. Marpe, and T. Wiegand, “Simplified Use of 8x8
Synplify Pro 7.1®, targeting the FPGA device XC2V4000 Transform – Proposal,” Joint Video Team (JVT) of ISO/IEC MPEG and
(BF957 package) from the Virtex-II family of Xilinx©. ITU-T VCEG, doc. JVT –J029.
[14] S. Gordon, D. Marpe, and T. Wiegand, “Simplified Use of 8x8
The ideal way to utilize the proposed design is to embed it Transform – Updated Proposal & Results,” Joint Video Team (JVT) of
to an overall system, where the reference software runs on ISO/IEC MPEG and ITU-T VCEG, doc. JVT –K028, Munich, Germany,
a Digital Signal Processor (DSP), and the computationally March 2004.
extensive operations performed in the hardware block. [15] S. Gordon, “Simplified Use of 8x8 Transform,” Joint Video Team
(JVT) of ISO/IEC MPEG and ITU-T VCEG, doc. JVT –I022, San
Diego, USA, September 2003.
6. ACKNOWLEDGEMENT [16] I. Amer, W. Badawy, and G. Jullien, “Towards MPEG-4 Part 10
System On Chip: A VLSI Prototype For Context-Based Adaptive
The authors would like to thank Advanced Technology Variable Length Coding (CAVLC),” accepted in IEEE Workshop on
Signal Processing Systems, Austin, Texas, USA, October 2004.
Information Processing Systems (ATIPS), Alberta [17] I. Amer, W. Badawy, and G. Jullien, “A VLSI Prototype for
Ingenuity Fund (AIF), Natural Science and Engineering Hadamard Transform with Application to MPEG-4 Part 10,” accepted in
Research Council of Canada (NSERC), Canadian IEEE International Conference on Multimedia and Expo, Taipei,
Microelectronics Corporation (CMC), Micronet R&D, Taiwan, June 2004.
[18] I. Amer, W. Badawy, and G. Jullien, “Hardware Prototyping for The
Alberta Informatics Circle of Research Excellence H.264 4x4 Transformation,” proceedings of IEEE International
(iCORE), Canada Foundation for Innovation (CFI), and Conference on Acoustics, Speech, and Signal Processing, Montreal,
the Department of Electrical and Computer Engineering at Quebec, Canada, Vol. 5, pp. 77-80, May 2004.
the University of Calgary for supporting this research. [19] I. Amer, W. Badawy, and G. Jullien, “A SystemC Model for the
MPEG-4 Part 10 4x4 DCT-like Transformation and Quantization,”
ISO/IEC JTC1/SC29/WG11 M10830, Redmond, USA, July 2004.
7. REFERENCES [20] I. Amer, W. Badawy, and G. Jullien, “A Hardware Block for the
MPEG-4 Part 10 4x4 Transformation and Quantization,” ISO/IEC
[1] A. M. Tekalp, Digital Video Processing, Prentice-Hall, Inc., New JTC1/SC29/WG11 M10829, Redmond, USA, July 2004.
Jersey, USA, 1995. [21] I. Amer, W. Badawy, and G. Jullien, “A SystemC model for 4x4
[2] “ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC,” Draft Text of Final Hadamard Transform and Quantization with application to MPEG-4 Part
Draft International Standard for Advanced Video Coding, [Online]. 10,” ISO/IEC JTC1/SC29/WG11 M10828, Redmond, USA, July 2004.
Available: [22] I. Amer, W. Badawy, and G. Jullien, “A Hardware Block for 4x4
http://www.chiariglione.org/mpeg/working_documents.htm, March Hadamard Transform and Quantization in MPEG-4 Part 10,” ISO/IEC
2003. JTC1/SC29/WG11 M10827, Redmond, USA, July 2004.
[3] I. E. G. Richardson, “H.264/MPEG-4 Part 10: Transform & [23] I. Amer, W. Badawy, and G. Jullien, “A SystemC model for 2x2
Quantization,” A white paper. [Online]. Available: Hadamard Transform and Quantization with Application to MPEG–4
http://www.vcodex.com, March 2003. Part 10,” ISO/IEC JTC1/SC29/WG11 M10826, Redmond, USA, July 04.
[4] I. E. G. Richardson, H.264 and MPEG-4 Video Compression: Video [24] I. Amer, W. Badawy, and G. Jullien, “A Hardware Block for 2x2
Coding for Next-generation Multimedia, John Wiley & Sons Ltd., Hadamard Transform and Quantization with Application to MPEG–4
Sussex, England, December 2003. Part 10,” ISO/IEC JTC1/SC29/WG11 M10825, Redmond, USA, July 04.
[5] G. Sullivan, P. Topiwala, and A. Luthra, “The H.264 Advanced [25] I. Amer, W. Badawy, and G. Jullien, “An IP Block for MPEG-4 Part
Video Coding Standard: Overview and Introduction to the Fidelity 10 Context-Based Adaptive Variable Length Coding (CAVLC),”
Range Extensions,” SPIE Conference on Application of Digital Image ISO/IEC JTC1/SC29/WG11 M10824, Redmond, USA, July 2004.
Processing XXVII, Colorado, USA, August 2004. [26] I. Amer, W. Badawy, and G. Jullien, “A Proposed Hardware
[6] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, Reference Model for Spatial Transformation and Quantization in H.264,”
“Overview of the H.264/AVC Video Coding Standard,” IEEE accepted in Journal of Visual Communication and Image Representation
Special Issue on Emerging H.264/AVC Video Coding Standard.

II - 1140