878 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO.

6, SEPTEMBER 2000

Scalable Rate Control for MPEG-4 Video

Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE

Abstract—This paper presents a scalable rate control (SRC)

scheme based on a more accurate second-order rate-distortion
model. A sliding-window method for data selection is used to miti-
gate the impact of a scene change. The data points for updating a
model are adaptively selected such that the statistical behavior is
improved. For video object (VO) shape coding, we use an adaptive
threshold method to remove shape-coding artifacts for MPEG-4
applications. A dynamic bit allocation among VOs is implemented
according to the coding complexities for each VO.
SRC achieves more accurate bit allocation with low latency and
limited buffer size. In a single framework, SRC offers multiple
layers of controls for objects, frames, and macroblocks (MBs). At
MB level, SRC provides finer bit rate and buffer control. At mul-
tiple VO level, SRC offers superior VO presentation for multimedia
applications. The proposed SRC scheme has been adopted as part
of the International Standard of the emerging ISO MPEG-4 stan-
dard [1], [2].
Index Terms—Bit allocation, MPEG, multiple video object, rate Fig. 1. Layer structure of a network-based multimedia system.
control, rate-distortion optimization, shape coding control.
pressed according to the application’s requirement and available
I. INTRODUCTION network resource; e.g., 10 frames per second (fps) playback rate
and 500-ms maximal accumulated delay. In this paper, our at-

T HE MAIN challenge in designing a multimedia applica-

tion over communication networks is how to deliver mul-
timedia streams to users with minimal replay jitters. In gen-
tention focuses on the issue of SRC that arises from efficient
management of network bandwidth with sufficient video quality
to support current multimedia applications.
eral, a network-based multimedia system can be conceptually In the development of an SRC scheme, we need to con-
viewed as a layer-structure system, which consists of applica- sider a common feature of employing an inter-frame coding
tion layer on the top, compression layer, transport layer, and between two consecutive video frames in several widely used
transmission layer, as shown in Fig. 1. To diminish the impact video compression schemes such as MPEG-1, MPEG-2, and
on the video quality due to the delay jitter and available net- H.263. Although the inter-frame coding scheme exploits the
work resources (e.g., bandwidth and buffers), traffic shaping similarity usually found in encoding two consecutive video
and scalable rate control (SRC) are qualified candidates at two frames, and achieves significantly coding efficiency, the output
different system levels. Traffic shaping is a transport-layer ap- with variable-length video bitstream is not well suited for a
proach, while SRC is a compression-layer approach, which is fixed-rate communication channel. To better utilize network
the focus of this paper. The basic concept behind the traffic- resources and transmit coded video bitstream as accurately
shaping approach is that before the encoded video bitstreams as possible, the network parameters and encoding parameters
are injected into the network for transmission, the traffic pattern should be jointly considered, and their relationship should
is already shaped with the desired characteristics, such as max- be modeled accurately. Technically speaking, rate control is
imal delay bounds and peak instantaneous rate [3]–[6]. There- a decision-making process where the desired encoding rate
fore, all the system components along the network path from for a source video can be met accurately by properly setting
the sender to the receiver can be configured to meet the quality a sequence of quantization parameters (QP). To cope with
of service (QoS) as desired by allocating the appropriate re- various requirements of different coding environments and
sources a priori. On the other hand, SRC approach is a compres- applications, a rate-control scheme needs to provide sufficient
sion-layer technique where the source video sequence is com- flexibility and scalability. For example, multimedia applications
are categorized into two groups, which are variable-bit-rate
Manuscript received September 15, 1998; revised February 8, 2000. This (VBR) application and constant-bit-rate (CBR) application.
paper was recommended by Associate Editor C. W. Chen. For VBR applications, rate control attempts to achieve the
H.-J. Lee is with Multimedia Technology Laboratory, Sarnoff Corporation, optimum quality for a given target rate. In CBR and real-time
Princeton, NJ 08543 USA (e-mail: [email protected]).
T. Chiang is with National Chiao Tung University, Taipei, Taiwan, R.O.C. applications, a rate-control scheme must satisfy low-latency
(e-mail: [email protected]). and buffer constraints. In addition, the rate-control scheme has
Y.-Q. Zhang was with Multimedia Technology Laboratory, Sarnoff Corpora- to be applicable to a variety of sequences and bit rates. Thus,
tion, Princeton, NJ 08543 USA. He is now with Microsoft Research, Beijing
100080, China (e-mail: [email protected]). a rate-control scheme must be scalable for various bit rates
Publisher Item Identifier S 1051-8215(00)07555-8. (e.g., 10 kbits/s to 1 Mbits/s), various spatial resolutions (e.g.,
1051–8215/00$10.00 © 2000 IEEE
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 879

QCIF to CCIR-601), various temporal resolutions (e.g., 7.5–30 our scheme, not only more accurate bit-rate control with buffer
fps), various coders (e.g., DCT and wavelet), and various regulation is achieved, but scalability is also preserved for all
granularities of video object (VO) (e.g., single VO to multiple test video sequences in various applications. For example, in the
VOs, frame-layer to macroblock (MB)-layer). object-based video coding of the emerging ISO MPEG-4 inter-
In developing a rate-control technique, there are two widely national standard, it is very important to appropriately allocate
used approaches: 1) an analytical model-based approach and bits among different VOs. In allocating bits among VOs, video
2) an operational rate-distortion (R-D) based approach. In the contents and coding complexity must be considered. Otherwise,
model-based approach, various distribution and characteristics over- and under-runing of the bit budget can occur. In general,
of signal source models with associated quantizers are consid- a rate-control scheme should spend more bits in the VO of user
ered. Based on the selected model, a closed-form solution is de- interests (e.g., foreground VO) than in other areas (e.g., back-
rived using optimization theory. Such a theoretical optimization ground VO). Without employing proper bit allocation, for ex-
solution cannot be implemented easily because there is only a fi- ample, the background VO could have excellent quality, while
nite discrete set of quantizers and the source signal model varies the foreground VO could suffer from lots of annoying distortion,
spatially. Alternatively, an operational R-D based approach is even though bits are evenly distributed. By considering video
used in practical coding environment. For example, to minimize contents and coding complexity in our quadratic R-D model,
the overall coding distortion subject to a total bit budget con- our rate-control scheme with joint buffer control can dynami-
straint, lots of techniques based on dynamic programming or cally and appropriately allocate the bits among VOs to meet the
Lagragian multiplier for optimization solutions have been de- overall bit-rate requirement with uniform video quality.
veloped [7]–[12]. These methods share the similar concept of Another unique merit of MPEG-4 VO-based coding from
data pre-analysis. By analyzing the R-D characteristics of fu- the other video coding standards like MPEG-1, MPEG-2,
ture frames, the bit-allocation strategy is determined afterwards. and H.263 is that an encoder can separately encode any
The Lagragian multiplier is a well-known technique for optimal VO from the rest and transmit these individual elementary
bit allocation in image and video coding, but with an assump- bitstreams independently. In this category of applications, our
tion that the source consists of statistically independent com- proposed joint-buffer rate-control scheme seems imposing
ponents. Thus, an inter-frame based coding may not find the fairly restrictive conditions that each VO shares the same buffer
Lagragian multiplier approach applicable because of the tem- management. As a matter of fact, the proposed rate-control
poral dependency. Although Ramchandran [9] takes frame de- scheme can quite easily handle this scenario. A simple way is
pendencies into account in bit-rate control, its potentially high to consider each individual VO as a “frame” so that each VO
complexity with increasing the operating R-D points make it can operate on its own (i.e., separate buffer control and separate
unsuitable for the applications requiring interactivity or low en- R-D model and so on). In this case, the frame-level rate control
coding delay. In [13], [14], Ding investigated a joint encoder and can be directly applied with minor modifications (e.g., initial
channel rate-control scheme for VBR video over ATM networks setting of the buffer fullness). Note that since there is no joint
and claimed that the rate-control scheme has to balance both is- buffer control in this case, before an encoder starts encoding
sues of consistent video quality in the encoder side and bitstream each VO, the application needs to specify the buffer condition
smoothness for statistical multiplexing gain in the network side. for each VO.
Tao et al. [15] proposed a parametric R-D model for MPEG en- With its precision of the R-D model and ease of implemen-
coders, especially for the picture-level rate control. Based on tation, our rate-control scheme with the following new con-
the bit-rate mquant model, the desired mquant is calculated and cepts and techniques has been adopted as part of the rate-con-
used for encoding every MB by combining with appropriate trol scheme in the International Standard of the emerging ISO
quantization matrix entry in a picture. A normalized parametric MPEG-4 standard:
R-D model based approach [16] has been also developed for
H.263-compatible video codecs. By providing good approxima- 1) a more accurate second-order R-D model for the target
tions of all 31 rate-distortion relations, the authors claim that the bit-rate estimation;
proposed model offers an efficient and less memory requirement 2) a sliding-window method for smoothing the impact of
approach to approximate the rate and distortion characteristics scene change;
for all QPs. Recently, Vetro and Sun [17], [18] and Ribas-Cor- 3) an adaptive selection criterion of data points for better
bera and Lei [19], [20] also proposed rate-control techniques for model updating process;
MPEG-4 object-level and MB-level video coding, respectively. 4) an adaptive threshold shape control for better use of bit
However, most of the aforementioned techniques only focus on budget;
a single coding environment, either frame level, object level, or 5) a dynamically bit-rate allocation among VOs with dif-
macro level. None of these techniques demonstrates its applica- ferent coding complexities.
bility to MPEG-4 video coding including the above three coding The proposed rate-control scheme provides a scalable solution,
granularities simultaneously. meaning that our rate-control technique offers a general frame-
In this paper, based on a newly revised quadratic R-D model, work for multiple layers of controls for objects, frames, and
our SRC proposes a single framework which is designed to MBs in various coding contexts.
meet both VBR without delay constraints and CBR with low- The rest of the paper is organized as follows. Section II re-
latency and buffer constraints. With this scalable framework views the theoretical foundation of the proposed rate control,
based on a new R-D model and several new concepts [21] in and characterizes new features for generalizing and improving
880 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

Fig. 2. Block diagram of the proposed SRC.

the R-D models. In Section III, detailed descriptions of our gen- A. Scalable Quadratic Rate Distortion Model
eralized rate control are presented. Some fundamental research To illustrate the rationale of quadratic R-D function mod-
problems for rate-control scheme in various coding granular- eling, we summarize the result derived in [22], [23]. Assuming
ities are also addressed. In Section IV, extensive experiments that the source statistics are Laplacian distributed
are conducted to evaluate the performance of the scheme. This
paper concludes with Section V.
where
II. FRAMEWORK OF SRC
the distortion measure is defined as , then
In this section, we describe the framework of the proposed
SRC scheme which provides an integrated approach with three there is a closed-form solution for the R-D functions as derived
in [24]
different coding granularities, including frame level, object
level, and MB level. The theoretical foundation behind the
proposed rate-control scheme is based on the R-D model, where
the distortion is measured in terms of quantization parameter.
The block diagram of the proposed SRC is depicted in Fig. 2, where
where the proposed rate control consists of four stages: 1)
initialization stage; 2) pre-encoding stage; 3) encoding stage;
and 4) post-encoding stage.
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 881

The R-D function is expanded into a Taylor series where

total bit budget;
bits budget used for the first I frame;
number of P frames;
the bit budget used for all P frames, and
bit budget used for nontexture information.
Based on the above observation, we present a new model to Then the and can be obtained based on the technique [23].
evaluate the target bit rate before performing the actual encoding Let and , where
process. The new model is formulated in the equation as follows: , and is the number of selected data samples.
Then

Although the above R-D model provides the theoretical foun-

dation of the rate-control scheme, the major drawback is its lack Based on these two model parameters and , the quanti-
of considering the following two factors. zation level and target bit rate can be computed before
1) The R-D model is not scalable with video contents. The encoding the next frame.
model was developed based on the assumption that each
video frame has similar coding complexity, resulting in B. Initialization Stage
similar video quality for each frame by properly target
bit-rate estimation. By introducing an index for video In the initialization stage, the major tasks the encoder has to
coding complexity such as mean absolute difference complete with respect to the rate control include:
(MAD), the R-D model becomes scalable. 1) initializing the buffer size based on latency requirement;
2) The R-D model does not exclude the bit counts used 2) subtracting the bit counts of the first I-frame from total bit
for coding the overhead including video/frame syntax, counts;
motion vectors and shape information. Although the bit 3) initializing the buffer fullness in the middle level.
count used for motion vectors, for example, is video Without loss of generality, we assume that the video sequence
content dependent, its variation in bit count is relatively is encoded first as an I frame, and subsequently P frames. In this
smaller, compared to the texture information. stage, the encoder encodes the first I-frame using an initial QP
To enhance our R-D model with more accuracy, a simple pre- value specified as an input parameter. Then the remaining avail-
diction is used to predict those bit counts using the last coded able bits for encoding the subsequent P frames can be calculated
frame as a reference. These bits used for nontexture information as
are considered as constant numbers irrespective of its distortion
and excluded from the target bit-rate estimation. To accurately (1)
estimate the target bit rate with scalability, the original quadratic
R-D formula is modified by introducing two new parameters: where
MAD and nontexture overhead ( ). remaining available bits for encoding subsequent
P frames at the coding time instant (e.g., initially
);
duration of the video sequence in the unit of seconds
where (e.g, 10 s);
total number of bits used for encoding the current itrate for the sequence in the unit of bits per second
frame ; (e.g., 10 kbits/s);
denotes the bits used for header, motion vectors, and number of bits used for the first I frame.
shape information, Thus, the channel output rate is , where is the number
MAD, computed using motion-compensated of P frames in the sequence or in a GOP. The buffer size is set
residual for the luminance component (i.e., based on the latency requirement specified by the user. The de-
component); fault buffer size is (i.e., the maximum accumulated delay
quantization level used for the current frame ; is 500 ms.). The target buffer fullness is set at the middle level
, first- and the second-order coefficients. of the buffer (i.e., ) is to have more rooms to effectively
To solve the target bit rate, we assume the video is encoded offset the potential buffer overflow or underflow caused by sud-
first as an I frame, and subsequently P frames. The scheme has denly increasing or decreasing bit count in a segment of video
been extended to variable GOP structure and B frames as well sequence. It is clear that higher target buffer level may cause the
buffer overflow, while lower target buffer level may cause the
buffer underflow. Although stuffing useless bits is a straight-
forward way to handle it, the waste of network resource, e.g.,
bandwidth and network buffer is its major disadvantage.
882 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

C. Pre-Encoding Stage 2) activating the MB-layer rate control if desired.

In the pre-encoding stage, the tasks of the rate-control scheme In the encoding stage, if either the frame- or object-level rate
include: 1) target bits estimation,; 2) further adjustment of the control is activated, the encoder compresses each video frame
target bits based on the buffer status for each VO; and 3) quanti- or VO using QP as computed in the pre-encoding stage. How-
zation parameter calculation. The target bit count is estimated in ever, some low-delay applications may require strict buffer reg-
the following phases including: 1) frame-level bit rate; 2) object ulations, less accumulated delay, and better spatially percep-
level if desired; and 3) MB-level bit-rate estimation if desired. tual quality. An MB-level rate control is necessary. However,
At the frame level, the target bit count for a P frame at time an MB-level rate control is costly at low rate since there is
, , is estimated as additional overhead if quantization parameter is changed fre-
quently within a frame. For instance, in the MPEG-4, the MB
type requires one to three more bits to indicate the existence of
the differential quantization parameter (i.e., dquant). Further-
where more, two bits need to be sent for dquant. In the worse case, for
remaining bit counts at time ; the same prediction mode, additional 5 bits need to be trans-
emaining number of P frames at time ; mitted in order to change QP. In the case of encoding at 10
actual bits used for the P frame at time (i.e., the pre- kbits/s, 7.5 fps, QCIF resolution, the overhead can be as high
vious P frame). as kbits/s. If only 33 MBs are encoded, the
Note that is the weighting factor to determine impact from the overhead kbits/s. Thus, there will be about
previous frame on the target bit estimation of the current frame. 10% loss in compression efficiency at low bit-rate encoding. At
It can be determined adaptively or set as a constant number. The high bit rate, the overhead bit count is less significant than the
default value of is 0.05 in our experiments. residual bit count. The details of the MB-level rate control will
To get a better target bit-rate estimation, we need to consider be presented in the next section.
buffer fullness. Hence, the target bit-rate estimation can be fur-
E. Post-Encoding Stage
ther adjusted with the following equation:
In the post-encoding stages, the encoder needs to complete
(2) the following tasks: 1) updating the correspondent quadratic
R-D model for the entire frame or an individual VO; 2) per-
where is the current buffer fullness at time and is the forming the shape-threshold control to balance the bit usage
buffer size. The adjustment in the above equation aims to keep between shape information and texture information; and 3)
the buffer fullness in the middle level to reduce any chance of performing the frame-skipping control to prevent the potential
buffer overflow or underflow. To achieve constant video quality buffer overflow and/or underflow.
for each video frame or VO, the encoder must allocate a min- 1) R-D Model Update: After the encoding stage, the en-
imum number of bits, which is denoted as , where and coder has to update each VO’s respective R-D model based on
are application’s bit rate and frame rate of the source video, the following formula, for example, for VO :
respectively. That is

(3) (4)

Then, the final adjustment is made to predict the impact of

where is the actual bit count used for object , and is the
on the future buffer fullness. A safety margin, denoted as
overhead bit count used for syntax, motion and shape coding.
, of the buffer, which is pre-set before encoding, is enforced
Note that in the case of MB rate control, the quantization pa-
to avoid the potential buffer overflow and buffer underflow. To
rameter is defined as the average of all encoded MBs. To make
avoid buffer overflow, if , then the
our R-D model more accurate to reflect the video contents, the
target bit rate is decreased and becomes
R-D model update process is consisted of the following three
steps. The motivation and technical details of these three steps
will be described in the next section.
To avoid buffer underflow, on the other hand, if Step 1—Selection of Data Points: Those data points selected
, then the target bit rate is increased and becomes by the encoder are used to update the R-D model. The quality
and quantity of the data set are critical to accuracy of the model.
With respect to the quantity of the data set, generally speaking,
more data points are likely to yield a more accurate model at the
where is the channel output rate.
expense of higher complexity. With respect to the quality of the
data set, some objective indices, such as MAD or SAD, of the
D. Encoding Stage
current data point can be used. By considering these factors, a
In the encoding stage, the major tasks that the encoder has to sliding-window based data selection mechanism is developed,
complete include: and its details will be presented in the next section. Although
1) encoding the video frame (object) and recording all actual inserting an I frame is an more effective way to handle scene
bit rate; change, its higher complexity, e.g., resuming the buffer control
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 883

Fig. 3. Distribution of the bit-count difference.

Fig. 4. Scenario of frame-skipping control.

to prevent buffer overflow, is its major drawback. Note that since linear regression technique, the two model parameters and
our rate control is a GOP basis, the insertion of I-frame can can be obtained as
be viewed as the resume of another GOP round, thus our rate
control still applicable.
Step 2—Calculation of the Model Parameters and :
Based on these two values, the theoretical target bit rate can
be calculated for each data point within the sliding window ob- (5)
tained in step 1. For those selected data points, the encoder col-
lects quantization levels and actual bit-rate statistics. Using a
884 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

3) predictive frame-skipping control.

In object-based rate control, we would like to address the fol-
(6) lowing research problem: 1) calculating the target bit rate among
the VOs; 2) balancing the bit budget between the shape infor-
where is the number of selected past frames, and are mation and the texture information without introducing notice-
the actual average quantization levels, and actual bit counts in able distortion; and 3) encoding the VOs with proper temporal
the past, respectively. resolution so that the quality of the composite video frame is
Step 3—Removal of the Outliers from the Data Set: After the sufficient.
new model parameters and are derived, the encoder per- To solve these problems, we propose the following solutions
forms further refinement step to remove some bad data points. based on our experimental results and some theoretical implica-
Then the actual bit rate for each past frame is known and the tions.
target bit rate is recalculated based on the new model parame- 1) dynamic target bit-rate distribution among VOs;
ters obtained in Step 2, the actual bit rate , and the average 2) adaptive shape-threshold control
quantization level . By applying the refinement process, more
representative data points are selected, and the final model pa- A. Sliding-Window Data-Point Selection
rameters can be derived based on these new data points. The The sliding-window mechanism is used to adaptively smooth
detail of this process will be presented in the next section. the impact of a scene change for certain number of frames in
2) Shape-Threshold Control: Next, we discuss the issue of updating the R-D model. If the complexity changes significantly
shape-threshold control. The issue arises from the fact that only (i.e., high motion scenes), a smaller window with more recent
a limited bit budget is available for object-based video coding. data points is used. By using such a mechanism, the encoder
In the object-based video coding, the development of an effi- can intelligently select those representative data points for R-D
cient and effective way to allocate limited bit budget used for model updates. The selection of data points is based on the ratio
coding shape and texture information becomes a very important of “scene change” between the current frame (object) and the
research problem. In MPEG-4, there are two ways to control previous encoded frame (object). Note that the sliding-window
the bit count used for the shape information: 1) size conver- mechanism with data-point selection is to diminish the effect
sion process and 2) shape-threshold setting. The size conver- of partially deals with scene change, not completely solve the
sion process adopted by MPEG-4 is used to reduce the amount abrupt scene change.
of shape information for rate control. The size conversion can To quantify the amount of scene change, various indices,
be carried out on an MB basis [25], [26]. The shape-threshold such as MAD or SAD, or their combinations can be used.
setting, on the other hand, is a controllable parameter, and is car- A sophisticated weighting factor can be considered, too. To
ried out on an object basis. The threshold value could be either a make our rate-control scheme easy to implement with lower
constant or a variable. In this paper, an adaptive shape-threshold complexity, only MAD is used as an index to quantify the
control is developed with its details presented in the next section. amount of scene change. If a segment of video contains a trend
3) Frame-Skipping Control: The objective of the of higher motion scene (i.e., increasing coding complexity),
frame-skipping control is to prevent buffer overflow. Once then a smaller number of data points with recent data are
the encoder predicts that encoding the next frame would selected. On the other hand, a larger number of data points with
cause buffer overflow, the encoder skips the encoding of the historic data are selected for a lower motion scene of video
next frame. The buffer occupancy will be decreased by the contents. Algorithmically, if MAD MAD , where is
channel transmission rate at the expense of lower frame rate. the time instant of coding, the size of the sliding window is
Although the frame skipping is an effective way to prevent MAD MAD MAX SLIDING WINDOW , otherwise
buffer overflow, the overall perceptual quality will be reduced it is MAD MAD MAX SLIDING WINDOW , where
significantly, especially in the case of continuous frame skip- MAX_SLIDING_WINDOW is a preset constant (e.g., 20 in
ping. To fight the problem with continuous frame skipping, a our experiments).
frame-skipping mechanism is proposed and will be presented
in details in the next section. B. Statistical Removal of Data Outliers
The R-D model is further enhanced and calibrated by em-
III. NEW FEATURES OF THE PROPOSED SRC
ploying a statistical method of rejecting some erroneous data
In this section, we introduce several new concepts with its points. Those erroneous data points are defined, in the statis-
associated technical merits in our rate-control scheme and elab- tical sense, as the data points whose prediction errors between
orate those techniques leading to a more accurate and scalable the actual bit rate and the target bit rate is larger than standard
target bit-rate estimation. deviations (e.g., as a rule of thumb, is set to one in our exper-
In our frame-level rate control, to make the quadratic R-D iments). In updating the R-D model after encoding a frame, the
model more accurate and scalable for different coding require- removal of the outliers from the data set can diminish the impact
ments, the following new concepts and mechanisms are intro- of bad data points on the model update process. In other words,
duced: after new model parameters and are derived, the encoder
1) sliding-window data-point selection; performs this refinement step to remove some erroneous data
2) statistical removal of data outliers; points. Since the actual bit rate for each past frame is known,
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 885

Fig. 5. Scenario of shape-threshold settings.

Fig. 6. Block diagram of the proposed MB-level rate control.

then the target bit rate is recalculated based on the new model count difference is depicted in Fig. 3, where the -axis denotes
parameters. By applying this statistical technique, more repre- the bit-count difference of a coded P frame from the average
sentative data are selected, and the final model parameters can coding bit count in a sequence or GOP, while the -axis denotes
be derived based on new data. Note that, to avoid the removal the number of P frames. In this example, the average bit count
of all data points, the latest data point is always selected in the for a P-frame is calculated as (24 kbits sec kbit)/100
data set. frames kbits, given 1 kbit is used for coding the first
I frame. If the target buffer level is in the middle level (i.e.,
C. Predictive Frame-Skipping Control 12 kbits), then the encoder has a upper or lower half of 6/2.39
Ideally, if the input bit rate to the buffer is equal to the channel frames for buffering any sudden bit count increasing, or de-
output rate, then the buffer fullness will keep its middle level, creasing, respectively. Considering the buffer-overflow case, if
which is the target buffer level. Otherwise, it changes its full- a threshold value is properly set, then there is only a slight
ness upwards or downwards which may potentially cause buffer chance of overflowing the buffer. In this case, if the threshold
overflow or underflow, respectively. To effectively prevent the is set as 80% of the buffer size (i.e., 4.8 kbits), the consecutive
buffer overflow without losing overall video quality due to con- frame skipping occurs very rarely from the statstical viewpoint.
tinuous frame skipping, we proposed a predictive frame-skip- This empirical assumption by setting a proper threshold demon-
ping control. By setting a safety threshold, if an encoder predicts strates although our predictive frame-skipping scheme cannot
to have a potential buffer overflow, then it skips next frame (ob- guarantee no consecutive frame skipping occurring; however,
ject) and subtracts the buffer fullness from the channel output there exist fairly high confidence that consecutive frame skip-
rate. The safety threshold is selected based on the distribution ping rarely occurs.
of bit-count difference of a predicted frame (object) from the The proposed frame-skipping mechanism is described as fol-
average bit count of coded frame (object) in a video sequence lows. Before encoding the next frame, the encoder first exam-
or GOP. For example, a containership video sequence with 300 ines the current buffer occupancy and the estimated target bit
qcif frames is coded as the first I-frame and subsequent P-frame rate for the next frame. If the current buffer fullness plus the es-
coding pattern at 10 fps and 24 kbits/s. Its distribution of the bit- timated frame bits for the next frame is larger than some pre-de-
886 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

(a) (b)

(c) (d)

(e) (f)

Fig. 7. Frame-level rate control: the buffer occupancy for various test conditions. SVORC: (a) silent qci f at 10 fps and 24 kbits/s; (b) hall qci f at 7.5 fps and
10 kbits/s; (c) mad qci f at 10 fps and 24 kbits/s; (d) madl qci f at 7.5 fps and 10 kbits/s; (e) news cif at 15 fps and 112 kbits/s. (f) news cif at 7.5 fps and
48 kbits/s.
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 887

TABLE I
VBR EXPERIMENTAL RESULTS OF THE FRAME-LEVEL RATE CONTROL

TABLE II
CBR EXPERIMENTAL RESULTS OF THE FRAME-LEVEL RATE CONTROL

termined threshold, called the safety margin, for instance 80% However, this simple scheme suffers several serious problems.
of the buffer size, the next frame will be skipped. Note that the For example, the background VO may have bits left unused,
safety margin is used to reduce continuous frame skipping and while the foreground VO requires more. Therefore, we propose
can be adaptively changed based on the content, or a pre-deter- a new bit-allocation method to adaptively adjust the bit budget
mined constant for simplicity. An example, as shown in Fig. 4, for each VO. Based on the coding complexity and perceptual
demonstrates our frame-skipping control. Before encoding the importance, the distribution of the bit budget is proportional to
frame , the encoder uses the actual bit count used for the the square of MAD of a VO, which is obtained empirically [27].
frame to predict the next frame . Then the buffer As long as the target bit rate is known at the frame level, say ,
occupancy will exceed 80% of the buffer size, resulting in skip- the target bit rate for VO at time , , is
ping the frame and decreasing the buffer occupancy by the
channel output rate. The frame-skipping condition can be for- (7)
mulated as follows.
where MAD MAD MAD MAD and
WHILE (buffer fullness + are the number of VOs in the coding frame at time .
actual bitcounts for last frame 0 channel output rate  In addition to the distribution of bits among VOs in the
buffer size3 skip_margin) f spatial domain, we also need consider the composition of
Skip the next frame; VOs in the temporal domain. Since each VO has different
buffer fullness = buffer fullness 0 channel output rate coding complexity (e.g., low or high motion), to obtain better
coding efficiency, it is fairly straightforward to encode the
VOs at different frame rates, then the decoder can display
D. Dynamic Target Bit-Rate Distribution Among VOs the composite video. However, our experimental results show
To estimate the target bit rate for each VO, a straightfor- that the significant quality deterioration is experienced in the
ward way is to allocate an equal number of bits to each VO “gluing” boundary of VOs. Thus, encoding the VOs at the same
without considering its complexity and perceptual importance. frame rate is a better alternative to yield better video quality.
888 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

TABLE III
EXPERIMENTAL RESULTS OF MULTIPLE VO’S RATE CONTROL

E. Adaptive Shape-Threshold Control IF ( ) THEN

To avoid using excessive bits for motion and shape infor- )
mation instead of texture and to balance the bit usage without ELSE
introducing noticeable distortion, the threshold values can be
set adaptively based on the previous coding information. The
proposed adaptive threshold shape control is described as fol-
F. MB-Level Rate Control
lows.
For a VO, if the actual bit count used for nontexture in- The block diagram of the proposed MB rate control [28] is
formation (e.g., shape information) exceeds its estimated bit depicted in Fig. 6. Once the MB-level rate control is activated
budget, the encoder will increase the threshold value to re- to provide more strict buffer regulations and higher bit-rate en-
duce the bit count for nontexture coding at the expense of coding, the following steps are taken. Based on the target bit
shape accuracy of a VO. On the other hand, if the above con- rate and the quantization level calculated in frame (object) level,
dition does not hold, then the encoder decreases the threshold the encoder starts to perform the encoding for every MB. The
value to get a better video shape accuracy. A scenario of model is now updated on an MB basis, instead of frame (object)
the threshold setting by the proposed adaptive shape control basis. Since our rate-control scheme is scalable with the MAD,
mechanism is depicted in Fig. 5. Initially, the threshold, , the flow of the MB rate control is the same as the frame-level
for a VO is set to zero, then it is increased by if the rate control with a smaller update step: 1) target bit-rate calcu-
target bit rate is larger than lation; 2) quantization parameter calculation; and 3) update the
bits used for object in the previous frame), other- R-D model. Assuming that the following information are avail-
wise it is decreased by . To maintain the accuracy of able before encoding the first MB of a frame (object): weighting
the video shape to a certain degree, (i.e, to avoid a negative factor for MB and MAD for MB . The specific tech-
threshold or a large ), the is bounded between 0 and nique in calculating the weighting factor can be selected at the
. The shape-threshold control mechanism is described as designer’s choice [29]. Thus, the MB-level rate control consists
follows: of the following steps:
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 889

1) Target Bit-Rate Calculation for an MB : Given for IF ( MB is skipped) THEN

the entire frame or for VO , then

or (8) END
;

IV. EXPERIMENTAL RESULTS

where is the number of MBs in a frame (object).
To evaluate the performance of the proposed rate-control
2) Quantization Parameter Calculation: Solve QP using
scheme, extensive experiments for three different coding
and . Note that is not considered in the modeling.
granularities with various test conditions are conducted. The
3) Update R-D Model: In the first step, the formula can
performance results of the proposed SRC scheme for the frame
be derived from the case of equal weighting among MBs.
level, object level, and MB level are reported in the following
Assuming that weighting is 1 for all MBs, then the problem is
sections.
formulated as follows:
A. Frame-Level Rate Control
For the frame-level rate control, the experiments are
or conducted in two cases of VBR without any buffer restric-
tion and contant bit-rate CBR with the limited buffer size
, i.e., the maximum accumulated latency
Since all MBs are equally important, every MB is quantized is set to be 500 ms, as required by MPEG-4. Tables I and II
with the same QP. Thus demonstrate that in VBR and CBR both cases, our frame-level
rate control achieves its accuracy in target frame rate and
target bit rate. In addition, in CBR case, on average, less than
1% variation of target bit rate and around 3% variation of the
implies
target frame rate are experienced. Fig. 7 illustrates that during
the encoding process, the buffer occupancy is around 50% of
the buffer size with a small variation, and it also shows that
Solving with a total target rate of , we have with our SRC algorithm, buffer overflow and/or underflow is
improbable. The results demonstrate that our SRC scheme can
achieve very accurate target bit-rate allocation, and satisfy the
buffer with low latency (i.e., 500 ms) for all test sequences.
In addition, there are only a few video frames which are skipped
in order to meet the buffer constraints, implying that the target
The complete MB rate-control scheme is presented as follows: frame rate is also obtained.

/ starting encoding a frame (VO) / B. Object-Level Rate Control

calculate and , where is the For multiple VO rate control, the performance results are re-
target bit count for this frame and ported in Table III. The top three rows report the experiment
is a threshold for eliminating in- results obtained in lower bit-rate coding context (e.g., 24–48
significant MBs with low MADs. kbits/s), comparing to the results reported in the bottom three
IF ( ) THEN rows with higher bit-rate coding context (e.g., 64–192 kbits/s.)
By examining the results of the number of encoded frames and
compute the weighted sum of all existing the target bit-rate allocation, only small variations with target
MAD as frame rate and target bit rate are experienced. As aforemen-
. tioned, the allocation of target bit rate among VOs is an impor-
/ start encoding each MB / tant research issue. As reported in the last column of Table III,
FOR ( ) DO with our target bit-rate allocation based on the MAD , our rate-
quantization parameter calculation control scheme demonstrates its capability of automatically and
IF ( ) THEN appropriately allocating the target bit rate among VOs. For ex-
ample, in coding the video sequence at 10 fps and 25
ELSE kbits/s, most of the frame (object)-level bit budget (i.e., 68%
of the texture bit budget) is used to encode the VO with the
encode the MB higher coding complexity (e.g., foreground VO), while only
update the R-D model 32% of the bit budget is used to encode the VO with lesser
coding complexity (e.g., background VO). With its appropriate
target bit distribution, each VO has a very similar visual quality
for both low bit rate and high bit-rate coding contexts, even
890 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

(a) (b)

(c) (d)

(e) (f)
Fig. 8. Object-level rate control: the buffer occupancy for various test conditions. MVORC: (a) akiyoq ci f at 10 fps and 24 kbits/s; (b) akiyo qci f at 15 fps and
64 kbits/s; (c) coastguardq ci f at 10 fps and 48 kbits/s; (d) news ci f at 15 fps and 192 kbits/s. (e) containerq ci f at 10 fps and 24 kbits/s. (f) container qci f
at 15 fps and 64 kbits/s.
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 891

(a) (b)

(c) (d)

(e) (f)

Fig. 9. MB-level rate control: the buffer occupancy for various test conditions under the maximum accumulated delays are 500 ms (left column) and 250 ms
(right column). (a), (b): MBRC coastguard qci f at 10 fps and 24 kbits/s. (c), (d): MBRC container qci f at 10 fps and 24 kbits/s. (e), (f): mad qci f at 10 fps
and 24 kbits/s.
892 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

(g) (h)

(i) (j)

Fig. 9. (Continued). MB-level rate control: the buffer occupancy for various test conditions under the maximum accumulated delays are 500 ms (left column)
and 250 ms (right column). (g), (h): news ci f at 7.5 fps and 192 kbits/s. (i), (j): silent qci f at 10 fps and 24 kbits/s. (i), (j): silent qci f at 10 fps and 24 kbits/s.

TABLE IV
EXPERIMENTAL RESULTS OF MB-LAYER RATE CONTROL GIVEN THE MAXIMUM ACCUMULATED DELAYS ARE 500 AND 125 MS
LEE et al.: SCALABLE RATE CONTROL FOR MPEG-4 VIDEO 893

though there exist fairly large dB differences up to 5, 3, 7, 3, The proposed SRC has shown the following advantages: 1) low
12, and 11 dB between objects in the respective tests. It is ob- latency and the limited buffer constrains are satisfied for CBR
vious that the background object has pretty much static with applications; 2) the VBR quality is maintained; 3) both the target
very little difference from its previous frame. In this case, the bit rate and the target frame rate are obtained within a negligible
peak signal-to-noise-ratio (PSNR) value is not a sufficient in- error; and 4) easy extension to the multiple VOs and MB layer.
dicator for visual quality. Our experimental results show that The proposed SRC has been adopted in the International Stan-
equal or similar PSNR values in this case create more coding dard of the emerging MPEG4 standard.
artifacts in background objects and in a composite frame. So,
even the PSNR values are fairly different among VOs; however, REFERENCES
their composite video frame shows a better visual quality. [1] Coding of Moving Pictures and Associated Audio ISO/IEC 14 496-2,
Next the buffer occupancy is examined, and plotted in Fig. 8. MPEG-4 Committee Draft ISO/IEC JTC1/SC29/WG11, Oct. 1997.
[2] Coding of Moving Pictures and Associated Audio ISO/IEC 14 496-2,
In our experiments, a single joint buffer control is used and the MPEG-4 Committee Draft ISO/IEC JTC1/SC29/WG11, Oct. 1998.
maximum accumulated delay is set to be 500 ms for the en- [3] H. Zhang and D. Ferrari, “Rate-controlled static priority queueing,” in
coding process. As shown in Fig. 8, the buffer occupancy is Proc. IEEE INFOCOM’93, Apr. 1993, pp. 227–236.
maintained at around 50%–60% of the buffer size with a small [4] D. Ferrari and D. Verma, “A scheme for real-time channel establish-
ment in wide-area networks,” IEEE J. Select. Areas Commun., vol. 8,
variation, as expected, even for the case that there are up to six pp. 368–379, Apr. 1990.
VOs (e.g., container video sequence). It also demonstrates that [5] L. Zhang, “Virtual clock: A new traffic control algorithm for packet
with our rate-control scheme, the buffer overflow or underflow switching networks,” in Proc. ACM SIGCOMM’90, Sept. 1990, pp.
19–29.
does not occur for all test conditions of high- and low-bit rate [6] S. J. Golestani, “A stop-and-go queueing framework for congestion
coding contexts. management,” in Proc. ACM SIGCOMM’90, Sept. 1990, pp. 8–18.
[7] Y. Shoham and A. Gersho, “Efficient bit allocation for an arbitrary set
of quantizers,” IEEE Trans. Acoust., Speech, Signal Processing, vol. 36,
C. MB-Level Rate Control pp. 1445–1453, 1988.
[8] S. W. Wu and A. Gersho, “Rate-constrained optimal block-adaptive
Table IV reports the results of MB layer rate control under coding for digital tape recoding of HDTV,” IEEE Trans. Circuits Syst.
two maximum accumulated latencies which are set to be 500 Video Technol., vol. 1, Mar. 1991.
and 250 ms, respectively. Compared to the frame-level rate con- [9] K. Ramchandran, A. Ortega, and M. Vetterli, “Bit Allocation for depen-
dent quantization with applications to multiresolution and MPEG video
trol, the MB-level rate control generates more accurate target coders,” IEEE Trans. Image Processing, vol. 3, pp. 533–545, Sept. 1994.
bit estimation, resulting in the higher frame rate, and lower la- [10] A. Ortega, K. Ramchandran, and M. Vetterli, “Optimal trellis-based
tency delay at the expense of slightly quality loss. For the case of buffered compression and fast approximations,” IEEE Trans. Image
Processing, vol. 3, pp. 26–40, Jan. 1994.
the maximum accumulated delay equal to 250 ms, a larger—but [11] J. Choi and D. Park, “A stable feedback control of the buffer state using
within the tolerance—fluctuation of buffer occupancy is experi- the controlled Langrange multiplier method,” IEEE Trans. Image Pro-
enced, as expected, since a smaller buffering is used to smooth cessing, vol. 3, pp. 546–558, Sept. 1994.
[12] L. J. Lin and A. Ortega, “Bitrate control using Piecewise approximated
out the bit rate fluctuation of encoding process (see Fig. 9). rate-distortion characteristics,” IEEE Trans. Circuits Syst. Video
However, under this strict test condition, no buffer overflow oc- Technol., vol. 8, pp. 446–459, Aug. 1998.
curs, proving the robustness and stability of our MB rate-control [13] W. Ding, “Rate control of MPEG video coding and recording by rate-
quantization modeling,” IEEE Trans. Circuits Syst. Video Technol., vol.
scheme. The results show that our MB-level rate control can be 6, pp. 12–20, Feb. 1996.
applied for the various applications with higher bit rate and strict [14] , “Joint encoder and channel rate control of VBR video over ATM
buffer regulations. networks,” IEEE Trans. Circuits Syst. Video Technol., vol. 7, Aug. 1997.
[15] B. Tao, H. A. Peterson, and B. W. Dickinson, “A Rate-Quantization
model for MPEG Encoders,” in Proc. 1997 Int. Conf. Image Processing,
V. SUMMARY AND CONCLUSIONS Oct. 1997, pp. 338–341.
[16] K. H. Yang, A. Jacquin, and N. S. Jayant, “A normalized rate-distor-
In this paper, we proposed a single, integrated and SRC tion model for H.263-compatible codecs and its application to quantizer
scheme for different coding granularity including the frame selection,” in Proc. 1997 Int. Conf. Image Processing, Oct. 1997, pp.
41–44.
level, the object level, and the MB level. Our scheme not only [17] Joint rate control for multiple-VO, MPEG96/M1890 ISO/IEC
provides a framework for the very low bit-rate control, but also JTC/SC29/WG11, Apr. 1997.
comes up with several new techniques to improve the accuracy [18] A. Vetro, H. F. Sun, and Y. Wang, “MPEG-4 rate control for multiple
video objects,” IEEE Trans. Circuits Syst. Video Technol., vol. 9, pp.
of the R-D model. Those new methods or concepts are listed 186–199, Feb. 1999.
below: [19] J. Ribas-Corbera and S. M. Lei, “Contribution to rate control Q2 exper-
iment: A quantizer control tool for achieving target bitrates accurately,”
1) a more accurate R-D model which is scalable with MAD; Coding of Moving Pictures and Associated Audio MPEG96/M1812
2) a dynamically bit-rate allocation among VOs with various ISO/IEC JTC/SC29/WG11, Sevilla, Spain, Feb. 1997.
coding complexities; [20] , “Rate control in DCT video coding for low-delay communica-
tions,” IEEE Trans. Circuits Syst. Video Technol., vol. 9, pp. 172–185,
3) a sliding-window mechanism for smoothing the impact of Feb. 1999.
scene change; [21] H.-J. Lee, T. Chiang, and Y.-Q. Zhang, “Scalable rate control for very
4) an adaptive selection criterion of data points for the R-D low bitrate video,” in Proc. 1997 Int. Conf. Image Processing, vol. II,
Oct. 1997, pp. 768–771.
model update process; [22] T. Chiang, “A rate control scheme using a new rate-distortion model,”
5) an adaptive threshold setting for rate reduction in shape JCoding of Moving Pictures and Associated Audio MPEG95/0436
coding; TC1/SC29/WG11, Dallas, TX, Nov. 1995.
[23] T. Chiang and Y.-Q. Zhang, “A new rate control scheme using a new
6) an effective frame-skipping control for the prevention of rate-distortion model,” IEEE Trans. Circuits Syst. Video Technol., pp.
the potential buffer-overflow problem. 246–250, Feb. 1997.
894 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000

[24] A. Viteribi and J. Omura, “A new rate control scheme using a new Tihao Chiang (S’90–M’95–SM’99) was born
rate-distortion model,” in Principle of Digital Communication and in Cha-Yi, Taiwan, R.O.C., in 1965. He received
Coding. New York: McGraw-Hill, 1979. the B.S. degree from National Taiwan University,
[25] Coding of Moving Pictures and Associated Audio MPEG96, MPEG-4 Taipei, Taiwan, in 1987, the M.S. degree from
video verification model V5.0 ISO/IEC JTC1/SC29/WG11, Nov. 1996. Columbia University in 1991, and the Ph.D. degree
[26] Coding of Moving Pictures and Associated Audio MPEG96, MPEG-4 from Columbia University, New York, in 1995, all
video verification model V8.0 ISO/IEC JTC1/SC29/WG11, July 1997. in electrical engineering.
[27] H.-J. Lee, T. Chiang, and Y.-Q. Zhang, “Multiple-VO Rate Control,” In 1995, he joined David Sarnoff Research Center,
ISO/IEC JTC/SC29/WG11 MPEG97/M2554, July 1997. Princeton, NJ, as a Member of Technical Staff. He
[28] T. Chiang, H.-J. Lee, and Y.-Q. Zhang, “Macroblock Layer Rate Con- was later promoted to Technology Leader and then
trol,” ISO/IEC JTC/SC29/WG11 MPEG97/M2555, July 1997. Program Manager. While at Sarnoff, he led a team
[29] S. Ryoo, J. Shin, and E. Jang, “Rate control tool: Based on human visual of world-class researchers and developed an optimized MPEG-2 software en-
sensitivity (HVS) for low bitrate coding,” ISO/IEC JTC/SC29/WG11 coder. For his work in the encoder and MPEG-4 areas, he received two Sarnoff
Coding of Moving Pictures and Associated Audio MPEG96/M0566, Achievement Awards and three Sarnoff team awards. In September 1999, he
Munich, Germany, Jan. 1996. joined the faculty at National Chiao-Tung University, Taiwan, R.O.C. Since
1992, he has actively participated in ISO’s MPEG digital video-coding stan-
dardization process, with particular focus on the scalability/compatibility issue.
He is currently the co-chair for encoder optimization on the MPEG-4 com-
mittee, has made more than 40 contributions to the MPEG committee over
the past eight years, and has co-authored the rate-control technology that was
adopted as part of the MPEG-4 International Standards in 1998. He holds two
U.S. patents and more than 10 pending patents. He published over 20 technical
journal and conference papers in the field of video and signal processing. His
main research interests include compatible/scalable video compression, stereo-
scopic video coding, and motion estimation.

Ya-Qin Zhang (F’97) was born in Taiyuan, China,

in 1966. He received the B.S. and M.S. degrees from
the University of Science and Technology of China
(USTC) in 1983 and 1985, respectively, and the Ph.D.
degree from George Washington University, Wash-
ington, DC, in 1989, all in electrical engineering.
In 1999, he joined Microsoft Research, Beijing,
China, as the Assistant Managing Director. Pre-
viously, he was the Director of the Multimedia
Hung-Ju Lee (M’96) received the B.S. degree from Technology Laboratory, Sarnoff Corporation,
Tatung Institute of Technology, Taipei, Taiwan, in Princeton, NJ. During 1989–1994, he was with GTE
1987, and the M.S. and Ph.D. degrees from Texas Laboratories Inc., Waltham, MA, and Contel Technology Center, Washington,
A&M University, College Station, TX, in 1993 and DC. He has authored and co-authored over 150 refereed papers and 30
1996, respectively, all in computer science. U.S. patents in digital video, Internet, multimedia, wireless, and satellite
In 1996, he joined Sarnoff Corporation, Princeton, communications. Many of the technologies he and his team developed have
NJ, as a Member of Technical Staff. He actively become the basis for start-up ventures, commercial products, and international
participates in ISO’s MPEG digital video stan- standards. His recent focus has included MPEG2/DTV, MPEG4/VLBR, and
dardization process, focusing particularly on the Internet. He has been an active contributor to the ISO/MPEG and ITU
wavelet-based visual texture coding and scalable standardization efforts in digital video and multimedia.
rate control for MPEG-4 video. He received Sarnoff Dr. Zhang served as the Editor-In-Chief for the IEEE TRANSACTIONS ON
Technical Achievement Award in 1998 for his contributions on the development CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY from July 1997 to July 1999.
of MPEG-4 rate control. His current research interests include image and video He serves on the Editorial boards of seven other professional journals and over
coding, and network resource management for multimedia applications. a dozen conference committees.