Zexia H.

264 Hardware Encoder in VHDL

H.264 Hardware Encoder in VHDL
Notes and usage information
The H.264 hardware encoder is designed as a modular system with small, eicient, low !ower
com!onents doing well deined tas"s. The !rinci!le design aim was to ma"e an scala#le encoder
or mega!ixel images suita#le or use in camera heads and low !ower recorders.
The encoder is not designed to #e all things to all !eo!le, #ut rather designed to eiciently
im!lement a non$interlaced %ase &roile with no limit to the num#er o streams or 'ideo resolution.
(s such ew generic !arameters are !ro'ided, #ut com!onents can #e re!laced as needed to
customi)e the codec to a s!eciic a!!lication. *or exam!le, only +(VL+ encoding is !erormed,
#ut this is !erormed #y the h264ca'lc and h264header modules only. , re-uired, these can #e
re!laced #y custom modules to !erorm another orm o encoding.
A tour of the Components
( diagram o the !rinci!le com!onents is gi'en on the next !age.
Video comes in at the to!, and is usually written to .external/ 0(1 to #uer it tem!orarily. 2hen
needed it is read into the !rediction com!onents such as intra4x4.
3ut!uts rom the !rediction com!onents !our through the transorm loo!4 coretransorm, -uantise,
de-uantise, in'transorm, reconstruct. *or intra encoding, these reconstructed !ixels are re-uired
immediately to !redict the next #loc", in addition they are also written to .external/ 0(1 or use
or the next inter coded rame.
%ecause o the eed#ac", es!ecially or intra encoding, this transorm loo! is timing$critical, since
the latency is im!ortant as well as the through!ut. Transorm modules need all data in #eore they
can out!ut the irst out!ut !ixel, #ut the delay #etween the last !ixel in and the irst !ixel out is
minimal.
*or #loc"s which use D+ as well as (+ com!onents, a 2x2 D+ transorm .Hadamard transorm/ is
also !ro'ided as !art o the eed#ac" loo!. ,n order to s!eed the !rocess, the intra5x5cc module
.which encodes chroma or intra encoding/ out!uts the sums o each #loc" as a se!arate D+ data
stream which is ed into the irst dctransorm.
3ut!ut rom -uantise is ed to the #uer which delays and reorders the #loc"s or out!ut to the
ca'lc module which encodes the data. Header data is mixed in ater ca'lc, and the stream is turned
into a #yte stream #y to#ytes, which also stus 67 #ytes to !re'ent startcode emulation. The out!ut
rom to#ytes is a 8(L, and a done signal is asserted at the end.
,t is u! to higher le'el code to add (nnex % startcodes #etween rames .66 66 66 69/, or else count
and #uer the #ytes out!ut and add a header in m!4 or rt! ormat, or other ormat as re-uired.
1ulti!le streams may #e simultaneously encoded #y the H.264 encoder: these may #e o dierent
resolutions. There is no design limit to the 'ideo resolution.
The com!onents are inde!endently u!gradea#le i re-uired.
Zexia (ccess Ltd ; 2665 9 o 6 H.264 Hardware Encoder
Zexia H.264 Hardware Encoder in VHDL
Zexia (ccess Ltd ; 2665 2 o 6 H.264 Hardware Encoder
input: YCC pixel values
output: encoded NAL stream
Principal components in H.264 encoder
Note that RAM is usually implemented off-chip
0(1
&$!rediction ,$!rediction
+ore transorm
%uer
D+
transorm
D+
transorm
,n'erse Transorm
0econstruction
+(VL+
To
%ytes
Headers
+ontrol
<
Timing
=uantise
De$-uantise
Zexia H.264 Hardware Encoder in VHDL
Target hardware
The design has #een com!iled or the ollowing de'ices4
>ilinx ?!artan 7 amily $ .79@4 ?lices/
(ltera +yclone ,,, amily A .26,@B4 LEs/
,t is li"ely to com!ile successully or most other *&C( and (?,+ technologies.
This encoder is #eing #uilt into a commercial a!!lication which uses a ?!artan 7( 9466 which is
a#out 4 times the si)e o the re-uirement -uoted a#o'e.
8ote that the +yclone ,,, is rather larger than the ?!artan 7 due to the use o small or unlatched
0(1 elements in the design which the ?!artan can ma! to distri#uted 0(1 #ut the +yclone needs
to im!lement in discrete logic. ( modiication to intra4x4 and intra5x5cc com!onents to !ermit
T3&, to ha'e a two$cloc" latency rather than one would !ermit latched 0(1 in this situation.
Parameters: PP and SP
,t is necessary to include &icture &arameters .&&/ and ?tream &arameters .?&/ to s!eciy the details
o the encoder or the decoder to use. These are usually encoded as se!arate 8(L units which can
#e transmitted immediately #eore the irst 8(L unit o image stream.
?ome recommended ?tream &arameters .?&/ are4
profile_idc 01000010 ( 66)
constrained_set0_flag 0 ( 0)
constrained_set1_flag 0 ( 0)
constrained_set2_flag 0 ( 0)
constrained_set3_flag 0 ( 0)
reserved_zero_4bits 0000 ( 0)
level_idc 00101000 ( 40)
seq_parameter_set_id 1 ( 0)
log2_max_frame_num_minus4 1 ( 0)
pic_order_cnt_tpe 011 ( 2)
num_ref_frames 010 ( 1)
gaps_in_frame_num_value_allo!ed_flag 0 ( 0)
pic_!idt"_in_mbs_minus1 000010110 ( 21) ##
pic_"eig"t_in_map_units_minus1 000010010 ( 1$) ##
frame_mbs_onl_flag 1 ( 1)
direct_%x%_inference_flag 1 ( 1)
frame_cropping_flag 0 ( 0)
vui_parameters_present_flag 0 ( 0)
DD 0e!lace the !icEwidth and !icEheight with a!!ro!riate 'alues, these are in 96$!ixel$units and
thus the !arameters here encode an image o 7B2x255.
(s a 8(L, these can #e encoded as .hex #ytes/4
6$ 42 00 2% &' 0( %2 ()
Zexia (ccess Ltd ; 2665 7 o 6 H.264 Hardware Encoder
Zexia H.264 Hardware Encoder in VHDL
?ome recommended &icture &arameters .&&/ are4
pic_parameter_set_id 1 ( 0)
seq_parameter_set_id 1 ( 0)
entrop_coding_mode_flag 0 ( 0)
pic_order_present_flag 0 ( 0)
num_slice_groups_minus1 1 ( 0)
num_ref_idx_l0_active_minus1 1 ( 0)
num_ref_idx_l1_active_minus1 1 ( 0)
!eig"ted_pred_flag 0 ( 0)
!eig"ted_bipred_idc 00 ( 0)
pic_init_qp_minus26 1 ( 0)
pic_init_qs_minus26 1 ( 0)
c"roma_qp_index_offset 1 ( 0)
debloc*ing_filter_control_present_flag 0 ( 0)
constrained_intra_pred_flag 0 ( 0)
redundant_pic_cnt_present_flag 0 ( 0)
(s a 8(L these can #e encoded as .hex #ytes/4
6% +, 3% %0
?o encoding #oth using (nnex % ormat .ie, with startcode o 66 66 66 69/, gi'es4
00 00 00 01 6$ 42 00 2% &' 0( %2 ()
00 00 00 01 6% +, 3% %0
These #ytes can #e !ut at the start o the stream .and re!eated i needed #eore any ,D0 rame/.
Clocks
There are two cloc"s in use #y the modules, +LF which is nominally the !ixel cloc" rate, with a
design re-uency o 661H) or #elow, and +LF2 which is a dou#le rate cloc" and should run at
exactly twice +LF rate. +LF is used in the h264ca'lc module and also #y the #ac" end which
emits the #yte stream. +LF2 is used #y the !rediction and transorm logic which wor"s at higher
data rates than the !ixel rate. (s a result, B92 dou#le rate cloc"s are a'aila#le or each macro#loc"
.2B6 !ixels/ or the !rediction and transorm logic eed#ac" loo!.
Top level
( s"eleton to! le'el is !ro'ided, to allow a real one to #e written or your a!!lication. ( sim!le to!
le'el is !ro'ided or simulation which reads and writes iles and can dum! intermediate data and an
annotated out!ut #it stream i re-uired.
Zexia (ccess Ltd ; 2665 4 o 6 H.264 Hardware Encoder
Zexia H.264 Hardware Encoder in VHDL
RAM
(t minimum, an entire uncom!ressed reerence image must #e #uered in 0(1 to allow inter
!redictionGDH. Isually the incoming image is #uered as well as the reerence image, so two co!ies
might #e needed. , encoding multi!le streams, images rom all streams need to #e #uered.
De!ending on resolution, this might #e a lot o memory, and thus it is antici!ated that this is
im!lemented o$chi!.
GDH o course, you could use intra !rediction only and thus a'oid this o'erhead, #ut com!ression is
then usually around 9649, rather than the B649 -uoted or inter com!ressed streams. The actual
com!ression will 'ary with the contents o the !icture stream.
Prediction components
The intra !rediction currently a'aila#le .intra4x4 and intra5x5cc/ only considers a su#set o
!ossi#le modes, and uses a sim!le ?(D .sum o a#solute dierences/ com!arison. This ma"es the
intra rames a little larger than they otherwise might #e, #ut tests against reerence sotware .which
can choose a wider range o modes and other #etter com!arison com!utations/ shows only a ew
!ercent im!ro'ement.
(lso, only a sim!le inter !rediction .!$rames/ com!onent is a'aila#le at !resent. Zexia has others
a'aila#le #ut they cannot #e released at !resent under an o!en source license: they will !ro#a#ly #e
released once commercial agreements ex!ire. , you want to wor" on this, !lease dro! me an email,
since a !ool o good inter !rediction com!onents will enhance this codec and ,Jd #e !leased to hel!.
?ince the target is %ase &roile, no attem!t has #een made to encode %$rames, howe'er the same
ca'lc and transorm loo! can #e used so itJs Kust a case o modiying the ront end !rediction and
header generation.
Patents
H.264 is co'ered #y !atents, you will need a license rom 1&EC$L(. (ccording to the 1&EC$L(
we# site .htt!4LLwww.m!egla.com/, there is no royalty !aya#le on less than 966,666 units.
The author "nows o no other !atents which co'er this encoder, #ut that doesnJt mean to say there
are none .see the disclaimer #elow/.
Contact
To contact the author and maintainer o this encoder, (ndy Henson, email "264-zexia.co.u*
Copyright !S"#style license$
2ritten #y (ndy Henson
+o!yright .c/ 2665 Zexia (ccess Ltd
(ll rights reser'ed.
$$
0edistri#ution and use in source and #inary orms, with or without modiication, are !ermitted
!ro'ided that the ollowing conditions are met4
D 0edistri#utions o source code must retain the a#o'e co!yright
notice, this list o conditions and the ollowing disclaimer.
Zexia (ccess Ltd ; 2665 B o 6 H.264 Hardware Encoder
Zexia H.264 Hardware Encoder in VHDL
D 0edistri#utions in #inary orm must re!roduce the a#o'e co!yright
notice, this list o conditions and the ollowing disclaimer in the
documentation andLor other materials !ro'ided with the distri#ution.
D 8either the name o the Zexia (ccess Ltd nor the
names o its contri#utors may #e used to endorse or !romote !roducts
deri'ed rom this sotware without s!eciic !rior written !ermission.
$$
TH,? ?3*T2(0E ,? &03V,DED %M ZE>,( (++E?? LTD NN(? ,?JJ (8D (8M E>&0E?? 30
,1&L,ED 2(00(8T,E?, ,8+LID,8C, %IT 83T L,1,TED T3, THE ,1&L,ED
2(00(8T,E? 3* 1E0+H(8T(%,L,TM (8D *,T8E?? *30 ( &(0T,+IL(0 &I0&3?E
(0E D,?+L(,1ED. ,8 83 EVE8T ?H(LL ZE>,( (++E?? LTD 30 (8DM HE8?38 %E
L,(%LE *30 (8M D,0E+T, ,8D,0E+T, ,8+,DE8T(L, ?&E+,(L, E>E1&L(0M, 30
+38?E=IE8T,(L D(1(CE? .,8+LID,8C, %IT 83T L,1,TED T3, &03+I0E1E8T 3*
?I%?T,TITE C33D? 30 ?E0V,+E?: L3?? 3* I?E, D(T(, 30 &03*,T?: 30 %I?,8E??
,8TE00I&T,38/ H32EVE0 +(I?ED (8D 38 (8M THE30M 3* L,(%,L,TM, 2HETHE0
,8 +38T0(+T, ?T0,+T L,(%,L,TM, 30 T30T .,8+LID,8C 8ECL,CE8+E 30
3THE02,?E/ (0,?,8C ,8 (8M 2(M 3IT 3* THE I?E 3* TH,? ?3*T2(0E, EVE8 ,*
(DV,?ED 3* THE &3??,%,L,TM 3* ?I+H D(1(CE.
%nd
Zexia (ccess Ltd ; 2665 6 o 6 H.264 Hardware Encoder