Implementation of LZW Algorithm for Binary

Lossless Data Compression

Mohammad Raza', Suvendra Sahu'
"Department of Electronics and Communication Engineering, N.J.£. T Greater Noida
[email protected]
'[email protected]

Abstract- When high-speed media or channels are process of compression into two separate components:
used, high-speed data compression is desired. Software modeling and coding. Modeling is the process of
implementations are often not fast enough. In this paper, constructing representation. Coding entails mapping the
we present the very high speed hardware description
modeler's representation of the source into a
language (VHDL) modeling environment ofLempel-Ziv-
compressed representation. The process of converting a
Welch (LZW) algorithm for binary data compression to
ease the description, verification, simulation and text, such as a file on disk, a string in memory, or a stream
hardware realization. The VHDL model defines a main of characters, into a compact representation is called
block, which describe the LZW algorithm for binary encoding or compression. Decoding or decompression
data compression through a behavioral and structural restores the original text from its compressed
description. The LZW algorithm for binary data representation [8-9].
compression comprises of two modules compressor and
decompressor. The input of compressor is l-bit bit stream Huffman coding is constrained to represent every
read in according to the clock cycle. The output is an 8-bit event using an integral number of bits [10-11].
integer stream fed into the decompressor, which is an Updating a Huffman tree is much more time
index that represents the memory location of the bit
consuming. Like Huffman, Shannon-Fano coding
string stored in the dictionary. The output of
requires to recognize the statistical properties of the
decompressor is 1-bit bit stream. Once detecting the
particular approaches for input, output, main block and source file i.e. the probability of occurrence of a code
different modules, the VHDL descriptions are run must be found first before the compression start [12-
through a VHDL simulator, followed by the timing 13]. But in most cases, it is not possible to determine
analysis for the validation, functionality and the statistical properties of the file due to the
performance of the designated model that supports the randomness ofthe occurrence
effectiveness ofthe model for the application.
Dictionary compression (also referred to as Ziv-
Keywords- Binary Data Compression, LZW, VHDL,
Simulation Lempel compression or textual substitution) removes
data redundancy by replacing repeated input substrings
1. INTRODUCTION by references (also called indices or pointers) to earlier
copies of the identical substring [14-17]. A dictionary of
Despite the fact that computer memory costs have characters, words or phrases that are expected to occur
decreased dramatically over the past few years, data frequently is maintained and a recurring substring is
storage still remains, and will probably always remain, encoded by the index of its corresponding dictionary
an important cost factor for many large scale data base entry. Compression is achieved by choosing indices so
applications [1-3]. Compressing data in a database that on average they require less space than the phrase
system is attractive for two reasons: data storage they encode.
reduction and performance improvement. Storage
reduction is a direct and obvious benefit, while LZW compression implements a variation of LZ78
performance improves because smaller amounts of due to Welch and initializes the dictionary with the input
physical data need to be moved for any particular character set [18]. LZW compression is a loss less,
operation on the database [4-6] One of the most adaptive and dynamic dictionary compression technique
important developments in the study of data [19-20]. LZW compress text, executable code, and
compression is the modem paradigm first presented by similar data files to about one-half their original size.
Rissanen and Langdon [7]. This paradigm divides the LZW also performs well when presented with extremely

NIET Journal of Engineering & Technology, Vol. 5, 2014 67

 redundant data files, such as tabulated numbers, are filled up before all the input bits are read if the
computer source code, and acquired signals. number of memory locations allocated is not enough.
Compression ratios of5: I are common for these cases This make the dictionary "congested" and as a result
the compression cease. The number of bits used is
In this paper we present a model of LZW algorithm
directly proportional to the file size to be compressed.
for binary data using VHDL. The goal of this work is to
Unlike design in software compressor, the software is
ease hardware realization to achieve high-speed data
able to check the file size and allocate more bits
compression and to evaluate the feasibility of using
for bigger file size. Since the compressor is hardware,
VHDL for rapid design and prototyping of the same
where the compressor itself unable to check the file

The use of VHDL for modeling is especially size, the number of bits used is predefined. In this

appealing since it provides a formal description of the research, 8 bits are used to design an 8-bit compressor

system and allows the use of specific description styles thus the memory locations allocated is 28=256
to cover the different abstraction levels (architectural, memory locations.
register transfer and logic level) employed in the design 8·bH

-
in!eger
II bH Olllp lit Ilit

-
[21-23]. In the computation of method, the problem is I~
ream stream stre am

first divided into small pieces, each can be seen as a

Compressor ecompresso
submodule in VHDL Following the software

r
I
verification of each submodule the synthesis is then
activated. It performs the translations of hardware
description language code into an equivalent netlist of
digital cells. The synthesis helps integrate the design
."....,...---,:.1hf-1[
work and provides a higher feasibility to explore a far
Output
wider range of architectural alternative [24-27]. The select
Fig. 1: Structural view ofYHDL model
method provides a systematic approach for hardware
realization, facilitating the rapid prototyping of the
Since the decompressor decompress the output of
energy meter.
compression, the compression output is used as the
input for decompression. The decompress or adds a
II. MATERIALS AND METHODS
new string to the dictionary each time it reads a new
This research is to develop a model of LZW code. It translates each incoming code into a string and
algorithm for binary data compression using VHDL. sends it to the output. The input to the decompressor is
The model is capable of compressing l-bit bit stream an integer stream consists of 8 bits. The output of the
of binary data. decompression is a bit streams. The desired output of
decompression is binary data, which is same as the
The application ofthe model can be used in various original binary data.
data compression devices thus saving memory
Three control signals, clock, selector and output
space and reduce the transmission time. A structural
select control the operation. Clock signal is used to
view ofthe model is shown in Fig. 1
control reset, compression, or decompression Table 1
The input of compressor is a bit streams, shows the function and its corresponding control signal.
where binary data is read in one by one according to the
TABLE I
clock cycle. The input bit stream is compressed, or
Selector And Its Corresponding Function
encoded to another form. The output is an index that
represents the memory location of the bit string stored x y FUNCTION
in the dictionary called a codeword. The output of
0 0 RESET
compression is an integer stream, which consists of a
predefined number of bits. The number of bits used 0 1 COMPRESSION
in the compression determines the number of memory
1 0 DECOMPRESSION
locations allocated in the dictionary. The selection of
number of bits is important as the memory locations 1 1 UNUSED

68 NIET Journal of Engineering & Technology, Vol. 5, 2014

 entity Izw is
The last control signal is output select. When
port (
output select is "I", the output is enabled and the
Bit stream: in STD LOGIC:
compressor output the current value of the output, Integer_stream: out STD _ LOGIC_VECTOR (7downto 0);
whether it is in compression or decompression. This Integer_stream _dec in STD_ LOGIC_VECTOR (7downto 0);
Bit_stream_de: out STD_LOGIC;
signal is necessary because when the compressor
output_sel: out STD_LOGICE;
searches the dictionary to find whether the input clock: in STD_LOGIC;
string is in the dictionary, the searched result is either Selector: in STD_LOGIC_VECTOR (ldownto 0)
found or not found. If the string is not found, the );
end Izw
compressor input next byte, concatenates it with the
Architecture defines a body for a component entity.
previous string, and searches the dictionary again.
An architecture body specifies the behavior between
Hence, the compressor does not have an output after
inputs and outputs. The architecture name is not the same
every new cycle. The output is only enabled when a
as the component name. The architecture name of the
string is not found in the dictionary.
model is lzw archi. Architecture lzw archi is tied to
VHDL language is able to design in hierarchies. entity lzw.
Reusing components, error management and
All the signals are declared and also the type of
verification allows describing complex circuitry
variable is defined. A partial signal declaration VHDL
efficiently. In VHDL model, three packages std_
code is given below.
logic_I 164, std_logic_ arith and std_logic_ unsigned
are defined from the IEEE library. These packages variable element : integer :=0;
include definition of standard logic data types, variable counter : integer :=1;
variable ML_max : integer :=2;
arithmetic operations involving standard logic data : integer :=0;
variable ele_num
types, integer data types, integer to standard logic variable found : integer :=1;
vector conversion and vice versa. The standard logic variable ML : array256;
variable ocode : integer;
is the type declaration for bit stream and standard
variable ncode : integer;
logic vector is the type declaration for integer stream. variable ML_de : array256slv;
variable str : std_logic_vector(7 downto 0);
The VHDL design entity for the LZW algorithm variable hitent : integer:
variable endpro :integer :=0;
for binary data compression is built by constructing a
number of entities with certain behavior associated
begin
with them. The modeling styles of entities that are
being used: structural, dataflow, and behavioral. process(clock)

The model is basically consists of 4 main parts, i.e.

entity declaration, architecture declaration, signal
declaration, and clock and control setting.
begin

The entity declaration specifies the name of the if (clock'event and clock='1 ')then

entity being modeled. All the interface ports, which if(selector="OO")then--resec case
include input and output, are declared in the entity
elsif (selector="Ol ")then-compression
declaration. The entity name of the model is lzw.
elsif (selector+" 1O")then-decompression
There are 7 interface ports that are declared which
contributes a total of 22 input and output. Bit stream end if:

is the input of l-bit bit stream for compression. end if;

Integer stream consists of 8-bit for compression

The program starts its execution with process
output. Integer_stream _ de consists of 8-bit for "clock" high. The control signal "selector" has three
decompression input and Bit_stream_de is the output functions run at anyone time. These unctions are reset,
of I-bit bit stream. The following VHDL code is the compression and decompression. The VHDL code
entity lzw declaration. formation of clock and control setting is given below.

NIET Journal of Engineering & Technology, Vol. 5, 2014 69

 III. RESULTS AND DISCUSSIONS

A test bench is written to perform simulation by

adding the stimulus and defining the clock cycle to
verify the correctness of the model !> clock
JlJlJlJUU1JlJ1JUlIlJlJlJUlJL
A part of the YHDL code of stimulus is given below.
At the initialization of the simulation, the selector is set
to "00" to reset the operation. When the time run until
21 Ons, the selector selects either compression or
decompression depends on which simulation is desire

selector <="00" after Ons, "0 I" 210ns;

c1ock<=not(c1ock) after 100 ns;

Bit_stream<='I' after 210ns '0' after 4lOns, '0' after 6 IOns, 'I' after 810ns

The simulation is performed using the string of 50

binary bits. The simulation waveform of compression
block and decompression block is shown in Fig. 2 and
Fig. 3 respectively and their corresponding variables
and functions are tabulated in Table 2 and Table 3
respectively.

TABLE 2
•e Integer_stream -:::J[J,-1 _
Variable and its function in compression "to clock 1JlnJ1.rlJ"
Name Function «) output_stl ~

Bit_stream Input bit stream .:> selector

Integr _stream Output integer stream

Clock Clock signal Fig. 2: The Simulation waveform of compression block

Output_sel Output enable

Selector Function selector Input:

1,0,0,1,0,1,1,0,1,1,0,1,1,0,0,0,1,0,0,1,0,0,1,0,1,0,0,
TABLE 3 1,0, I, I ,1,0,
Variable and its function in decompression

Name Function 1,0,1,1,1,0,1,1,1,0,1,0,0,1,0,1,1

Bit_stream_de Output bit stream Output:

Integr _stream_de Input integer stream
Clock Clock signal 1,0,0,2,1,5,7,3,4,3,2,10,12,7,5,4,6,17,16,13, I
Output_sel Output enable
Selector Function selector
In Fig. 3, the clock cycle is 200 ns per cycle.
The selector is "00" from Ons to l l Ons. After liOns,
the selector is set to "10" to switch to decompression
In Fig. 2, the clock cycle is 200 ns per cycle. The mode. Hence, the first input integer is read at liOns
selector is "00" from Ons to 21 Ons. Selector "00" is and after every 8 clock cycles, or 1600 ns, the next
the reset case. After 210 ns, the selector is set to "01" integer is read The gap between subsequent inputs is 8
to switch to compression mode. Hence, the first input clock cycles due to 8-bit integer. The decompressor
bit is read at 210 ns, and after every new cycle, the read one bit in one clock cycle. Like compression, the
next bit is read. The output is enabled when output is enabled when output_sel is high, or equal to
output_sel is high, or equal to "1". For every high "I". For every high state of output_sel, the
state of output_sel, the corresponding value of corresponding value of Bit_ stream_de is the output of
Integer_stream is the output of compression. decompression

70 NIET Journal of Engineering & Technology, Vol. 5, 2014

 cI(I('k
. to. 11II0' ••••• .......
JUum!' J:nnnnMMIUlJl.JU1J1l'.IlIUU'J1MnllIUUIIllIlllUlJUl.
REFERENCES
Inttger_strtam_de [I] Reaz, M.B.I., Choong F. and Mohd-Yasin F. 2006 VHDL
e Bit stream _de modelling for classification of power quality disturbance
o output_sel
11 employing wavelet transform, artificial neural network and
• selector

dock
.... [2]
fuzzy ligic. simulation.

Mohd- Yasin,
82( 12):867-881

F., A.L. Tan and M.1. Reaz, 2004.The FPGA

.• Integer_stream_de 1
prototyping of Iris recognition for biometric identification
06 Bit stream _de I
.outpul_scl employing neural network. In the Proceedings of the
selector International Conference on Microelectronics, ICM, pp: 458-
461

[3] Marufuzzaman Mohd., M.B.!. Reaz, M.S. Rahman and M.A.

Mohd. Ali, 2010. Hardware prototyping of an intelligent current
dq PI controller for FOC PMSM drive. In the Proceedings of the
6th International Conference on Electrical and Computer
Engineering(lCECE 20 I 0), pp: 86-88.
Inttger_strum_de
[4] Arifin, M.A.B.T., M. Mamun, M.A.S. Bhuiyan and H. Husain,
Q Bit strum _de
2012. Design of A Low Power and Wide Band True Single-Phase
CI outpul_sel

• selector
Clock Frequency Divider. Australian Journal of Basic and
Applied Sciences, 6(7): 73-79.

[5] Kader,W.M .. H.Rashid,M. Mamun and M.A.S. Bhuiyan,

20 12.Advancement of cmos Schmitt Trigger Circuits. Modern
Applied Science,6( 12): 51-58

[6] Rosli, K.A., M. Mamun, M.A.S. Bhuiyan and H. Husain, 2012.A

Fig. 3: Simulation waveform of decompression block. Low Loss Wide Swing cascode Current Mirror in 0.18-im
CMOS Technology. Journal of Applied Sciences Research, 8(8):
4096-4102.
Input:
[7] Rissanen, J.J. and G.G. Langdon, 1981.Universal Modeling and
1,0,0,2,1,5,7,3,4,3,2, I0, 12,7,5,4,6, 17, 16, 13,1 Coding. IEEE Trans. InfTheory,27(1): 12-23.

[8] Bhuiyan, M.A.S., M. Reaz and M. Ali, 2012. A Review On On-

Output:
Chip Antenna For 2.4 Ghz Ism Band Rfid Tag. In the Proceedings
1,0,0,1,0,1,1,0,1,1,0,1,1,0,0,0,1,0,0,1,0,0,1,0,1,0,0, of the Ist International Conference on Engineering and Built
1,0,1,1,1,0, Environment (ICEBE 20 12), pp: 1-6.23

[9] Huffman, D.A., 1952. A Method for the Construction of

1,0,1,1,1,0,1,1,1,0,1,0,0,1,0,1,1
Minimum-Redundancy Codes. In the Proceedings of the lnst.
The simulation results of input and output of each Radio Eng., pp: 1098-110 I

block confirms the correctness of the code. The model is [10] Reaz, M.B.I., M.1. Ibrahimy, F. Mohd-Yasin, C.S. Wei and M.
also compared with a C model ofLZW algorithm where Kamada, 2007 Single core hardware module to implement
encryption in TECB mode. Infonnacije MIDEM, 37(3): 165-
the same input is being fed and the output is synonymous 171.
with the result obtained from VHDL model.
[II] Jing, Y, 2011. The Combinational Application of LZSS and
IV. CONCLUSIONS LZW Algorithms for Compression Based on Huffman. In the
Proceedings of the International Conference on Electronics and
In this paper, we present the very high speed Optoelectronics (lCEOE-20 II), pp: 397-399

hardware description language (VHDL) modeling [12] Shannon, C.E., 1948. A Mathematical Theory of
environment of Lempel-Ziv-Welch (LZW) algorithm Communication. Bell System Technical Journal,27: 379-423,
623-656.
for binary data compression to ease the description,
verification, simulation and hardware realization. By [13] Fano, R.M., 1961.Transmission of Information. Wiley, ew
York.
simulating with binary data and comparing with C model
of LZW algorithm the proposed approach of VHDL [14] Rodeh, M., YR. Pratt and S. Even, 1981. Linear Algorithm for
Data Compression via String Matching. J. ACM, 28( I): 16-24
modeling of LZW algorithm for binary data
[15] Storer, J.A. and T.G. Szymanski, 1982. Data Compression via
compression is successfully designed, implemented and
Textual Substitution. J. ACM, 29(4): 928-951.
tested. Currently, we are conducting the further research
[16] Ziv, J. and A. Lempel, 1977. A Universal Algorithm for
that considers further reductions in the hardware
Sequential Data Compression. IEEE Trans. Inf. Theory, 23(3):
complexity in terms of synthesis and ease hardware 337-343.
realization.

NIET Journal of Engineering & Technology, Vol. 5, 2014 71

-- _._---------_.
 [17] Ziv, J. and A. Lempel, 1975. Compression of Individual [26] Akter, M., M.B.I. Reaz, F.M. Yasin and F. Choong,200S.
Sequences via Variable-Rate Coding. IEEE Trans. Inf. Theory, Hardware Implementations of Image Compressor for Mobile
24(5): 530-536. Communications. Journal of Communications Technology and
[IS] Welch, T.A., 1984. A Technique for High- Performance Data Electronics, 53(8): 899-910.
Compression. IEEE Computer, 17(6): S-I9. [27] Reaz, M.B.I., M.T. Islam, M.S. Sulaiman, M.A.M. Ali, H.
[19] Bin, L., N. Guiqiang, L. Jianxin and Z. Xue, 20 II. BWT-based Sarwar and S. Rafique, 2003. FPGA Realization Of
Data Preprocessing for LZW .. In the Proceedings of the Multipurpose F[R Filter,. In the Proceedings of the Parallel and
International Conference on Multimedia and Signal Processing Distributed Computing, Applications and Technologies
(CMSP- 2011), pp: 37-40. (PDCAT-2003), pp: 912-915.

[20] Tao, T. and A. Mukherjee, 2005. Pattern Matching in LZW

Mohammad Raza received his B.Tech.
Compressed Files. IEEE Transactions on Computers, 54(8):
degree in Electronics and
929-93S.
Communication Engineering from the
[21] Romli, N.B., M. Mamun, M.A.S. Bhuiyan and H. Husain, 2012.
Uttar Pradesh technical university,
Design of a Low Power Dissipation and Low Input Voltage
Lucknow in 2011. Currently he is
Range Level Shifter in Cedec O.IS- m CMOS Process, World
Applied Sciences Journal, 19(8): 1140-114S.
pursuing M. Tech degree in VLSI
Design from Noida Institute of
[22] Yasin, F.M., A.L. Tan and M.l. Reaz, 2004. The FPGA
Engineering & Technology, Greater Noida. At present he
Prototyping of Iris Recognition for Biometric Identification
Employing Neural Network .. In the Proceedings of the 16th is doing his thesis work on implementation of improved
IEEE International Conference on Microelectronics, pp: 45S- LZRW 1 algorithm using VHDL. His topic of interest is
461. data compression.
[23] Chen, S., B. Mulgrew and P.M. Grant,1993. A Clustering Suvendra Sahu received M.Tech
Technique for Digital Communications Channel Equalization
degree in VLSI Design from NIT
using Radial Basis Function Networks. [EEE Trans. Neural
Tiruchirappalli Tamil Nadu. Currently,
Networks, 4(4): 570-590.
he is an Assistant Professor in
[24] Amin, M.S., M.B.I. Reaz and J. Jalil, 2013. Design ofa novel
Electronics and Communication
adder-less Barker matched filter for RF[D. lnt J. Circ. Theor.
Engineering department at NIET.
App\.. doi: 10.1 002/cta.IS95
Greater Noida. He is also M.Tech
[25] Reaz, M.B.\., F. Choong, M.S. Sulaiman and F.M. Yasin, 2007.
(VLSI) coordinator and teaching various courses that
Prototyping of Wave let Transform Artificial Neural Network and
include Analog Circuit Design, Interconnects and Data
Fuzzy Logic for Power Quality Disturbance Classifier. Journal
of Electric Power Components and Systems, 35( I): 1-17. Compression Techniques.

72 NIET Journal of Engineering & Technology, Vol. 5, 2014