AV121: Data Structures and Algorithms

Kurian John

Lecture 02
Lecture 02 – Plan

 Review of Topics Covered

 Binary Trees and n-ary Trees

April-May 2023 AV121 - Data Structures and Algorithms 2

Review of Topics Covered
 Set
 Map
 Tree basics
 Tree Traversals

April-May 2023 AV121 - Data Structures and Algorithms 3

Binary Trees

0 1
0 1
0
1
0
1 0 1 0
0
0
0 1 1 1 1
0
1 1 1 0
1
1

Is this a binary tree?

April-May 2023 AV121 - Data Structures and Algorithms 4
Binary Trees
 The arbitrary number of children in general trees is often
unnecessary—many real-life trees are restricted to two
branches
→ Expression trees using binary operators
→ An ancestral tree of an individual, parents, grandparents, etc.
→ Lossless encoding algorithms

 There are also issues with general trees:

→ There is no natural order between a node and its children

April-May 2023 AV121 - Data Structures and Algorithms 5

Binary Trees
 A binary tree is a restriction where each node has exactly
two children:
→ Each child is either empty or another
binary tree
→ This restriction allows us to label the
children as left and right

→ We can also refer to the two sub-trees as

 The left-hand sub-tree, and
 The right-hand sub-tree

April-May 2023 AV121 - Data Structures and Algorithms 6

Binary Trees

April-May 2023 AV121 - Data Structures and Algorithms 7

Binary Trees

Each node
can have at
most two
child nodes

April-May 2023 AV121 - Data Structures and Algorithms 8

Binary Trees
 A full node is a node where both the left and right sub-trees are non-empty trees

full nodes neither leaf nodes

April-May 2023 AV121 - Data Structures and Algorithms 9

Binary Trees
 A full binary tree is where each node is:
→ A full node, or
→ A leaf node

 These have applications in

→ Expression trees
→ Huffman encoding

April-May 2023 AV121 - Data Structures and Algorithms 10

Binary Trees
 Revisiting Expression Trees

Postfix expression?

→ Internal nodes store operators → Humans think in in-order

→ Leaf nodes store literals or variables → Computers often think in post-order:
→ No nodes have just one sub tree → Both operands must be loaded into
registers
→ The order is not relevant for
→ The operation is then called on those
• Addition and multiplication (commutative) registers
→ Order is relevant for
• Subtraction and division (non-commutative)
April-May 2023 AV121 - Data Structures and Algorithms 11
Binary Trees
 Revisiting Expression Trees

3 4 a × b c + + × d 5 ÷ 6 e – + +
→ Internal nodes store operators
→ Leaf nodes store literals or variables
→ No nodes have just one sub tree
→ The order is not relevant for
• Addition and multiplication (commutative)
→ Order is relevant for
• Subtraction and division (non-commutative)
April-May 2023 AV121 - Data Structures and Algorithms 12
Perfect Binary Trees
 A perfect binary tree of height h is a binary tree where
→ All leaf nodes have the same depth h
→ All other nodes are full

Height

April-May 2023 AV121 - Data Structures and Algorithms 13

Perfect Binary Trees
 A perfect binary tree of height h is a binary tree where
→ All leaf nodes have the same depth h
→ All other nodes are full

0 1 2 3 4 Height

April-May 2023 AV121 - Data Structures and Algorithms 14

Perfect Binary Trees

Is this a binary tree?

April-May 2023 AV121 - Data Structures and Algorithms 15
Perfect Binary Trees – Properties
 A perfect binary tree of height h and number of nodes n
has the following properties:
Property Value
Number of nodes, n 2(h+1) - 1
Height, h Θ (ln (n) )
Number of leaf nodes 2h
Average depth of a node Θ (ln (n) )

April-May 2023 AV121 - Data Structures and Algorithms 16

Complete Binary Trees
 A perfect binary tree has ideal properties but restricted in
the number of nodes: n = 2(h + 1) – 1 for h = 0, 1, …
→ 1, 3, 7, 15, 31, 63, 127, 255, 511, 1023, ....

 We require binary trees which are

→ Similar to perfect binary trees, but
→ Defined for all n

April-May 2023 AV121 - Data Structures and Algorithms 17

Complete Binary Trees
 A Complete Binary Tree is a binary tree where all the levels are
completely filled except possibly the last level and the last level has
all keys as left as possible.
 A complete binary tree is just like a full binary tree, but with two
major differences:
→ Every level must be completely filled
→ All the leaf elements must lean towards the left.
 The last leaf element might not have a right sibling – doesn’t have
to be a perfect binary tree

 Filled at each depth from

left to right

April-May 2023 AV121 - Data Structures and Algorithms 18

Complete Binary Trees
 Recursive definition: a binary tree with a single node is a complete
binary tree of height h = 0 and a complete binary tree of height h is
a tree where either:
→ The left sub-tree is a complete tree of height h – 1 and the right sub-tree is a
perfect tree of height h – 2, or
→ The left sub-tree is perfect tree with height h – 1 and the right sub-tree is
complete tree with height h – 1

Height of a complete binary tree with n nodes is h = ⌊lg(n)⌋

April-May 2023 AV121 - Data Structures and Algorithms 19

Complete Binary Trees
 Advantage: Can be easily stored as an array
→ Traverse the tree in breadth-first order, placing the entries into the array

April-May 2023 AV121 - Data Structures and Algorithms 20

Complete Binary Trees
 Advantage: Can be easily stored as an array
→ For inserting a node, keep adding to the array

April-May 2023 AV121 - Data Structures and Algorithms 21

Complete Binary Trees
 Advantage: Can be easily stored as an array
→ For removing a node, remove the last element in the array

April-May 2023 AV121 - Data Structures and Algorithms 22

Complete Binary Trees
 Why did we skip the first element of the array?
→ The children of the node with index k are in 2k and 2k + 1
→ The parent of node with index k is in k ÷ 2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

April-May 2023 AV121 - Data Structures and Algorithms 23

Complete Binary Trees
 Why did we skip the first element of the array?
→ The children of the node with index k are in 2k and 2k + 1
→ The parent of node with index k is in k ÷ 2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

parent = k >> 1;
left_child = k << 1;
right_child = left_child | 1;

April-May 2023 AV121 - Data Structures and Algorithms 24

Complete Binary Trees
 So why not store all kinds of binary trees as arrays?

April-May 2023 AV121 - Data Structures and Algorithms 25

Complete Binary Trees
 So why not store all kinds of binary trees as arrays?

In the worst case, an exponential amount of memory is required

April-May 2023 AV121 - Data Structures and Algorithms 26
Application of Binary Trees – Huffman Coding
 ASCII uses same number of bits to encode each character
 Huffman codes use different number of bits to encode different
letters
→ More bits for rare letters and fewer bits for common letters
→ Why? Loss-less compression!

 Huffman coding assigns codes to characters such that the length of

the code depends on the relative frequency or weight of the
corresponding character.
 Huffman codes are of variable-length, and prefix-free (no code is
prefix of any other). Any prefix-free binary code can be visualized as
a binary tree with the encoded characters stored at the leaves.

 A Huffman coding tree or Huffman tree is a full binary tree in which

each leaf of the tree corresponds to a letter in the given alphabet.

April-May 2023 AV121 - Data Structures and Algorithms 27

Application of Binary Trees – Huffman Coding
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6

 Problems
0 306 1
→ Tree Building
120 186
1 → Encoding
E
0
1 → Decoding
79 107
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 28

Application of Binary Trees – Huffman Coding
 Encoding DEED

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 29

Application of Binary Trees – Huffman Coding
 Encoding DEED

101

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 30

Application of Binary Trees – Huffman Coding
 Encoding DEED

101

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 31

Application of Binary Trees – Huffman Coding
 Encoding DEED

101 0

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 32

Application of Binary Trees – Huffman Coding
 Encoding DEED

101 0 0 101

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 33

Application of Binary Trees – Huffman Coding
 Decoding 111111001110111101

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 34

Application of Binary Trees – Huffman Coding
 Decoding 111111001110111101

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 35

Application of Binary Trees – Huffman Coding
 Decoding 111111001110111101

M U

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 36

Application of Binary Trees – Huffman Coding
 Decoding 111111001110111101

M U C

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 37

Application of Binary Trees – Huffman Coding
 Decoding 111111001110111101

M U C K

0 306 1
120 186
1
E
0
107
1
79
0 1 0
65 1
37 42 42 0
U D L
32
0 33 1
C
24
9 M
0 1
2 7
Z K

April-May 2023 AV121 - Data Structures and Algorithms 38

Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6
0 306 1 1. Prepare a collection of n initial
1 Huffman trees, each of which is
120 186
E
0 a single leaf node. Put the n
1 trees onto a priority queue
79 107 organized by weight
0 1 0 (frequency).
65 1 2. Remove the first two trees (the
37 42 42 0 ones with lowest weight). Join
U D L
these two trees to create a
32
0 33 1
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
0 1
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 39
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6

1. Prepare a collection of n initial

Huffman trees, each of which is
120
a single leaf node. Put the n
E
trees onto a priority queue
organized by weight
(frequency).
37
2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 40
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6

1. Prepare a collection of n initial

Huffman trees, each of which is
120
a single leaf node. Put the n
E
trees onto a priority queue
organized by weight
(frequency).
37
2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 41
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6

1. Prepare a collection of n initial

Huffman trees, each of which is
120
a single leaf node. Put the n
E
trees onto a priority queue
organized by weight
(frequency).
37
2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 33 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 42
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6

1. Prepare a collection of n initial

Huffman trees, each of which is
120
a single leaf node. Put the n
E
trees onto a priority queue
organized by weight
(frequency).
37
65 2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 33 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 43
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6

1. Prepare a collection of n initial

Huffman trees, each of which is
120
a single leaf node. Put the n
E
trees onto a priority queue
79 organized by weight
(frequency).
37
65 2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 33 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 44
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6

1. Prepare a collection of n initial

Huffman trees, each of which is
120
a single leaf node. Put the n
E
trees onto a priority queue
79 107 organized by weight
(frequency).
37
65 2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 33 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 45
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6
306 1. Prepare a collection of n initial
Huffman trees, each of which is
120 186 a single leaf node. Put the n
E
trees onto a priority queue
79 107 organized by weight
(frequency).
37
65 2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 33 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 46
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6
306 1. Prepare a collection of n initial
Huffman trees, each of which is
120 186 a single leaf node. Put the n
E
trees onto a priority queue
79 107 organized by weight
(frequency).
37
65 2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 33 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 47
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6
306 1. Prepare a collection of n initial
Huffman trees, each of which is
120 186 a single leaf node. Put the n
E
trees onto a priority queue
79 107 organized by weight
(frequency).
37
65 2. Remove the first two trees (the
42 42
ones with lowest weight). Join
U D L
32 33 these two trees to create a
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
weights of the two child trees.
2 7
3. Put this new tree into the
K
priority queue.
Z
4. Repeat steps 2-3 until all of the
partial Huffman trees have
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 48
Application of Binary Trees – Huffman Coding
 Tree building
Letter E D L U C M K Z
Frequency 120 42 42 37 32 24 7 2
Code 0 101 110 100 1110 11111 111101 111100
Bits 1 3 3 3 4 5 6 6
0 306 1 1. Prepare a collection of n initial
1 Huffman trees, each of which is
120 186
E
0 a single leaf node. Put the n
1 trees onto a priority queue
79 107 organized by weight
0 1 0 (frequency).
65 1 2. Remove the first two trees (the
37 42 42 0 ones with lowest weight). Join
U D L
these two trees to create a
32
0 33 1
C
new tree whose root has the
24 two trees as children, and
9 M whose weight is the sum of the
0 1
During Traversal: While moving to the left weights of the two child trees.
child, write 0 to the array. While moving to the
2 7
3. Put this new tree into the
right child, write 1 to the array priority queue.
Z K
4. Repeat steps 2-3 until all of the
Array will contain the code when a leaf node is
partial Huffman trees have
encountered
April-May 2023 AV121 - Data Structures and Algorithms been combined into one 49
N-ary Trees
 Generalization of binary trees
→ A tree where each node had N sub-trees, any of which may be empty trees

Ternary trees Quaternary tree

h +1
N −1 h
Number of nodes in a perfect N-ary tree of height h, n =  N =
k

k =0 N −1
h = log N (n( N − 1) + 1) − 1
A perfect binary tree has approximately log 2 (N) times the height
of an n-ary tree with the same number of nodes
April-May 2023 AV121 - Data Structures and Algorithms 50
N-ary Trees
 Trie
→ A trie (derived from
retrieval) is a multiway
tree data structure used
for storing strings over an a d
alphabet. It is used to
store a large number of n a o
strings.
→ Pattern matching can be d t d
done efficiently using
tries

and ant dad do

April-May 2023 AV121 - Data Structures and Algorithms 51

N-ary Trees
 Trie - the root represents the start of each valid word, and the
different sub-trees represent next letters in valid words

→ Consider the words in the

phrase
“The fable then faded from my
thoughts and memory.”
→ All 26 sub-trees are only
shown for the root node, but
all nodes have 26 sub-trees
→ Some nodes are marked as
terminal indicating the end of
a valid word
→ These terminal points could
be used to store a list
of all places in a document
where the word occurs
 Ultimate index to a book!

April-May 2023 AV121 - Data Structures and Algorithms 52

 Questions?

April-May 2023 AV121 - Data Structures and Algorithms 53

Lecture 02 – Review

 Binary Trees
 Definitions
 Perfect and Complete Binary Trees, Representation
 Applications – Expression / Huffman Trees
 N-ary Trees, Tries

April-May 2023 AV121 - Data Structures and Algorithms 54