Advanced Set Representation.

AVL Trees

Advanced Set Representation Methods

Problem with BSTs: worst case operation may take O(n) time. One solution: AVL tree: binary search tree with a balance condition: For every node in an AVL tree T, the height of the left (TL) and right (TR) subtrees can differ by at most 1: |hL - hR| 1

AVL trees. 2-3(-4) Trees. Union-Find Set ADT

Balance condition must be easy to maintain, and it

ensures that the tree depth is O(log n) Requiring that the left and right subtrees have the same height does not suffice (tree may not be shallow)
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

Two BSTs

AVL Tree Height

Claim: height of an AVL tree storing n keys is O(log n) Proof

Which BST is AVL?

depth=0 depth=1 depth=2 depth=3

Let n(h) denote the number of nodes of an AVL tree of height h. It is easy to see that n(1) = 1 and n(2) = 2 For n > 2, an AVL tree of height h contains:

the root node, one AVL subtree of height h-1 and another of height h-2.

For every node in an AVL tree T, the height of the left (TL) and right (TR) subtrees can differ by at most 1: |hL - hR| 1
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 3

That is, n(h) = 1 + n(h-1) + n(h-2) Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). By induction n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), , n(h) > 2in(h-2i) Solving the base case we get: n(h) > 2 h/2-1 Taking logarithms: h < 2log n(h) +2

Thus the height of an AVL tree is O(log n)

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 4

Insertion in an AVL Tree

Insertion is as in a binary search tree Always done by attaching an external node. Example:

AVL tree rebalancing

Two kinds of rotation:

single rotation (left or right) double rotation

left-right: left rotation around the left child of a node followed

by a right rotation around the node itself

right-left: right rotation around the left child of a node followed

by a left rotation around the node itself

Rotation features:

After insertion the tree may need rebalancing

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 5

Nodes not in the subtree of the node rotated are unaffected A rotation takes constant time Before and after the rotation tree is still BST Code for left rotation is symmetric to code for a right rotation

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

AVL Single Rotations

Single Rotation Right Example

Single Rotation

k1 < k2 all elements in subtree A are smaller than k1 all elements in subtree C are larger than k2 all elements in subtree B are in between k1 and k2
Note that subtrees B and C are empty

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

Single Rotation Left Example

AVL Implementation Detail

typedef struct { ElementT element; AVLPtr left; AVLPtr right; int height; } AVLNode; typedef AVLNode *AVLPtr;

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

10

AVL Single Rotations

void snglRotRight( AVLPtr *k2 ) { AVLPtr k1 ; k1 = (*k2)->left ; (*k2)->left = k1->right ; k1->right = *k2 ; (*k2)->height = max( height( (*k2)->left ), height( (*k2)->right) ) +1 ; k1->height = max( height( k1>left ), (*k2)->height ) + 1 ; *k2 = k1 ; /* assign new root */ } /* snglRotLeft is symmetric */
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 11

Single Rotation not enough

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

12

AVL Double Rotation

Right-left k2 < k1, k3 < k1, k3 < k2 right rotation around the left child of a node followed by a left rotation around the node itself Rotate to make k2 topmost node
void dblRotLeft( AVLPtr k3 ) { /* rotate between k1 and k2 */ snglRotRight ( (*k3)->left ); /* rotate between k3 and k2 */ snglRotLeft ( k3 ); }
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 13

AVL Double Rotation

Left-right: k1 < k2, k1 < k3, k2 < k3 left rotation around the left child of a node followed by a right rotation around the node itself Rotate to make k2 topmost node

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

14

AVL Right-left Double Rotation Example 1

AVL Right-left Double Rotation Example 2

Subtrees W, X, Y, Z are empty

Subtree X is empty

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

15

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

16

AVL Double Rotation example

AVL Trees. When to use rotations

Balance factor: height(TL) height(TR) Use

A symmetric case of double rotation

Tree before insertion of 27
k3 k1 k2 27

Tree after insertion of 27 and re-balancing

k1

k2

k3
17

Single rotation left: when a node is inserted in the right subtree of the right child (B) of the nearest ancestor (A) with balance factor -2 Single rotation right: when a node is inserted in the left subtree of the left child (B) of the nearest ancestor (A) with balance factor +2. Right-left double rotation: when a node is inserted in the left subtree of the right child (B) of the nearest ancestor (A) with balance factor -2. Left-right double rotation: when a node is inserted in the right subtree of the left child (B) of the nearest ancestor (A) with balance factor +2.
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 18

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

AVL Trees. Deletions

AVL Trees. Deletions. Rebalancing (I)

Let A be the node where balance must be restored. If the deleted node was in A's right subtree, then let B be the root of A's left subtree. Then: B has balance factor 0 or +1 after deletion -- then perform a single right rotation B has balance factor -1 after deletion -- then perform a left-right rotation If the deleted node was in A's left subtree, then let B be the root of A's right subtree. Then: B has balance factor 0 or -1 after deletion -- then perform a single left rotation B has balance factor +1 after deletion -- then perform a right-left rotation
19 DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 20

A node is deleted using the standard inorder successor (predecessor) logic for binary search trees Imbalance is fixed using rotations Identify the parent of the actual node that was deleted, then:

If the left child was deleted, the balance factor at

the parent decreased by 1. If the right child was deleted, the balance factor at the parent increased by 1.
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

AVL Trees. Deletions. Rebalancing (II)

Unlike insertion, one rotation may not be enough to restore balance to the tree. If this is the case, then locate the next node where the balance factor is "bad" (call this A): If A's balance factor is positive, then let B be A's left child

AVL Trees. Deletions. Rebalancing (III)

If A's balance factor is negative, then let B be A's right child

If B's right subtree height is larger than B's left subtree

height, then perform a single left rotation. If B's right subtree height is smaller than B's left subtree height, then perform a right-left rotation. If B's right subtree height is equal to B's left subtree height, then perform either rotation.

If B's left subtree height is larger than B's right subtree

height, then perform a single right rotation. If B's left subtree height is smaller than B's right subtree height, then perform a left-right rotation. If B's left subtree height is equal to B's right subtree height, then perform either rotation.
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 21

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

22

AVL trees demo

Running times for AVL tree operations

a single restructure is O(1)

Demos from:
http://webpages.ull.es/users/jriera/Docencia/AVL/A VL%20tree%20applet.htm http://www.site.uottawa.ca/~stan/csi2514/applets/a vl/BT.html

23

using a linked-structure binary tree height of tree is O(log n), no restructures needed initial find is O(log n) restructuring up the tree, maintaining heights is O(log n) initial find is O(log n) restructuring up the tree, maintaining heights is O(log n)

find is O(log n) insert is O(log n)

Replacement for AVL trees: Red-Black trees (not discussed here)

remove is O(log n)

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

24

2-3 Trees
2-3 tree properties:

2-3 trees

Each interior node has two or three children. Each path from the root to a leaf has the same length. A tree with zero or one node(s) is a special case of a 2-3 tree. Elements are placed at the leaves If element a is to the left of element b, then a < b must hold. Ordering of elements based on one field of a record: a key. At each interior node: key of the smallest descendant of the second child and, if there is a third child, key of the smallest descendant of third child.
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 25

A 2-3 tree of k levels has between 2k - 1 and 3k - 1 leaves, i.e.

representing a set of n elements requires at least

1+log3 n levels and no more than 1+log2 n levels. Thus, path lengths in the tree are O(log n).

Representing sets with 2-3 trees:

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

26

2-3 trees. Insertion 2 children

2-3 trees. Insertion 3 children

If parent node has only two children:

insert in order, adjust keys

If parent node node already has three children:

We cannot have four nodes, and thus we have to

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 27

split parent in two nodes node and node. The two smallest elements among the four children of node stay with node, The two larger will become the children of node Continue process up the tree Special case when splitting the root

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

28

2-3 trees. Insertion

2-3 trees. Deletion

Leaf deletion could leave parent with only 1 child

Parent=root: delete node and let its lone child be

the new root Otherwise, let p be the parent of node

if p has another child, and that child of p has three

children, then we can transfer the proper one to node; done; if the children of p, have only two children, transfer the lone child of node to an adjacent sibling of node, and delete node; p has only one child, repeat all the above, recursively, with p in place of node.

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

29

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

30

2-3 trees. Deletion

2-3-4 trees.
2-3-4 tree refer to how many links to child nodes can potentially be contained in a given node. For non-leaf nodes, three arrangements :

A node with one data item always has two children A node with two data items always has three children A node with three data items always has four children In short, a non-leaf node must always have one more child than it has data items.

Symbolically, if the number of child links is L and the number of data items is D, then L = D + 1 Empty nodes are not allowed.
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 31 DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 32

2-3-4 trees. Node kinds.

2-3-4 trees. Properties and example.

All children in the subtree rooted at child 0 have key values less than key 0. All children in the subtree rooted at child 1 have key values greater than key 0 but less than key 1. All children in the subtree rooted at child 2 have key values greater than key 1 but less than key 2. All children in the subtree rooted at child 3 have key values greater than key 2.

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

33

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

34

2-3-4 trees. Example.

2-3-4 Trees. Insertion

Insertion procedure:

similar to insertion in 2-3 trees items are inserted at the leafs since a 4-node cannot take another item, 4-nodes are split up during insertion process

Strategy

on the way from the root down to the leaf: split up all 4-nodes "on the way" insertion can be done in one pass (remember: in 2-3 trees, a reverse pass might be necessary)
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 36

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

35

2-3-4 Trees. Deletion

Deletion procedure:

Demo: 2-3-4 trees

similar to deletion in 2-3 trees items are deleted at the leafs swap item of internal node with inorder successor note: a 2-node leaf creates a problem

Demo from: http://www.cs.unm.edu/~rlpm/499/ttft.html Local: Demos\2-3-4trees\TTFT.jar

Strategy (different strategies possible)

on the way from the root down to the leaf: turn 2-nodes (except root) into 3-nodes deletion can be done in one pass (remember: in 2-3 trees, a reverse pass might be necessary)
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 37

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

38

Disjoint Sets with the UNION and FIND Operations

Applicable to problems where:

Union-find set ADT

Example statements (see previous slide) Operations:

start with a collection of objects, each in a set by itself; combine sets in some order, and from time to time ask which set a particular object is in If set S has an equivalence relation (reflexive, symmetrical, transitive) defined on it, then the set S can be partitioned into disjoint subsets S1, S2,, ... S with S =S

Equivalence classes:

Equivalence problem:

U
k

given a set S and a sequence of statements of the form a b process the statements in order in such a way that at any time we are able to determine in which equivalence class a given element belongs

union(A, B) takes the union of the components A and B and calls the result either A or B, arbitrarily. find(x), a function that returns the name for the component of which x is a member. initial(A, x) creates a component named A that contains only the element x.
40

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

39

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

Union-find set ADT. Implementations

Examples
List based

List-based

Each set is stored in a sequence represented with

a linked-list Each node should store an object containing the element and a reference to the set name

Tree based

Each element is stored in a node, which contains a

pointer to a set name A node v whose set pointer points back to v is also a set name Each set is a tree, rooted at a node with a selfreferencing set pointer
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 41 DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

Tree based

42

Union-find set. List based implementation

const int n = /* appropriate value */ typedef int NameT; typedef int ElementT; typedef struct ufset { struct /* headers for set lists */ { int count:; int firstElement ; } setHeaders[ n ] ; struct /* table giving set containing each member */ { NameT setName ; int nextElement ; } setNames[ n ] ; } UFSetT;

Union-find sets. Tree representation

Maintain S as a collection of trees (a forest), one per partition Initially, there are n trees, each containing a single element find(i) returns the root of the tree containing i union(i, j) merges the trees containing i and j Typical traversals not needed, so no need for pointers to children, but we need pointers to parent Parent pointers can be stored in an array: parent[i] (=-1 for root)

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

43

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

44

Union-find sets. Tree implementation

Union-find sets. Speeding up

Union by size (rank):

When performing a union, make the root of smaller tree point to the root of the larger

Implies O(n log n) time for performing n unionfind operations:

Each time we follow a pointer, we are going to a subtree of size at least double the size of the previous subtree Thus, we will follow at most O(log n) pointers for any find.
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 46

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

45

Union-find sets. Speeding up

Path compression:

Using union by rank and path compression

After performing a find, compress all the pointers on the path just traversed so that they all point to the root

Implies O(n log* n) time for performing n union- find Time bound of O(m(n)) where (n) is a very slowly growing function
DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 47 DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 48

Union-find ADT demo

A very quickly growing function and its inverse

For integers k0 and j 1, define Ak(j):

Demo from: http://www.cs.unm.edu/~rlpm/499/uf.html Local: Demos\UnionFind\UnionFind.jar

j + 1, if k = 0 Ak ( j ) = ( j +1) Ak 1 ( j ), if k 1

where Ak-10(j)=j, Ak-1(i)(j)= Ak-1(Ak-1(i-1)(j)) for i 1. k is called the level of the function and i in the above is called iterations.

Ak(j) strictly increase with both j and k. Let us see how quick the increase is

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

49

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

50

A very quickly growing function and its inverse

Inverse of Ak(n):(n)

Let us see Ak(1): for k=0,1,2,3,4.

A0(1)=1+1=2 A1(1)=21+1=3 A2(1)=21+1 (1+1)-1=7 A3(1)=A2(1+1)(1)=A2(2)(1)=A2(A2(1))=A2(7)=27+1(7+1)

-1=28 8-1=2047 A4(1)=A32(1)=A3(A3(1)) =A3(2047)=A2(2048)(2047) >> A2(2047) =22048 2048-1 >22048 =(24)512 =(16)512 >>1080 . (estimated number of atoms in universe)

(n)=min{k: Ak(1) n} (n)= 0 for 0 n 2 1 n =3 2 for 4 n 7 3 for 8 n 2047 4 for 2048 n A4(1). Extremely slow increasing function. (n) 4 for all practical purpose.
51 DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos 52

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

Reading

AHU, chapter 5, sections 5.4, 5.5 Preiss, chapter: Search Trees section AVL Search Trees CLR, chapter 22, sections 1-4 Notes

DSA - lecture 4 - T.U.Cluj-Napoca - M. Joldos

53