INDEXING

Jehan-François Pâris
Spring 2015
Overview
 Three main techniques
 Conventional indexes
 Think of a page table, …

 B and B+ trees
 Perform better when records are constantly

added or deleted
 Hashing
Conventional
indexes
Indexes
 A database index is a data structure that
improves the speed of data retrieval operations
on a database table at the cost of additional
writes and storage space to maintain the index
data structure.

Wikipedia
Types of indexes
 An index can be
 Sparse
 One entry per data block

 Identifies the first record of the block

 Requires data to be sorted

 Dense
 One entry per record

 Data do not have to be sorted

Respective advantages
 Sparse
 Occupy much less space

 Can keep more of it in main memory

Faster access

 Dense
 Can tell if a given record exists without

accessing the file

 Do not require data to be sorted
Indexes based on primary keys
 Each key value corresponds to a specific record
 Two cases to consider:
 Table is sorted on its primary key
 Can use a sparse index

 Table is either non-sorted or sorted on

another field
 Must use a dense index
Sparse Index
Alan . Ahmed … …
Dana . Amita … …
Gina . Brenda … …
Carlos … …
Dana … …
Dino … …
Emily … …
Frank … …
Dense Index
Ahmed Ahmed … …
Amita Frank … …
Brenda Brenda … …
Carlos Dana … …
Dana
Dino Emily … …
Emily Dino … …
Frank Carlos … …
Amita … …
Indexes based on other fields
 Each key value may correspond to more than one
record
 clustering index
 Two cases to consider:
 Table is sorted on the field
 Can use a sparse index

 Table is either non-sorted or sorted on another field

 Must use a dense index
Sparse clustering index
Austin . Ahmed Austin …
Dallas . Frank Austin …
Laredo . Brenda Austin …
Dana Dallas …
Emily Dallas …
Dino Dallas …
Carlos Laredo …
Amita Laredo …
Dense clustering index
Austin
Austin
Ahmed Austin …
Austin
Amita Laredo …
Brenda Austin …
Dallas
Dallas
Carlos Laredo …
Dallas Dana Dallas …
Laredo Dino Dallas …
Laredo Emily Dallas …
Frank Austin …
Another realization
Austin Ahmed Austin …
Dallas . Amita Laredo …
Laredo . Brenda Austin …
Carlos Laredo …
We save space
Dana Dallas …
and add one extra
Dino Dallas …
level of indirection
Emily Dallas …
Frank Austin …
A side comment

 "We can solve any problem by introducing an

extra level of indirection, except of course for the
problem of too many indirections."

 David John Wheeler

Indexing the index
 When index is very large, it makes sense to
index the index
 Two-level or three-level index
 Index at top level is called master index
 Normally a sparse index
Two levels

AKA
Master Index
Top Index
Updating indexed tables
 Can be painful
 No silver bullet
B-trees and B+
trees
Motivation
 To have dynamic indexing structures that can evolve
when records are added and deleted
 Not the case for static indexes
 Would have to be completely rebuilt

 Optimized for searches on block devices

 Both B trees and B+ trees are not binary
 Objective is to increase branching factor (degree
or fan-out) to reduce the number of device
accesses
Binary vs. higher-order tree
 Binary trees:  Higher-order trees:
 Designed for in-  Designed for
memory searches searching data on
 Try to minimize the block devices
 Try to minimize the
number of memory
accesses number of device
accesses
 Searching within

a block is cheap!
B trees
 Generalization of binary search trees
 Not binary trees
 The B stands for Bayer (or Boeing)
 Designed for searching data stored on block-
oriented devices
A very small B tree

Bottom nodes are leaf nodes: all their

pointers are NULL
 In reality
In Key In Key In Key In Key In
tree tree tree tree tree
ptr Data ptr ptr Data ptr ptr Data ptr ptr Data ptr ptr

To 7 To 16 To -- --
Null Null
Leaf leaf Leaf Null Null
Organization
 Each non-terminal node can have a variable number
of child nodes
 Must all be in a specific key range
 Number of child nodes typically vary between d
and 2d
 Will split nodes that would otherwise have

contained 2d + 1 child nodes

 Will merge nodes that contain less than d child

nodes
 Searching the tree

keys < 7 keys > 16

7 < keys < 16

Balancing B trees
 Objective is to ensure that all terminals nodes be
at the same depth
Insertions
 Assume a tree where each node can contain three pointers (non represented)
 Step 1:

 Step 2:

 Step 3:

1
 Split node in middle

1 2

1 2 3 2

1 3
Insertions
 Step 4:

 Step 5: 2
1 3 4
 Split
 Move up

2
1 3 4 5

2 4
1 3 5
Insertions
 Step 6:
2 4
1 3 5 6

 Step 7:
2 4
1 3 5 6 7
Step 7 continued
2 4
1 3 6
 Split 4 7

2 4 6
1 3
 Promote
5 7
Step 7 continued

2 4 6
1 3
5 7
 Split after
the promotion 4

2 6
1 3 5 7
Two basic operations
 5 6 7
Split:
 When trying to add to a full node
6
 Split node at central value
5 7
 Promote:
 Must insert root of split
node higher up
 May require a new split
B+ trees
 Variant of B trees
 Two types of nodes
 Internal nodes have no data pointers
 Leaf nodes have no in-tree pointers
 Were all null!
B+ tree nodes
In In In In In In
tree Key tree Key tree Key tree Key tree Key tree
ptr ptr ptr ptr ptr ptr

Key Key Key Key Key Key

Data ptr Data ptr Data ptr Data ptr Data ptr Data ptr
More about internal nodes
 Consist of n -1 key values K1, K2, …, Kn-1 ,and n tree
pointers P1, P2, …, Pn :
 < P1,K1, P2, K2, P3, …, Pn-1, Kn-1,, Pn>
 The keys are ordered K1 < K2 < … < Kn-1
 For each tree value X in the subtree pointed at by tree
pointer Pi, we have:
X > Ki-1 for 1 ≤ i ≤ n
X ≤ Ki for 1 ≤ i ≤ n - 1
Warning
 Other authors assume that
 For each tree value X in the subtree pointed
at by tree pointer Pi, we have:
X ≥ Ki-1 for 1 ≤ i ≤ n
X < Ki for 1 ≤ i ≤ n - 1
 Changes the key value that is promoted when
an internal node is split
Advantages
 Removing unneeded pointers allows to pack
more keys in each node
 Higher fan-out for a given node size
 Normally one block

 Having all keys present in the leaf nodes allows

us to build a linked list of all keys
Properties
 If m is the order of the tree
 Every internal node has at most m children.
 Every internal node (except root) has at least ⌈m ⁄
2⌉ children.
 The root has at least two children if it is not a leaf
node.
 Every leaf has at most m − 1 keys
 An internal node with k children has k − 1 keys.
 All leaves appear in the same level
Best cases and worst cases
 A B+ tree of degree m and height h will store
 At most mh – 1(m – 1) = mh – m records
 At least 2⌈m ⁄ 2⌉h – 1 records
Searches
 def search (k) :
return tree_search (k, root)
Searches
def tree_search (k, node) :
if node is a leaf :
return node
elif k < k_0 :
return tree_search(k, p_0)
…
elif k_i ≤ k < k_{i+1}
return tree_search(k, p_{i+1})
…
elif k_d ≤ k
return tree_search(k, p_{d+1});
Insertions
 def insert (entry) :
 Find target leaf L
 if L has less than m – 2 entries :
 add the entry

else :
 Allocate new leaf L'

 Pick the m/2 highest keys of L and move them to L'

 Insert highest key of L and corresponding address leaf

into the parent node

 If the parent is full :

 Split it and add the middle key to its parent node

 Repeat until a parent is found that is not full
Deletions
 def delete (record) :
 Locate target leaf and remove the entry
 If leaf is less than half full:
 Try to re-distribute, taking from sibling (adjacent

node with same parent)

 If re-distribution fails:

 Merge leaf and sibling

 Delete entry to one of the two merged leaves

 Merge could propagate to root

Insertions
 Assume a B+ tree of degree 3

 Step 1:

 Step 2:

 Step 3:

1
 Split node in middle

1 2

1 2 3 2

1 2 3
Insertions
 Step 4:

 Step 5: 2
1 2 3 4
 Split
 Move up

2
1 2 3 4 5

2 4
1 2 3 4 5
Insertions
 Step 6:
2 4
1 2 3 4 5 6

 Step 7:
2 4
1 2 3 4 5 6 7
Step 7 continued
2 4
1 2 3 4 6
 Split 5 6 7

2 4 6
1 2
3 4
 Promote
5 6 7
Step 7 continued

2 4 6
1 3
5 7
 Split after
the promotion 4

2 6
1 3 5 7
Importance
 B+ trees are used by
 NTFS, ReiserFS, NSS, XFS, JFS, ReFS, and
BFS file systems for metadata indexing
 BFS for storing directories.
 IBM DB2, Informix, Microsoft SQL Server,
Oracle 8, Sybase ASE, and SQLite for table
indexes
 Not on
An interesting variant Spring 2015
first quiz
 Can simplify entry deletion by never merging
nodes that have less than ⌈m ⁄ 2⌉ entries
 Wait instead until there are empty and can be
deleted
 Requires more space
 Seems to be a reasonable tradeoff assuming
random insertions and deletions
Hashing
Fundamentals
 Define m target addresses (the "buckets")
 Create a hash function h(k) that is defined for
all possible values of the key k and returns an
integer value h such that 0 ≤ h ≤ m – 1

Key h(k)
The idea

Key

Hash
value
is
Bucket
address
Bucket sizes
 Each bucket consists of one or more blocks
 Need some way to convert the hash value into a
logical block address
 Selecting large buckets means we will have to
search the contents of the target bucket to find the
desired record
 If search time is critical and the database
infrequently updated, we should consider
sorting the records inside each bucket
Bucket organization
 Two possible solutions
 Buckets contain records
 When bucket is full, records go to an

overflow bucket
 Buckets contain pairs <key, address>
 When bucket is full, pairs <key, address>

go to an overflow bucket
 Buckets contain records

Assume each
bucket contains Overflow bucket
two records
Buckets contain records
A record

KEY

A bucket can
Many
contain many
more
more keys KEY records
than records
Finding a good hash function
 Should distribute records evenly among the
buckets
 A bad hash function will have too many
overflowing buckets and too many empty or
near-empty buckets
A good starting point
 If the key is numeric
 Divide the key by the number of buckets
 If the number of buckets is a power of two,

this means selecting log2 m least significant

bits of key
 Otherwise
 Transform the key into a numerical value
 Divide that value by the number of buckets
Looking further
 Hashing works best when the number of buckets
is a prime number

 If performance matters, consult

 Donald Knuth's Art of Computer Programming
 http://en.wikipedia.org/wiki/Hash_function
Selecting the load factor
 Percentage of used slots
 Best range is between 0.5 and 0.8

 If load factor < 0.5

 Too much space is wasted

 If load factor > 0.8

 Bucket overflows start becoming a problem
 Depending on how evenly the hash function

distributes the keys among the buckets

Dynamic hashing
 Conventional hashing techniques work well when
the maximum number of records is known ahead
of time
 Dynamic hashing lets the hash table grow as the
number of records grow
 Two techniques:
 Extendible hashing
 Linear hashing
Extendible hashing
 Represent hash values as bit strings:
 100101, 001001, …
 Introduce an additional level of indirection, the
directory
 One entry per key value
 Multiple entries can point to the same bucket
Extendible hashing
 We assume a three-bit key

Directory
K = 010 Records with
000 d=1
001
key = 0*
010
K = 111 Records with d = 1
001
100
key = 1*
101
110
Both buckets are at same depth d
101
Extendible hashing
 When a bucket overflows, we split it

Directory
K = 000 Records with d = 2
000
001
key = 00*
010
K = 111 Records with d = 1
001
100
key = 1*
101 Records with d = 2
K = 010
110
K = 011 key = 01*
101
Explanations (I)
 Choice of a bucket is based on the most
significant bits (MSBs) of hash value
 Start with a single bit
 Will have two buckets
 One for MSB = 0

 Other for MSB = 1

 Depth of bucket is 1
Explanations (II)
 Each time a bucket overflows, we split it
 Assume first bucket overflows
 Will add a new bucket containing records

with MSBs of hash value = 01

 Older bucket will keep records with MSBs

of hash value = 00
 Depths of these two bucket is 2
Explanations (III)
 At any given time, the hash table will contain
buckets at different depths
 In our example, buckets 00 and 01 are at
depth 2 while bucket 1 is at depth 1
 Each bucket will include a record of its depth
 Just a few bits
Discussion
 Extendible hashing
 Allows hash table contents
 To grow, by splitting buckets

 To shrink by merging buckets

but
 Adds one level of indirection
 No problem if the directory can reside in

main memory
Linear hashing
 Does not add an additional level of indirection
 Reduces but does not eliminate overflow buckets
 Uses a family of hash functions
 hi(K) = K mod m

 hi+1(K) = K mod 2m
 hi+2(K) = K mod 4m
…
How it works (I)
 Start with
 m buckets
 hi(K) = K mod m
 When any bucket overflows
 Create an overflow bucket
 Create a new bucket at location m
 Apply hash function hi+1(K)= K mod 2m to the contents
of bucket 0
 Will now be split between buckets 0 and m
How it works (II)
 When a second bucket overflows
 Create an overflow bucket
 Create a new bucket at location m + 1
 Apply hash function hi+1(K)= K mod 2m to the
contents of bucket 1
 Will now be split between buckets 1 and

m+1
How it works (III)
 Each time a bucket overflows
 Create an overflow bucket
 Apply hash function hi+1(K)= K mod 2m to the contents of
the successor s + 1 of the last bucket that was split
 Contents of bucket s + 1 will now be split between

buckets s and m + s – 1
 The size of the hash table grows linearly at each split until
all buckets use the new hash function
Advantages
 The hash table goes linearly
 As we split buckets in linear order, bookkeeping is
very simple:
 Need only to keep track of the last bucket s that
was split
 Buckets 0 to s use the new hash function

hi+1(K)= K mod 2m
 Buckets s + 1 to m – 1 still use the old hash
function hi(K)= K mod m
Example (I)
 Assume m = 4 and one record per bucket
 Table contains two records

Hash value = 0

Hash value = 2
Example (II)
 We add one record with hash value = 2

Overflow bucket
Hash value = 2 Hash value = 2

New bucket Hash value = 4

We assume that the contents of bucket 0 were

migrated to bucket 4
Multi-key indexes
 Not covered this semester