Csci 1933: Introduction to

Algorithms and Data Structures

Spring 2022
 Announcements

• Project 2 due tonight (3/3)

• Lab 6 milestones due tomorrow (3/4)

• Project 3 will be assigned soon (likely

during Spring Break)
Midterm results
Lab Section numbers
 Distribution of midterm 1 scores
# of students

Test score
Max: 97/100
Min: 10/100
Mean: 66/100
Median: 68.5/100
Std. dev: 20
 Average points per Problem

Averages Average Points Percentage

Problem 1 14.22916667 71.15%

Problem 2 8.470238095 56.47%

Problem 3 7.43452381 49.56%

Problem 4 13.45238095 67.26%

Problem 5 22.35714286 74.52%

 Distribution of midterm 1 scores
F D C B A
# of students

Max: 97/100
Min: 10/100 Test score
Mean: 66/100
Median: 68.5/100
Std. dev: 20 Note: I only curve the final course grade, not individual exam scores. However,
if I were to assign final letter grades based on this score, here’s how it would
look
 Test regrade policy
• Email cs ci1933-s [email protected] with a
summary of what you’d like regraded
• Attach a picture of the relevant part of your
exam to your email
• Email needs to be sent within 24 hrs from now
(before 9pm on Friday, 3/4)
Roadmap for next section of course
• Algorithm complexity analysis (continued)
• Sorting algorithms
• Abstract data types and data structures that
implement them
– Lists, stacks, queues, dictionaries
Brief review of key concepts covered last time
Intro to analysis of algorithm complexity
 How do we decide which algorithm is “best”?

• Possible metrics:
– Correctness!
– How much time does it take? (time complexity)
– How much memory does it require? (space
complexity)
(we can precisely quantify these)

– How easy is it to implement?

(this is often just as/more important)
 “Big Oh” Notation

• To say "Algorithm A has a worst-case time requirement proportional to

n"
– We say A is O(n)
– Read "Big Oh of n"
• For the other two algorithms
– Algorithm B is O(n2)
– Algorithm C is O(1)

Carrano and Henry, Data Structures and Abstractions with Java.

 Formalities of Big Oh notation
• What does it mean to say an algorithm’s complexity
is of O(g(n)) ?
• Formal definition:
– assume an algorithm requires f(n)
operations/space (e.g. f(n) = 2n+1)
– The algorithm is of order at most g(n) (O(g(n)))
if there exists a positive real number c and a positive
integer N exist such that
f(n) ≤ c • g(n) for all n ≥ N
Intuitive interpretation: c*g(n) is an upper bound on f(n) when n is
large enough
Carrano and Henry, Data Structures and Abstractions with Java.
 More formalities of Big Oh notation

• The following identities hold for Big Oh notation:

– O(k f(n)) = O(f(n)) (for a constant k)

– O(f(n)) + O(g(n)) = O(f(n) + g(n))
– O(f(n)) O(g(n)) = O(f(n) g(n))

Carrano and Henry, Data Structures and Abstractions with Java.

Exercise: estimating time
complexity

Answer: O(n2) algorithm

Carrano and Henry, Data Structures and Abstractions with Java.
 Exercise: estimating time complexity
sum=0;
for(int k=1; k <= n; k++) {
for (int i=1; i <= n; i++) {
for(int j=1; j <= n; j++) {
sum = sum+1;
}
}
}
 Sorting algorithms
• Algorithm #1
• Find the minimum value in the list “Selection” sort
• Swap it with the value in the first position
• Repeat the steps above for the remainder of the list (starting at the second position and
advancing each time)

• Algorithm #2
• Iterate from 1st to last element of the array “Insertion” sort
• Compare the current element to its predecessor
• If curr. element smaller, compare it to elements before; move greater elements right one
position to make space for smaller element

• Algorithm #3
“Merge sort”
- Divide an array into halves
- Sort the two halves (recursively)
- Merge them into one sorted array

• Algorithm #3 “Quicksort”
– Pick an element from the list (pivot).
– Reorder the list so that all elements less than the pivot come before the pivot and so that all
elements greater than the pivot come after it (equal values can go either way).
– Repeat the process on the sub-list of lesser elements and the sub-list of greater elements.
Summary of various sorting
algorithms

The time efficiency of various algorithms in Big

Oh notation

Carrano and Henry, Data Structures and Abstractions with Java.

Selection Sort
• Find the minimum value
in the array

• Swap it with the value in

the first position

• Repeat the steps above

for the remainder of the
array (starting at the
second position and
advancing each time)

Scan steps:

Swap steps:

Carrano and Henry, Data Structures and Abstractions with Java.

 Selection Sort Complexity
• Each step: One scan + One swap

– Scan steps: n + (n-1) + (n-2) … + 1 = 1 + 2 … + n

= n(n+1)/2 → O(n2)
– One swap each position = n-1 total swaps → O(n)

– Total = O(n2) + O(n) = O(n2)

 Selection sort questions to think
about
• We’ve discussed the worst-case complexity—
how about best-case?

• What’s the worst-case space complexity of

selection sort?
New material
 Overview of today’s material

• Analysis of algorithm complexity

• Merge sort
• Quicksort

• Intro to abstract data types

 Merge Sort
• Divide an array into halves
Sort the two halves
Merge them into one sorted array

• “divide and conquer” algorithm

(Convenient to use recursion)
 Merge Sort example

Fig. 12-3 The effect of the recursive calls and the

merges during a merge sort.
Carrano and Henry, Data Structures and Abstractions with Java.
 Merge Sort

Fig. 12-3 The effect of the recursive calls and the

merges during a merge sort.
Carrano and Henry, Data Structures and Abstractions with Java.
 Merge Sort pseudocode
//function that sorts arr from position l to r

sort(int arr[], int l, int r)

if l is less than r:

find the middle point of the array

sort first half: sort(arr, l, middle)

sort second half: sort(arr, middle +1,r)

merge the sorted halves of the array

}
}
Merge sort implementation

// Main function that sorts arr[l..r] using

// merge()
void sort(int arr[], int l, int r)
{
if (l < r)
{
// Find the middle point
int m = (l+r)/2;

// Sort first and second halves

sort(arr, l, m);
sort(arr , m+1, r);

// Merge the sorted halves

merge(arr, l, m, r);
}
}

(merge needs to be implemented)

Merge sort visualization: https://www.youtube.com/watch?v=es2T6KY45cA

Merge sort—merge function
example
Merge sort—merge function pseudocode
Algorithm merge(a, tempArray, first, mid, last)
// Merges the adjacent subarrays a[first..mid] and a[mid + 1..last].

beginHalf1 = first endHalf1 = mid

beginHalf2 = mid + 1
endHalf2 = last

// While both subarrays are not empty, compare an entry in one subarray with
// an entry in the other; then copy the smaller item into the temporary array
index = 0 // Next available location in tempArray
while ( (beginHalf1 <= endHalf1) and (beginHalf2 <= endHalf2) ) {
if (a[beginHalf1] <= a[beginHalf2]) {
tempArray[index] = a[beginHalf1]
beginHalf1++
}

else {
tempArray[index] = a[beginHalf2]
beginHalf2++
}

index++
}

// One subarray has been completely copied to tempArray.

// Copy remaining entries from other subarray to tempArray
// Copy entries from tempArray to array a

Carrano and Henry, Data Structures and Abstractions with Java.

Exercise: Merge Sort time
complexity analysis

• Total levels:
• For each level (how many comparisons?):

Fig. 12-3 The effect of the recursive calls and the

merges during a merge sort.
Carrano and Henry, Data Structures and Abstractions with Java.
Merge sort time complexity
analysis
Merge Sort time complexity
analysis

• Total levels: log2 (n)

• For each level: merging requires ~n comparisons

 time complexity: O(n*log(n)) What about

Fig. 12-3 The effect of the recursive average/best
calls and thecase?
merges during a merge sort.
Carrano and Henry, Data Structures and Abstractions with Java.
 Merge sort summary
• Time complexity - O(n*log(n)) in all cases
• Disadvantages? Space complexity?
 Merge sort summary
• Time complexity - O(n*log(n)) in all cases
• Disadvantages? Space complexity?
• It requires a temporary array for merging operation– O(n)
(compare to many “in-place” sorting algorithms, O(1))
 Overview of today’s material

• Analysis of algorithm complexity

• Merge sort
• Quicksort

• Intro to abstract data types

 The Quick Sort algorithm
(1) Pick a pivot element
(2) Partition the array:
• Elements less than the pivot are placed to the
left
• Elements greater than the pivot are placed to the
right (Note: array is not necessarily cut in half)
(3) Sort the left and right sub-arrays (e.g.
recursively)

Carrano and Henry, Data Structures and Abstractions with Java.

 Quick Sort pseudo-code

Algorithm quickSort (a, first, last)

Choose a pivot

Partition the array about the pivot

pivot Index = index of pivot

quickSort (a, first, pivotIndex - 1)

quickSort (a, pivotIndex + 1, last)
}
Quick Sort example
Quick Sort: partitioning step

A partition strategy for quick sort … continued→

Carrano and Henry, Data Structures and Abstractions with Java.

Quick Sort: partitioning step

A partition strategy for quick sort.

Carrano and Henry, Data Structures and Abstractions with Java.

 Quick Sort pseudo-code
Algorithm quickSort (a, first, last)
Choose a pivot

Partition the array about the pivot

pivot Index = index of pivot

quickSort (a, first, pivotIndex - 1)

quickSort (a, pivotIndex + 1, last)
}

QuickSort visualization: https://www.youtube.com/watch?v=aXXWXz5rF64,

start: 56 seconds
 Exercise: Quick Sort complexity
analysis
• Consists of one main procedure (partitioning), which is called
recursively
• 1 partitioning step, complexity?

• How many levels of partitioning will there be?

• Ideal case: we pick pivots that split the array in half

• Worst case?
Exercise: Quick Sort complexity
analysis
Quick Sort time complexity analysis
• Consists of one main procedure (partitioning), which is called
recursively
• 1 partitioning step, complexity?
• ~ O(n) (should require no more than n comparisons)

• How many levels of partitioning will there be?

• Ideal case: we pick pivots that split the array in half
log2(n) partitionings (similar to merge sort)

• Worst case?
We pick a pivot that is larger/smaller than all other elements at each step
~ n partitionings

Total worst-case complexity: O(n)*O(n) = O(n2)

Average-case complexity: O(n log n)

Can we avoid the worst-case?

 A smarter Quick Sort pivot
selection scheme
Idea: let’s pick a pivot at each for each partition that we know is
not the largest or the smallest
One possibility:
• select three elements randomly (e.g. first, middle, last)
• use the median (middle) value as the pivot

Carrano and Henry, Data Structures and Abstractions with Java.

 Quick Sort summary
• Fast sorting algorithm in most cases
• Usually requires less space than merge sort
(average case: O(log n) vs. O(n))
• Variations of Quick Sort are often used in the
“real world”
• Example: remember built-in Java class Arrays,
with static sort methods?
Arrays.sort(intArray);
(quicksort for primitives, mergesort for
reference types)
Carrano and Henry, Data Structures and Abstractions with Java.
 Summary of various sorting
algorithms

Bubble sort O(n2) O(n) O(n2)

The time efficiency of various algorithms in Big

Oh notation

Carrano and Henry, Data Structures and Abstractions with Java.

Quicksort vs. bubblesort:
https://www.youtube.com/watch?v=aXXWXz5rF64 start
1:45
Bubble sort:
Step through the list, comparing each pair of adjacent items, swap if they are in the wrong order
Repeat this process iteratively until no swaps are needed
Worst-case time complexity: O(n2 )

Quicksort vs. mergesort:

https://www.youtube.com/watch?v=es2T6KY45cA start: 2:10
 Take-home messages on Sorting
• You now know 3 classic sorting algorithms (selection sort,
merge sort, quicksort), their complexities, strengths,
weaknesses
(it’s a good idea to know these by name)
• Which algorithm you use may depend on the context
(example: is space efficiency important?)
• You should be able to implement these on standard data
structures we cover later (e.g. lists with arrays or linked lists)
• In general: you should be able to assess the complexity of a
given algorithm
 Overview of today’s material

• Analysis of algorithm complexity

• Merge sort
• Quicksort

• Intro to abstract data types

Abstract data types and data
structures
Quick review of where we’ve been/where we’re
going
• Essentials of Java programming
• Basics of object-oriented programming
– Modular design
– Encapsulation
– Inheritance
• Now: we want to start applying these concepts to
solve useful and general problems
• One useful problem that shows up everywhere: how
do we efficiently/robustly manage large collections
of data (e.g. student records, user accounts, internet
traffic, genome data, …)
 Motivation for data structure design

• Note: data have very specific structures/organization

• These patterns reappear in different applications

We should be able to design “types” (i.e. classes)

that help us organize/process/interact with these
data!
• Design should be motivated by natural structure
of the data + how we want to interact with it
• These types should be general so we can reuse
them in a lot of different settings
 Example: the human genome project
• 3.1 billion base pairs (A, T, C, or G) in each cell
• Organized into 20-25,000 genes
We’re also studying not
just these sequences,
but how the
corresponding proteins
interact
(~200 million potential
interactions)

The human protein-protein

interaction network

Rual et al. Towards a proteome-scale map of the human protein–protein interaction network. Nature 437, 1173-1178 (20 October 2005)
A simpler type of data: my weekend to-do list

(1) Register kids for summer camps (Sat 8am)

(2) Take Lucas to soccer
(3) Clean the house
(4) Take Nora to hockey
(5) Read over student’s paper
(6) Go out to dinner (Sat 7pm)

What kind of structure/behavior

does a “list” have?
 A closer look at a “list”

• special organizational properties

– has a natural order
– add entries
– remove entries (from anywhere in the list)
– look at any entry
– count how many

• These are also useful characteristics for managing

large amounts of data
 Abstract Data Type: definition

• A list is an example of an abstract data type (ADT)

• What is an ADT?: a set of operations that can be performed,

along with rules for how the class behaves under those
operations
NOT: an implementation, not even a type of storage
(not even language-specific, “abstract”)

Data structure: specific implementation of an ADT (e.g. in Java)

 What’s the process of defining an abstract data
type and implementing a data structure?
• Much of computer science— defining an idea for a
problem solution, ultimately mapping this to a language
(e.g. Java) that a computer can understand
• Steps:
More abstract (1) Develop specifications for ADT
• What type of data will be stored?
• What are the operations that are possible? How do
they change the data?
(2) Experiment with abstract functionality, iterate spec
definition until complete
Less abstract (3) Formalize specs of the ADT
(4) Implementation of data structure and testing
Developing specifications for a List ADT

• What type of data will be stored?

(not application-specific, but abstractly)

• What are the operations that are

possible?
 Specifications for the List ADT
• Data:
– a collection of objects in a specific order, having the
same type
– the number of objects in the collection
• Operations
– Add new entry – at end, or anywhere
– Remove an item
– Remove all items
– Replace an entry
– Look at any entry
– Look for an entry of a specific value
– Count how many entries
– Check if list is empty, full
See ListInterface.java
– Display all the entries
Carrano and Henry, Data Structures and Abstractions with Java.
Implementation of List data structure
• Primitive structures with any meta-information we need
• We’ll start with an array

• What member variables will we need?

• integer to keep track of max size
• integer to keep track of current size

• Methods? (need to follow specifications!)

public boolean add(T newEntry) / / add to end of list public T
remove(int position) / / remove a specific item public int getLength()
/ / return number of elements in list
Class Layout

public class AList<T> implements ListInterface<T>{

private T[] l i s t ;
p r i v a t e i n t length;
private s t a t i c i n t MAX_SIZE=50; // default max

public A L i s t ( ) {
this(MAX_SIZE); // c a l l next constructor
}
public AList( int maxSize) {
length = 0;
// necessary cast to generic type
list= (T[])new Object[maxSize];
}
/* Rest of implementation goes here */
}