0% found this document useful (0 votes)

159 views

Computational Thinking A Primer For Programmers and Data Scientists G Venkatesh Madhavan Mukund

Uploaded by

Prasun Babu

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

159 views

Computational Thinking A Primer For Programmers and Data Scientists G Venkatesh Madhavan Mukund

Uploaded by

Prasun Babu

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 187

lOMoARcPSD|44901534

Computational Thinking A Primer for Programmers and Data

Scientists - G Venkatesh, Madhavan Mukund
IITM online degree Data science and programming (Indian Institute of Technology
Madras)

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by Prasun Babu ([email protected])
lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Computational
Thinking
A Primer for Programmers and Data

Scientists

G Venkatesh
Madhavan Mukund

Downloaded by Prasun Babu ([email protected])

4.3 Compound conditions . . . . . . . . . . . . . . . . . . . . . . . 61

4.4 Looking for a data element . . . . . . . . . . . . . . . . . . . . 69

5 Datatypes 73
5.1 Sanity of data . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.2 Basic datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.3 Compound datatypes . . . . . . . . . . . . . . . . . . . . . . . 79
5.4 Subtypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.5 Transforming the data element . . . . . . . . . . . . . . . . . . 83
5.6 Datatypes for the elements in the dataset . . . . . . . . . . . . . 86

6 Filtering: dynamic conditions 91

6.1 Maximum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.2 Minimum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.3 Examples using maximum or minimum . . . . . . . . . . . . . 96
6.4 Combining static, dynamic and state conditions . . . . . . . . . 97

7 Pseudocode 103
7.1 Basic iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.2 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.3 Compound conditions . . . . . . . . . . . . . . . . . . . . . . . 108
7.4 Filtering with dynamic conditions . . . . . . . . . . . . . . . . 109

8 Procedures and Parameters 111

8.1 Pseudocode for procedures . . . . . . . . . . . . . . . . . . . . 111
8.2 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
8.3 Checking for a single outstanding student . . . . . . . . . . . . 113
8.4 Side-effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

II Computational Thinking for Data Science 117

9 Element ←→ Dataset 119

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

9.1 Sequenced iterations . . . . . . . . . . . . . . . . . . . . . . . 120

9.2 Compare with average . . . . . . . . . . . . . . . . . . . . . . 122
9.3 Classifying/Ranking the elements . . . . . . . . . . . . . . . . 125

10 Element ←→ Element 133

10.1 Nested iterations . . . . . . . . . . . . . . . . . . . . . . . . . 134
10.2 Reducing the number of comparisons . . . . . . . . . . . . . . 140
10.3 Examples of nested iterations . . . . . . . . . . . . . . . . . . . 145

11 Lists 151
11.1 List as a set or collection . . . . . . . . . . . . . . . . . . . . . 152
11.2 Using the list collections . . . . . . . . . . . . . . . . . . . . . 152

12 Procedures with Side Effects 153

13 Sorted Lists 155

13.1 Keeping lists in sorted form may speedup subsequent processing 156
13.2 How do we sort? . . . . . . . . . . . . . . . . . . . . . . . . . 156
13.3 Using the insertion sort procedure . . . . . . . . . . . . . . . . 156

14 Dictionaries 157

15 Dictionaries with Lists 159

16 Graphs 161

17 Graphs, Lists and Dictionaries 163

III Higher Level Computational Thinking 165

18 Recursion 167

19 Recursion Trees 169

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

20 Spanning Trees 171

21 Object Oriented Computing 173

22 Functional Computing 175

23 Concurrency 177

24 Input and Output 179

25 Real Time Systems 181

26 Bottom-up Computing 183

27 Summary & Further Reading 185

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Preface

How do we introduce computing to a lay person? This was the question that
confronted us when we volunteered to offer such a course in the foundation year
of the online BSc degree in Programming and Data Science at IIT Madras. We
searched for a suitable book or any material to use, but found nothing useful. Our
conception of what needs to be taught in the course seemed to differ widely from
what is currently on offer. We figured that "computational thinking" means differ-
ent things to different people. For some it is just another name for programming
in a suitable language. Those with some knowledge of computer science see it as
the study of algorithms (as distinct from the programs used to implement them).
Then there are those who equate it to solving logical or mathematical puzzles
using computers. None of these descriptions seem satisfactory.
Luckily, one of us had experimented with teaching computational thinking to
school students under the banner of his startup company Mylspot Education Ser-
vices. These sessions used data sets with each data element written out on a
separate card. Students try to find the answer to a computational question by work-
ing through the cards by hand. For instance, they could look at students’ marks
cards to award prizes, or look at shopping bills to identify the loyal customers,
or look at cards containing words from a paragraph to identify which person a
pronoun refers to.
The idea of using this hands-on method to explain the key concepts in compu-
tational thinking was appealing on many counts. The practical utility becomes
immediately obvious when interesting questions about commonly occurring and
familiar data sets can be answered. Many of the key abstractions underlying
computational thinking seem to surface naturally during the process of moving
cards and noting down intermediate values. Finally, the difference in complexity
of two alternate solutions can be directly experienced when tried out by hand on a
real data set.
The challenge for us was to do all this in an online course format with recorded
videos, where there is no possibility of interacting with students during the session.
Our response to this was to employ a dialogue format, where the two of us would
work together, posing questions and finding answers by working through the cards.
The NPTel team in IIT Madras rose to the challenge by setting up the studio with
one camera facing down on the work area and another recording our faces. The
feedback we got from students who watched the first few recordings was excellent,
and this gave us encouragement that the method seemed to be working.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Getting Started With

Computational Thinking

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

1. Introduction

1.1 What is Computational Thinking? . . . . . . . . . . . . . 13

1.1.1 Stepwise approach to problem solving . . . . . . . . 14
1.1.2 Finding re-usable patterns . . . . . . . . . . . . . . 15
1.2 Sample datasets . . . . . . . . . . . . . . . . . . . . . . . 16
1.2.1 Classroom dataset . . . . . . . . . . . . . . . . . . 16
1.2.2 Shopping bill dataset . . . . . . . . . . . . . . . . . 18
1.2.3 Words dataset . . . . . . . . . . . . . . . . . . . . . 25
1.2.4 Train timetable dataset . . . . . . . . . . . . . . . . 28
1.2.5 Numerical expressions dataset . . . . . . . . . . . . 28
1.2.6 Rectangles dataset . . . . . . . . . . . . . . . . . . 28
1.3 Organisation of the book . . . . . . . . . . . . . . . . . . 28
1.3.1 Basics: getting started with computational thinking . 28
1.3.2 Computational thinking for Data Science . . . . . . 29
1.3.3 Higher level computational thinking . . . . . . . . . 30
1.4 A guide to the digital companion . . . . . . . . . . . . . . 31

1.1 What is Computational Thinking?

We see computations all around us. When we scan the shelves in a supermarket for
a particular item, we unconsciously follow a previously learnt searching method
based on our knowledge of where different items are stored. A very similar search
may be witnessed when we look for an item in the kitchen, when we look for
some book on our bookshelves, or search for a document in our folders. Likewise,
when we make our to-do list in the morning and check it off one by one as we
carry out the tasks in it, the tracking system in use resembles the one where we
make a shopping list and score out the items which we have found and purchased,
or the one where we make an invitation list for a party and tick off the friends we
have already called.
Consider now the process of taking out of clothes from a bucket and hanging
them on multiple clothes lines. This process is strikingly similar to the one that
we will use when we pick out books from a box and place them on a multiple
shelves, or when we are given some travel items and are asked to pack them into
a set of suitcases. As a last example consider finding a road route to take when

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

14 Chapter 1. Introduction

lOMoARcPSD|44901534

18 Chapter 1. Introduction

The entire data set is contained in the following table:

No Name Gen DoB Town/City MA PH CH Tot

0 Bhuvanesh M 7 Nov Erode 68 64 78 210
1 Harish M 3 Jun Salem 62 45 91 198
2 Shashank M 4 Jan Chennai 57 54 77 188
3 Rida F 5 May Chennai 42 53 78 173
4 Ritika F 17 Nov Madurai 87 64 89 240
5 Akshaya F 8 Feb Chennai 71 92 84 247
6 Sameer M 23 Mar Ambur 81 82 87 250
7 Aditya M 15 Mar Vellore 84 92 76 252
8 Surya M 28 Feb Bengaluru 74 64 51 189
9 Clarence M 6 Dec Bengaluru 63 88 73 224
10 Kavya F 12 Jan Chennai 64 72 68 204
11 Rahul M 30 Apr Bengaluru 97 92 92 281
12 Srinidhi F 14 Jan Chennai 52 64 71 187
13 Gopi M 6 May Madurai 65 73 89 227
14 Sophia F 23 July Trichy 89 62 93 244
15 Goutami F 22 Sep Theni 76 58 90 224
16 Tauseef M 30 Dec Trichy 87 86 43 216
17 Arshad M 14 Dec Chennai 62 81 67 210
18 Abirami F 9 Oct Erode 72 92 97 261
19 Vetrivel M 30 Aug Trichy 56 78 62 196
20 Kalyan M 17 Sep Vellore 93 68 91 252
21 Monika F 15 Mar Bengaluru 78 69 74 221
22 Priya F 17 Jul Nagercoil 62 62 57 181
23 Deepika F 13 May Bengaluru 97 91 88 276
24 Siddharth M 26 Dec Madurai 44 72 58 174
25 Geeta F 16 May Chennai 87 75 92 254
26 JK M 22 Jul Chennai 74 71 82 227
27 Jagan M 4 Mar Madurai 81 76 52 209
28 Nisha F 10 Sep Madurai 74 83 83 240
29 Naveen M 13 Oct Vellore 72 66 81 219

Figure 1.2: Classroom dataset

BNo Shop Customer

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

lOMoARcPSD|44901534

1.2 Sample datasets 25

Item Category Qty Price Cost Total

Broom Household 1 70 70 340
Dustpan Household 1 45 45
Floor Cleaner Household 1 125 125
Tissue Paper Household 2 50 100

BNo Shop Customer

28 SV Stores Neeraja
Item Category Qty Price Cost Total
Milk Dairy/Food 3 24 72 92
Chips Packed/Food 1 20 20

BNo Shop Customer

29 SV Stores Vignesh
Item Category Qty Price Cost Total
Face Wash Toiletries 1 89 89 514
Shampoo Toiletries 1 140 140
Maggi Packed/Food 1 85 85
Chips Packed/Food 1 20 20
Chocolates Packed/Food 4 10 40
Air Freshener Toiletries 2 70 140

BNo Shop Customer

30 SV Stores Ahmed
Item Category Qty Price Cost Total
Chocolates Packed/Food 1 10 10 106
Curd Dairy/Food 2 32 64
Bananas Fruits/Food 4 8 32

60 with Preposition 4
61 his Pronoun 3
62 thin Adjective 4
63 long Adjective 4
64 cane . . . Noun 4

Figure 1.6: Words dataset

1.2.4 Train timetable dataset

1.2.5 Numerical expressions dataset

1.2.6 Rectangles dataset

1.3 Organisation of the book

The book is organised into three parts - the first part introduces the basic concepts
underlying computational thinking, the second part gets into the use of computa-
tional thinking in data science, and the last part takes the reader through a tour of
some of the advanced concepts in computational thinking.

1.3.1 Basics: getting started with computational thinking

If we are asked to find the one thing that differentiates a computer from a human
being, the answer that will most likely come up is - repetition. A computer can
do repetitive tasks easily, while a human will get bored when asked to repeat
something, and will surely make mistakes.
It is thus quite natural to start the exploration into the world of computing with
repetition. The key concept that we will use for this is the iterator introduced in
Chapter 2. The need to keep track of what is going on while repeating something
leads us naturally to the concept of a variable in Chapter 3. To be able to process
the data selectively, we bring in the concept of filtering in Chapter 4. To maintain
and keep track of different types of things with a reasonable level of sanity, we
introduce the concept of datatype in Chapter 5.
By allowing the filtering condition to keep changing during the iteration (repeti-
tion), we are able to carry out quite sophisticated computations, for instance to
search for something, or to find the best candidate meeting some criterion. These

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

1.3 Organisation of the book 29

investigations form the content of Chapter 6.

To be able to visualise these activities, we use a flowchart, that is first introduced
in Chapter 2 and developed with increasing complexity through the following
chapters. When we find the flowchart too cumbersome to draw, we introduce the
concept of pseudocode in Chaoter 7 to communicate textually what is contained
in the flowchart. Even the pseudocode can become needlessly long and messy, so
we organise it using procedures in Chapter 8, and find a way to re-use the same
pseudocode for different purposes using parameters.

1.3.2 Computational thinking for Data Science

Data science deals with sets of data drawn from the physical world. We could use
statistical tools to draw general inferences about these data sets. Such tools may
work on the entire data set or some random samples drawn from it. Before we
can apply these statistical tools, the data sets may need to be first re-organised and
formatted into tables.
We could also directly draw many useful (non-statistical) inferences by examining
the structure of the dataset captured through the tables above. Drawing such
inferences would require us to systematically process the entire dataset to look for
interesting relationships between the elements. This processing usually falls into
some well defined patterns of computational thinking, which we shall explore in
great detail through the chapters in the second part of this book.
We first take up in Chapter 9 the relationship between one data element and some
aggregate measure of the entire dataset. For instance, we could find students who
are doing well by comparing each student with the average.
We then take up the problem of determining the pairwise relationship between
data elements. This requires us to introduce nested iterations in Chapter 10.
Nested iterations are costly in terms of the time they take to complete execution,
so it is useful to find simple heuristics that could cut down the number of pairwise
comparisons that are to be carried out.
In Chapter 11, we look at scanning the dataset to collect information of relevance to
us and storing this information in a collection called a list. By examining multiple
lists together we are able to draw some useful inferences about the relationships
present in the data. The list could also be ordered for ease of processing (called
sorted lists), as shown in Chapter 13.
In Chapter 12, we discuss the problems that arise with procedures that have side
effects, and the care that needs to be taken while using such procedures. Chapter
13 on sorted lists will take advantage of this.
We then turn to clustering or binning the data elements into multiple bins through

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

30 Chapter 1. Introduction

what is called a dictionary described in Chapter 14. Combining dictionaries with

lists gives us yet more power to organise the information in a variety of interesting
ways. This organisation method described in Chapter 15 resembles the use of a
table, but improves on it.
Finally, we try to create a visual representation of the relationship using something
called a graph in Chapter 16. Questions about the dataset turn out to have answers
in terms of properties of these graphs. This gives us a powerful tool to cut the
clutter in the data and focus only on what is relevant to answer the question at
hand. Chapter 17 explains the different methods of representing a graph using
tables or lists and dictionaries, so that the properties we need to study can be
determined by suitable procedures on these representations.

1.3.3 Higher level computational thinking

To bring computational thinking closer to what we do in real life, we will need

what are called higher order computational concepts.
We first discuss the use of recursion, in which a procedure P can call itself (self
recursion), or can call other procedures, one of which calls P (mutual recursion).
It turns out that recursion allows many natural search procedures to be written
much more neatly. For instance to find a way to go from one station A to another
B using a sequence of trains, we could explore all the trains passing through A,
and for each of these trains, we could explore all the stations C on its route, which
in turn will explore trains passing through C, and so on, till we reach station B.
Recursion is discussed in Chapter 18. Chapter 19 looks at the tree structure that
is generated by the recursive procedure calls. The reverse also works - namely if
we need to use a tree structure to store intermediate information, then this could
be accomplished by the use of an appropriate recursion.
The tree structure that emerges when we use recursion to search a graph is called
a spanning tree, which is like the skeleton of the graph and can be very useful in
determining some of the graph properties. Spanning trees are discussed in Chapter
20.
We then move on to the observation that in real life, procedures are typically
carried out by someone or something - they don’t hang around on their own. The
idea of packaging procedures along with the data they need into objects is called
encapsulation. It lies at the core of object oriented thinking, a specific form
of computational thinking. In object oriented thinking, we ask an object to do
something using its data - it does so as best as it can, and returns some useful
result. The object may also consult other objects to get this task done. If all this
looks very familiar, it is because this is how humans do things. All the forms the
content of Chapter 21.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

1.4 A guide to the digital companion 31

In Chapter 22, we take a diversion into functional computational thinking, in

which we describe the solution to a problem through a set of rules, each of which
is an expression to be computed. The right rule to select depends on the pattern of
the input data, and so the selection involves pattern matching.
We have missed yet another significant facet of real life computing. In real life,
activities do not necessarily go on sequentially, on the other hand, most of the
times activities can take place simultaneously or in parallel with one another. This
facet is called concurrency. How do we think about concurrent computations?
This is discussed in Chapter 23. Since input and output from a computer system
is an inherently concurrent activity, it provides many use cases for us to study in
Chapter 24. We wrap up our discussion of concurrency in Chapter 25 by looking
at an example real time system consisting of a client connected to a remote web
server.
The procedures introduced up to now are all top down (i.e. start from a problem
definition and work out a solution). In Chapter 26, we take a bottom up approach
to problem solving - a technique made popular through machine learning. In this
approach, we try to generalise from a few example data elements and check if the
generalised model works on new data elements. If it does not work that well, then
we adapt the generalised model to fit the new data. In this way, the program is
generated through a process of learning from examples.
Chapter 27 concludes by summarising the contents of the book while providing
some tips for further reading in each of the areas.

1.4 A guide to the digital companion

Summary of chapter

Difference between logical thinking and computational think-

ing

Computational thinking is often mistaken for logical thinking. While we

need to think logically if we have to do any computational thinking, and
logical thinking does involve a sequence of reasoning steps, it would be
wrong to conclude that the two are the same.
The first major difference between the two is that logical thinking is deductive
while computational thinking is inductive. Deductive thinking goes from

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

2.1 Going through a dataset . . . . . . . . . . . . . . . . . . . 35

2.1.1 Starting . . . . . . . . . . . . . . . . . . . . . . . . 36
2.1.2 Picking and examining one data element . . . . . . . 36
2.1.3 Marking the data element as seen . . . . . . . . . . 36
2.1.4 Stopping . . . . . . . . . . . . . . . . . . . . . . . 37
2.1.5 Description of the iterator . . . . . . . . . . . . . . 37
2.2 Flowcharts . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.2.1 Main symbols used in flowcharts . . . . . . . . . . . 38
2.2.2 The generic Iterator flowchart . . . . . . . . . . . . 39
2.3 Iterator flowchart for cards . . . . . . . . . . . . . . . . . 40

2.1 Going through a dataset

The most common form of computation is a repetition. To find the total of the
physics marks of all the students, we can start with a total value of zero, then
repeatedly select one student and add the student’s physics marks to the total.
Similarly to determine the number of occurrences of a certain word W in the
paragraph, we can repeatedly examine the words one by one, and increment the
count if the examined word is W. Consider now that we have to arrange students
in a class by their heights. We can do this by making them stand in a line and
repeating the following step: pick any two students who are out of order (taller
person is before a shorter person), and simply ask them to exchange their places.
The pattern of doing something repetitively is called an iterator. Any such
repetition will need a way to go through the items systematically, making sure that
every item is visited, and that no item is visited twice. This in turn may require
the items to be arranged or kept in a certain way. For instance, to arrange students
by height, we may first have to make them stand in one line, which makes it easier
for us to spot students who are out of order with respect to height. We have to
know what to do at any step, and also how to proceed from one item to the next
one. Finally, we will need to know when to stop the iterator.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

36 Chapter 2. Iterator

2.1.1 Starting

The steps involved to setup the required context for the iterator to function is
collectively called initialisation. When we work with cards, the initialisation
typically consists of arranging all the cards in a single pile (may or may not be
in some order). For instance the marks cards set can be in any order at the start,
while for the paragraph words cards set, the words would need to be in the same
sequence as they appear in the paragraph.
We may also need to maintain other cards where we write down what we are
seeing as we go through the repetition. But this forms part of the discussion in the
next chapter. Once we have setup the context for the iterator, we can proceed to
the steps that need to be repeated.

2.1.2 Picking and examining one data element

Within the repeated step, we will first have to pick one element to work on. For an
arbitrarily ordered pile of cards, any one card can be picked from the pile - the top
card, the bottom card, or just any card drawn from the pile at random. If the card
pile is ordered (as in the words data set), then it has to be the top card that has to
be picked from the pile.
The card that is picked is kept aside for examination. Specific fields in the card
may be read out and their values processed. These values may be combined with
other values that are written down on other cards (which we will come to in the
next chapter). The values on the card could also be altered, though we do not
consider examples of this kind in this book. since altering the original data items
from the data set can lead to many inadvertent errors.
When we use datasets in tabular form (rather than physical cards), picking a card
corresponds to extracting one row of the table. Extracting would mean that the
table is reduced by one row (the one extracted). In the case of the ordered datasets,
we will always extract the top row of the table.

2.1.3 Marking the data element as seen

Once we are done examining the chosen element, we need to move on to the next
data element to process. But that would be exactly the same as what we did in the
previous step. For instance with cards, we just have to pick another card from the
pile and examine it. So can we just repeat the previous step many times?
In principle we could, but we have not really said what we would do with the card
(or extracted table row) that we have kept aside for examination. Would we just

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

2.1 Going through a dataset 37

return that card to the pile of cards after doing whatever we need to do with its
field values? If we did that, then we will eventually come back to the same card
that we have picked earlier and examine it once again. Since we don’t want to
repeat the examination of any card, this means that we cannot return the examined
card to the original pile.
What we can do is to create a second pile into which we can move the card that
we have examined. Think about this second pile as the pile of "seen" cards, while
the original pile would be the pile of "unseen" cards. This ensures that we will
never see the same card twice, and also ensures that all the cards in the dataset will
definitely be seen. Thus the simple process of maintaining two piles of cards - the
original "unseen" pile and the second "seen" pile ensures that we systematically
visit all the cards in the pile, without visiting any card twice.
Note that the second pile will also need to be initialised during the initialisation
step of the iterator. If we are working with cards, this just consists of making some
free space available to keep the seen cards. If we are working with the tabular
form, it means that there is a second empty "seen" table kept next to the original
"unseen" table of rows, and at each step we extract (remove) one row from the
"unseen" table and add the row to the "seen" table.

2.1.4 Stopping

Finally, we have to know when we should stop the iteration. With cards, this
naturally happens when there are no further cards to examine. Since we are
moving cards from one pile (the "unseen" pile) to another (the "seen" pile), the
original "unseen" pile keeps getting smaller and smaller and eventually there will
be no more cards in it to examine. We have to stop at this stage as the step of
picking a card to examine cannot be performed anymore.
In the tabular form, extracting a row makes the table smaller by one row. Eventu-
ally, the table will run out of rows and we have to stop because we cannot perform
the extraction step.

2.1.5 Description of the iterator

To summarise, the iterator will execute the following steps one after the other:
• Step 1: Initialisation step: arrange all the cards in an "unseen" pile
• Step 2: Continue or exit? if there are no more cards in the "unseen" pile,
we exit otherwise we continue.
• Step 3: Repeat step: pick an element from the "unseen" pile, do whatever
we want to with this element, and then move it to another "seen" pile.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

38 Chapter 2. Iterator

• Step 4: Go back to Step 2

Note that the step that checks whether we should continue is written before the
step in which we are picking a card for examination, unlike how we described it
earlier. This accommodates the strange situation in which the pile given to us is
empty to start with. Something like this is never likely to happen when we deal
with natural data (which will need to have at least one element), but could happen
with other intermediate data sets that we nay generate and iterate over. It is just a
convenient way of representing all the possible situations that can occur.
Note also that the step that causes repetition is Step 4, which directs us to go back
to the previous step (Step 2) and do it all over again. We could also have written
here "Repeat Steps 2 and 3 until we reach exit in step 2" - which is in essence
what going back to Step 2 means.

2.2 Flowcharts

The step-wise description of the iterator that we saw in the previous section can be
visualised nicely using a diagram called a flowchart. The flowchart was developed
to describe manual decision making processes - for instance to diagnose what
went wrong with a piece of electrical equipment or in an instruction manual to
describe the steps for installing a piece of equipment. They are also useful to
explain a protocol or procedure to another person, who could be a customer or a
business partner.
Our eye is trained to look for visually similar patterns (this is how we identify
something to be a tree or a cat for example). The flowchart thus gives us a visual
aid that may be useful to spot similar looking patterns much more easily. This in
turn may allow us to re-use a solution for one problem to another similar problem
situation from a different domain.

2.2.1 Main symbols used in flowcharts

We will mainly be using only 4 symbols from those that are typically found in
flowcharts. These symbols are shown in Figure 2.1 along with the explanation of
where they need to be used. The box with rounded edges is a terminal symbol
used to denote the start and end of the flowchart. The rectangular box is used
to write any activity consisting of one or more steps. The diamond is used to
represent a decision, where a condition is checked, and if the condition is true,
then one of the branches is taken, otherwise the other branch is taken. The arrow
is used to connect these symbols.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

2.2 Flowcharts 39

terminal symbol the start or end of the flowchart

process symbol any activity that needs to be performed

decision symbol check some condition and branch based on true/false

shows progress from one step to another

Figure 2.1: Four key symbols of flowcharts

2.2.2 The generic Iterator flowchart

Using the flowchart symbols in Figure 2.1 we can now draw a generic flowchart
for the iterator using the description in Section 2.1.5. The result is shown in Figure
2.2. After start comes the initialisation step, followed by a decision if the iteration
is to continue or stop. If it is to stop it goes to the end terminal. Otherwise, it goes
to the repeat step after which it returns to the decision box.
We can visualise easily the iteration loop in the diagram which goes from the
decision box to the repeated steps and back. Any diagram which has such a loop
can be modeled using an iterator pattern.

Start

Initialisation step

Continue No (False)
End
iteration?

Yes (True)
Step to be repeated

Figure 2.2: Generic iterator flowchart

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

40 Chapter 2. Iterator

2.3 Iterator flowchart for cards

The flowchart for a generic iterator can be modified to create one specifically for
cards. We need to keep all the cards in an "unseen" pile, and have some free space
for the "seen" pile. The decision step involves checking if there are any more
cards in the "unseen" pile. If so, we pick one card from the "unseen" pile, examine
it, and then move it to the "seen" pile. The result is shown in Figure 2.3.

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

More cards in No (False)

"unseen" pile? End

Yes (True)
Examine one card from the "unseen"
pile and and move it into the "seen" pile

Figure 2.3: Iterator flowchart for cards

Summary of chapter

We can iterate over any collection. Specifically the collection could be a set,
or a relation (set of ordered pairs), or a more complex multi-dimensional
relation (set of ordered tuples). For instance, we could iterate over the
collection of all pairs of cards drawn from the marks card data set. A more
interesting example may be iteration over all possible subsets of the marks
card data set.
Iterators and complexity: examining the structure of the iterator immedi-
ately gives us some clue about the order of complexity of the procedure we
are implementing using the iterator. If we are examining all elements of the
card set, it will be O(N). If over all pairs, it will be O(N 2 ), over all k-tuples,
it will be O(N k ), and over all subsets, it will be O(2N ).

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

2.3 Iterator flowchart for cards 41

Iterators and programming language design: Some notes on iterating

over different kinds of collections: a number N can be viewed as the template
for a higher order function that can apply its first argument N times to its
second argument. This can be very neatly encoded in the lambda calculus, a
formal language that is used to model all possible computational functions.
Which means that the number N has a computational meaning, which is
"iterating N times". Likewise a generic list without any elements is an iterator
that goes through the list doing things with each element. In this, we can
construct much more interesting "higher order" analogues to data objects,
which make the objects come alive and be active.
Iterators found in programming languages: How programming languages
model iteration: Fortran, Python, C, C++, Java, Javascript, Haskell

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

3. Variables

3.1 Generic flowchart for iterator with variables . . . . . . . 43

3.2 Counting . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 Sum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3.1 Generic flowchart for finding the sum . . . . . . . . 45
3.3.2 Find the total distance travelled and total travel time
of all the trains . . . . . . . . . . . . . . . . . . . . 47
3.3.3 Find the total area of all the rectangles . . . . . . . . 47
3.4 Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.4.1 Generic flowchart for average . . . . . . . . . . . . 48
3.4.2 Harder example: Centre of gravity of rectangles . . . 49
3.4.3 Collecting values in a list . . . . . . . . . . . . . . . 49
3.5 Accumulator . . . . . . . . . . . . . . . . . . . . . . . . . 50

In the previous chapter, we saw how the iterator can be used to repeat some activity
many times. Specifically, it can be used to go through the cards systematically,
ensuring that we visit each card exactly once. We now look at doing something
with the iterator. In order to get any result through iteration, we will need to
maintain some extra information. The main vehicle to do this is the variable.
When we work with cards, we can use an extra card to note down this intermediate
information, so the extra card could be independently numbered or named and
would represent a variable. Alternatively, to allow for the card to be better used,
we can note down multiple pieces of intermediate information by giving each
variable a name, and noting down the intermediate information against each of
the named variables on the card. Unlike a field of the card, whose value does not
change, and so is called a constant, the value of the variable can keep changing
during the iteration, which would explain why we call it a variable.
If we are using tables to represent the information, then we need some extra space
somewhere to record the variable names and their intermediate values.

3.1 Generic flowchart for iterator with variables

We now modify the iterator flowchart for cards shown in Figure 2.3 to include
variables to store intermediate information. The basic structure of the iterator
remains the same, since we have to examine each card exactly once. But we

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

44 Chapter 3. Variables

will need to initialise the variables before the iteration loop starts, and update the
variables using the values in the chosen card at each repeat step inside the iteration
loop. The result is shown in Figure 3.1 below.

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise the variables

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Update the variables using the values in X

Figure 3.1: Generic flowchart for iterator with variables

3.2 Counting
Lets start with the simplest of problems - counting the number of cards in the pile.
To find the number of cards, we can go through the cards one at a time (which the
iterator will do for us). As we go through each card, we keep track of the count of
the cards we have seen so far. How do we maintain this information? We can use
a variable for this (written on an extra card). Let us name this variable count.
The iterator has an initialisation step, a repeat step and a way to determine when to
stop. What would we initialise the variable count to at the start of the iteration?
Since the variable count stores the number of cards seen so far, it needs to be
initialised to 0, since at the start we have not seen any cards. At the repeat step,
we need to update the value of the variable count by incrementing it (i.e. by
increasing its value by 1). There is nothing extra to be done to determine when to
stop, since we will stop exactly when there are no more cards to be seen.
The resulting flowchart is shown in Figure 3.2.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

3.3 Sum 45

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise count to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Increment count

Figure 3.2: Flowchart for counting cards

3.3 Sum
The counting example in the previous section did not require us to examine the
contents of the cards at all. We merely moved the cards from one pile to another
while keeping track of how many we have seen in a variable called count. We
now look at something a bit more interesting - that of finding the sum of the values
of some field on the card. In the case of the classroom dataset, we could find
the sum of the total marks scored by all students put together, or we could find
the total marks of all students in one subject - say Maths. In the shopping bill
dataset, we could find the total spend of all the customers put together. In the
words dataset, we could find the total number of letters in all the words.

3.3.1 Generic flowchart for finding the sum

How do we modify the flowchart in Figure 3.2 to find the sum of some field value?
First, we observe that the overall structure of the flowchart is exactly the same.
We need to systematically go through all the cards, visiting each card exactly once.
This is achieved by putting all the cards in the "unseen" pile, and moving it to the
"seen" pile after examining it. However, instead of just keeping count of the cards,

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

46 Chapter 3. Variables

we need to do something else.

Instead of the variable count, we could maintain a variable sum to keep track of
the total of field values of the cards seen so far. When we have seen no cards, sum
is 0, so we can initialise sum to 0 at the initialisation step of the iterator. So far,
everything is pretty straightforward and not very different from the flowchart for
counting.
The key step that needs to be different is the update step inside the iteration loop.
During this step, we have to examine the card we have picked (lets call it card X)
and pick out the value of the relevant field. For a field named F, let X.F denote the
value of the field named F in the card named X. Then the update step will require
us to add X.F to sum.
The resulting flowchart is shown in Figure 3.3. The flowchart for sum is exactly
the same as that for counting, except that we have named the element picked in
the update step as X, and in the update step, we add the value of the relevant field
X.F to the variable sum.
To find the total marks of all students, we can replace X.F in the flowchart above
with X.totalMarks. Similarly, to find the total spend of all customers, we can
replace X.F by X.billAmount, and for finding the total number of all letters, we
can use X.letterCount.

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise sum to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Add X.F to sum

Figure 3.3: Generic flowchart for sum

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

3.4 Average 47

3.3.2 Find the total distance travelled and total travel time of
all the trains

3.3.3 Find the total area of all the rectangles

3.4 Average

Suppose now that we wish to find the average value of some field from all the
cards in the dataset. For example we could find the average physics marks of all
students, or the average bill amount, or the average letter count.
How do we determine the average? We need to find the total value of the field from
all the cards and divide this total by the number of cards. We used the flowchart
in Figure 3.3 to find the value of the total represented through the variable sum.
Likewise, the number of cards was represented by the variable count whose value
was found using the flowchart in Figure 3.2. We can get the average now by
simply dividing sum by count.
The problem with the procedure describe above is that it requires us to make two
passes through the cards - once to find sum and once again to find count. Is it
possible to do this with just one pass through the cards? If we maintain both
variables sum and count at the same time while iterating through the cards, we
should be able to determine their values together. The flowchart shown in Figure
3.4 below does this.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

48 Chapter 3. Variables

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise sum to 0 and count to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Add X.F to sum and Increment count

Store sum/count in average

Figure 3.4: Flowchart for finding average

3.4.1 Generic flowchart for average

The flowchart above works, but seems a bit unsatisfactory. The computation of
average is inside the iteration loop and so keeps track of the average of all the
cards seen so far. This is unnecessary. We can just find the average after all the
cards are seen. The generic flowchart for finding average shown in Figure 3.5
does exactly this.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

3.4 Average 49

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise sum to 0 and count to 0

More cards in No (False)

"unseen" pile?

Yes (True)
Pick one card X from the "unseen" pile Store sum/count
and and move it into the "seen" pile in average

Add X.F to sum and Increment count End

Figure 3.5: Generic flowchart for finding average

We can use this generic flowchart for finding the average physics marks of all
students - just replace X.F in the flowchart above by X.physicsMarks. Similarly
to find the average letter count, we can replace it by X.letterCount.

3.4.2 Harder example: Centre of gravity of rectangles

3.4.3 Collecting values in a list

What are the intermediate values that we can store in a variable? Is it only numbers,
or can we also store other kinds of data in the variables? We will discuss this
in more detail in Chapter 5. But for now, let us consider just the example of
collecting field values into a variable using what is called a list. An example of a
list consisting of marks is [46, 88, 92, 55, 88, 75]. Note that the same element can
occur more than once in the list.
The flowchart shown below in Figure 3.6 collects all the Physics marks from all
the cards in the list variable marksList. It is initialised to [] which represents the
empty list. The operation append M to marksList adds a mark M to the end of
the list - i.e. if marksList is [a1 , a2 , a3 , ..., ak ] before we append M, then it will
be [a1 , a2 , a3 , ..., ak , M] after the append operation.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

50 Chapter 3. Variables

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise marksList to []

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Append X.physicsMarks to marksList

Figure 3.6: Flowchart for collecting all the Physics marks

3.5 Accumulator

Note the similarity between the flowchart for sum and that for collecting the list of
physics marks. In both cases we have a variable that accumulates something - sum
accumulates (adds) the total marks, while marksList accumulates (appends) the
physics marks. The variables were initialised to the value that represents empty -
for sum this was simply 0, while for marksList this was the empty list [].
In general, a pattern in which something is accumulated during the iteration is
simply called an accumulator and the variable used in such a pattern is called an
accumulator variable.
We have seen two simple examples of accumulation - addition and collecting
items into a list. As another example, consider the problem of finding the product
of a set of values. This could be easily done through an accumulator in which
the accumulation operation will be multiplication. But in all of these cases, we
have not really done any processing of the values that were picked up from the
data elements, we just accumulated them into the accumulator variable using the
appropriate operation (addition, appending or multiplication).
The general accumulator will also allow us to first process each element through

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

3.5 Accumulator 51

an operation (sometimes called the map operation) which is then followed by the
accumulation operation (sometimes also called the reduce operation).
For instance consider the problem of finding the total number of items purchased
by all the customers in the shopping bill dataset. Each shopping bill has a list
of items bought, of which some may have fractional quantities (for example 1.5
kg of Tomatoes or 0.5 litres of milk), and the remaining are whole numbers (e.g.
2 shirts). Since it is meaningless to add 1.5 kg of tomatoes to 0.5 litres of milk
or to 2 shirts, we could simply take one item row in the shopping bill to be a
single packed item - so the number of item rows will be exactly the number of
such packed items purchased. This is not very different from what happens at a
supermarket when we buy a number of items which are individually packed and
sealed. The watchman at the exit gate of the supermarket may check the bags to
see if the number of packages in the bill match the number of packages in the
shopping cart.
In this example, the map operation will take one bill and return the number of
packages (item rows) in the bill. The reduce operation will add the number of
packages to the accumulator variable, so that the final value of the accumulator
variable will be the total number of packed items purchased.

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False)
Select card X ?

Yes (True)
Update the variables using the values in X

Figure 4.1: Generic flowchart for iterator with filtering

4.2 Examples

Let us now consider examples of applying filtering to find something useful from
the datasets.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.2 Examples 55

4.2.1 Total spend of one person

Suppose we want to find the total spend of one of the customers (lets say Ahmed).
We can do this by examining all the shopping bills, filtering it by the customer
name field (checking if the customer name is Ahmed) and then accumulating the
bill amounts in an accumulator variable (lets call this variable ahmedSpend). The
flowchart for this is shown in Figure 4.2.

Start

Initialisation: Arrange shopping bill cards in

"unseen" pile; keep free space for "seen" pile

Initialise ahmedSpend to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False) X.customerName
is Ahmed ?

Yes (True)
Add X.billAmount to ahmedSpend

Figure 4.2: Flowchart for finding the total spend of Ahmed

4.2.2 Number of verbs

Suppose we want to find the number of verbs in the words dataset. This is similar
to the customer spend problem, and we should be able to just modify the flowchart
4.2 to create one for this problem. The variable we need to maintain can be called
numVerbs. It can be initialised to 0 at the initialisation step.
We need to make two more changes - change the filtering condition (replacement
for Is X.customerName Ahmed?), and change the accumulation step (replace-
ment for Add X.billAmount to ahmedSpend).

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

56 Chapter 4. Filtering

We are looking only for verbs and can ignore all other cards. So the filtering
condition should be: X.partOfSpeech is Verb ? The update operation is pretty
straightforward, we just increment verbCount (or equivalently, we add 1 to
verbCount). The resulting flowchart is shown in 4.3.

Start

Initialisation: Arrange words cards in "un-

seen" pile; keep free space for "seen" pile

Initialise verbCount to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False) X.partOfSpeech
is Verb ?

Yes (True)
Increment verbCount

Figure 4.3: Flowchart for finding the number of verbs

4.2.3 Sum of both girls’ marks and boys’ marks

Suppose now that we want to find the total marks of the boys and girls separately.
One approach would be to do a filtered iteration checking for boys (flowchart is
similar to Figure 4.3), filter condition being X.gender is M. We store the sum in
a variable boysTotalMarks. We then proceed to find the girlsTotalMarks in
exactly the same way, except that we check if X.gender is F.
The issue with this method is that it needs two passes through the cards, once for
the boys and once again for the girls. Can we find both in a single pass? For this,
we first note that when we check for the filtering condition (X.gender is M), we
are using only one branch - the branch for Yes (True). The other branch for No

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.2 Examples 57

(False) is not used, it simply takes us back to the beginning of the loop. Now if
the result of the check X.gender is M returns No (False), then it must be the case
that the gender is F (since gender can only have one of two values M or F). So we
could put some activity on that branch. This is illustrated in Figure 4.4.

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise boysTotalMarks to 0
and girlsTotalMarks to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Add No (False)
X.totalMarks to X.gender is M ?
girlsTotalMarks
Yes (True)
Add X.totalMarks to boysTotalMarks

Figure 4.4: Flowchart for finding total marks for both boys and girls

4.2.4 Total revenues of each shop

In the shopping bills dataset, we could try to find the total revenues earned by each
shop. Note that there are three shops - SV Stores, Big Bazaar and Sun General,
so we have to find the total value of all the bills generated from each of these
shops. Let svr, bbr and sgr be variables that represent the total revenues earned
by SV Stores, Big Bazaar and Sun General respectively. We can run one pass
with multiple filtering conditions, one following another, first check for SV Stores,
then Big Bazaar, and finally for Sun General. This is shown in Figure 4.5.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

58 Chapter 4. Filtering

Start

Initialisation: Arrange shopping bill cards in

"unseen" pile; keep free space for "seen" pile

Initialise svr to 0, bbr to 0 and sgr to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Add Yes (True) X.shopName

X.billAmount
is SV Stores ?
to svr
No (False)

Add Yes (True) X.shopName is

X.billAmount Big Bazaar ?
to bbr

No (False)

Add Yes (True)

X.billAmount X.shopName is
to sgr Sun General ?

No (False)

Figure 4.5: Flowchart for finding the revenues of all the shops

Note that to make the diagram easier to read, the Yes (True) and No (False)
branches are switched as compared to the single customer spend flowchart we saw
in Figure 4.2.
Observe the flowchart carefully. The straight path through all the filtering condi-
tions in which the shop name is not SV Stores, Big Bazaar or Sun General is not
actually possible. However, for completeness it is always good to express all the
conditions in the flowchart. If for instance the shop name is wrongly written, or a
new shop name is introduced, the flowchart will still work correctly.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.2 Examples 59

4.2.5 Check if a student is at the median

Suppose we want to check if a specific student S is near the median of total marks
- i.e. if the students are arranged in increasing order of total marks, the student is
somewhere in the middle. Is it possible to do this without first ordering the cards?
Note that the student will be at the median (or in the middle) if the number of
students scoring less than the student is more or less equal to those scoring equal
to or greater than the student. This can be easily determined by keeping a variable
balanced for the difference between these two numbers. If we see a student
below, we decrement balanced, and if we see a student equal to or above, we
increment balanced. The value of balanced can be used to determine whether
the student is in the middle (for instance any value which is between -5 and 5 may
be an acceptable definition of middle).
The flowchart in Figure 4.6 implements this, in which ST is the constant that holds
the student S’s total marks. The flowchart just returns the variable balanced and
leaves it to the user to interpret whether the student is in the middle or not.

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise balanced to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Increment No (False)
X.totalMarks < ST ?
balanced

Yes (True)
Decrement balanced

Figure 4.6: Flowchart for checking if a student is at the median

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

60 Chapter 4. Filtering

4.2.6 Number of trains on a specific week day

4.2.7 Number of operators in an expression

4.2.8 Harder Example: Number of words in the sentences

Lets consider a slightly harder example now if finding the number of words in all
the sentences of the words dataset.
There are many ways in which this example differs from the ones we have seen
so far. Firstly, when we pick a card from the pile, we will always need to pick
the topmost (i.e. the first) card. This is to ensure that we are examining the cards
in the same sequence as the words in the paragraph. Without this, the collection
words will not reveal anything and we cannot even determine what is a sentence.
Secondly, how do we detect the end of a sentence? We need to look for a word
that ends with a full stop symbol. Any such word will be the last word in a
sentence. Finally, there are many sentences, so the result we are expecting is not a
single number. As we have seen before we can use a list variable to hold multiple
numbers. Lets try and put all this together now.
We use a count variable to keep track of the words we have seen so far within
one sentence. How do we do this? We initialise count to 0 at the start of each
sentence, and increment it every time we see a word. At the end of the sentence
(check if the word ends with a full stop), we append the value of count into
a variable swcl which is the sentence word count list. Obviously, we have to
initialise the list swcl to [] during the iterator initialisation step.
The resulting flowchart is shown in the Figure 4.7 below.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.3 Compound conditions 61

Start

Initialisation: Arrange words cards in order in

"unseen" pile; keep free space for "seen" pile

Initialise count to 0 and swcl to []

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick the first card X from the "unseen"
pile and and move it into the "seen" pile

Increment count

No (False) X.word ends

with full stop ?

Yes (True)
Append count to swcl and Set count to 0

Figure 4.7: Flowchart for finding the word count of all sentences

Note that we do increment of count before the filtering condition check since the
word count in the sentence does not depend on whether we are at the end of the
sentence or not. The filtering condition decides whether the count needs to be
reset to 0 (to start counting words of the next sentence), but before we do the reset,
we first append the existing count value to the list of word counts swcl.

4.3 Compound conditions

In the previous examples, the filtering was done on a single field of the card. In
many situations, the filtering condition may involve multiple fields of the card.
We now look at examples of such compound conditions.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

62 Chapter 4. Filtering

4.3.1 Sum of Chennai girls’ marks

As the first example, lets say we want to find the total marks of all the girls from
Chennai in the classroom dataset. This requires us to check for two conditions -
gender is F and town/city is Chennai. We can build the flowchart for this in the
same way that we did the shop revenues flowchart in 4.5, where the conditions are
checked one after another. The resulting flowchart is shown in Figure 4.8, where
the accumulator variable cgSum holds the required sum of the Chennai girls total
marks. Filtering conditions of this kind can be called nested since the loop for
one condition contains the loop for the next.

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise cgSum to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False)
X.gender is F ?

Yes (True)

No (False) X.townCity
is Chennai ?

Yes (True)
Add X.totalMarks to cgSum

Figure 4.8: Flowchart using nested conditions

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.3 Compound conditions 63

4.3.2 Compound conditions instead of nested conditions

As we saw in the Chennai girls sum example in Figure 4.8, when we need to check
two different fields of the card, we can use nested conditions where one condition
is checked after another. This however seems unnecessarily complicated for
something that looks a lot simpler. Can we not simply check for both conditions
together within the filter decision box?
The way to check for more complex decisions (maybe using multiple data fields)
is to use compound conditions. Compound conditions are created from the basic
conditions by using the operators AND, OR or NOT (called Boolean operators).
The redrawn flowchart for finding the sum of all Chennai girls’ total marks using
a compound condition in the filtering decision box is shown in Figure 4.9 below.

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise cgSum to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False) X.gender is F AND

X.townCity is Chennai ?

Yes (True)
Add X.totalMarks to cgSum

Figure 4.9: Flowchart using compound condition

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

64 Chapter 4. Filtering

4.3.3 Boys born in the first half of the year

Compound decisions may be required even when we have to look at only one field
of the card, for instance when we have to check if the value of that field is within
a certain range. Consider the problem of finding the number of students who have
marks between 60 and 80 in Maths. We have to check if the Maths marks is above
60 (first condition) and that it is below 80 (second condition).
Say we have to find the number of students who were born in the first half of
the year, i.e. any day from Jan 1 to Jun 30. However, we have a problem here -
while we could compare a students marks with 60 or 80, can we really compare
a student’s date of birth with two given dates? This will require date to be a
number or something else that can be compared. We will discuss this issue in
more detail in Chapter 5. For now, assume that d1 ≤ d2 works for two dates d1
and d2 . The flowchart in Figure 4.10 uses the accumulator countBFH with an
appropriate compound condition.

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise countFBH to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False) X.gender is M AND

Jan 1 ≤ X.birthDate AND
X.birthDate ≤ Jun 30 ?

Yes (True)
Increment countBFH

Figure 4.10: Flowchart for number of boys born in the first half of the year

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.3 Compound conditions 65

4.3.4 Number of high scorers

Let us say we want to find the number of students who are able to score high
marks in at least one subject, where high marks means higher than 90. How would
we go about finding this? We have to check the Maths marks, the Physics marks
and the Chemistry marks to see if any one of them is above 90 - so we cannot
use compounding using the AND Boolean operator as we did in the last example.
Luckily, there is another Boolean operator OR which does exactly this - checks if
one or more of the component conditions are true. The required flowchart using
OR for finding the count of highscorers countHS is shown in Figure 4.11.

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise countHS to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False) X.mathsMarks > 90 OR

X.physicsMarks > 90 OR
X.chemistryMarks > 90 ?

Yes (True)
Increment countHS

Figure 4.11: Flowchart for counting the high scorers

4.3.5 Sequence of conditions is not the same as AND or OR

We saw that nested conditions can be replaced by AND. At first glance, we may
think that a sequence of conditions making the same update to a variable but with

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

66 Chapter 4. Filtering

different conditions behaves like an OR. Consider for example the flowchart stub
in Figure 4.12. This is clearly the same as taking the OR of the three conditions
as shown in 4.13.

No (False)
X.mathsMarks > 90 ?

Yes (True)
Set highScorer to True

No (False)
X.physicsMarks > 90 ?

Yes (True)
Set highScorer to True

No (False)
X.chemistryMarks > 90 ?

Yes (True)
Set highScorer to True

Figure 4.12: Part of flowchart showing sequence of conditions

X.mathsMarks > 90 OR No (False)

X.physicsMarks > 90 OR
X.chemistryMarks > 90 ?

Yes (True)
Set highScorer to True

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.3 Compound conditions 67

Figure 4.13: Part of flowchart showing OR of conditions

We can quickly check that two flowchart stubs in Figures 4.12 and 4.13 are exactly
the same. In both cases, the Boolean variable highScorer is set to True if at least
one of the three conditions holds.
Now consider the sequence of conditions shown in the stub 4.14. Is this the same
as the compound condition shown in the stub 4.15?

No (False)
X.mathsMarks > 90 ?

Yes (True)
Increment countHS

No (False)
X.physicsMarks > 90 ?

Yes (True)
Increment countHS

No (False)
X.chemistryMarks > 90 ?

Yes (True)
Increment countHS

Figure 4.14: Part of flowchart showing sequence of conditions

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

68 Chapter 4. Filtering

X.mathsMarks > 90 OR No (False)

X.physicsMarks > 90 OR
X.chemistryMarks > 90 ?

Yes (True)
Increment countHS

Figure 4.15: Part of flowchart showing OR of conditions

The answer is No ! The flowchart stubs are clearly not equivalent. To see this,
consider the case where a student has scored more than 90 in two subjects - say
Maths and Physics. Then the stub 4.14 will increment countHS twice once for
the condition X.mathsMarks > 90 and again for the condition X.physicsMarks
> 90. What will the stub in 4.15 do? It will increment countHS only once (since
there is only one compound condition that is checked). So the resulting countHS
of 4.14 will be 1 more than that of 4.15.
Why is it that the update to a Boolean variable highScorer worked, while
incrementing the variable countHS did not? The difference between the two is
that setting the Boolean variable highScorer to True twice is just the same as
setting it once, whereas incrementing the variable countHS twice is not the same
as incrementing it once.
This argument also tells us that if the conditions are mutually disjoint (i.e. it is
never possible for any two of them to be true at the same time), then the sequence
of conditions with exactly the same update operation can be replaced by an OR of
the conditions.
In general, a sequence of conditions may or may not resemble OR - we have to
analysed the updates more carefully before determining if they are the same or
not.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.4 Looking for a data element 69

4.3.6 Number of rectangles containing a point

4.4 Looking for a data element

Suppose we want to search for any one card that satisfies some property specified
through a condition.
We could in principle just iterate through all the cards looking for a card which
satisfies the desired condition. When such a card is found, it is set aside, but we
continue through all the remaining cards till the iterator finishes its job. While this
works, it seems wasteful to continue with the iteration when we have found what
we want. Can’t we just stop the iterator when we have finished what we want to
do, which is finding some card? The problem is that we have not said how we can
stop an iterator - we can start it, we can do something in each step and we can
wait till it is over. So far, we are not allowed to stop it mid way.
Let us look at this through an example.

4.4.1 Search for a high scoring student

Let us say that we want to search for a high scoring student, defined as someone
who has scored more than 90 in all the subjects. The filtering condition we are
looking at can be written as X.mathsMarks > 90 AND X.physicsMarks > 90 AND
X.chemistryMarks > 90, which does not change across iterations (except of course
that we are applying the same condition to a different X each time). We can write
a simple filtered iterator that checks for this condition and accumulates all the
student names that satisfies this condition in a accumulator variable that is a list
called highscorers. But this is wasteful as we are not asking to find all students
that satisfies the condition, we are only asking for any one student who does.
If we want to stop the flowchart when the condition is satisfied, all we will need to
do is to exit the loop in the iteration and go from there straight to the End terminal
in the flowchart. This is shown in the Figure 4.16 below.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

70 Chapter 4. Filtering

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise highScorers to []

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False) X.mathsMarks > 90 AND

X.physicsMarks > 90 AND
X.chemistryMarks > 90 ?

Yes (True)
Append X.name to highScorers

Figure 4.16: Flowchart for searching for a high scorer

Note that rather than go back to the start of the iteration after the append step in
the last activity box, we have exited the iteration loop by going directly to the End
terminal.

4.4.2 Simplifying the flowchart using compound condition

The flowchart in Figure 4.16 is not very satisfactory as there are two places from
which the iterator can exit - either when the iterator is done with its job (visited all
the cards) or when the item we are searching for is found. In this example, the step
immediately following the completion of the iterator is the End of the procedure,
so it did not turn out that bad. However, when we have to do something with the
values we generate from the iteration (as we saw for example in the example to
find the average in Figure 3.5), we have to know exactly where to go when we
exit from the loop.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

4.4 Looking for a data element 71

Can we make the exit happen at the same decision box where the iterator checks
for end of iteration? For this we need to extend the exit condition to include
something that checks if the required card has been found. We can do this by
using a new Boolean variable called found, which is initialised to False and turns
True when the required card is found. We then add found to the exit condition
box. This is illustrated in the flowchart in 4.17.

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise highScorers to [] and found to False

More cards in "unseen" pile No (False)

End
AND NOT found ?

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False) X.mathsMarks > 90 AND

X.physicsMarks > 90 AND
X.chemistryMarks > 90 ?

Yes (True)
Append X.name to highScorers
Set found to True

Figure 4.17: High scorer search with single exit point

4.4.3 Generic flowchart for find/search

We can now write the generic flowchart for searching for a card, which can be
adapted for other similar situations. The generic flowchart is shown in Figure
4.18.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

72 Chapter 4. Filtering

Start

Initialisation: Arrange student marks cards in

"unseen" pile; keep free space for "seen" pile

Initialise variables, found to False

More cards in "unseen" pile No (False)

End
AND NOT found ?

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False)
X is the required card ?

Yes (True)
Update variables
Set found to True

Figure 4.18: Generic flowchart for searching for a card

4.4.4 Harder Example: Find a rectangle that is cut in half by

a given line

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5. Datatypes

5.1 Sanity of data . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.1.1 Classroom dataset . . . . . . . . . . . . . . . . . . 75
5.1.2 Shopping bill dataset . . . . . . . . . . . . . . . . . 76
5.1.3 Words dataset . . . . . . . . . . . . . . . . . . . . . 76
5.1.4 Trains dataset . . . . . . . . . . . . . . . . . . . . . 77
5.1.5 Expressions dataset . . . . . . . . . . . . . . . . . . 77
5.1.6 Rectangles dataset . . . . . . . . . . . . . . . . . . 77
5.2 Basic datatypes . . . . . . . . . . . . . . . . . . . . . . . . 77
5.2.1 Boolean . . . . . . . . . . . . . . . . . . . . . . . . 77
5.2.2 Character . . . . . . . . . . . . . . . . . . . . . . . 78
5.2.3 Integer . . . . . . . . . . . . . . . . . . . . . . . . 78
5.3 Compound datatypes . . . . . . . . . . . . . . . . . . . . 79
5.3.1 Strings . . . . . . . . . . . . . . . . . . . . . . . . 79
5.3.2 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.3.3 Records . . . . . . . . . . . . . . . . . . . . . . . . 80
5.4 Subtypes . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.4.1 Subtypes of Character . . . . . . . . . . . . . . . . 81
5.4.2 Subtypes of Integer . . . . . . . . . . . . . . . . . . 81
5.4.3 Subtypes of String . . . . . . . . . . . . . . . . . . 82
5.5 Transforming the data element . . . . . . . . . . . . . . . 83
5.5.1 Date . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.5.2 Fractional values: quantity, price and marks . . . . . 84
5.6 Datatypes for the elements in the dataset . . . . . . . . . 86
5.6.1 Student datatype . . . . . . . . . . . . . . . . . . . 86
5.6.2 Word datatype . . . . . . . . . . . . . . . . . . . . 87
5.6.3 Shopping bill datatype . . . . . . . . . . . . . . . . 87
5.6.4 Train datatype . . . . . . . . . . . . . . . . . . . . 89
5.6.5 Station datatype . . . . . . . . . . . . . . . . . . . . 89
5.6.6 Expression datatype . . . . . . . . . . . . . . . . . 89
5.6.7 Rectangle datatype . . . . . . . . . . . . . . . . . . 89

In the datasets that we have considered so far, the field names and values where
all correctly written. This is usually not the case in real life, where the actual data
is entered by humans who are prone to making mistakes. Field names may be

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

74 Chapter 5. Datatypes

misspelt, data values may have errors in them, and the field names and values may
not be properly matched.
The problem with erroneous data is that it creates a significant overload on the
computational process. Before we process any data element, we would need to
first check the sanity of the data in the card or table row. Every such possible error
will need a decision box in the flow chart to identify the error. Then we will need
to know what to do if an error is found - do we just ignore the card, or do we try to
rectify the error and then go ahead. What are the rules of rectification, i.e. which
values would replace the wrong ones?
The flowcharts or pseudocode will be so full of all these kinds of decisions that the
main logic/pattern of the problem solution will be lost in all the noise. Worse, when
so much checking has to be done, it is quite possible that some will be left out, or
the wrong checking logic is implemented in the flowchart or pseudocode. This
would let an erroneous data element to pass through the flowchart or pseudocode
and give rise to all kinds of unexpected behaviour.
Even if the data elements are correctly written, we could have made errors in how
the values in the data element are used within our procedures. For instance, we
could attempt to add two name fields, or multiply one date field value with another,
or subtract one word from another - all of which are meaningless operations that
are very likely to produce some nonsense values. These values may be used later
in the procedure, resulting in totally unpredictable outcomes.
Would it not be nice if we could simply specify what it means for the data
element to be sane, and what operations are allowed to be performed on it? The
development environment of programming system that processes our procedures
should be able to take these specifications and automatically check that they are
complied with. If not, they can flag errors at processing time, so that the errors
can be rectified. This would greatly relieve the writing of the flowcharts or pseudo
code, which would no longer need to check for any of these kinds of errors.
In this chapter, we will first look at the different conditions that we can place on
the field values in our datasets and the operations allowed to be performed on
them. This will lead us to the concept of a datatype, which lets us define clear
specifications to be written about the data items and the operations permitted on
them.

5.1 Sanity of data

Let us first look at some of types of fields we can find in the datasets. Some of the
fields hold numbers - like marks, prices, amounts, distances etc. Many of them,
for example names, are sequences of alphabet characters, perhaps separated by

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.1 Sanity of data 75

a space or some special character (like a full stop). We can also see items like
the date field which seem to be like numbers, but are written with non-numeric
characters - for example Jun 1. We now go through all of these data elements
systematically, identifying valid values and operations for each of them.

5.1.1 Classroom dataset

There are four types of fields in the classroom data element:

• Marks: which is a natural number taking values in the range 0 to 100 (in
the case of subjects), and 0 to 300 in the case of total marks. he marks field
should allow addition and subtraction, but multiplication or division of one
mark by another does not make any sense. We should also be allowed to
compare two numbers - check if they are equal, less than, greater than etc.
In terms of the field names itself, we expect the subject field names to be
valid subjects.
• Name, City: Is a sequence of characters which need to be from the alphabet
(A-Z or a-z) along with some spacing characters, like blank, full stop or
hyphen. We can compare one name or city with another to check for
equality, but cannot do any other operation with it. We also expect that the
city/town name is from the list of recognised names for cities/towns.
• Gender: is a single character with two possible values - M or F. Again, the
only operation allowed should be equality checking.
• Date of birth: is a date, which we have abbreviated to hold only the month
and the date in the month. The month can take one of 12 values, one for
each month (Jan, Feb, ..., Dec) and the date can take numerical values from
1 to 31 only. We should be able to compare two dates for equality, and to
check if one comes before another (so less than, greater than etc should be
possible).
There is yet another implicit constraint that the classroom data element must
satisfy, which is that the total marks must be equal to the sum of the three subject
marks. If this is not the case, then it implies either that one or more of the subject
marks are wrongly entered, or that a mistake was made in computing the total.
Note that we do not really need to provide the total - it can always be computed
from the subject marks. However, we often do provide such redundant fields in
data elements precisely for the purpose of sanity checking. If we did not have the
total marks written, then some errors in the entry of marks could go undetected.
For example, while we will immediately detect a wrong entry with value -5 or
105 (since both these are outside the range of valid values), we will not be able to
detect an entry error where the Physics marks (say 45) is wrongly entered also in
the Maths field (which should be say 65). But if total is provided, we can catch
this error since the total will correctly add 65 for Maths and not 45. Note that

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

76 Chapter 5. Datatypes

this does not ensure that we catch all such errors though - for instance the Maths
and Physics marks may get erroneously interchanged. This will not be caught by
providing the total, since the total remains the same.

5.1.2 Shopping bill dataset

The shopping bills dataset has these types of fields:

• Quantity, price, amount: are fractional numbers, with possibly 2 decimal
places (but not more). We can set some upper limits for each of these.
Addition and subtraction is allowed, but not multiplication or division.
• Names of customer, shop and item categories: Is exactly like the name field
in the classroom dataset.
In this example as well, we have some redundant values. The amount on each line
item should be equal to price multiplied by quantity. Then the total bill amount
must equal the sum of all the line item amounts.

5.1.3 Words dataset

The words dataset has these types of fields:

• Letter count: is a number. We can set an upper limit for this based on the
longest word in the dictionary. Addition and subtraction is allowed, but not
multiplication or division.
• The word itself and the category: Is exactly like the name field, but addi-
tional special characters (like comma, semicolon etc) may be allowed for
the word. The word should be of the right category, which can also be
checked using a dictionary.
Here too we have a redundant value. The letter count needs to be exactly equal to
the alphabet letters in the word.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.2 Basic datatypes 77

5.1.4 Trains dataset

5.1.5 Expressions dataset

5.1.6 Rectangles dataset

5.2 Basic datatypes

The most natural way of preventing wrong data entries or operations on the data
fields is to associate what is called a datatype which each of them.
A datatype (or simply type) is a way of telling the computer (or another person)
how we intend to use a data element:
• What are the values (or range of values) that the element can take ?
• What are the operations that can be performed on the data element ?
Thus when we specify that a variable is of a specific type, we are describing the
constraints placed on that variable in terms of the values it can store, and the
operations that are permitted on it.
There are three basic datatypes that we have used in our datasets. These are:
• Boolean: for holding for instance the True or False values of the filtering
conditions
• Integer: for holding numerical values like count, marks, price, amount etc
• Character: for holding single character fields like gender which has M and
F as values
We now look at each of these in turn.

5.2.1 Boolean

An element of the Boolean datatype takes one of only two values - True of False.
So any attempt to store any value other than True or False in a variable of Boolean
datatype should result in an error.
What are the operations that should be allowed on elements of this datatype? We
need to be able to combine Boolean datatype values and also check if a given
Boolean value is True or False. The table given below lists some of the possible
operations possible on the Boolean datatype.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

78 Chapter 5. Datatypes

Operation Number of args Resulting type

AND 2 Boolean
OR 2 Boolean
NOT 1 Boolean
= 2 Boolean

5.2.2 Character

An element of the Character datatype take the following kinds of values:

• Alphabet: Capital letters A,B,...,Z or small letters a,b,...,z
• Numeral: Digits 0,1,...,9
• Special characters: Separators like blank, comma, fullstop, semicolon,
colon, hyphen, underscore, slash, backslash, exclamation mark, question
mark; other special characters like $,#, %, @, ...
The only operation that makes sense for characters is check for equality, which
returns Boolean.

Operation Number of args Resulting type

= 2 Boolean

5.2.3 Integer

An Integer datatype element take the values ...,-3,-2,-1,0,1,2,3,... (i.e. the integer
can be a negative number, zero or a positive number.
We can add, subtract, multiply integers. We can also compare two integers. It may
or may not be possible to divide one integer by another, unless we define a special
integer operation that takes only the quotient after division. These operations are
shown in the table below:

Operation Number of args Resulting type

+ 2 Integer
− 2 Integer
× 2 Integer
÷ 2 Integer
= 2 Boolean
> 2 Boolean
< 2 Boolean

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.3 Compound datatypes 79

5.3 Compound datatypes

We can construct more involved datatypes from the basic datatypes using datatype
constructors. The resulting datatypes are called compound datatypes. The three
main kinds of compound datatypes that we have used in our datasets are:
• strings: are just sequences of characters of arbitrary length. Our datasets
have names of people, shops and cities, shopping item names, categories,
and words from a paragraph - all of which are strings.
• lists: are a sequence of elements, typically all of the same datatype. Our
datasets have for example lists of items in the shopping bill.
• records: are a set of named fields along with their associated value which is
of some specific datatype. Each card in our dataset is an example of a record
- the student marks card, the shopping bill, the word from the paragraph.
We examine each of them in more detail below.

5.3.1 Strings

A string datatype is made by "stringing" together a sequence of characters. There

can be any number of characters in the string. There is also no restriction on the
types of characters that may be present in the string - we could have letters from
the alphabet, numerals or special characters. It is not necessary that the string that
we make should make any sense when seen as a word. We will later introduce the
notion of subtype to place appropriate restrictions on the string datatype to make
it more suitable to be used for example as a name.
What are the operations that are allowed on strings? We should be able to append
or concatenate one string to another to make a longer string. We should also
be able to check if one string is equal to another, or is present at the beginning
or at the end or somewhere within another string. The table below shows these
operations:

Operation Number of args Resulting type

append 2 String
= 2 Boolean
startsWith 2 Boolean
endsWith 2 Boolean
contains 2 Boolean

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

80 Chapter 5. Datatypes

5.3.2 Lists

Can we collect together several elements, each of which is of some datatype?

There are two mechanisms to do this: lists and records.
In lists, the elements in the collection are not named, and have to be identified
by their position in the list. For instance, in the list L = [a1 , a2 , a3 , ..., ak ], we can
pick out a3 , by going down the list L from the beginning till we get to the third
element.
Typically, all elements in a list are of the same datatype, though this is not strictly
necessary. There is also no restriction on the length of the list.
What are the operations that should be allowed on lists? Firstly, we should be able
to check if the list is the empty list []. We should be able to make a longer list
by putting two lists together (append one list to another), or by inserting a new
element at the beginning of the list. We should also be able to take out the first
element from the list.

Operation No of args Resulting type What it does

append 2 List Appends one list to another
add 2 List Add an element at start of list
= 2 Boolean Check if two lists are equal
isEmpty 1 Boolean Check if the list is empty
head 1 Element’s type First element of the list
tail 1 List List with first element removed

Note that the last two operations head and tail are possible only if the list is not
empty.

5.3.3 Records

Unlike lists, a record is a collection of named fields, with each field having a
name and a value. The value can be of any other datatype. Typically there is no
restriction of any kind in terms of the number of fields or on the datatypes of the
fields.
What operations would we want to perform on the record? The one that we will
need to use for our datasets is that of picking out any field using its name. This
operation is simply denoted by ".", where X.F returns the value of the field F from
the record X. The result type is the same as the type of the field F.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.4 Subtypes 81

5.4 Subtypes
A subtype of a datatype is defined by placing restrictions on the range of values
that can be taken, limiting the kinds of operations that can be performed, and
maybe adding more constraints on the element to ensure that there is sufficient
sanity in the data values.
We now look at each of the subtypes that we have used in our datasets.

5.4.1 Subtypes of Character

The gender field in the classroom dataset is an example of a subtype of the
Character basic datatype. This field is allowed only one of two character values,
M or F. The equality operation of the Character datatype can also be applied to
the Gender subtype.

5.4.2 Subtypes of Integer

There are a number of places in our datasets where we have used integers with
restrictions. These are listed below:
• Sequence no: should have only positive values (negative values and 0 is
disallowed). Let us name this datatype SeqNo. Since we are not likely to
have a very large number of data items in the dataset, we could also set a
MAX value for the upper limit of the range of data values. None of the
operations that are allowed on integers (+, −, ×, ÷) make sense for SeqNo.
The only meaningful operation is the = operation that checks if two SeqNo
elements have the same value.
• Marks: should have values starting from 0 (i.e. negative numbers are not
allowed). For the subject marks, we call the datatype Marks and let its
value range from 0 to 100 (since in our dataset, subject marks cannot exceed
100). For the total marks field, we can call the datatype TMarks and let
its values range from 0 to 300. While we can add and subtract marks,
multiplication and division of one mark with another mark makes no sense
at all. All the comparison operations (=, > and <) can now be applied to
marks.
• Count: allows 0 or any positive integer value (negative numbers are disal-
lowed) upto a sufficiently high limit MAX. The words dataset has a field
letter count. The variable count was also used to hold the number of cards
seen so far in iterations. We can simply write Count to represent this
datatype. We can add or subtract counts, but multiplication and division
does not make sense. Comparison opertaions are all possible.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

82 Chapter 5. Datatypes

The table below summarises all of these restrictions.

Subtype Values allowed Operations allowed

SeqNo 0..MAX =
Marks 0..100 +, −, =, <, >
TMarks 0..300 +, −, =, <, >
Count 0..MAX +, −, =, <, >

5.4.3 Subtypes of String

We have used strings in our datasets with many restrictions. These are listed
below:
• Names: should have only alphabet characters and could have blank or
fullstop characters additionally to separate the parts of the name. Let us call
this datatype Names. All the string operations can be allowed for Names.
We could also add some new operations - for instance to extract the first
and last name.
• City: is exactly the same as Names. We could call this subtype City.
• Words: All strings can be considered as words, including those with nu-
merals and special characters. Let us call this subtype Words. All string
operations are valid for Words
• Word category: The values need to be specific strings from the set WordCat-
egoryNames = {"Noun", "Pronoun", "Verb", "Adjective", ...}. Let us call
this datatype WordCategory. Except =, all the other operations should be
disallowed for this subtype.
• Items: values need to be specific strings which represent legitimate item
names, ItemNames = {"Carrots", "Soap", "Socks", ...}. As for WordCate-
gory, only the = operation is admissible. Let us call this datatype Items.
• Item category: Values need to be specific strings form the set ItemCategory-
Names = {"Vegetables/Food", "Toiletries", "Footwear/Apparel", ...}. Again,
except =, all other operations should be disallowed. Let us call this subtype
ItemCategory.
The table below summarises all of these restrictions.

Subtype Values allowed Ops allowed

Names Only alphabet chars with blank, fullstop All
City Only alphabet chars with blank, fullstop All
Words All chars allowed All
WordCategory WordCategoryNames Only =
Items ItemNames Only =
ItemCategory ItemCategoryNames Only =

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.5 Transforming the data element 83

5.5 Transforming the data element

We have more or less taken care of most of the fields present in our datasets. But
there are a few tricky ones that we have not yet dealt with - the date of birth field
in the classroom dataset, the fractional quantity and prices in the shopping bills.
Even for the marks field which consists of whole numbers, the average marks
could be fractional.
In each of these cases, we have many options for storing the data. Some of these
options may not really suitable as they may not permit the kinds of operations that
we want to perform on them. We may have to first transform the data element
to ensure that we can easily store it, while allowing the operations we want to
perform on it.

5.5.1 Date

Let us first consider the date of birth field. We have chosen only to represent the
month and date (within the month) values, and have ignored the year of birth. This
is sufficient for the questions we want to ask (mostly to determine birthday within
the year).
Now, a typical value of this field, for instance "6 Dec", looks like a string. So it is
probably natural for us to store this field as a string, with suitable restrictions on
values that can be taken. However, using a string is not very satisfactory since it
will not allow us to compare one date D1 with another D2 to check if D1 < D2
(which would mean that D1 comes earlier in the year than D2). We may also want
to subtract D2 from D1 (to find how many days are there between birth date D1
and D2). All of this means that we need something other than string to represent
date.
We can apply the following simple transformation to the date field to make it
suitable for the comparison and subtraction operations. Since we are only looking
at the month and date within the month, we can represent the date using a single
whole number representing the number of days starting from Jan 1, which is
taken as day 0. So Jan 6 would be represented by the number 5, Jan 31 would
be represented by 30 and Feb 1 would be represented by 31. What should be the
number representing Mar 1? This depends on whether the year is a leap year (in
which Feb will have 29 days) or not. This additional bit of information is required
to correctly store the date of birth as a whole number.
Let us say it is not a leap year, so that Feb has 28 days. Then Mar 1 will be
represented by the number 59 (31 days in Jan and 28 days in Feb adds to 59).
What is the number that should be associated with Dec 6? Note that the number

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

84 Chapter 5. Datatypes

used to represent a date is just the number of days in the calendar year preceding
that date. We can thus determine the associated date for all the dates in the calendar.
Dec 31 will have the number 364 associated with it (since that is the number of
days preceding Dec 31 in the calendar). So Dec 6 should be represented by 339.
Since date is stored as a whole number, we can perform comparison operations on
it easily. To check if one date D1 precedes another D2, we just have to check that
the number corresponding to D1 is less than that corresponding to D2. Similarly
to find the number of days between two calendar dates, we can just subtract their
associated whole numbers.
We will also need two convenience operations to go between strings and numbers.
The first store(D) will convert a date string D (say "6 Dec") into its number
equivalent - this will require us to add up all the days in the months Jan to
Nov and add 5 to it to get the number 339 representing "6 Dec". In the other
direction, print(N) will take a number in the range 0 to 364, and print the date
string corresponding to it. This is important to do, otherwise any external user
will have to go through the tedious and error prone task of finding out which date
we are referring to using a number. Thus, store("6 Dec") = 339, and print(339) =
"6 Dec".
Note now that after this transformation, date just becomes an integer, so we can
perform any integer like operations on it. Of course, multiplying two dates does
not make any sense, so we should place restrictions on the operations that can be
performed. The values also have to be constrained to lie within the range 0 to 364.
To take care of these constraints, we could define date as a subtype of integer. All
the operations needed to work on the date field can now be shown in the table
below:

Operation No of args Result type What it does

= 2 Boolean Checks if two dates are equal
< 2 Boolean Checks if one date precedes another
− 2 Integer Gap between two dates
store 1 Integer Stores string in a date subtype
print 1 String Converts date subtype into a string

5.5.2 Fractional values: quantity, price and marks

In the shopping bill dataset, the price, quantity, cost and final bill amount could
all be fractional values. We could try storing these in an integer after rounding off
the fractional value to the nearest integer, but this means that some information is
lost.
There is yet another basic datatype that could be used for storing fractional values,

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.5 Transforming the data element 85

which is called float. However, this is a very elaborate datatype that can allow a
very wide range of values from the very small or the very large (both of which
would need exponents to write). We have some simple values with at most two
fractional decimal places, so the use of float may be excessive for us. Is there
something simple we could do?
The simplest way of dealing with this would be to simply multiply the data values
by 100. This will make the value into an integer (for instance 62.75 will be
converted into 6275, which is an integer). Like in the date example, we will need
two convenience functions store(62.75) which will return 6275 to store it as an
integer subtype, and print(6275) which will convert the integer subtype into a
string "62.75" which can be presented when needed.
We can now define Quantity, Price, Cost and Amount as subtypes of Integer.
What operations should be allowed on these subtypes? Clearly addition should be
allowed, since we can accumulate quantities and costs. Subtraction could also be
allowed to determine for instance price difference between two items.
But multiplication and division do not make any sense for these quantities. Luckily,
this is rather convenient for us, because if we needed to multiply two fractional
numbers, then our convenient mechanism of multiplying them by 100 would fail
! Note that print ( store(a) × store (b) ) is not the fractional value a × b, since it
multiplies both a and b by 100 and so when we take their product, it is multiplied
by 10000 in the subtype, and print only reduces this product by 100, so the result
is 100 times too large. Division does exactly the reverse, it makes the number 100
times too small after division. Since we don’t need to multiply or divide, we do
not have to worry about these anomalies introduced due to our transformation.
We can also compare any two of these subtypes using =, < or >. The operations
allowed on the subtypes Quantity, Price, Cost and Amount are shown in the table
below:

Operation No of args Result type What it does

+ 2 Integer Add two elements
− 2 Integer Subtract two elements
= 2 Boolean Compares for equality
< 2 Boolean Compared for less than
> 2 Boolean Compared for greater than
store 1 Integer Stores string in the subtype
print 1 String Converts subtype into a string

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

86 Chapter 5. Datatypes

5.6 Datatypes for the elements in the dataset

We now have suitable datatypes for all the fields in our datasets. Let us put these
together to make the right datatypes for each of the data elements in our dataset.
The main instrument for us to construct the datatype for the entire data element
is the record, since each data element has some named fields which takes values
from some datatype. Let us see what kind of records are produced for each of the
datasets.

5.6.1 Student datatype

Let us take a relook at the classroom dataset element, which is the marks card of
one student.

Figure 5.1: Grade card data element

We note that the card carries the following fields, so we can create a record
datatype StudentMarks whose field names along with their respective datatypes is
shown below:
• uniqueId: SeqNo
• studentName: Names

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.6 Datatypes for the elements in the dataset 87

• gender: Gender
• dateOfBirth: Date
• townCity: City
• mathsMarks: Marks
• physicsMarks: Marks
• chemistryMarks: Marks
• totalMarks: TMarks
What should be the datatype for the entire classroom dataset ? Clearly it is simply
the compound datatype: List of StudentMarks.

5.6.2 Word datatype

Let us now turn to the Words dataset, an element of which is pictured below.

Figure 5.2: Words data element

Clearly, we can make a record datatype ParagraphWord for this, whose field names
and respective types are as shown below:
• uniqueId: SeqNo
• word: Words
• partOfSpeech: WordCategory
• letterCount: Count
What should be the datatype for the entire Words dataset ? Clearly it is simply the
compound datatype: List of ParagraphWord.

5.6.3 Shopping bill datatype

Consider now the shopping bill dataset element pictured below.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

88 Chapter 5. Datatypes

Figure 5.3: Shopping bill data element

We can attempt to make a record for this. The fields which are obvious are shown
below along with their respective datatypes.
• uniqueId: SeqNo
• storeName: Names
• customerName: Names
• billAmount: Amount
But what do we do with the items purchased? Note that there are a number of
purchased items, and that the number may vary from bill to bill. The obvious
datatype to use for this is a List. But a list of what type?
Each purchased item has the item name, category name/sub category name,
quantity purchased, price per unit and cost of item purchased. So the item itself
looks like a record. Let us try to make a record datatype called BillItem with fields
and datatypes as shown below:
• itemName: Items
• category: ItemCategory
• quantity: Quantity
• price: Price
• cost: Cost

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

5.6 Datatypes for the elements in the dataset 89

Now that we have the datatype for a line in the shopping bill, we can make a
List of BillItem and use that as a field in the shopping bill. The final record
ShoppingBill will have the following fields:
• uniqueId: SeqNo
• storeName: Names
• customerName: Names
• items: List of BillItem
• billAmount: Amount
What should be the datatype for the entire shopping bill dataset ? Clearly it is
simply the compound datatype: List of ShoppingBill. Note that the shopping bill
dataset is a list, each of its elements has within it another list of type BillItem. So
it is a list of lists.

5.6.4 Train datatype

5.6.5 Station datatype

5.6.6 Expression datatype

5.6.7 Rectangle datatype

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

6. Filtering: dynamic conditions

6.1 Maximum . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.1.1 Collect all the max elements . . . . . . . . . . . . . 92
6.2 Minimum . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.3 Examples using maximum or minimum . . . . . . . . . . 96
6.3.1 Find the length of the longest sentence . . . . . . . . 96
6.3.2 Harder Example: Find the boundary rectangles . . . 97
6.4 Combining static, dynamic and state conditions . . . . . 97
6.4.1 Largest noun in the third sentence . . . . . . . . . . 98
6.4.2 Checking if a customer is loyal . . . . . . . . . . . . 100
6.4.3 Example: . . . . . . . . . . . . . . . . . . . . . . . 102

The filtering conditions in the examples considered in Chapter 4 were all of

the type where we compared a element’s field with a constant value. In other
words, the filtering conditions were constant and not varying across successive
repeat steps of the iterator. In this chapter, we look at situations where we have
to compare the element’s field values with the value of a variable (rather than
a constant value). This would make the filtering condition itself change as the
variable changes its value, i.e. the filtering condition is dynamic.

6.1 Maximum
We first consider the problem of finding the maximum value of some (numerical)
field in the dataset. The method we are going to use for finding the maximum is
this: we keep a variable called max which will carry the maximum value seen so
far (for the desired field). At the end of the iteration, all the cards will have been
seen, and max will carry the maximum value of the field. When we pick up a card
X, we check if its field X.F has a value higher than the current value of max. If so,
max will need to be updated with this new high value X.F. Note that this means
that the value of max can only keep increasing as we proceed through the cards.
What should the initial value of max be? We could start with something that
is lower than all the field values. In our dataset, we don’t have any fields with
negative values, so we can safely initialise max to 0 at the start.
The generic flowchart for finding the maximum of field F is shown in Figure 6.1.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

92 Chapter 6. Filtering: dynamic conditions

Start

Initialisation: Arrange all the cards in "un-

seen" pile; keep free space for "seen" pile

Initialise max to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False)
X.F > max ?

Yes (True)
Set max to X.F

Figure 6.1: Flowchart for finding max value of field F

Note that the filtering condition is not static anymore, since the card’s field is
compared with a variable (max in this case), and not with a constant. The way to
view this is: the filtering condition in the beginning admits all cards (i.e. filters
out nothing). As the iteration progresses, it filters out cards with field values lower
than max, and picks only those that contribute to increasing the max.
We can use the generic flowchart above to find the maximum value of different
(numerical) fields by substituting that field in place of F in the generic flowchart
above - for instance to find the maximum Maths marks, we use X.mathsMarks in
place of X.F in the flowchart. Similarly, to find the bill with the highest spend, we
use X.billAmount and to find the longest word, we can use X.letterCount.

6.1.1 Collect all the max elements

The flowchart in Figure 6.1 produces the maximum value of field F, but it does
not tell us which cards contribute to this max value. For example, it will give us
the maximum total marks, but will not say which students scored these marks.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

6.1 Maximum 93

To remedy this, we can keep a second variable called maxCardId, which keeps
track of the unique sequence number of the card that holds the maximum field
value out of cards seen so far. What should maxCardId be initialised to? Any
value that will definitely not occur in the dataset should be fine. The simplest such
integer is -1, which can never be any card’s sequence number, which can only
have 0 or positive values. The required flowchart is shown below in Figure 6.2.

Start

Initialisation: Arrange all the cards in "un-

seen" pile; keep free space for "seen" pile

Initialise max to 0, maxCardId to -1

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

No (False)
X.F > max ?

Yes (True)
Set max to X.F and maxCardId to X.uniqueId

Figure 6.2: Finding the card that has the maximum value of a field

Is this flowchart correct? Suppose that the first 4 cards have fields shown below:

UniqueId 9 2 7 11
ChemistryMarks 65 82 77 82

Then max takes value 0 after initialisation and 65, 82, 82, 82 after the first, second,
third and fourth cards are read and processed. The first card sets max to 65. At the
second card, X.chemistryMarks > max, so max gets set to 82. In the third and
fourth cards, the Chemistry marks is not greater than 82, and so they are skipped.
What will the value of maxCardId be? It will be -1 after initialisation. After
the first card, it is set to 9. The second card sets it to 2, and since the third and

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

94 Chapter 6. Filtering: dynamic conditions

fourth card are skipped (filtered out), they don’t change the value of maxCardId.
So, the procedure is correct in the limited sense that it holds one student whose
Chemistry marks is the highest. But this is not so satisfactory since the fourth
card has exactly the same marks as the second card, and it should receive equal
treatment. This would mean that we have to keep both cards, and not just one. We
would need a list variable to hold multiple cards - let us call this list maxCardIds.
The modified flowchart is shown in Figure 6.3.

Start

Initialisation: Arrange all the cards in "un-

seen" pile; keep free space for "seen" pile

Initialise max to 0, maxCardIds to []

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Append No (False)
[X.uniqueId] X.F > max ?
to maxCardIds
Yes (True)
Set max to X.F and maxCardIds to [X.uniqueId]

Figure 6.3: Flowchart for finding all the maximum cards

Note that the list variable maxCardIds needs to be initialised to the empty list []
at the start. If we find a card N with a bigger field value than the ones we have
seen so far, the old list needs to be discarded and replaced by [N]. But as we have
seen above, if we see another card M with field value equal to max (82 in the
example above), then we would need to append [M] to the list maxCardIds.
We can apply this flowchart to find all the words with the greatest number of
letters in the Words dataset. For this, replace X.F with X.letterCount.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

6.2 Minimum 95

6.2 Minimum

In the last section, we saw how to find the maximum value of some field, and
the cards that hold that maximum value. Suppose now that we want to find the
minimum value instead of the maximum value. Does the same method work?
In principle it should. Instead of keeping a variable called max, we can keep
a variable called min which holds the minimum value of the required field. In
the update step, instead of checking whether the field F has a value larger than
max, we now check if the field value is lower than min. So far it looks pretty
straightforward.
The key question is: what do we initialise min with? In the case of max, we could
initialise with 0, since in our dataset there are no negative values in the numeric
fields. Since we know that min will keep decreasing as we go through the cards,
we could in principle start with the maximum value of the field across all the
cards - but this requires us to find the maximum first ! Rather than do that, we
just set min to some high enough value MAX that is guaranteed to be higher than
the maximum value of the field. Usually, the field’s subtype would have some
constraint on the range of values allowed - so we can simply take the upper end of
the range as the value of MAX.
In the classroom dataset, we saw that the subject marks had a maximum value of
100, so MAX would be 100 if are trying to find the minimum marks in a subject,
while the total marks had a maximum value of 300, so MAX can be set to 300 if are
trying to find the minimum total marks.
The flowchart in Figure 6.4 produces the minimum value of field F and the list of
cards carrying this minimum value.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

96 Chapter 6. Filtering: dynamic conditions

Start

Initialisation: Arrange all the cards in "un-

seen" pile; keep free space for "seen" pile

Initialise min to MAX, minCardIds to []

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Append No (False)
[X.uniqueId] X.F < min ?
to minCardIds
Yes (True)
Set min to X.F and minCardIds to [X.uniqueId]

Figure 6.4: Flowchart for finding all the minimum cards

6.3 Examples using maximum or minimum

We now look at examples of situations where the maximum or minimum is

combined with other conditions to make the filtering really dynamic.

6.3.1 Find the length of the longest sentence

Let us now consider a slightly more difficult problem - that of finding the length of
the longest sentence (i.e. the sentence with the largest number of words). For this,
we have to count the words in each sentence, and as we do this we have to keep
track of the maximum value of this count. The flowchart in Figure 4.7 gives us the
word count of all the sentences. Can we modify it to give us just the maximum
count rather than the list of all the sentence word counts? The required flowchart
is shown in Figure 6.5.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

6.4 Combining static, dynamic and state conditions 97

Start

Initialisation: Arrange words cards in order in

"unseen" pile; keep free space for "seen" pile

Initialise count to 0 and max to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick the first card X from the "unseen"
pile and and move it into the "seen" pile

Increment count

No (False) X.word ends

with full stop ?

Yes (True)
No (False)
Set count to 0 count > max ?

Yes (True)
Set max to count

Figure 6.5: Flowchart for finding length of largest sentence

6.3.2 Harder Example: Find the boundary rectangles

6.4 Combining static, dynamic and state conditions

We now look at finding something (for example the maximum or minimum) only
within a range of cards and not for the whole set of cards.
The obvious one is to pick cards which satisfy some (static) filtering condition - for
example we could find the largest verb in the paragraph by iterating over the cards,

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

98 Chapter 6. Filtering: dynamic conditions

filtering to check if the word’s part of speech is Verb, and within the verbs, we
find the word with the highest letter count. All we will need to do is to replace the
filtering condition X.F > max in Figure 6.1 with the condition X.partOfSpeech
is Verb AND X.letterCount > max. This is thus an example of filtering using
a combination of a static condition (X.partOfSpeech is Verb) along with a
dynamic condition (X.letterCount > max). As another example consider the
problem of finding the bill with the highest spend for a given customer named N.
We could check for the customer name (static condition X.customerName is N)
along with the dynamic condition X.billAmount > max.
Sometimes, we have to go beyond using a static condition to select the range of
cards to process. Suppose that as we go over the cards, the iteration passes through
distinct phases (each of which we can call a state). The state of the iteration
can come about because of certain cards that we have seen in the past. Without
state, we will have to use filtering only to check the values of the current card
(independent of what we have seen in cards in the past). In the examples that
follow, we will look at a few examples of the use of state - for instance to keep
track of the sentence number we are currently in, or to keep track of the last bill
seen of a certain customer.
A good example of the use of state is through the Boolean variable found for
exiting during the middle of an iteration that we say in Figure 4.18. The iteration
goes through two states - the state before the item was found (when found is
False) and the state after it is found (when found is True). However, in that
example, we did not do anything in the latter state, except exit the iterator. So the
use of state is not of much use in that example.
We could argue that keeping track of max is also like maintaining a state (the value
of these variables). But this is not very useful as a way to understand what is
going on - there are too many states (values of max), and the transition from one
state to another is not very logical (max can change to anything depending on the
card we are seeing). Compare this with the case of the sentence number - there
are only a few states (number of sentences in the paragraph), and the transition is
quite meaningful (sentence number increments by one during each transition).
So maintaining the state of iteration in a variable (or variables) is a way of
simplifying complex dynamic conditions (wherever possible) by separating the
part that depends on history (the state) from those that depend only on the current
card.

6.4.1 Largest noun in the third sentence

Let us see an example where we do a combination of filtering conditions - static,

dynamic and state. Let us say we want to find the longest noun in the third

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

6.4 Combining static, dynamic and state conditions 99

sentence. This requires us to select all the nouns - an example of static filtering
(since we will be comparing the field partOfSpeech with the constant "Noun").
But it also requires us to keep track of which sentence we are currently in (the
state), for which we will need to compare a variable snum for sentence number,
which is incremented each time we see a full stop. It also requires us to find the
maximum length word - which is a dynamic condition.
Since we are not interested in looking beyond the third sentence, we should exit
the iteration loop once we have finished with the third sentence. The exit condition
needs to take care of this (as we did in Figure 4.18 for exit after finding something).
The flowchart for doing all this is shown in Figure 6.6.

Start

Initialisation: Arrange words cards in order in

"unseen" pile; keep free space for "seen" pile

Initialise snum to 0 and max to 0

More cards in "unseen" No (False)

pile AND snum < 3? End

Yes (True)
Pick the first card X from the "unseen"
pile and and move it into the "seen" pile

snum is 2 AND X.partOfSpeech is No (False)

Noun AND X.letterCount > max ?

Yes (True)
Set max to X.letterCount

No (False)
X.word ends with full stop ?

Yes (True)
Increment snum

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

100 Chapter 6. Filtering: dynamic conditions

Figure 6.6: Length of the largest noun in the third sentence

Observe the flowchart closely. There are a number of tricks that have been used.
Firstly, to exit the flowchart after the third sentence, we could have used a Boolean
variable found as we did in the generic find flowchart in Figure 4.18. However,
this is not really necessary, as there is a good equivalent in the form of checking a
condition of the state (snum < 3). Note that the state represents if we are in the
first sentence (snum=0), second (snum=1) or third sentence (snum=2). Secondly,
note that all the conditions are being checked in a single decision box - check for
a state (snum=2 meaning that we are in the third sentence), the word is a noun
(static condition), and that the word is longer than max (the dynamic condition).
Only if all these are satisfied will max be updated. Thirdly, both branches of this
decision box converge to the check for end of sentence, which if true causes snum
to be incremented.

6.4.2 Checking if a customer is loyal

In the case of the words dataset, the words were assumed to be in an order
according to their occurrence in the paragraph. So using the count of the number
of sentences as a state variable made some sense. What kind of state information
can we carry when the cards are not in any order?
Let us now consider an example of cards that may not be any order - the shopping
bill dataset. Say we want to find out if a certain customer named N is loyal -
defined as follows: N is loyal if all the bills that have N as the customer name
all share the same shop name. This would mean that customer N shops only in
one shop (i.e. is loyal to that shop). How do we check if the given customer N is
loyal? As we iterate through the cards, we keep looking for the customer name
N. If we find such a card, we change the state (variable shop initially empty) to
the name of the shop on the card (say S). As we go through the remaining cards,
we keep looking for a card with customer name N (static condition) and also for
state not being empty (i.e. we have seen a previous card of the same customer),
and check if the card’s shop name will change the state again. If it will, then the
customer is not loyal and we can exit the iteration.
The flowchart is shown below in 6.7. The Boolean variable loyal starts with the
value True, and becomes false when a second shop is encountered for the same
customer name.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

6.4 Combining static, dynamic and state conditions 101

Start

Initialisation: Arrange shopping bill cards in order

in "unseen" pile; keep free space for "seen" pile

Initialise shop to "" and loyal to True

More cards in No (False)

"unseen" pile End
AND loyal?

Yes (True)
Pick the first card X from the "unseen"
pile and and move it into the "seen" pile

No (False)
X.customerName is N ?

Yes (True)

Set shop to No (False) shop 6= "" AND

X.shopName X.shopName 6= shop ?

Yes (True)
Set loyal to False

Figure 6.7: Flowchart for checking if a customer is loyal

A few points to note about the flowchart. The exit condition is strengthened with a
check for loyal. If we have found that the customer is not loyal, then the iteration
exits (nothing further to do). Within the iterator, we first check for the static
condition - whether the customer name is N. If not, we skip the card. If it is N,
then we check if the state shop is previously set and if so if it is different from the
current shop name - which cause loyal to be set to False. In any event, we set
the state shop to the shop name on the card. This is merely for convenience (so
we don’t have to write another decision box). If shop was "" earlier, then it will
take the shop name from the card. If the shop name remains the same, assigning it
to shop will not change the state. If the shop name is different, then shop is set to

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

102 Chapter 6. Filtering: dynamic conditions

the latest shop name (i.e. the state changes), but this is never used again because
we will exit the loop.

6.4.3 Example:

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

7. Pseudocode

7.1 Basic iteration . . . . . . . . . . . . . . . . . . . . . . . . 105

7.2 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

7.3 Compound conditions . . . . . . . . . . . . . . . . . . . . 108

7.4 Filtering with dynamic conditions . . . . . . . . . . . . . 109

Flowcharts provide an appealing pictorial representation of a computational pro-

cess. Their layout makes it easy to visualize the overall structure of the computa-
tion.
However, there are also some disadvantages with using flowcharts. The first is size:
complex processes generate large flowcharts. As the visual representation grows,
it is no longer possible to fit the entire picture on a single page, and digesting the
overall structure becomes harder. The second drawback is that pictures are not
convenient for collaboration. Computational processes are typically developed
in teams, and it is important to be able to share descriptions of these processes
in a format that can be edited and updated easily. A natural consequence of
collaboration is that the description will change over time, with updates made
by more than one person. It is important to be able to compare changes between
versions, which is difficult with flowcharts.
Let us revisit our simple flowchart for counting the number of cards in a pile.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

104 Chapter 7. Pseudocode

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise count to 0

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Increment count

Figure 7.1: Flowchart for counting cards

We can, instead, describe the process in words, as a sequence of steps.

Counting cards
Step 0 Start
Step 1 Initialize count to 0
Step 2 Check cards in Pile 1
Step 3 If no more cards, go Step 8
Step 4 Pick a card X from Pile 1
Step 5 Move X to Pile 2
Step 6 Increment count
Step 7 Go back to Step 2
Step 8 End

However, this text notation does not emphasize the logical structure of the compu-
tation. For instance, Steps 2, 3 and 7 together describe how we iterate over the
cards till all of them are processed. Steps 4, 5 and 6 are the steps that we perform
repeatedly, in each iteration.
Instead of using plain English to describe the steps, is is better to develop special-
ized notation that captures the basic building blocks of computational processes.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

7.1 Basic iteration 105

This specialized notation is called a programming language. We would expect it

to have more direct ways of describing conditional and repeated execution.
We will gradually build up such a programming language to describe the processes
that we encounter in our exploration of computational thinking. A computational
process written using a programming language is referred to as code and is
typically translated into low level basic instructions that can be executed by a
computer. The language we develop will be less formal than a real programming
language, so we refer to our notation as pseudocode. to emphasize that it is not
“real” code.

7.1 Basic iteration

Let us begin by representing our process to count cards using pseudocode.

Start
count = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
Increment count
}
End
Pseudocode 7.1: Counting cards

In this example, we have a few basic elements of pseudocode.

• The statement “count = 0” assigns a value 0 to the variable count. Not
surprisingly, this is called an assignment statement.
• The special word while signals an iteration where we repeatedly perform a
block of actions till some condition becomes false.
The condition to be checked is enclosed in brackets after the word while—
in this case, the condition is “Pile 1 has more cards”. If this condition
evaluates to true, the next round of iteration begins. If it is false, the
iteration terminates.
The scope of the iteration is demarcated by a pair of matching curly brackets,
or braces, “{“ and “}”. All statements between this pair of matching braces
will be executed in each iteration. Such a set of statements, enclosed within
a pair of matching braces, is often referred to as a block of code.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

106 Chapter 7. Pseudocode

With minor changes, we can modify this pseudocode to add up the values in a
particular field F across all the cards. Instead of incrementing a variable count
with each iteration, we update a variable sum by adding the value X.F to sum for
each card X that we read. Here is the pseudocode for the flowchart from Figure 3.3.

Start
sum = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
Add X.F to sum
}
End
Pseudocode 7.2: Generic sum

We can merge and extend these computations to compute the average, as we had
earlier seen in the flowchart of Figure 3.5.

Start
count = 0
sum = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
count = count + 1
sum = sum + X.F
}
average = sum/count
End
Pseudocode 7.3: Average

This pseudocode represents a variant of the computation described in Figure 3.4.

In that flowchart, average is updated with each card that we read, inside the
body of the iteration. In the pseudocode above, we only update count and sum
when we iterate over the cards. When we exit the iteration, we move to the first
statement after the closing brace that defines the scope of the iteration and use a
single assignment statement to store the final value of sum/count in average.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

7.2 Filtering 107

We have made one more change with respect to Pseudocode 7.1 and 7.2. The
increment to count and the update to sum are also now described using assignment
statements. The statement “count = count + 1” is not to be read as an equation—
such an equation would never be true for any value of count! Rather, it represents
an update to count. The expression on the right hand side reads the current value
of count and adds 1 to it. This new value is then stored back in count. In the
same way, the assignment “sum = sum + X.F” takes the current value of sum, adds
X.F to it, and stores the resulting value back in sum.

7.2 Filtering
The next step is to add a conditional statement that will allow us to do filtering.
Consider the flowchart for finding total marks for both boys and girls, shown in
Figure 4.4. Here is the corresponding pseudocode.

Start
boysTotalMarks = 0
girlsTotalMarks = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.gender == M) {
boysTotalMarks = boysTotalMarks + X.totalMarks
}
else {
girlsTotalMarks = girlsTotalMarks + X.totalMarks
}
}
End
Pseudocode 7.4: Total marks of both boys and girls

We have introduced a new word if to describe conditional execution to be per-

formed once, unlike while which captures steps to be performed repeatedly till the
given condition becomes false. As before, the condition to be checked appears in
brackets after the word if. In this case, we are checking whether the field gender
on the card, X.gender, is equal to M. Notice that we use an unusual symbol “==”
to denote equality, to avoid confusion with the operator “=” that assigns a value to
a variable. If the condition is true, the statements within the first set of braces are

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

108 Chapter 7. Pseudocode

executed. The block labelled else is executed if the statement is false. There may
be no else block. For instance, here is an example where we only sum the total
marks of girls. If X.gender is not equal to F, we don’t perform any action within
the body of the iteration.

Start
girlsTotalMarks = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.gender == F) {
girlsTotalMarks = girlsTotalMarks + X.totalMarks
}
}
End
Pseudocode 7.5: Total marks of girls

7.3 Compound conditions

We can combine conditions using AND, as we have done in flowcharts. Here is

the pseudocode for the flowchart in Figure 4.9 that computes the total marks of
girls from Chennai.

Start
cgSum = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.gender == F AND X.townCity == Chennai)) {
cgSum = cgSum + X.totalMarks
}
}
End
Pseudocode 7.6: Total marks of girls from Chennai

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

7.4 Filtering with dynamic conditions 109

7.4 Filtering with dynamic conditions

As a final example, let us look at filtering with dynamic conditions. Figure 6.2
describes a flowchart to find the card that has the maximum value of a field. Here
is the corresponding pseudocode.

Start
max = 0
maxCardID = -1
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.F > max) {
max = X.F
maxCardID = X.uniqueId
}
}
End
Pseudocode 7.7: Card with maximum value of a field

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

8. Procedures and Parameters

8.1 Pseudocode for procedures . . . . . . . . . . . . . . . . . 111

8.2 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 112
8.3 Checking for a single outstanding student . . . . . . . . . 113
8.4 Side-effects . . . . . . . . . . . . . . . . . . . . . . . . . . 114

We have seen that some patterns are repeated across different computations.
Understanding the structure of these patterns helps us build up abstractions such
as filtered iterations, which can be applied in many contexts.
At a more concrete level, variants of the same computation may be required in for
the same context. For example, the computation described in Figure 6.1 allow us
to find the maximum value of any field F in a pile of cards. We can instantiate F to
get a concrete computation—for instance, for the Scores dataset, if we choose F
to be chemistryMarks, we would get the maximum marks in Chemistry across
all the students.
We can think of such a generic pattern as a unit of computation that can be
parcelled out to an external agent. Imagine a scenario where we have undertaken
a major project, such as building a bridge. For this, we need to execute some
subtasks, for which we may not have the manpower or the expertise. We outsource
such tasks to a subcontractor, by clearly specifying what is to be performed,
which includes a description of what information and material we will provide the
subcontractor and what we expect in return.
In the same we, we can package a unit of computation as a “contract” to be
performed on our behalf and invoke this contract whenever we need it. This
packaged code that we contract out to be performed independently is called
a procedure. We pass the information required to execute the computation as
parameters—for instance, to tell a procedure that computes a generic maximum
which field F we are interested in. The procedure then returns the value that
corresponds to our expectation of the task to be performed, in this case, the
maximum value of field F across all the cards.

8.1 Pseudocode for procedures

Let us write the flowchart in Figure 6.1 to illustrate our pseudocode for procedures.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

112 Chapter 8. Procedures and Parameters

Procedure findMax(field)
max = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.field > max) {
max = X.field
}
}
return(max)
End findMax
Pseudocode 8.1: Procedure for generic maximum

We use the word Procedure to announce the start of the procedure and signal the
end of the code with the word End. The words Procedure and End are tagged
by the name of the procedure—in this case, findMax. We indicate the parameters
to be passed in brackets, after the name of the procedure in the first line. At the
end of its execution, the procedure has to send back its answer. This is achieved
through the return statement which, in this case, sends back the value of the
variable max.
Here is a typical statement invoking this procedure.
maxChemistryMarks = findMax(chemistryMarks)
The right hand side calls the procedure with argument chemistryMarks. This
starts off the procedure with the parameter field set to chemistryMarks. At the
end of the procedure, the value max is returned as the outcome of the procedure
call, and is assigned in the calling statement to the variable maxChemistryMarks.
Thus, the call to the procedure is typically part of an expression that is used to
compute a value to be stored in a variable.

8.2 Parameters

In the previous example, the parameter passed was a field name. We can also
pass a value as a parameter. Suppose we want compute the maximum value
in a particular field for students of a particular gender. This would require two
parameters, one specifying the field to examine and the other describing the gender,
as shown below.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

8.3 Checking for a single outstanding student 113

Procedure findMax(field,genderValue)
max = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.field > max AND X.gender == genderValue) {
max = X.field
}
}
return(max)
End findMax
Pseudocode 8.2: Procedure for maximum of a specific gender

To compute the maximum Physics marks among the boys in the class, we would
write a statement like the following.
maxPhysicsMarksBoys = findMax(physicsMarks,M)
The arguments are substituted by position for the parameters in the procedure,
so field is assigned the name physicsMarks and genderValue takes the
value M. As before, the value returned by the procedure is stored in the vari-
able maxPhysicsMarksBoys.

8.3 Checking for a single outstanding student

To illustrate the use of procedures, let us examine the problem of determining
whether there is a single student who is the best performer across all the subjects.
If the same student has achieved the top mark in all subjects, then this student’s
total should be the same as the sum of the highest marks in all the subjects. We
can call the procedure in Pseudocode 8.1 multiple times to determine this.

Start
maxMaths = findMax(mathsMarks)
maxPhysics = findMax(physicsMarks)
maxChemistry = findMax(chemistryMarks)
maxTotal = findMax(totalMarks)
subjTotal = maxMaths + maxPhysics + maxChemistry

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

114 Chapter 8. Procedures and Parameters

if (maxTotal == subjTotal) {
singleTopper = True
}
else {
singleTopper = False
}
End
Pseudocode 8.3: Check if there is a single outstanding student

8.4 Side-effects
What happens when a variable we pass as an argument is modified within a
procedure? Consider the following procedure that computes the sum 1+2+· · ·+n
for any upper bound n. The procedure decrements the parameter value from n
down to 1 and accumulates the sum in the variable sum.

Procedure prefixSum(upperLimit)
sum = 0
while (upperLimit > 0) {
sum = sum + upperLimit
upperLimit = upperLimit - 1
}
return(sum)
End prefixSum

Pseudocode 8.4: Prefix sum

Part II

Computational Thinking for

Data Science

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

9. Element ←→ Dataset

9.1 Sequenced iterations . . . . . . . . . . . . . . . . . . . . . 120

9.1.1 Flowchart for sequenced iterations . . . . . . . . . . 121
9.1.2 Pseudocode for sequenced iterations . . . . . . . . . 122
9.2 Compare with average . . . . . . . . . . . . . . . . . . . . 122
9.2.1 Students performing well . . . . . . . . . . . . . . . 123
9.2.2 Are the girls doing better than the boys? . . . . . . . 123
9.2.3 Harder Example: Find the rectangles near the "cen-
tre" of the figure . . . . . . . . . . . . . . . . . . . 125
9.3 Classifying/Ranking the elements . . . . . . . . . . . . . 125
9.3.1 Assigning grades to students . . . . . . . . . . . . . 125
9.3.2 Awarding 3 prizes to the top students . . . . . . . . 126
9.3.3 Harder Example: Assign a precedence number to an
operator in an expression . . . . . . . . . . . . . . . 131

There are many ways to make sense of the data that is given to us. In descriptive
statistics, we try to create some summary information about the dataset using
aggregate values such as the mean (same as average) of the data. This is usually
accompanied by information about how dispersed the data is around the mean,
and whether the data is symmetric around the mean or is skewed to one side.
As we have seen in Part I, computational methods allow us to collect such aggre-
gate values (and perhaps many more) from the entire dataset, by making one or
more passes through it using iterators and (accumulator) variables to store the
results.
In this section, we will see how we can make further sense of the dataset by
identifying where each element stands with respect to these aggregate values
collected from the entire dataset.
As an example, consider the procedure to find the maximum Chemistry marks
and the element(s) that contribute to this value. The procedure not only finds the
maximum Chemistry marks but also creates a relationship between the maximum
elements and the rest of the dataset - all the elements will have Chemistry marks
below these maximum elements. Likewise, when we find the minimum, all the
elements will be positioned above the elements with minimum marks.
As another example, say we have collected the average total marks of all the
students, then we can separate the students into two groups - those who are above

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

120 Chapter 9. Element ←→ Dataset

average and those who are below average. We could also form three groups -
those around the average marks (say within average plus or minus 5 marks), those
significantly above average (with total marks that is at least 5 above average),
and those significantly below average (with total marks that is at least 5 below
average).
If we know the maximum, minimum and average, we could make many more
groups - lets say groups A, B, C, D and E (which are grades that can be awarded
to the students). Those in the average ±5 can be given C grade. We can divide
the range between the maximum and average + 5 into two equal parts, with the
higher half getting A and the lower half getting B. Similarly, the range between
minimum and average − 5 can be divided into two equal halves, with the higher
being awarded D and the lower E.
We can also use filtering to find averages for subsets of the data. For example, say
we find the average of the boys and the average of the girls total marks. Then by
comparing these averages, we can say if the girls are doing better than the boys
in this class. This may not be entirely satisfactory because there could be one
boy with high marks who skews the boys’ average to the higher side. A better
method may be to find the number of boys who are above the overall average and
compare it with the number of girls who are above average. But this is not really
accurate, since the class may have many more boys than girls - so what we have
to do is to find the percentage of girls who are above average and compare it with
the percentage of boys who are above average.
Similarly, in the shopping bill dataset, we can find the average bill value and
then proceed to categorise a customer as low spenders, average spenders or high
spenders depending on whether the average bill value for that customer (use
filtering) is much below average, around the average or much above average
respectively.
In the words dataset, we could find the average frequency and average letter count
of the words. Then we could determine which words are occurring frequently (i.e.
above the average frequency) or are long (i.e. above the average lettercount).

9.1 Sequenced iterations

As we saw in the examples above, to find the relation between a data element
and the entire dataset, we will need at least two iterations - executed in sequence.
The first iteration is used to find the aggregate values such as the average, and the
second (or later) iterations are used to position an element (or subset) in relation
to the entire dataset.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

9.1 Sequenced iterations 121

9.1.1 Flowchart for sequenced iterations

The generic flowchart for doing this is shown in Figure 9.1. The variables used
in the first iteration are called aggregate variables to distinguish them from the
variables we will use in the second iteration.

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise the aggregate variables

More cards in No (False)

"unseen" pile?

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Update the aggregate vari-

ables using the values in X

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise variables

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile
and and move it into the "seen" pile

Update variables using the values

in X and the aggregate variables

Figure 9.1: Generic flowchart for two iterations in sequence

If you observe the flowchart carefully and compare it with the flowcharts that we

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

122 Chapter 9. Element ←→ Dataset

saw in Part I, you will notice that there are two iteration stubs embedded in the
flowchart (identified through the initialisation, decision to end the iteration, and
update steps). Whereas in most of the iterators seen earlier the iterator exited and
went straight to the end terminal, in this flowchart, the first iteration exits and goes
to the initialisation of the second iteration. It is the second iteration that exits by
going to the end terminal. That is precisely why we say that the two iterations
are in sequence. The second iteration will start only after the first iteration is
complete.

9.1.2 Pseudocode for sequenced iterations

The pseudocode corresponding to this flowchart is shown below (where Pile 1 is
the "Unseen" Pile, Pile 2 is the "Seen" pile).

Start
Arrange all cards in Pile 1
Initialise the aggregate variables
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
Update aggregate variables using the field values of X
}
Arrange all cards in Pile 1
Initialise the variables
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
Update variables using the aggregate variables and the field values
of X
}
End
Pseudocode 9.1: Sequenced iterations

9.2 Compare with average

To start with, we first look at comparing each element with the average to see
which side of the average it lies. By using appropriate filtering conditions, we

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

9.2 Compare with average 123

could determine the average only for a subset of the data elements, and compare
these with the average of the whole, which may reveal something about that
subset.

9.2.1 Students performing well

Let us say we want to find out which students are doing well. How do we define
"doing well"? We could start with a simple definition - say doing well is the same
as above average. The pseudocode pattern for sequential iteration can be modified
to do this as shown below:

Start
Arrange all cards in Pile 1
sum = 0, count = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
sum = sum + X.totalMarks
count = count + 1
}
average = sum/count
Arrange all cards in Pile 1
aboveList = []
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.totalMarks > average) {
Append [X] to aboveList
}
}
End
Pseudocode 9.2: Above average students

9.2.2 Are the girls doing better than the boys?

As we discussed at the beginning of the chapter, to find out if the girls are doing
better than the boys, we could find the percentage of girls who are above the

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

124 Chapter 9. Element ←→ Dataset

overall average and compare it with the percentage of boys who are above average.
This is better than just checking if the girls average is higher than the boys average,
since the girls average could be skewed higher due to a single high scoring girl
student (or alternatively, the boys average may be skewed lower due to a single
low scoring boy student).
The pseudocode for finding the above average students can be modified using
filters to find the count of the above average boys and above average girls as shown
below:

Start
Arrange all cards in Pile 1
sum = 0, count = 0, numBoys = 0, numGirls = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
sum = sum + X.totalMarks
count = count + 1
if ( X.gender == Girl ) {
numGirls = numGirls + 1
} else {
numBoys = numBoys + 1
}
}
average = sum/count
Arrange all cards in Pile 1
boysAbove = 0, girlsAbove = 0
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (X.totalMarks > average) {
if ( X.gender == Girl ) {
girlsAbove = girlsAbove + 1
} else {
boysAbove = boysAbove + 1
}
}
}
girlsAreDoingBetter = False

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

9.3 Classifying/Ranking the elements 125

if ( girlsAbove/numGirls > boysAbove/numBoys ) {

girlsAreDoingBetter = True
}
End
Pseudocode 9.3: Are the girls doing better than the boys?

9.2.3 Harder Example: Find the rectangles near the "centre"

of the figure

9.3 Classifying/Ranking the elements

In this section, we look at using sequenced iterations in situations where we need
to classify (or grade) the elements in the dataset. As before, we first find the
aggregate values (such as average, max and min) from the entire dataset in the first
pass through the dataset. In the second pass, we filter the elements using boundary
conditions derived from the aggregate values and place them in different buckets.

9.3.1 Assigning grades to students

Suppose we want to assign grades to the students based on the total marks scored
in the classroom dataset. Grades can be A (for the excellent students), B (for
slightly above average students), C (for slightly below average students) and D
(for students performing poorly).
A logical division (that is often used by teachers) is to divide the range between
the minimum and maximum marks into four equal intervals, each of which is
assigned a different grade. Say min and max are the minimum and maximum
total marks respectively. Let delta = (max − Min)/4. Then we assign D grade to
students lying between min and min + delta, C to students lying between min +
delta and min + 2 × delta, B to min + 2 × delta and min + 3 × delta and
finally A to min + 3 × delta and max (= min + 4 × delta). The pseudocode to
do this is shown below:

physicsCutoff, chemistryCutoff)
if (card.mathsMarks >= mathsCutoff OR
card.physicsMarks >= physicsCutoff OR
card.chemistryMarks >= chemistryCutoff) {

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

9.3 Classifying/Ranking the elements 129

return(True)
}
else {
return(False)
}
end subjectTopper

Pseudocode 9.6: Check if student is a subject topper

We can now write the code for the main iteration to compute the three top students
overall.
We have to maintain not only the three highest marks, but also the identities of the
students with these marks, so we have variables max, secondmax and thirdmax
to record the marks and variables maxid, secondmaxid and thirdmaxid to
record the identities of the corresponding students.
We record the third highest mark in each subject by calling the procedure topThreeMarks
three times, once per subject. We then iterate over all the cards. We first check if
the card represents a subject topper; if not we don’t need to process it. If the card
is a subject topper, we compare its total marks with the three highest values and
update them in the same way that we had done in the procedure topThreeMarks,
except that we also need to update the corresponding identities.

Start
max = 0
secondmax = 0
thirdmax = 0
maxid = -1
secondmaxid = -1
thirdmaxid = -1
mathsThird = topThreeMarks(mathsMarks)
physicsThird = topThreeMarks(physicsMarks)
chemistryThird = topThreeMarks(chemistryMarks)
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
if (subjectTopper( X, mathsThird,
physicsThird, chemistryThird) {
if (X.totalMarks > max) {

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

130 Chapter 9. Element ←→ Dataset

thirdmax = secondmax
thirdmaxid = secondmaxid
secondmax = max
secondmaxid = maxid
max = X.totalMarks
maxid = X.id
}
if (max > X.totalMarks AND X.totalMarks > secondmax) {
thirdmax = secondmax
thirdmaxid = secondmaxid
secondmax = X.totalMarks
secondmaxid = X.id
}
if (secondmax > X.totalMarks AND
X.totalMarks > thirdmax) {
thirdmax = X.field
thirdmaxid = X.id
}
}
}
End
Pseudocode 9.7: Procedure for top three values

We could add more criteria for awarding the top three prizes. For instance, we
could insist that there must be at least one boy and one girl in the top three—it
should not be the case that all three are boys or all three are girls. To handle this,
we could examine the top three students we have found using the process we have
described and check their gender. If there are both boys and girls in the list, we
are done. Otherwise, we discard the third highest card and repeat the computation
for the reduced pile.
If we repeat this process enough times, are we guaranteed to find a suitable set
of three students to award prizes? What if all the subject toppers are of the same
gender? What if there are ties within the top three totals?
As you can see, framing the requirement for a computation can also be a compli-
cated process. We need to anticipate extreme situations—sometimes called corner
cases—and decide how to handle them. If we do not do a thorough analysis
and our code encounters an unanticipated corner case, it will either get stuck or
produce an unexpected outcome.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

9.3 Classifying/Ranking the elements 131

9.3.3 Harder Example: Assign a precedence number to an op-

erator in an expression

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

10. Element ←→ Element

10.1 Nested iterations . . . . . . . . . . . . . . . . . . . . . . . 134

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise the comparison variables

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile

Initialisation: move all the cards from "unseen"

to "temp" pile and place X in the "seen" pile

No (False) More cards in

"temp" pile?

Yes (True)
Pick one card Y from the "temp" pile
and move it into the "unseen" pile

Update the variables using the values in X and Y

Figure 10.1: Generic flowchart for nested iterations

Observer the flowchart carefully. Unlike in the case of the sequenced iterations,
where the exit from the first iteration took us to the initialisation step of the second
iteration, here it is in reverse. The exit of the second (inner) iteration takes us to
the exit condition box of the first (outer) iteration.
Note also that we are properly keeping track of the cards that have been visited.
The "unseen" pile is supposed to keep track of cards that remain to be processed
in the outer iteration. We now also need to keep track of cards visited during
the inner iteration. To do that, we move all the cards from the "unseen" pile to
a "temp" pile. So for the inner iteration, the "temp" pile holds all the unvisited
cards, and the "unseen" pile holds the cards visted in the inner iteration, but not
yet visited in the outer iteration. At the end of the inner iteration, all the cards are
transferred back to the "unseen" pile, and the outer iteration can then proceed as if

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

10.1 Nested iterations 137

the "unseen" pile were never disturbed.

10.1.2 Pseudocode for nested iterations

The pseudocode for the generic nested iteration is shown below, where Pile 1 is
the "unseen" pile, Pile 2 is the "seen" pile and Pile 3 is the "temp" pile.

Start
Arrange all cards in Pile 1, Initialise the variables
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move all the remaining cards from Pile 1 to Pile 3
Move X to Pile 2
while (Pile 3 has more cards) {
Pick a card Y from Pile 3
Move Y to Pile 1
Update variables using the field values of X and Y
}
}
End
Pseudocode 10.1: Nested iterations

10.1.3 Alternate flowchart for nested iterations

The trick we used to manage to keep track of cards in both the outer and the inner
iteration was to move the "unseen" cards to a third "temp" pile, and move cards
one by one in the inner iteration from the "temp" pile back to the "unseen" pile.
There is another way that we can produce all the pairs. Instead of comparing all
the cards that are picked from the "unseen" pile with only the remaining cards in
the "unseen" pile, we can compare all the cards that have been picked with only
the cards in the "seen" pile. This will do the job as well !
Let us see why this is so. Consider a pair (A,B), with A coming before B in the
outer iteration. When we pick the a card A, "seen" pile does not contain B, so
(A,B) is not produced at this stage. But after this, A is moved to the "seen" pile so
that when we pick the card B, B is now compared all the elements in the "seen"
pile - so in particular we will compare B with A. Note that this way of doing things
will produce the pair (B,A), whereas the earlier one produced the pair (A,B).

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

138 Chapter 10. Element ←→ Element

The generic flowchart for nested iterations discussed above is depicted in Figure
10.2 below.

Start

Initialisation: Arrange cards in "unseen"

pile; keep free space for "seen" pile

Initialise the comparison variables

More cards in No (False)

"unseen" pile? End

Yes (True)
Pick one card X from the "unseen" pile

Initialisation: move all the cards from "seen"

to "temp" pile and place X in the "seen" pile

No (False) More cards in

"temp" pile?

Yes (True)
Pick one card Y from the "temp"
pile and move it into the "seen" pile

Update the variables using the values in X and Y

Figure 10.2: Alternative flowchart for nested iterations

Observer the difference between this flowchart and the earlier one. Here, we move
all the cards (except the current card X) from the "seen" pile to a "temp" pile. So
for the inner iteration, the "temp" pile holds all the unvisited cards. At the end of
the inner iteration, all the cards are transferred back from the "temp" pile to the
"seen" pile. The outer iteration can then proceed as if the "seen" pile were never
disturbed.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

10.1 Nested iterations 139

10.1.4 Alternate pseudocode for nested iterations

The pseudocode for this alternate nested iteration is shown below, where Pile 1 is
the "unseen" pile, Pile 2 is the "seen" pile and Pile 3 is the "temp" pile.

Start
Arrange all cards in Pile 1, Initialise the variables
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move all cards from Pile 2 to Pile 3
Move X to Pile 2
while (Pile 3 has more cards) {
Pick a card Y from Pile 3
Move Y to Pile 2
Update variables using the field values of X and Y
}
}
End
Pseudocode 10.2: Another way to do nested iterations

10.1.5 List of all pairs of elements

As the first example, let us just return all the pairs of elements as a list pairsList.
The pseudocode to do this 9using the alternate method above) is shown below.

Start
Arrange all cards in Pile 1, Initialise pairsList to []
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move all cards from Pile 2 to Pile 3
Move X to Pile 2
while (Pile 3 has more cards) {
Pick a card Y from Pile 3
Move Y to Pile 2
Append [ (X,Y) ] to pairsList
}
}

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

140 Chapter 10. Element ←→ Element

End
Pseudocode 10.3: List of all the pairs

10.2 Reducing the number of comparisons

As we saw above, the number of comparison pairs in nested iterations grows
quadratically with the number of elements. This becomes a major issue with large
datasets. Can we somehow reduce the number of comparisons to make it more
manageable?
In the example above where we are trying to find two students with the same
birthday, the problem has a deeper structure that we can exploit. Note that two
students will share the same birthday only if firstly they share the same month of
birth. Which means that there is simply no sense in comparing two students if
they are born in different months. This should result in a substantial reduction in
comparisons.
In order to take advantage of this, we have to first separate the students into bins,
one for each month. Let us name the bins Bin[1], Bin[2], ..., Bin[12]. So for
example, Bin[3] will be the list of all students who are born in March. We should
first process all the cards in the classroom dataset and put each of them in the
correct bin. Once this is done, we can go through the bins one by one and compare
each pair of cards within the bin.
The pseudocode to do this is shown below. It uses a procedure processBin(bin)
to do all the comparisons within each bin, whose body will be exactly the same
as the Same birthday pseudocode that we saw earlier, except that instead of
processing the entire classroom dataset, it will process only the cards in Bin[i]
and return the value of found. The procedure month(X) returns the month part of
X.dateOfBirth as a numeral between 1 and 12.

Start
Arrange all cards from the classroom dataset in Pile 1
Initialise Bin[i] to [] for all i from 1 to 12
while (Pile 1 has more cards) {
Pick a card X from Pile 1
Move X to Pile 2
Append [X] to Bin[month(X)]
}

found = False, i = 1
while (i < 13 AND NOT found) {
found = processBin(Bin[i])
i=i+1
}

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

else {
Append [X] to binA
}
}

studyPairs = []
Append processBin (binA) to studyPairs
Append processBin (binB) to studyPairs
Append processBin (binC) to studyPairs
Append processBin (binD) to studyPairs
End
Pseudocode 10.8: Study group pairing using binning

Note that through the use of a simplifying heuristic (that we are more likely to find
pairs within the bins rather than across the bins), we can reduce the comparisons
substantially.

10.3.2 Matching pronouns to nouns

Let us now turn our attention to the words dataset, which are words drawn from a
paragraph. Typically, a paragraph is broken down into sentences, each of which
could be a simple sentence (i.e. has a single clause) or a compound sentence
(made of multiple clauses connected by a comma, semicolon or a conjunction).
The sentence itself will have a verb, some nouns and perhaps some adjectives
or adverbs to qualify the nouns and verbs. In most sentences, the noun will be
replaced by a short form of the noun, called a pronoun. For instance the first
sentence may use the name of a person as the noun (such as Swaminathan), while
subsequent references to this person may be depicted using an appropriate personal
pronoun (he, his, him etc).
Now, the question we are asking here is this: given a particular personal pronoun
(such as the word "he"), can we determine who this "he" refers to in the paragraph?
Clearly, "he" has to refer to a male person - so we can look at an appropriate
word that represents a name (and hopefully a male name). If we look back from
the pronoun word "he" one word at a time, we should be able to find the closest
personal noun that occurs before this pronoun. Maybe that is the person which is
referred to by "he"? This is a good first approximation to the solution. So let us
see how to write it in pseudocode form.
Note that unlike in the other datasets, in the words dataset, the order of the words

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

10.3 Examples of nested iterations 149

matters (since the meaning of a sentence changes entirely or becomes meaningless

if the words are jumbled). Note also that for a given pronoun, we have to search
all the cards before that pronoun to look for a personal noun, and that we will
search in a specific order - we start from the word just before the pronoun and
start moving back through the words one by one till we find the desired noun.
The pseudocode given below attempts to do find all the possible matches for
each pronoun in a list possibleMatches. It uses the Boolean procedures
isProperName that checks if a noun is a proper name, and personalPronoun
that checks if the pronoun is a personal pronoun (code for both not written here
!). Recall that the "seen" cards will be the words before the current word, while
the "unseen" words will be the ones after the current word. So, the alternate
pseudocode in 10.2 is the one we need to use.

Start
Arrange all words cards in Pile 1 in order
possibleMatches = []
while (Pile 1 has more cards) {
Pick a card X from Pile 1
if (X.partOfSpeech == Pronoun AND
personalPronoun(X.word)) {
Move all cards from Pile 2 to Pile 3 (retaining the order)
while (Pile 3 has more cards) {
Pick the top card Y from Pile 3
Move Y to the bottom of Pile 2
if (Y.partOfSpeech == Noun AND
isProperName(Y.word)) {
Append [(X,Y)] to possibleMatches
}
}
}
Move X to the top of Pile 2
}
End
Pseudocode 10.9: Possible matches for each pronoun

There are a number of interesting things going on in this pseudocode. Firstly

note that we are not carrying out the inner iteration for all the words, we start an
inner iteration only when the current word is a personal pronoun. So, this already
reduces the number of comparisons to be done.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

150 Chapter 10. Element ←→ Element

Secondly, when we are at such a personal pronoun, we start looking in the "seen"
pile (Pile 2 moved to Pile 3). To make sure that we are looking through the "seen"
pile in such a way that we are reading backwards in the paragraph, the Pile 2
has to be maintained in reverse order of the words in Pile 1. This is ensured by
placing the card picked up on the top of Pile 2. While transferring cards from Pile
2 to Pile 3, we need to preserve this reverse order. The inner iteration should not
disturb the order of cards, so when we are done examining a card from Pile 3, we
place it at the bottom of Pile 2, so that after the inner iteration is done, Pile 2 will
be exactly as before. The code to move card X to Pile 2 is moved after the inner
iteration, so that it does not affect the "seen" pile (Pile 2) over which we need to
search. It also needs to be done whether we are executing the inner loop or not.
Finally, note that we do not stop when we find the proper name noun Y matching
the pronoun X, since this match may in some cases not be the correct one (but we
keep the pair as the first possible match in our list). If we are interested only in
finding the first possible match, we can use a Boolean variable found which is set
to True when the match is found, and the inner iteration can exit on this condition.

10.3.3 Finding connecting trains

10.3.4 Finding matching brackets in an expression

10.3.5 Finding overlapping rectangles

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

11. Lists

11.1 List as a set or collection . . . . . . . . . . . . . . . . . . 152

11.1.1 Example: Collect all students born in May . . . . . 152

11.1.2 Example: Collect all students from Chennai . . . . . 152

11.1.3 Example: Collect all the bills of one customer . . . . 152

11.1.4 Example: Collect all the items sold of one category . 152

11.1.5 Example: Collect the first verb from each sentence . 152

11.1.6 Example: Collect all trains between two stations . . 152

11.1.7 Example: Find all rectangles at the leftmost boundary 152

11.1.8 How do we store the same element in two different

lists ? . . . . . . . . . . . . . . . . . . . . . . . . . 152

11.1.9 Using indexing: Lists of indices rather than lists of

elements . . . . . . . . . . . . . . . . . . . . . . . 152

11.2 Using the list collections . . . . . . . . . . . . . . . . . . . 152

11.2.1 Example: Students scoring over 80 in all subjects . . 152

11.2.2 Example: Are the Chennai students doing better than

the Bangalore students? . . . . . . . . . . . . . . . 152

11.2.3 Example: Which shop has the most loyal customers? 152

11.2.4 Example: Is there a word that is both a noun and a

verb? . . . . . . . . . . . . . . . . . . . . . . . . . 152

11.2.5 Example: Finding the rectangles at the boundary . . 152

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

152 Chapter 11. Lists

11.1 List as a set or collection

11.1.1 Example: Collect all students born in May

11.1.2 Example: Collect all students from Chennai

11.1.3 Example: Collect all the bills of one customer

11.1.4 Example: Collect all the items sold of one category

11.1.5 Example: Collect the first verb from each sentence

11.1.6 Example: Collect all trains between two stations

11.1.7 Example: Find all rectangles at the leftmost boundary

11.1.8 How do we store the same element in two different lists

11.1.9 Using indexing: Lists of indices rather than lists of ele-

ments

11.2 Using the list collections

11.2.1 Example: Students scoring over 80 in all subjects

11.2.2 Example: Are the Chennai students doing better than

the Bangalore students?

11.2.3 Example: Which shop has the most loyal customers?

11.2.4 Example: Is there a word that is both a noun and a

verb?

11.2.5 Example: Finding the rectangles at the boundary

Summary of chapter

Exercises
Downloaded by Prasun Babu ([email protected])
lOMoARcPSD|44901534

12. Procedures with Side Effects

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

13. Sorted Lists

13.1 Keeping lists in sorted form may speedup subsequent pro-

cessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

13.1.1 Example: Assign grades to students . . . . . . . . . 156

13.1.2 Example: Match pronoun and noun . . . . . . . . . 156

13.1.3 Example: Find transit station for going from one city
to another . . . . . . . . . . . . . . . . . . . . . . . 156

13.1.4 Example: Find the subject noun for a verb in a sentence 156

13.2 How do we sort? . . . . . . . . . . . . . . . . . . . . . . . 156

13.2.1 Insertion sort . . . . . . . . . . . . . . . . . . . . . 156

13.2.2 Pseudocode for insertion sort . . . . . . . . . . . . . 156

13.3 Using the insertion sort procedure . . . . . . . . . . . . . 156

13.3.1 Pseudocode: Assigning grades to students . . . . . . 156

13.3.2 Pseudocode: Matching pronoun to noun . . . . . . . 156

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

156 Chapter 13. Sorted Lists

13.1 Keeping lists in sorted form may speedup sub-

sequent processing

13.1.1 Example: Assign grades to students

13.1.2 Example: Match pronoun and noun

13.1.3 Example: Find transit station for going from one city
to another

13.1.4 Example: Find the subject noun for a verb in a sentence

13.2 How do we sort?

13.2.1 Insertion sort

13.2.2 Pseudocode for insertion sort

13.3 Using the insertion sort procedure

13.3.1 Pseudocode: Assigning grades to students

13.3.2 Pseudocode: Matching pronoun to noun

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

14. Dictionaries

14.0.1 Finding the frequency of all the words

14.0.2 Are the words with high frequency short?

If we take a look at the whole paragraph in the words dataset, we can observe
that there are quite a number of short words, and some of these seem to occur
more than once. So this leads us to ask the hypothetical question: "Are the high
frequency words mostly short?". How do we determine this? What does "mostly"
mean?
The usual way to settle questions of the kind: "Do elements with property A
satisfy property B?" is to put all the elements into 4 buckets - A and B, A and
not B, not A and B, not A and not B. This is shown in the table below, where the
top row is B and the bottom row is not B, the left column is not A and the right
column is A:

not A and B A and B

not A and not B A and not B

We then count the number of elements for each box of the table, and represent them
using percentages of the total number of elements. If the majority of elements fall
along the diagonal (the sum of the percentages of the boxes (not A and not B) and
(A and B) is much higher than the sum of percentages in the boxes (A and not B)
and (not A and B)), then we know that A and B go mostly together, and so the
hypothesis must be true.
We are now ready to carry out this procedure as shown in the pseudocode below.
The variables HFHL, HFLL, LFHL and LFLL represent high frequency and high
length, high frequency and low length, low frequency and high length and low
frequency and low length, where high and low represent above average and below
average values respectively.

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

158 Chapter 14. Dictionaries

14.0.3 Finding the total spend of each customer

14.0.4 Do the high spenders shop at the biggest shop?

14.0.5 Total distance and total time of all the trains

14.0.6 Are the longer distance trains the fastest?

Summary of chapter

Exercises

Downloaded by Prasun Babu ([email protected])

lOMoARcPSD|44901534

15. Dictionaries with Lists

Downloaded by Prasun Babu ([email protected])

Effective Pandas. Patterns For Data Manipulation (Treading On Python) - Matt Harrison - Independently Published (2021)
100% (12)
Effective Pandas. Patterns For Data Manipulation (Treading On Python) - Matt Harrison - Independently Published (2021)
392 pages
PD 2 Preview
0% (3)
PD 2 Preview
32 pages
Intrusion Detection Honeypots
From Everand
Intrusion Detection Honeypots
Chris Sanders
3/5 (2)
Audio, Video, and Media in the Ministry
From Everand
Audio, Video, and Media in the Ministry
Clarence Floyd Richmond
No ratings yet
Computational Thinking v0.1 13-Oct-2020 PDF
100% (6)
Computational Thinking v0.1 13-Oct-2020 PDF
185 pages
Data Science Project - An Inductive Learning Approach, Verri
No ratings yet
Data Science Project - An Inductive Learning Approach, Verri
238 pages
CT Book
No ratings yet
CT Book
188 pages
An Introduction To Computing
100% (1)
An Introduction To Computing
157 pages
book
No ratings yet
book
471 pages
CSE_DAA_LN_UG20
No ratings yet
CSE_DAA_LN_UG20
115 pages
Aaanotes
No ratings yet
Aaanotes
156 pages
Data Science Master
No ratings yet
Data Science Master
11 pages
Book 1 65
No ratings yet
Book 1 65
65 pages
Johan Sannemo - Principles of Algorithmic Problem Solving
No ratings yet
Johan Sannemo - Principles of Algorithmic Problem Solving
351 pages
Ics 2013
No ratings yet
Ics 2013
883 pages
AI Book
No ratings yet
AI Book
121 pages
Competitive Programming 3 PDF
100% (1)
Competitive Programming 3 PDF
448 pages
2301.01188v4
No ratings yet
2301.01188v4
450 pages
Design and Analysis of Algorithms
No ratings yet
Design and Analysis of Algorithms
124 pages
Principles of Algorithmic Problem Solving PDF
100% (1)
Principles of Algorithmic Problem Solving PDF
351 pages
Introduction To Computing
No ratings yet
Introduction To Computing
270 pages
Datawranglingpy Screen v1.0.3 20230206
No ratings yet
Datawranglingpy Screen v1.0.3 20230206
442 pages
I Fundamentals 1
No ratings yet
I Fundamentals 1
593 pages
CP Guide Contents
No ratings yet
CP Guide Contents
7 pages
Python Code For Artificial Intelligence
No ratings yet
Python Code For Artificial Intelligence
298 pages
Python For Finance
No ratings yet
Python For Finance
289 pages
Main
No ratings yet
Main
321 pages
Aipython
No ratings yet
Aipython
314 pages
Aipython
No ratings yet
Aipython
318 pages
Deep R Programming
No ratings yet
Deep R Programming
456 pages
Week 1
No ratings yet
Week 1
5 pages
Mehlhorn K., Sanders P. Concise Algorithmics, The Basic Toolbox 124ñ PDF
No ratings yet
Mehlhorn K., Sanders P. Concise Algorithmics, The Basic Toolbox 124ñ PDF
124 pages
AI Python
No ratings yet
AI Python
312 pages
Algorithmic Problem Solving: Johan Sannemo 2020
No ratings yet
Algorithmic Problem Solving: Johan Sannemo 2020
404 pages
Aipython PDF
No ratings yet
Aipython PDF
320 pages
Aipython Inteligencia Artificial
No ratings yet
Aipython Inteligencia Artificial
320 pages
Artificial Intelligence, Second Edition, Python Code
No ratings yet
Artificial Intelligence, Second Edition, Python Code
307 pages
Certified Professional Diploma in Data Science-1
No ratings yet
Certified Professional Diploma in Data Science-1
43 pages
ae9P57Rv6Fm9p iW2kV5MV5gc3iJ4zE
No ratings yet
ae9P57Rv6Fm9p iW2kV5MV5gc3iJ4zE
30 pages
Aipython
No ratings yet
Aipython
388 pages
Aipython
No ratings yet
Aipython
350 pages
Python
100% (3)
Python
540 pages
Data Science & AIML Coursework
No ratings yet
Data Science & AIML Coursework
10 pages
CM1035 Ads
No ratings yet
CM1035 Ads
29 pages
Unlocking Statistics for the Social Sciences
From Everand
Unlocking Statistics for the Social Sciences
Norma Sinclair
No ratings yet
Gray Hat Hacking the Ethical Hacker's
From Everand
Gray Hat Hacking the Ethical Hacker's
Çağatay Şanlı
5/5 (1)
ChatGPT for Business: Strategies for Success
From Everand
ChatGPT for Business: Strategies for Success
Matthew C. Smith
No ratings yet
The Linux Terminal for Advanced Users - The Command Line Made Easy: First Edition
From Everand
The Linux Terminal for Advanced Users - The Command Line Made Easy: First Edition
Michael Basler
No ratings yet
Securing ChatGPT: Best Practices for Protecting Sensitive Data in AI Language Models
From Everand
Securing ChatGPT: Best Practices for Protecting Sensitive Data in AI Language Models
Matthew C. Smith
No ratings yet
A To Z of Internet: Everything You Wanted to Know
From Everand
A To Z of Internet: Everything You Wanted to Know
Bittu Kumar
No ratings yet
Advanced college algebra study guide
From Everand
Advanced college algebra study guide
Harrison Cook
No ratings yet
ADVANCED COLLEGE ALGEBRA STUDY GUIDE
From Everand
ADVANCED COLLEGE ALGEBRA STUDY GUIDE
Harrison K Cook
No ratings yet
Data Empowerment: Harnessing Advanced Mathematical and Statistical Methods for Data Science and Machine Learning
From Everand
Data Empowerment: Harnessing Advanced Mathematical and Statistical Methods for Data Science and Machine Learning
NAGARAJU CHEVURU
No ratings yet
Mastering Python Advanced Concepts and Practical Applications
From Everand
Mastering Python Advanced Concepts and Practical Applications
Aissa Younes
No ratings yet
Content Creation Revolution with chatGPT
From Everand
Content Creation Revolution with chatGPT
Maria Cowen
No ratings yet
Software Patterns Made Easy
From Everand
Software Patterns Made Easy
Justice Nanhou
No ratings yet
Grow with Python Programming: From Basics to Advanced
From Everand
Grow with Python Programming: From Basics to Advanced
Mark Fliks
No ratings yet
Advanced Multiplayer Game Development with Ureal Engine 5: A Comprehensive Guide to C++ Scripting
From Everand
Advanced Multiplayer Game Development with Ureal Engine 5: A Comprehensive Guide to C++ Scripting
Vladimir Kiselev
No ratings yet
Blog Smarter, Not Harder: SEO, Blogging, and AI Strategies to Skyrocket Your Traffic
From Everand
Blog Smarter, Not Harder: SEO, Blogging, and AI Strategies to Skyrocket Your Traffic
Jay Nans
No ratings yet
Mathematics N4: FET College Nated, #6
From Everand
Mathematics N4: FET College Nated, #6
Efetobo Emede
No ratings yet
Glossary of Petroleum Industry Common Terms
No ratings yet
Glossary of Petroleum Industry Common Terms
10 pages
Ear Irrigation Procedure Rationale
No ratings yet
Ear Irrigation Procedure Rationale
4 pages
Biphasic liquid dosage form SUSPENSION Fy G1G2
No ratings yet
Biphasic liquid dosage form SUSPENSION Fy G1G2
54 pages
IOTDE-PIP-CL-001 Rev 0 Check List For Plot Plan
100% (1)
IOTDE-PIP-CL-001 Rev 0 Check List For Plot Plan
3 pages
Script Fibonacci and Golden Ratio
No ratings yet
Script Fibonacci and Golden Ratio
3 pages
Cambridge Ordinary Level
No ratings yet
Cambridge Ordinary Level
20 pages
Pure Math Holiday 2024
No ratings yet
Pure Math Holiday 2024
2 pages
KILN Technical
No ratings yet
KILN Technical
37 pages
Clarithromycin
No ratings yet
Clarithromycin
3 pages
226N Data Sheet - FBE Coating
No ratings yet
226N Data Sheet - FBE Coating
2 pages
Pages from Tord Berglundh editor William V Giannobile editor Mariano Sanz editor Niklaus P Lang editor - Lindhes Clinical Periodontology and Implant Dentistry Volume I-II-Wiley-Blackwell 2021(4)
No ratings yet
Pages from Tord Berglundh editor William V Giannobile editor Mariano Sanz editor Niklaus P Lang editor - Lindhes Clinical Periodontology and Implant Dentistry Volume I-II-Wiley-Blackwell 2021(4)
63 pages
Download Complete Speech and Language Processing An Introduction to Natural Language Processing Computational Linguistics and Speech Recognition Second Edition Daniel Jurafsky PDF for All Chapters
No ratings yet
Download Complete Speech and Language Processing An Introduction to Natural Language Processing Computational Linguistics and Speech Recognition Second Edition Daniel Jurafsky PDF for All Chapters
29 pages
Audit Report: Standard - Food Safety Management System
100% (1)
Audit Report: Standard - Food Safety Management System
6 pages
Rainman Watermaker Installation Guidelines v2.0
No ratings yet
Rainman Watermaker Installation Guidelines v2.0
4 pages
INNOWOOD V Joint Shiplap Cladding Installation Manual JUN 2017 V2
No ratings yet
INNOWOOD V Joint Shiplap Cladding Installation Manual JUN 2017 V2
29 pages
Rapt 12
No ratings yet
Rapt 12
16 pages
1
No ratings yet
1
15 pages
Core Competency
No ratings yet
Core Competency
7 pages
Electrical Safety
No ratings yet
Electrical Safety
191 pages
Workshop 7
No ratings yet
Workshop 7
1 page
XLR Ju 1
No ratings yet
XLR Ju 1
7 pages
Application and Interpretations Standard Level Syllabus
No ratings yet
Application and Interpretations Standard Level Syllabus
6 pages
Introduction of ZOOMLION ZE335G Crawler Excavator (Middle East)
No ratings yet
Introduction of ZOOMLION ZE335G Crawler Excavator (Middle East)
2 pages
Titan 2000 Vet Spec
67% (3)
Titan 2000 Vet Spec
10 pages
9th English
No ratings yet
9th English
2 pages
Science 4 Quarter 2 Week 6
No ratings yet
Science 4 Quarter 2 Week 6
57 pages
RHB 325 6
No ratings yet
RHB 325 6
28 pages
Essentials of Nuclear Medicine Imaging Expert Consult Online and Print 6e 6th Edition Fred A. Mettler Jr. Md Mph all chapter instant download
100% (16)
Essentials of Nuclear Medicine Imaging Expert Consult Online and Print 6e 6th Edition Fred A. Mettler Jr. Md Mph all chapter instant download
50 pages
Bowel's Reaction Series
No ratings yet
Bowel's Reaction Series
6 pages
Performance and Characteristics of Reverse Osmosis Membranes
No ratings yet
Performance and Characteristics of Reverse Osmosis Membranes
11 pages