(eBook PDF) Introduction to Data Mining, Global

Edition 2nd Edition download

https://ebooksecure.com/product/ebook-pdf-introduction-to-data-
mining-global-edition-2nd-edition/

Download full version ebook from https://ebooksecure.com

We believe these products will be a great fit for you. Click
the link to download now, or visit ebooksecure.com
to discover even more!

(eBook PDF) Introduction to Data Mining 2nd Edition by

Pang-Ning Tan

http://ebooksecure.com/product/ebook-pdf-introduction-to-data-
mining-2nd-edition-by-pang-ning-tan/

(eBook PDF) Introduction to Business Data Mining 1st

Edition

http://ebooksecure.com/product/ebook-pdf-introduction-to-
business-data-mining-1st-edition/

Introduction to Algorithms for Data Mining and Machine

Learning 1st edition - eBook PDF

https://ebooksecure.com/download/introduction-to-algorithms-for-
data-mining-and-machine-learning-ebook-pdf/

(eBook PDF) Data Mining and Predictive Analytics 2nd

Edition

http://ebooksecure.com/product/ebook-pdf-data-mining-and-
predictive-analytics-2nd-edition/
(eBook PDF) Handbook of Statistical Analysis and Data
Mining Applications 2nd Edition

http://ebooksecure.com/product/ebook-pdf-handbook-of-statistical-
analysis-and-data-mining-applications-2nd-edition/

(eBook PDF) Data Mining Concepts and Techniques 3rd

http://ebooksecure.com/product/ebook-pdf-data-mining-concepts-
and-techniques-3rd/

Big Data Mining for Climate Change 1st edition - eBook

PDF

https://ebooksecure.com/download/big-data-mining-for-climate-
change-ebook-pdf/

(eBook PDF) Introduction to Java Programming and Data

Structures, Comprehensive Version, 11th Global Edition

http://ebooksecure.com/product/ebook-pdf-introduction-to-java-
programming-and-data-structures-comprehensive-version-11th-
global-edition/

(eBook PDF) Introduction to Global Business:

Understanding the International Environment & Global
Business Functions 2nd Edition

http://ebooksecure.com/product/ebook-pdf-introduction-to-global-
business-understanding-the-international-environment-global-
business-functions-2nd-edition/
 Preface to the Second Edition 7

been updated to include discussions of mutual information and kernel-based

techniques.
The last chapter, which discusses how to avoid false discoveries and pro-
duce valid results, is completely new, and is novel among other contemporary
textbooks on data mining. It supplements the discussions in the other chapters
with a discussion of the statistical concepts (statistical significance, p-values,
false discovery rate, permutation testing, etc.) relevant to avoiding spurious
results, and then illustrates these concepts in the context of data mining
techniques. This chapter addresses the increasing concern over the validity and
reproducibility of results obtained from data analysis. The addition of this last
chapter is a recognition of the importance of this topic and an acknowledgment
that a deeper understanding of this area is needed for those analyzing data.
The data exploration chapter has been deleted, as have the appendices,
from the print edition of the book, but will remain available on the web. A
new appendix provides a brief discussion of scalability in the context of big
data.

To the Instructor As a textbook, this book is suitable for a wide range

of students at the advanced undergraduate or graduate level. Since students
come to this subject with diverse backgrounds that may not include extensive
knowledge of statistics or databases, our book requires minimal prerequisites.
No database knowledge is needed, and we assume only a modest background
in statistics or mathematics, although such a background will make for easier
going in some sections. As before, the book, and more specifically, the chapters
covering major data mining topics, are designed to be as self-contained as
possible. Thus, the order in which topics can be covered is quite flexible. The
core material is covered in chapters 2 (data), 3 (classification), 4 (association
analysis), 5 (clustering), and 9 (anomaly detection). We recommend at least
a cursory coverage of Chapter 10 (Avoiding False Discoveries) to instill in
students some caution when interpreting the results of their data analysis.
Although the introductory data chapter (2) should be covered first, the basic
classification (3), association analysis (4), and clustering chapters (5), can be
covered in any order. Because of the relationship of anomaly detection (9) to
classification (3) and clustering (5), these chapters should precede Chapter 9.
Various topics can be selected from the advanced classification, association
analysis, and clustering chapters (6, 7, and 8, respectively) to fit the schedule
and interests of the instructor and students. We also advise that the lectures
be augmented by projects or practical exercises in data mining. Although they
8 Preface to the Second Edition

are time consuming, such hands-on assignments greatly enhance the value of
the course.

Support Materials Support materials available to all readers of this book

are available on the book’s website.

• PowerPoint lecture slides

• Suggestions for student projects
• Data mining resources, such as algorithms and data sets
• Online tutorials that give step-by-step examples for selected data mining
techniques described in the book using actual data sets and data analysis
software

Additional support materials, including solutions to exercises, are available

only to instructors adopting this textbook for classroom use.

Acknowledgments Many people contributed to the first and second edi-

tions of the book. We begin by acknowledging our families to whom this book
is dedicated. Without their patience and support, this project would have been
impossible.
We would like to thank the current and former students of our data
mining groups at the University of Minnesota and Michigan State for their
contributions. Eui-Hong (Sam) Han and Mahesh Joshi helped with the initial
data mining classes. Some of the exercises and presentation slides that they
created can be found in the book and its accompanying slides. Students in
our data mining groups who provided comments on drafts of the book or
who contributed in other ways include Shyam Boriah, Haibin Cheng, Varun
Chandola, Eric Eilertson, Levent Ertöz, Jing Gao, Rohit Gupta, Sridhar Iyer,
Jung-Eun Lee, Benjamin Mayer, Aysel Ozgur, Uygar Oztekin, Gaurav Pandey,
Kashif Riaz, Jerry Scripps, Gyorgy Simon, Hui Xiong, Jieping Ye, and Pusheng
Zhang. We would also like to thank the students of our data mining classes at
the University of Minnesota and Michigan State University who worked with
early drafts of the book and provided invaluable feedback. We specifically
note the helpful suggestions of Bernardo Craemer, Arifin Ruslim, Jamshid
Vayghan, and Yu Wei.
Joydeep Ghosh (University of Texas) and Sanjay Ranka (University of
Florida) class tested early versions of the book. We also received many useful
suggestions directly from the following UT students: Pankaj Adhikari, Rajiv
Bhatia, Frederic Bosche, Arindam Chakraborty, Meghana Deodhar, Chris
Everson, David Gardner, Saad Godil, Todd Hay, Clint Jones, Ajay Joshi,
 Preface to the Second Edition 9

Joonsoo Lee, Yue Luo, Anuj Nanavati, Tyler Olsen, Sunyoung Park, Aashish
Phansalkar, Geoff Prewett, Michael Ryoo, Daryl Shannon, and Mei Yang.
Ronald Kostoff (ONR) read an early version of the clustering chapter
and offered numerous suggestions. George Karypis provided invaluable LATEX
assistance in creating an author index. Irene Moulitsas also provided assistance
with LATEX and reviewed some of the appendices. Musetta Steinbach was very
helpful in finding errors in the figures.
We would like to acknowledge our colleagues at the University of Minnesota
and Michigan State who have helped create a positive environment for data
mining research. They include Arindam Banerjee, Dan Boley, Joyce Chai, Anil
Jain, Ravi Janardan, Rong Jin, George Karypis, Claudia Neuhauser, Haesun
Park, William F. Punch, György Simon, Shashi Shekhar, and Jaideep Srivas-
tava. The collaborators on our many data mining projects, who also have our
gratitude, include Ramesh Agrawal, Maneesh Bhargava, Steve Cannon, Alok
Choudhary, Imme Ebert-Uphoff, Auroop Ganguly, Piet C. de Groen, Fran
Hill, Yongdae Kim, Steve Klooster, Kerry Long, Nihar Mahapatra, Rama Ne-
mani, Nikunj Oza, Chris Potter, Lisiane Pruinelli, Nagiza Samatova, Jonathan
Shapiro, Kevin Silverstein, Brian Van Ness, Bonnie Westra, Nevin Young, and
Zhi-Li Zhang.
The departments of Computer Science and Engineering at the University of
Minnesota and Michigan State University provided computing resources and
a supportive environment for this project. ARDA, ARL, ARO, DOE, NASA,
NOAA, and NSF provided research support for Pang-Ning Tan, Michael Stein-
bach, Anuj Karpatne, and Vipin Kumar. In particular, Kamal Abdali, Mitra
Basu, Dick Brackney, Jagdish Chandra, Joe Coughlan, Michael Coyle, Stephen
Davis, Frederica Darema, Richard Hirsch, Chandrika Kamath, Tsengdar Lee,
Raju Namburu, N. Radhakrishnan, James Sidoran, Sylvia Spengler, Bha-
vani Thuraisingham, Walt Tiernin, Maria Zemankova, Aidong Zhang, and
Xiaodong Zhang have been supportive of our research in data mining and
high-performance computing.
It was a pleasure working with the helpful staff at Pearson Education.
In particular, we would like to thank Matt Goldstein, Kathy Smith, Carole
Snyder, and Joyce Wells. We would also like to thank George Nichols, who
helped with the art work and Paul Anagnostopoulos, who provided LATEX
support.
We are grateful to the following Pearson reviewers: Leman Akoglu (Carnegie
Mellon University), Chien-Chung Chan (University of Akron), Zhengxin Chen
(University of Nebraska at Omaha), Chris Clifton (Purdue University), Joy-
deep Ghosh (University of Texas, Austin), Nazli Goharian (Illinois Institute of
Technology), J. Michael Hardin (University of Alabama), Jingrui He (Arizona
10 Preface to the Second Edition

State University), James Hearne (Western Washington University), Hillol Kar-

gupta (University of Maryland, Baltimore County and Agnik, LLC), Eamonn
Keogh (University of California-Riverside), Bing Liu (University of Illinois at
Chicago), Mariofanna Milanova (University of Arkansas at Little Rock), Srini-
vasan Parthasarathy (Ohio State University), Zbigniew W. Ras (University of
North Carolina at Charlotte), Xintao Wu (University of North Carolina at
Charlotte), and Mohammed J. Zaki (Rensselaer Polytechnic Institute).
Over the years since the first edition, we have also received numerous
comments from readers and students who have pointed out typos and various
other issues. We are unable to mention these individuals by name, but their
input is much appreciated and has been taken into account for the second
edition.

Acknowledgments for the Global Edition Pearson would like to thank

and acknowledge Pramod Kumar Singh (Atal Bihari Vajpayee Indian Institute
of Information Technology and Management) for contributing to the Global
Edition, and Annappa (National Institute of Technology Surathkal), Komal
Arora, and Soumen Mukherjee (RCC Institute of Technology) for reviewing
the Global Edition.
Contents

Preface to the Second Edition 5

1 Introduction 21
1.1 What Is Data Mining? . . . . . . . . . . . . . . . . . . . . . . . 24
1.2 Motivating Challenges . . . . . . . . . . . . . . . . . . . . . . . 25
1.3 The Origins of Data Mining . . . . . . . . . . . . . . . . . . . . 27
1.4 Data Mining Tasks . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.5 Scope and Organization of the Book . . . . . . . . . . . . . . . 33
1.6 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 35
1.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2 Data 43
2.1 Types of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.1.1 Attributes and Measurement . . . . . . . . . . . . . . . 47
2.1.2 Types of Data Sets . . . . . . . . . . . . . . . . . . . . . 54
2.2 Data Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
2.2.1 Measurement and Data Collection Issues . . . . . . . . . 62
2.2.2 Issues Related to Applications . . . . . . . . . . . . . . 69
2.3 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . 70
2.3.1 Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.3.2 Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . 72
2.3.3 Dimensionality Reduction . . . . . . . . . . . . . . . . . 76
2.3.4 Feature Subset Selection . . . . . . . . . . . . . . . . . . 78
2.3.5 Feature Creation . . . . . . . . . . . . . . . . . . . . . . 81
2.3.6 Discretization and Binarization . . . . . . . . . . . . . . 83
2.3.7 Variable Transformation . . . . . . . . . . . . . . . . . . 89
2.4 Measures of Similarity and Dissimilarity . . . . . . . . . . . . . 91
2.4.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
2.4.2 Similarity and Dissimilarity between Simple Attributes . 94
2.4.3 Dissimilarities between Data Objects . . . . . . . . . . . 96
2.4.4 Similarities between Data Objects . . . . . . . . . . . . 98
12 Contents

2.4.5 Examples of Proximity Measures . . . . . . . . . . . . . 99

2.4.6 Mutual Information . . . . . . . . . . . . . . . . . . . . 108
2.4.7 Kernel Functions* . . . . . . . . . . . . . . . . . . . . . 110
2.4.8 Bregman Divergence* . . . . . . . . . . . . . . . . . . . 114
2.4.9 Issues in Proximity Calculation . . . . . . . . . . . . . . 116
2.4.10 Selecting the Right Proximity Measure . . . . . . . . . . 118
2.5 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 120
2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

3 Classification: Basic Concepts and Techniques 133

3.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
3.2 General Framework for Classification . . . . . . . . . . . . . . . 137
3.3 Decision Tree Classifier . . . . . . . . . . . . . . . . . . . . . . 139
3.3.1 A Basic Algorithm to Build a Decision Tree . . . . . . . 141
3.3.2 Methods for Expressing Attribute Test Conditions . . . 144
3.3.3 Measures for Selecting an Attribute Test Condition . . . 147
3.3.4 Algorithm for Decision Tree Induction . . . . . . . . . . 156
3.3.5 Example Application: Web Robot Detection . . . . . . . 158
3.3.6 Characteristics of Decision Tree Classifiers . . . . . . . . 160
3.4 Model Overfitting . . . . . . . . . . . . . . . . . . . . . . . . . . 167
3.4.1 Reasons for Model Overfitting . . . . . . . . . . . . . . 169
3.5 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
3.5.1 Using a Validation Set . . . . . . . . . . . . . . . . . . . 176
3.5.2 Incorporating Model Complexity . . . . . . . . . . . . . 177
3.5.3 Estimating Statistical Bounds . . . . . . . . . . . . . . . 182
3.5.4 Model Selection for Decision Trees . . . . . . . . . . . . 182
3.6 Model Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 184
3.6.1 Holdout Method . . . . . . . . . . . . . . . . . . . . . . 185
3.6.2 Cross-Validation . . . . . . . . . . . . . . . . . . . . . . 185
3.7 Presence of Hyper-parameters . . . . . . . . . . . . . . . . . . . 188
3.7.1 Hyper-parameter Selection . . . . . . . . . . . . . . . . 188
3.7.2 Nested Cross-Validation . . . . . . . . . . . . . . . . . . 190
3.8 Pitfalls of Model Selection and Evaluation . . . . . . . . . . . . 192
3.8.1 Overlap between Training and Test Sets . . . . . . . . . 192
3.8.2 Use of Validation Error as Generalization Error . . . . . 192
3.9 Model Comparison∗ . . . . . . . . . . . . . . . . . . . . . . . . 193
3.9.1 Estimating the Confidence Interval for Accuracy . . . . 194
3.9.2 Comparing the Performance of Two Models . . . . . . . 195
3.10 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 196
3.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
 Contents 13

4 Association Analysis: Basic Concepts and Algorithms 213

4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
4.2 Frequent Itemset Generation . . . . . . . . . . . . . . . . . . . 218
4.2.1 The Apriori Principle . . . . . . . . . . . . . . . . . . . 219
4.2.2 Frequent Itemset Generation in the Apriori Algorithm . 220
4.2.3 Candidate Generation and Pruning . . . . . . . . . . . . 224
4.2.4 Support Counting . . . . . . . . . . . . . . . . . . . . . 229
4.2.5 Computational Complexity . . . . . . . . . . . . . . . . 233
4.3 Rule Generation . . . . . . . . . . . . . . . . . . . . . . . . . . 236
4.3.1 Confidence-Based Pruning . . . . . . . . . . . . . . . . . 236
4.3.2 Rule Generation in Apriori Algorithm . . . . . . . . . . 237
4.3.3 An Example: Congressional Voting Records . . . . . . . 238
4.4 Compact Representation of Frequent Itemsets . . . . . . . . . . 240
4.4.1 Maximal Frequent Itemsets . . . . . . . . . . . . . . . . 240
4.4.2 Closed Itemsets . . . . . . . . . . . . . . . . . . . . . . . 242
4.5 Alternative Methods for Generating Frequent Itemsets* . . . . 245
4.6 FP-Growth Algorithm* . . . . . . . . . . . . . . . . . . . . . . 249
4.6.1 FP-Tree Representation . . . . . . . . . . . . . . . . . . 250
4.6.2 Frequent Itemset Generation in FP-Growth Algorithm . 253
4.7 Evaluation of Association Patterns . . . . . . . . . . . . . . . . 257
4.7.1 Objective Measures of Interestingness . . . . . . . . . . 258
4.7.2 Measures beyond Pairs of Binary Variables . . . . . . . 270
4.7.3 Simpson’s Paradox . . . . . . . . . . . . . . . . . . . . . 272
4.8 Effect of Skewed Support Distribution . . . . . . . . . . . . . . 274
4.9 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 280
4.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

5 Cluster Analysis: Basic Concepts and Algorithms 307

5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
5.1.1 What Is Cluster Analysis? . . . . . . . . . . . . . . . . . 310
5.1.2 Different Types of Clusterings . . . . . . . . . . . . . . . 311
5.1.3 Different Types of Clusters . . . . . . . . . . . . . . . . 313
5.2 K-means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
5.2.1 The Basic K-means Algorithm . . . . . . . . . . . . . . 317
5.2.2 K-means: Additional Issues . . . . . . . . . . . . . . . . 326
5.2.3 Bisecting K-means . . . . . . . . . . . . . . . . . . . . . 329
5.2.4 K-means and Different Types of Clusters . . . . . . . . 330
5.2.5 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 331
5.2.6 K-means as an Optimization Problem . . . . . . . . . . 331
14 Contents

5.3 Agglomerative Hierarchical Clustering . . . . . . . . . . . . . . 336

5.3.1 Basic Agglomerative Hierarchical Clustering Algorithm 337
5.3.2 Specific Techniques . . . . . . . . . . . . . . . . . . . . . 339
5.3.3 The Lance-Williams Formula for Cluster Proximity . . . 344
5.3.4 Key Issues in Hierarchical Clustering . . . . . . . . . . . 345
5.3.5 Outliers . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
5.3.6 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 347
5.4 DBSCAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
5.4.1 Traditional Density: Center-Based Approach . . . . . . 347
5.4.2 The DBSCAN Algorithm . . . . . . . . . . . . . . . . . 349
5.4.3 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 351
5.5 Cluster Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 353
5.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 353
5.5.2 Unsupervised Cluster Evaluation Using Cohesion and
Separation . . . . . . . . . . . . . . . . . . . . . . . . . 356
5.5.3 Unsupervised Cluster Evaluation Using the Proximity
Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
5.5.4 Unsupervised Evaluation of Hierarchical Clustering . . . 367
5.5.5 Determining the Correct Number of Clusters . . . . . . 369
5.5.6 Clustering Tendency . . . . . . . . . . . . . . . . . . . . 370
5.5.7 Supervised Measures of Cluster Validity . . . . . . . . . 371
5.5.8 Assessing the Significance of Cluster Validity Measures . 376
5.5.9 Choosing a Cluster Validity Measure . . . . . . . . . . . 378
5.6 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 379
5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

6 Classification: Alternative Techniques 395

6.1 Types of Classifiers . . . . . . . . . . . . . . . . . . . . . . . . . 395
6.2 Rule-Based Classifier . . . . . . . . . . . . . . . . . . . . . . . . 397
6.2.1 How a Rule-Based Classifier Works . . . . . . . . . . . . 399
6.2.2 Properties of a Rule Set . . . . . . . . . . . . . . . . . . 400
6.2.3 Direct Methods for Rule Extraction . . . . . . . . . . . 401
6.2.4 Indirect Methods for Rule Extraction . . . . . . . . . . 406
6.2.5 Characteristics of Rule-Based Classifiers . . . . . . . . . 408
6.3 Nearest Neighbor Classifiers . . . . . . . . . . . . . . . . . . . . 410
6.3.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 411
6.3.2 Characteristics of Nearest Neighbor Classifiers . . . . . 412
6.4 Naı̈ve Bayes Classifier . . . . . . . . . . . . . . . . . . . . . . . 414
6.4.1 Basics of Probability Theory . . . . . . . . . . . . . . . 415
6.4.2 Naı̈ve Bayes Assumption . . . . . . . . . . . . . . . . . . 420
 Contents 15

6.5 Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . . . . 429

6.5.1 Graphical Representation . . . . . . . . . . . . . . . . . 429
6.5.2 Inference and Learning . . . . . . . . . . . . . . . . . . . 435
6.5.3 Characteristics of Bayesian Networks . . . . . . . . . . . 444
6.6 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . 445
6.6.1 Logistic Regression as a Generalized Linear Model . . . 446
6.6.2 Learning Model Parameters . . . . . . . . . . . . . . . . 447
6.6.3 Characteristics of Logistic Regression . . . . . . . . . . 450
6.7 Artificial Neural Network (ANN) . . . . . . . . . . . . . . . . . 451
6.7.1 Perceptron . . . . . . . . . . . . . . . . . . . . . . . . . 452
6.7.2 Multi-layer Neural Network . . . . . . . . . . . . . . . . 456
6.7.3 Characteristics of ANN . . . . . . . . . . . . . . . . . . 463
6.8 Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
6.8.1 Using Synergistic Loss Functions . . . . . . . . . . . . . 465
6.8.2 Using Responsive Activation Functions . . . . . . . . . . 468
6.8.3 Regularization . . . . . . . . . . . . . . . . . . . . . . . 470
6.8.4 Initialization of Model Parameters . . . . . . . . . . . . 473
6.8.5 Characteristics of Deep Learning . . . . . . . . . . . . . 477
6.9 Support Vector Machine (SVM) . . . . . . . . . . . . . . . . . . 478
6.9.1 Margin of a Separating Hyperplane . . . . . . . . . . . . 478
6.9.2 Linear SVM . . . . . . . . . . . . . . . . . . . . . . . . . 480
6.9.3 Soft-margin SVM . . . . . . . . . . . . . . . . . . . . . . 486
6.9.4 Nonlinear SVM . . . . . . . . . . . . . . . . . . . . . . . 492
6.9.5 Characteristics of SVM . . . . . . . . . . . . . . . . . . 496
6.10 Ensemble Methods . . . . . . . . . . . . . . . . . . . . . . . . . 498
6.10.1 Rationale for Ensemble Method . . . . . . . . . . . . . . 499
6.10.2 Methods for Constructing an Ensemble Classifier . . . . 499
6.10.3 Bias-Variance Decomposition . . . . . . . . . . . . . . . 502
6.10.4 Bagging . . . . . . . . . . . . . . . . . . . . . . . . . . . 504
6.10.5 Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . . 507
6.10.6 Random Forests . . . . . . . . . . . . . . . . . . . . . . 512
6.10.7 Empirical Comparison among Ensemble Methods . . . . 514
6.11 Class Imbalance Problem . . . . . . . . . . . . . . . . . . . . . 515
6.11.1 Building Classifiers with Class Imbalance . . . . . . . . 516
6.11.2 Evaluating Performance with Class Imbalance . . . . . . 520
6.11.3 Finding an Optimal Score Threshold . . . . . . . . . . . 524
6.11.4 Aggregate Evaluation of Performance . . . . . . . . . . 525
6.12 Multiclass Problem . . . . . . . . . . . . . . . . . . . . . . . . . 532
6.13 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 535
6.14 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
16 Contents

7 Association Analysis: Advanced Concepts 559

7.1 Handling Categorical Attributes . . . . . . . . . . . . . . . . . 559
7.2 Handling Continuous Attributes . . . . . . . . . . . . . . . . . 562
7.2.1 Discretization-Based Methods . . . . . . . . . . . . . . . 562
7.2.2 Statistics-Based Methods . . . . . . . . . . . . . . . . . 566
7.2.3 Non-discretization Methods . . . . . . . . . . . . . . . . 568
7.3 Handling a Concept Hierarchy . . . . . . . . . . . . . . . . . . 570
7.4 Sequential Patterns . . . . . . . . . . . . . . . . . . . . . . . . . 572
7.4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 573
7.4.2 Sequential Pattern Discovery . . . . . . . . . . . . . . . 576
7.4.3 Timing Constraints∗ . . . . . . . . . . . . . . . . . . . . 581
7.4.4 Alternative Counting Schemes∗ . . . . . . . . . . . . . . 585
7.5 Subgraph Patterns . . . . . . . . . . . . . . . . . . . . . . . . . 587
7.5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 588
7.5.2 Frequent Subgraph Mining . . . . . . . . . . . . . . . . 591
7.5.3 Candidate Generation . . . . . . . . . . . . . . . . . . . 595
7.5.4 Candidate Pruning . . . . . . . . . . . . . . . . . . . . . 601
7.5.5 Support Counting . . . . . . . . . . . . . . . . . . . . . 601
7.6 Infrequent Patterns∗ . . . . . . . . . . . . . . . . . . . . . . . . 601
7.6.1 Negative Patterns . . . . . . . . . . . . . . . . . . . . . 602
7.6.2 Negatively Correlated Patterns . . . . . . . . . . . . . . 603
7.6.3 Comparisons among Infrequent Patterns, Negative
Patterns, and Negatively Correlated Patterns . . . . . . 604
7.6.4 Techniques for Mining Interesting Infrequent Patterns . 606
7.6.5 Techniques Based on Mining Negative Patterns . . . . . 607
7.6.6 Techniques Based on Support Expectation . . . . . . . . 609
7.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 613
7.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618

8 Cluster Analysis: Additional Issues and Algorithms 633

8.1 Characteristics of Data, Clusters, and Clustering Algorithms . 634
8.1.1 Example: Comparing K-means and DBSCAN . . . . . . 634
8.1.2 Data Characteristics . . . . . . . . . . . . . . . . . . . . 635
8.1.3 Cluster Characteristics . . . . . . . . . . . . . . . . . . . 637
8.1.4 General Characteristics of Clustering Algorithms . . . . 639
8.2 Prototype-Based Clustering . . . . . . . . . . . . . . . . . . . . 641
8.2.1 Fuzzy Clustering . . . . . . . . . . . . . . . . . . . . . . 641
8.2.2 Clustering Using Mixture Models . . . . . . . . . . . . . 647
8.2.3 Self-Organizing Maps (SOM) . . . . . . . . . . . . . . . 657
8.3 Density-Based Clustering . . . . . . . . . . . . . . . . . . . . . 664
 Contents 17

8.3.1 Grid-Based Clustering . . . . . . . . . . . . . . . . . . . 664

8.3.2 Subspace Clustering . . . . . . . . . . . . . . . . . . . . 668
8.3.3 DENCLUE: A Kernel-Based Scheme for Density-Based
Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . 672
8.4 Graph-Based Clustering . . . . . . . . . . . . . . . . . . . . . . 676
8.4.1 Sparsification . . . . . . . . . . . . . . . . . . . . . . . . 677
8.4.2 Minimum Spanning Tree (MST) Clustering . . . . . . . 678
8.4.3 OPOSSUM: Optimal Partitioning of Sparse Similarities
Using METIS . . . . . . . . . . . . . . . . . . . . . . . . 679
8.4.4 Chameleon: Hierarchical Clustering with Dynamic
Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . 680
8.4.5 Spectral Clustering . . . . . . . . . . . . . . . . . . . . . 686
8.4.6 Shared Nearest Neighbor Similarity . . . . . . . . . . . 693
8.4.7 The Jarvis-Patrick Clustering Algorithm . . . . . . . . . 696
8.4.8 SNN Density . . . . . . . . . . . . . . . . . . . . . . . . 698
8.4.9 SNN Density-Based Clustering . . . . . . . . . . . . . . 699
8.5 Scalable Clustering Algorithms . . . . . . . . . . . . . . . . . . 701
8.5.1 Scalability: General Issues and Approaches . . . . . . . 701
8.5.2 BIRCH . . . . . . . . . . . . . . . . . . . . . . . . . . . 704
8.5.3 CURE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706
8.6 Which Clustering Algorithm? . . . . . . . . . . . . . . . . . . . 710
8.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 713
8.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719

9 Anomaly Detection 723

9.1 Characteristics of Anomaly Detection Problems . . . . . . . . . 725
9.1.1 A Definition of an Anomaly . . . . . . . . . . . . . . . . 725
9.1.2 Nature of Data . . . . . . . . . . . . . . . . . . . . . . . 726
9.1.3 How Anomaly Detection is Used . . . . . . . . . . . . . 727
9.2 Characteristics of Anomaly Detection Methods . . . . . . . . . 728
9.3 Statistical Approaches . . . . . . . . . . . . . . . . . . . . . . . 730
9.3.1 Using Parametric Models . . . . . . . . . . . . . . . . . 730
9.3.2 Using Non-parametric Models . . . . . . . . . . . . . . . 734
9.3.3 Modeling Normal and Anomalous Classes . . . . . . . . 735
9.3.4 Assessing Statistical Significance . . . . . . . . . . . . . 737
9.3.5 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 738
9.4 Proximity-based Approaches . . . . . . . . . . . . . . . . . . . . 739
9.4.1 Distance-based Anomaly Score . . . . . . . . . . . . . . 739
9.4.2 Density-based Anomaly Score . . . . . . . . . . . . . . . 740
9.4.3 Relative Density-based Anomaly Score . . . . . . . . . . 742
9.4.4 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 743
18 Contents

9.5 Clustering-based Approaches . . . . . . . . . . . . . . . . . . . 744

9.5.1 Finding Anomalous Clusters . . . . . . . . . . . . . . . 744
9.5.2 Finding Anomalous Instances . . . . . . . . . . . . . . . 745
9.5.3 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 748
9.6 Reconstruction-based Approaches . . . . . . . . . . . . . . . . . 748
9.6.1 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 751
9.7 One-class Classification . . . . . . . . . . . . . . . . . . . . . . 752
9.7.1 Use of Kernels . . . . . . . . . . . . . . . . . . . . . . . 753
9.7.2 The Origin Trick . . . . . . . . . . . . . . . . . . . . . . 754
9.7.3 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 758
9.8 Information Theoretic Approaches . . . . . . . . . . . . . . . . 758
9.8.1 Strengths and Weaknesses . . . . . . . . . . . . . . . . . 760
9.9 Evaluation of Anomaly Detection . . . . . . . . . . . . . . . . . 760
9.10 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 762
9.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769

10 Avoiding False Discoveries 775

10.1 Preliminaries: Statistical Testing . . . . . . . . . . . . . . . . . 776
10.1.1 Significance Testing . . . . . . . . . . . . . . . . . . . . 776
10.1.2 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . 781
10.1.3 Multiple Hypothesis Testing . . . . . . . . . . . . . . . . 787
10.1.4 Pitfalls in Statistical Testing . . . . . . . . . . . . . . . 796
10.2 Modeling Null and Alternative Distributions . . . . . . . . . . . 798
10.2.1 Generating Synthetic Data Sets . . . . . . . . . . . . . . 801
10.2.2 Randomizing Class Labels . . . . . . . . . . . . . . . . . 802
10.2.3 Resampling Instances . . . . . . . . . . . . . . . . . . . 802
10.2.4 Modeling the Distribution of the Test Statistic . . . . . 803
10.3 Statistical Testing for Classification . . . . . . . . . . . . . . . . 803
10.3.1 Evaluating Classification Performance . . . . . . . . . . 803
10.3.2 Binary Classification as Multiple Hypothesis Testing . . 805
10.3.3 Multiple Hypothesis Testing in Model Selection . . . . . 806
10.4 Statistical Testing for Association Analysis . . . . . . . . . . . 807
10.4.1 Using Statistical Models . . . . . . . . . . . . . . . . . . 808
10.4.2 Using Randomization Methods . . . . . . . . . . . . . . 814
10.5 Statistical Testing for Cluster Analysis . . . . . . . . . . . . . . 815
10.5.1 Generating a Null Distribution for Internal Indices . . . 816
10.5.2 Generating a Null Distribution for External Indices . . . 818
10.5.3 Enrichment . . . . . . . . . . . . . . . . . . . . . . . . . 818
10.6 Statistical Testing for Anomaly Detection . . . . . . . . . . . . 820
10.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 823
10.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 828
 Contents 19

Author Index 836

Subject Index 849

Copyright Permissions 859

This page is intentionally left blank
 1

Introduction
Rapid advances in data collection and storage technology, coupled with the
ease with which data can be generated and disseminated, have triggered the
explosive growth of data, leading to the current age of big data. Deriving
actionable insights from these large data sets is increasingly important in
decision making across almost all areas of society, including business and
industry; science and engineering; medicine and biotechnology; and govern-
ment and individuals. However, the amount of data (volume), its complexity
(variety), and the rate at which it is being collected and processed (velocity)
have simply become too great for humans to analyze unaided. Thus, there is
a great need for automated tools for extracting useful information from the
big data despite the challenges posed by its enormity and diversity.
Data mining blends traditional data analysis methods with sophisticated
algorithms for processing this abundance of data. In this introductory chapter,
we present an overview of data mining and outline the key topics to be covered
in this book. We start with a description of some applications that require
more advanced techniques for data analysis.

Business and Industry Point-of-sale data collection (bar code scanners,

radio frequency identification (RFID), and smart card technology) have al-
lowed retailers to collect up-to-the-minute data about customer purchases
at the checkout counters of their stores. Retailers can utilize this informa-
tion, along with other business-critical data, such as web server logs from
e-commerce websites and customer service records from call centers, to help
them better understand the needs of their customers and make more informed
business decisions.
Data mining techniques can be used to support a wide range of busi-
ness intelligence applications, such as customer profiling, targeted marketing,
22 Chapter 1 Introduction

workflow management, store layout, fraud detection, and automated buying

and selling. An example of the last application is high-speed stock trading,
where decisions on buying and selling have to be made in less than a second
using data about financial transactions. Data mining can also help retailers
answer important business questions, such as “Who are the most profitable
customers?”; “What products can be cross-sold or up-sold?”; and “What is
the revenue outlook of the company for next year?” These questions have in-
spired the development of such data mining techniques as association analysis
(Chapters 4 and 7).
As the Internet continues to revolutionize the way we interact and make
decisions in our everyday lives, we are generating massive amounts of data
about our online experiences, e.g., web browsing, messaging, and posting on
social networking websites. This has opened several opportunities for business
applications that use web data. For example, in the e-commerce sector, data
about our online viewing or shopping preferences can be used to provide per-
sonalized recommendations of products. Data mining also plays a prominent
role in supporting several other Internet-based services, such as filtering spam
messages, answering search queries, and suggesting social updates and connec-
tions. The large corpus of text, images, and videos available on the Internet
has enabled a number of advancements in data mining methods, including
deep learning, which is discussed in Chapter 6. These developments have led
to great advances in a number of applications, such as object recognition,
natural language translation, and autonomous driving.
Another domain that has undergone a rapid big data transformation is
the use of mobile sensors and devices, such as smart phones and wearable
computing devices. With better sensor technologies, it has become possible to
collect a variety of information about our physical world using low-cost sensors
embedded on everyday objects that are connected to each other, termed the
Internet of Things (IOT). This deep integration of physical sensors in digital
systems is beginning to generate large amounts of diverse and distributed data
about our environment, which can be used for designing convenient, safe, and
energy-efficient home systems, as well as for urban planning of smart cities.

Medicine, Science, and Engineering Researchers in medicine, science,

and engineering are rapidly accumulating data that is key to significant new
discoveries. For example, as an important step toward improving our under-
standing of the Earth’s climate system, NASA has deployed a series of Earth-
orbiting satellites that continuously generate global observations of the land
 23

surface, oceans, and atmosphere. However, because of the size and spatio-
temporal nature of the data, traditional methods are often not suitable for
analyzing these data sets. Techniques developed in data mining can aid Earth
scientists in answering questions such as the following: “What is the relation-
ship between the frequency and intensity of ecosystem disturbances such as
droughts and hurricanes to global warming?”; “How is land surface precipita-
tion and temperature affected by ocean surface temperature?”; and “How well
can we predict the beginning and end of the growing season for a region?”
As another example, researchers in molecular biology hope to use the large
amounts of genomic data to better understand the structure and function of
genes. In the past, traditional methods in molecular biology allowed scientists
to study only a few genes at a time in a given experiment. Recent break-
throughs in microarray technology have enabled scientists to compare the
behavior of thousands of genes under various situations. Such comparisons
can help determine the function of each gene, and perhaps isolate the genes
responsible for certain diseases. However, the noisy, high-dimensional nature
of data requires new data analysis methods. In addition to analyzing gene
expression data, data mining can also be used to address other important
biological challenges such as protein structure prediction, multiple sequence
alignment, the modeling of biochemical pathways, and phylogenetics.
Another example is the use of data mining techniques to analyze electronic
health record (EHR) data, which has become increasingly available. Not very
long ago, studies of patients required manually examining the physical records
of individual patients and extracting very specific pieces of information per-
tinent to the particular question being investigated. EHRs allow for a faster
and broader exploration of such data. However, there are significant challenges
since the observations on any one patient typically occur during their visits
to a doctor or hospital and only a small number of details about the health
of the patient are measured during any particular visit.
Currently, EHR analysis focuses on simple types of data, e.g., a patient’s
blood pressure or the diagnosis code of a disease. However, large amounts of
more complex types of medical data are also being collected, such as electrocar-
diograms (ECGs) and neuroimages from magnetic resonance imaging (MRI)
or functional Magnetic Resonance Imaging (fMRI). Although challenging to
analyze, this data also provides vital information about patients. Integrating
and analyzing such data, with traditional EHR and genomic data is one of the
capabilities needed to enable precision medicine, which aims to provide more
personalized patient care.
24 Chapter 1 Introduction

1.1 What Is Data Mining?

Data mining is the process of automatically discovering useful information in
large data repositories. Data mining techniques are deployed to scour large
data sets in order to find novel and useful patterns that might otherwise
remain unknown. They also provide the capability to predict the outcome of
a future observation, such as the amount a customer will spend at an online
or a brick-and-mortar store.
Not all information discovery tasks are considered to be data mining.
Examples include queries, e.g., looking up individual records in a database or
finding web pages that contain a particular set of keywords. This is because
such tasks can be accomplished through simple interactions with a database
management system or an information retrieval system. These systems rely
on traditional computer science techniques, which include sophisticated index-
ing structures and query processing algorithms, for efficiently organizing and
retrieving information from large data repositories. Nonetheless, data mining
techniques have been used to enhance the performance of such systems by
improving the quality of the search results based on their relevance to the
input queries.

Data Mining and Knowledge Discovery in Databases

Data mining is an integral part of knowledge discovery in databases
(KDD), which is the overall process of converting raw data into useful infor-
mation, as shown in Figure 1.1. This process consists of a series of steps, from
data preprocessing to postprocessing of data mining results.

Input Data Data

Postprocessing Information
Data Preprocessing Mining

Feature Selection
Filtering Patterns
Dimensionality Reduction
Visualization
Normalization
Pattern Interpretation
Data Subsetting

Figure 1.1. The process of knowledge discovery in databases (KDD).

Another Random Scribd Document
with Unrelated Content
If the man bitten by a snake be bold, he cuts off the bitten part.

Some have the bitten part branded.

Those who have no ulcer in the mouth suck the poison, and spit it out.

The powder of the fruit of the Nol Vel is also administered with water.

Sometimes emetics and purgative medicines are given.

A mixture of pepper and clarified butter is also believed to be efficacious.66

Other remedies for the cure of snake bite, are as follows:—

The patient is made to wear a cotton thread in the name of Charmālia Nāg,
Sharmalia Nāg, or Vasangi Nāg, and certain observances, as stated above,
are promised to the snake deity.67

The ends of peacock feathers are pounded and smoked in a chilum (clay
pipe) by the patient.68

A moharo (stone found in the head of a snake supposed to be a cure for

snake poison) is applied to the wound caused by the bite. It absorbs the
poison, and on being dipped into milk, transfers the poison to the milk.

Thus it can be used any number of times.69

There is a Girāsia in Lakhtar who is believed to cure patients suffering from

snake poison. As soon as a person is bitten by a snake, one of the garments
worn by him is taken to the Girāsia, who ties it into a knot and this cures the
patient.70

There is a Bāva in Rajpāra, a village near Anandpur. He and all the

members of his family are reputed to be able to cure snake-bites. When a
person is bitten by a snake, he or a friend goes to the Bāvā’s house and
informs him or any member of his family of the occurrence. The Bāva or
the person who receives the intimation folds into a knot a garment of the
informant, which he afterwards unfolds. As soon as this is done, the patient
is in great pain, loses his senses, is seized with convulsions and tells why
the snake bit him. Thereupon the relatives of the patient implore the pardon
of the snake, which is granted on condition that the patient should give alms
to the poor.71

In some places, Bhagats or devotees of Mātāji are invited to dinner along

with a number of exorcists, who are generally Rabāris. After they have
assembled at the house of the patient, they start out in a procession headed
by one who holds in his hand a bunch of peacock feathers, to bathe in a
river. On their way to and back from the river they sing songs in praise of
the goddess to the accompaniment of drums and other musical instruments.
After their return from the river, the whole party are treated to a feast,
which is supposed to cure the patient of the effects of the snake-bite.72

Some people believe that snakes, like evil spirits, can enter the bodies of
human beings. Such persons, when possessed, are supposed to have the
power of curing snake-bites.73

Every village has an exorcist who is a specialist in curing the effects of

snake-bites. When a person is bitten by a snake the exorcist is at once sent
for. He gives the patient Nim leaves and pepper to chew, to determine the
extent of the effect of the bite. Next he asks one of those present to bathe
and bring water in an unused earthen jar. He then recites incantations, and
sprinkles water from the jar over the body of the patient. If this does not
counteract the effects of the poison, he throws red-hot pieces of charcoal at
the patient, when the snake speaks through the patient and states that he bit
the patient because he committed a certain offence, and that he will leave
him if certain offerings are made. After he has ceased speaking, the patient
begins to shake and to crawl about like a snake, and is then cured. If the
man be doomed to death, the snake would say, “I have bitten him by the
order of the god of death, and I will not leave him without taking his life.”74

Sometimes the exorcist fans the patient with branches of the Nim tree,
reciting mantras, and thereupon the patient becomes possessed by the snake
and declares the cause of his offence.
Some exorcists present a magic epistle or charm asking the snake that bit
the patient to be present. The snake obeys the call, and appears before the
exorcist. The latter then asks the snake to suck the poison from the wound
of the patient, which is done by the snake, and the patient is then cured.75

In some places, the exorcist ties up the patient when the snake tells the
cause of the bite. Next the exorcist calls on the snake to leave the body of
the patient, who then begins to crawl about like a snake and is cured.

On some occasions, the exorcist slaps the cheek of the person who calls him
to attend the patient. It is said that the poison disappears as soon as the slap
is given.76

Some exorcists take a stick having seven joints and break them one by one.
As the stick is broken, the patient recovers, his recovery being complete
when the seventh joint is broken.77

It is believed that the Dhedas are the oldest worshippers of Nāgs or snakes.
When a person is bitten by a snake, he is seated near a Dheda, who prays
the snake to leave the body of the patient. It is said that in some cases this
method proves efficacious in curing the patient.78

It is stated that exorcists who know the mantra (incantation) for the cure of
snake-bites must lead a strictly moral life. If they touch a woman in child-
bed or during her period the mantra loses its power. This can be regained
through purification, bathing, and by reciting the mantra while inhaling the
smoke of burning frankincense. Some exorcists abstain from certain kinds
of vegetables and sweets, e. g., the Mogri (Rat-tailed radish), Julebi (a kind
of sweet), etc. They have also to abstain from articles of a colour like that of
a snake.79

A belief prevails that there is a precious stone in the head of the snake. Such
stones are called mohors. They are occasionally shown to the people by
snake-charmers, who declare that it is very difficult to procure them.
It is stated that on dark nights snakes take these mohors out of their head
and place them on prominent spots in order to be able to move about in the
dark by their light.80

It is believed that snakes give these mohors to those who please them. If
one tries to take a mohor by force, the snake swallows it and dissolves it
into water.81

As stated above, the mohor has the property of absorbing the poison from
snake-bites.

It is because a snake is believed to hold a precious stone in its head that it is

called manidhar, that is, holder of a jewel.82

It is believed by some people that the mohor shines the most when a
rainbow appears in the sky.83

According to the Purānas the pātāl or nether world is as beautiful as

heaven. It is inhabited by Nāgs or snakes in human form. The Nāg girls are
reputed to be so handsome that an extraordinarily beautiful girl is
commonly likened to a Nāg girl.

It is believed that in ancient times inter-marriages between Nāgs and human

beings were common.84

It is a common belief that Kshetrapāl, the guardian snake of fields, married

human brides. So to propitiate him, his image is installed on the marriage
altar, and the bride takes three turns round it when walking round the
sacrificial fire with the bridegroom.85

According to the Purānas, king Dasharath married a Nāg girl Sumitra.86

Similarly Indrajit, the son of Rāvan, the Lord of Lanka or Ceylon, married a
Nāg girl.87

At times snakes are seen in houses. They are believed to be the guardians of
the houses, and worshipped with offerings of lamps fed with ghi. After
worship, the members of the family pray to the snake, “Oh snake! Thou art
our guardian. Protect our health and wealth. We are thy children and live in
thy garden.”88

Some people believe that the spirits of deceased ancestors, on account of

the anxiety for the welfare of progeny, become snakes and guard the
house.89

1 The School Master of Ganod. ↑

2 The School Master of Todia. ↑
3 Mr. K. D. Desāi. ↑
4 The School Master of Dhānk. ↑
5 The School Master of Moti Parabdi. ↑
6 The School Master of Todia. ↑
7 Mr. K. D. Desāi. ↑
8 Mr. K. D. Desāi. ↑
9 The School Master of Chhatrāsa. ↑
10 The School Master of Todia. ↑
11 The School Master of Zinzuvāda. ↑
12 The School Master of Dadvi. ↑
13 The School Master of Vanod. ↑
14 The School Master of Kolki. ↑
15 The School Master of Dadvi. ↑
16 The School Master of Dhānk. ↑
17 The School Master of Kolki. ↑
18 The School Master of Dhānk. ↑
19 The School Master of Dhānk. ↑
20 The School Master of Bhāyāvadar. ↑
21 The School Master of Uptela. ↑
22 The School Master of Anandpur. ↑
23 The School Masters of Ganod and Khirāsara. ↑
24 The School Master of Dhānk. ↑
25 The School Master of Kolki. ↑
26 The School Master of Chhatrāsa. ↑
27 The School Master of Limbdi. ↑
28 The School Master of Limbdi. ↑
29 The School Master of Ganod. ↑
30 The School Master of Jasdan. ↑
31 The School Masters of Dhānk and Dadvi. ↑
32 The School Masters of Dhānk and Māvaiya. ↑
33 The School Master of Ganod. ↑
34 Mr. K. D. Desāi. ↑
35 The School Master of Dhānk. ↑
36 The School Master of Khirāsara. ↑
37 The School Master of Dhānk. ↑
38 The School Master of Vanod. ↑
39 The School Master of Chhatrāsa. ↑
40 The School Masters of Uptela and Limbdi. ↑
41 The School Master of Zinzuvāda. ↑
42 The School Master of Kotda Sangani. ↑
43 The Schoolmistress, Barton Female Training College, Rajkot. ↑
44 The School Master of Khirāsara. ↑
45 The School Master of Sānka. ↑
46 The School Master of Dhānk. ↑
47 The School Master of Jodia. ↑
48 The School Master of Ganod. ↑
49 The School Master of Jodia. ↑
50 The School Master of Mota Devalia. ↑
51 The School Master of Dhānk. ↑
52 The School Master of Ganod. ↑
53 The School Master of Vanod. ↑
54 The School Master of Ganod. ↑
55 The School Mistress, Barton Female Training College, Rājkot. ↑
56 The School Master of Dhānk. ↑
57 The School Master of Kotda Sangani. ↑
58 The School Master of Charādva. ↑
59 The School Master of Khirāsara. ↑
60 The School Master of Ganod. ↑
61 The School Master of Vanod. ↑
62 The School Master of Songadh. ↑
63 The School Master of Sanka. ↑
64 The School Master of Kolki. ↑
65 The School Master of Todia. ↑
66 The School Master of Dhānk. ↑
67 The School Master of Kotda Sangani. ↑
68 The School Mistress, Barton Female Training College, Rājkot. ↑
69 The D. E. Inspector, Hālār. ↑
70 The School Master of Lilāpur. ↑
71 The School Master of Anandpur. ↑
72 The School Master of Dhānk. ↑
73 The School Master of Dadvi. ↑
74 The School Master of Dhānk. ↑
75 The School Master of Chhatrāsa. ↑
76 The D. E. Inspector, Hālār. ↑
77 The School Master of Songadh. ↑
78 The School Master of Sānka. ↑
79 The School Mistress, Barton Female Training College, Rājkot. ↑
80 The School Master of Dhānk. ↑
81 The School Master of Chhatrāsa. ↑
82 Mr. K. D. Desāi. ↑
83 The School Master of Kolki. ↑
84 The School Master of Vanod. ↑
85 The School Master of Kolki. ↑
86 The School Mistress, Civil Station Girls’ School, Rājkot. ↑
87 The School Master of Zinzuvāda. ↑
88 The School Master of Vanod, and Mr. K. D. Desāi. ↑
89 The School Master of Sāyala. ↑
CHAPTER VIII.
 TOTEMISM AND FETISHISM.
The worship of totems is not known to prevail in Gujarāt, but the names of
persons and clans or families are occasionally derived from animals and
plants.

Instances of names derived from animals are given below:—

NAME. ANIMAL FROM WHICH NAME. ANIMAL FROM WHICH

DERIVED. DERIVED.
1. Hāthibhāi Hathi—an elephant. 8. Mankodia Mankoda—a black ant.
2. Vāghajibhāi Vāgh—a tiger. 9. Tido Tid—a locust.2
3. Nāgjibhāi Nāg—a snake. 10. Hansrāj Hansa—a goose.
4. Popatbhāi Popat—a parrot. 11. Vinchi3 Vinchi—a female scorpion.
5. Morbhāi Mor—a peacock. 12. Olo Olo—a species of bird.
6. Chaklibhat Chakli—a sparrow.1 13. Ajo Aja—a goat.
7. Kido Kidi—an ant. 14. Mena3 Mena—a species of bird.4

The Kāli Parāj or aboriginal tribes in Gujarāt give such names as Kāgdo
(crow), Kolo (Jackal), Bilādo (cat), Kutro (dog) to their children according
as one or other of these animals is heard to cry at the time of birth.5

The following are instances of names derived from plants:—

NAME. PLANT OR TREE FROM NAME. PLANT OR TREE FROM

WHICH DERIVED. WHICH DERIVED.
1. Gulāb6 Gulāb—the rose. 8. Dudhi3 Dudhi—Pumpkin.
2. Ambo Ambo—the mango. 9. Lavengi3 Laveng—Clove.
3. Tulsibāi3 Tulsi—the sweet basil. 10. Mulo Mulo—Radish.
4. Tulsidās Ditto. 11. Limbdo Limbdo—The Nim tree.
5. Kesharbāi3 Keshar—Saffron. 12. Mako Maki—Maize.8
 NAME. PLANT OR TREE FROM NAME. PLANT OR TREE FROM
WHICH DERIVED. WHICH DERIVED.
6. Galālbāi3 Galāl—Red powder.7 13. Champo Champa—Michelia
Champaca.9
7. Bili3 Bili—Aegle marmelos.

Instances of family or clan names derived from trees and animals are as
follows:—

NAME. DERIVATION. NAME. DERIVATION.

1. Untia Unt—camel. 6. Bhensdadia Bhensa—A buffalo.
2. Gadheda Gadheda—An ass. 7. Ghetiya Gheta—A sheep.
3. Dedakia Dedako—A frog. 8. Sāvaj A species of wild animals.10
4. Balada Balad—An ox. 9. Kakadia Kakadi—cucumber.11
5. Godhani Godho—A bull.

The cow, the she-goat, the horse, the deer, peacock, the Tilad or singing
sparrow, the goose, the Nāg or snake, the eagle, the elephant and the male
monkey are believed to be sacred by all Hindus. Of these, the greatest
sanctity attaches to the cow. Her urine is sipped for the atonement of sins.
The cow is also revered by the Pārsis.12

The mouth of the she-goat and the smell of the horse are considered
sacred.12

An elephant is considered sacred, because when the head of Ganpati was

chopped off by Shiva, the head of an elephant was joined to his trunk.13

The peacock is considered sacred on account of its being the conveyance of

Sarasvati, the goddess of learning.13

A male monkey is held holy, because it is supposed to represent the monkey

god Māruti.13
Some sanctity attaches to the rat also, as it is the conveyance of the god
Ganpati. He is called Māma or maternal uncle by the Hindus.14

The pig is held taboo by the Musalmāns.15

Brāhmans, Baniās, Bhātiās, Kunbis, Sutārs and Darjis abstain from flesh
and liquor.16

Some Brāhmans and Baniās do not eat tādiās (fruit of the palm tree) as they
look like human eyes.17

Some Brāhmans abstain from garlic and onions. Some do not eat Kodra
(punctured millet).18

The masur (Lentil) pulse is not eaten by Brāhmans and Baniās, because,
when cooked, it looks red like blood.19

The Humbad Baniās do not eat whey, milk, curdled milk and clarified
butter.20

The Shrāvaks abstain from the suran (Elephant foot), potatoes and roots
that grow underground.21

Mahomedans abstain from the suran, because “su” the first letter of the
word suran is also the first letter of their taboo’d animal the pig.22

There are some deities associated with the worship of animals. These
animals, with the deities with whom they are connected, are given below.

1. Pothio or the bull is believed to be the vehicle of god Shiva. In all

temples of Shiva its image is installed, facing the image of Shiva in the
centre of the temple.

2. Sinha or the lion is believed to be the vehicle of Pārvati, the consort of

Shiva. The lion is also connected with the demon planet Rāhu.

3. Hansa the goose is associated with Brahma the creator.

4. Gadhedo the ass is believed to be connected with Shitala, the goddess of
small pox.

5. Undar the mouse is the conveyance of Ganpati.

6. Mor the peacock is the conveyance of Sarasvati, the goddess of learning.

The peacock is also associated with Kārtik Swāmi.

7. Garud the eagle is the conveyance of the god Vishnu.

8. Pādo the male buffalo is the conveyance of Devis or goddesses.

9. Ghodo the horse is the conveyance of the Sun. The horse is also
associated with the planet Guru or Jupiter and Shukra or Venus.

10. Mrig the deer is supposed to be the conveyance of the Moon as well as
of Mangal or Mars.

11. Balad the ox is connected with Mars and Shani or Saturn.

12. Hāthi the elephant is supposed to be the conveyance of Indra. It is also

connected with Budha or Mercury.

13 The tiger is the conveyance of the goddess Ambāji.

The animals mentioned above are worshipped along with deities and
planets with whom they are associated23.

It is generally believed that the earth is supported by a tortoise. So,

whenever the goddess earth or Prithvi is worshipped, the tortoise is also
worshipped.24

In the temples of the Mātās cocks and hens, and in the temple of Kāl
Bhairav, dogs, are worshipped.25

For the propitiation of goddesses and evil spirits, male goats, he-buffaloes
and cocks are sacrificed.26
In his first incarnation, the god Vishnu was born as a fish, in the second as
an alligator, and in the third as a boar. For this reason the images of these
animals are worshipped.27

All the gods, goddesses and spirits mentioned in the preceding pages are
represented by idols made of stone, metal or wood. In addition to stone
idols of gods there are certain stones which are considered to represent gods
and worshipped as such. Some of these stones are described below.

All the stones found in the river Narbada are believed to represent the god
Shiva and worshipped.

There is a kind of stone found in the river Gandaki which is smooth on one
side and porous on the other. It is either round or square and about five
inches in length. This stone is called Shāligrām and is believed to represent
the god Vishnu. It is kept in the household gods and worshipped daily.

There is another kind of hard, white, porous stone found near Dwārka. It is
also worshipped along with the idol of Vishnu.

Sometimes tridents are drawn with red lead on stones to represent

goddesses.28

There is a tank near the Pir in Kutiāna in which bored stones are found
floating on the surface of the water. These stones are considered sacred.29

Certain stones are considered sacred on account of their supposed curative

properties. One of such stones is called Pāro. It is believed to be efficacious
in curing rheumatism.30

There is also a kind of red stone which is supposed to cure skin diseases.31

Each of the nine planets is supposed to be in touch with a stone of a

particular colour. For instance, the stone in touch with Shani or Saturn is
black, and that with Mangal or Mars is red. These stones are bored, and set
in rings which are worn by persons suffering from the influence of these
planets.
A kind of stone called Akik, found in abundance in Cambay, is considered
sacred by the Mahomedan saints, who wear garlands made of beads carved
out of these stones.32

In ancient times human sacrifices were offered on certain occasions. Now-

a-days, in place of a human being, a cocoanut or a Kolu (Cucurbita
maxima) is offered. At the time of making the offering, the cocoanut is
plastered with red lead and other holy applications and covered with a silk
cloth. The Kolu is offered by cutting it into two pieces with a stroke of a
knife or sword.33

Sometimes an image of the flour of Adad is sacrificed in place of a human

being.34

This sacrifice is generally made on the eighth or tenth day of the bright half
of Ashvin.

In place of human blood, milk mixed with gulāl (red powder) and molasses
is offered.35

In ancient times, when a well was dug, a human sacrifice was made to it if
it did not yield water, with the belief that this would bring water into the
well. Now-a-days, instead of this sacrifice, blood from the fourth finger of a
man is sprinkled over the spot.36

It is also related that in ancient times, when a king was crowned, a human
sacrifice was offered. Now-a-days, instead of this sacrifice, the king’s
forehead is marked with the blood from the fourth finger of a low caste
Hindu at the time of the coronation ceremony.37

There are a few stones which are supposed to have the power of curing
certain diseases. One of such stones is known as Ratvano Pāro. It is found
at a distance of about two miles from Kolki. It is marked with red lines. It is
bored and worn round the neck by persons suffering from ratawa38 (a
disease in which red spots or pimples are seen on the skin).
There is another stone called Suleimani Pāro which is supposed to have the
power of curing many diseases.39

There is a kind of white semi-circular stone which is supposed to cure eye

diseases when rubbed on the eyes and fever when rubbed on the body.40

Sieves for flour and corn, brooms, sambelus or corn pounders, and ploughs
are regarded as sacred.

Sieves are considered sacred for the following reasons.

1. Because articles of food such as flour, grain, etc., are sifted through
them.41

2. Because, on auspicious occasions, when women go to worship the

potter’s wheel, the materials of worship are carried in a sieve.

3. Because the fire used for igniting the sacrificial fuel is taken in a sieve, or
is covered with a sieve while it is being carried to the sacrificial altar.42

4. Because at the time of performing the ceremony when commencing to

prepare sweets for a marriage, a sieve is worshipped.43

5. Because, in some communities like the Bhātiās, the bride’s mother, when
receiving the bridegroom in the marriage booth, carries in a dish a lamp
covered with a sieve.44

The flour collected by Brāhmans by begging from door to door is supposed

to be polluted. But it is considered purified when it is passed through a
sieve.45

The sambelu is considered so sacred that it is not touched with the foot. If a
woman lie down during day time, she will not touch it either with her head
or with her foot.

One of the reasons why it is considered sacred is that it was used as a

weapon by Baldev, the brother of the god Krishna.
A sambelu is one of the articles, required for performing the reception
ceremony on a bridegroom’s entering the marriage pandal.46

It is believed that a fall of rain is expedited by placing a sambelu erect in a

dish when there is a drought.47

Among Shrigaud Brāhmans, on the marriage day, one of the men of the
bridegroom’s party wears a wreath made of a sambelu, a broom and other
articles. Some special marks are also made on his forehead. Thus adorned,
he goes with the bridegroom’s procession and plays jokes with the parents
of both the bride and bridegroom. His doing so is supposed to bless the
bridal pair with a long life and a large family.48

On the marriage day, after the ceremony of propitiating the nine planets has
been performed in the bride’s house, in some castes three, and in others one
sambelu, is kept near the spot where the planets are worshipped. Next, five
unwidowed women of the family hold the sambelus and thrash them five or
seven times on the floor repeating the words “On the chest of the ill-wisher
of the host.” The sambelus are bound together by a thread.49

If a woman has to take part in an auspicious ceremony on the fourth day of

her monthly period, she is made to thresh one maund of rice with a
sambelu. Her fourth day is then considered as the fifth50 and she becomes
eligible for taking part in the ceremony.51

The plough is considered sacred, because it is the chief implement for

cultivating the soil. It is worshipped on the full-moon day of Shrāvan which
is known as a Balev holiday, the worship being called Grahan-pujan.52

Some people consider the plough sacred because Sita, the consort of Rām,
was born of the earth by the touch of a plough.53 Others hold it sacred as it
was used as a weapon by Baldev, the brother of the god Krishna.

On account of the sanctity which attaches to the plough, it forms part of the
articles, with which a bridegroom is received in the marriage pandal by the
bride’s mother.54
It is related that king Janak ploughed the soil on which he had to perform a
sacrifice. Hence it has become a practice to purify with a plough the spot on
which a sacrifice is to be performed.55

In some places, on the Balev day, a number of persons gather together near
a pond, and each of them fills an earthen jar with the water of the pond.
Next, one of the party is made to stand at a long distance from the others
with a small plough in his hands. The others then run a race towards the
latter. He who wins the race is presented with molasses and a cocoanut.56

It is customary among Brāhmans to perform the worship known as Baleviān

after the performance of a thread ceremony. In Native States, the prime
minister and other State officials and clerks join the ceremony, the principal
function of the ceremony being performed by the prime minister. In
villages, this function is performed by the headman of the village. The party
go in procession to a neighbouring village or a pond where an earthen
image of Ganpati besmeared with red lead is installed on a red cloth two
feet square. Near this image are installed the nine planets, represented by
nine heaps of corn, on each of which is placed a betelnut. This is called the
installation of Baleviān. A plough about two feet in length is kept standing
near the Baleviān with its end buried in the ground. The prime minister or
the village headman worships the plough, after which, four Kumbhārs or
potters wash themselves, and holding four jars on their heads, run a race.
Each of the Kumbhārs is named after one of the four months of the rainy
season. He who wins the race is presented with the plough. The expenses of
the ceremony are paid from the State treasury or the village fund.57

According to a popular saying, a broom should not be kept erect or

trampled under foot. This indicates that brooms are held sacred.

When a newly-born infant does not cry, the leaves of a broom are thrown
into the fire and their smoke is passed over the child. It is said that this
makes the child cry.58

Some people consider brooms sacred, because they are used in sweeping
the ground58 (that is the earth, which is a goddess).
In some places, children suffering from cough are fanned with a broom.59

In some castes, a broom is worshipped on the marriage day.60

Many people deny any sanctity to a broom. A belief is common that if a

man sees a broom the first thing after getting up in the morning, he does not
pass the day happily.61

Some believe that if a broom be kept erect in the house, a quarrel between
the husband and wife is sure to follow. There is also a belief that if a person
thrashes another with a broom, the former is liable to suffer from a gland
under the arm.62

Fire is considered to be a deity by all Hindus. In all sacrifices, fire is first

ignited with certain ceremonies of worship.63 In all Brāhman families,
every morning before breakfast, a ceremony called Vaishvadeva is
performed, in which fire is worshipped and cooked rice is offered to it.64

The Agnihotris keep a constant fire burning in their houses and worship it
thrice a day, morning, noon and evening65.

The Pārsis consider fire so sacred that they do not smoke. Neither do they
cross fire. In their temples called Agiāris a fire of sandal wood is kept
constantly burning. It is considered a great mishap if this fire is
extinguished.65

Fire is specially worshipped on the Holi day, that is the full-moon day of the
month of Fālgun.66

Other special occasions on which it is worshipped are the thread ceremony,

the ceremony of installing a new idol in a temple, the first pregnancy
ceremony, and the ceremony performed at the time of entering a new
house.67

Fire is also worshipped in Mahārudra, Vishnuyāg, Gāyatri-purashcharan,

Nilōtsarga, Vāstupujan, Shatachandi, Lakshachandi, and the sacrifices
performed during the Navarātra and on the Dasara day.68

Fire is considered to be the mouth of God, through which he is supposed to

receive all offerings.69

The offerings made to fire generally consist of clarified butter, cocoanuts,

sesamum seed, the Java, chips of the wood of the Pipal and the Shami,
curdled milk and frankincense.70

The fire to be used for sacrifices and agnihotras is produced by the friction
of two pieces of the wood of the Arani,71 the Pipal, the Shami72 or the
bamboo while mantras or incantations are being recited by Brāhmans.73

1 The School Master of Dhānk. ↑

2 The School Master of Kotda Sangani. ↑
3 These are female names. ↑
4 The School Master of Kolki. ↑
5 The School Master of Hālār. ↑
6 Both male and female. ↑
7 The School Master of Dhānk. ↑
8 The School Master of Dadvi. ↑
9 The School Master of Kolki. ↑
10 The School Master of Dhānk. ↑
11 The School Master of Kotda Sangani. ↑
12 The School Master of Dhānk. ↑
13 The School Master of Todia. ↑
14 The School Master of Mota Devalia. ↑
15 The School Master of Kotda Sangani. ↑
16 The School Master of Kotda Sangani. ↑
17 The School Master of Vanod. ↑
18 The School Master of Zinzuvāda. ↑
19 The School Master of Songadh. ↑
20 The School Master of Pātanvāv. ↑
21 The School Master of Vāla. ↑
22 The School Master of Songadh. ↑
23 The School Master of Dhānk. ↑
24 The School Master of Vanod. ↑
25 The School Master of Vanod. ↑
26 The School Master of Dadvi. ↑
27 The School Master of Bāntva. ↑
28 The School Master of Ganod. ↑
29 The School Master of Devalia. ↑
30 The School Master of Chhatrāsa. ↑
31 The School Master of Jetpur. ↑
32 The School Master of Zinzuvāda. ↑
33 The School Masters of Kotda Sangani, Zinzuvāda and Gohelwād. ↑
34 The School Master of Hālār. ↑
35 The School Master of Kotda Sangani. ↑
36 The School Master of Devalia. ↑
37 The School Master of Todia. ↑
38 The School Master of Kolki. ↑
39 The School Master of Dhānk. ↑
40 The School Master of Jetpur. ↑
41 The School Master of Dhānk. ↑
42 The School Master of Dadvi. ↑
43 The School Master of Aman. ↑
44 The School Master of Todia. ↑
45 The School Master of Lilāpur. ↑
46 The School Master of Dhānk. ↑
47 The School Master of Zinzuvāda. ↑
48 The School Master of Zinzuvāda. ↑
49 The School Master of Todia. ↑
50 Among Hindus women in menses are considered impure for four days. ↑
51 The School Master of Lilāpur. ↑
52 The School Master of Dhānk and Kota Sangani. ↑
53 The School Master of Ganod. ↑
54 The School Master of Dadvi. ↑
55 The School Master of Lilāpur. ↑
56 The School Master of Zinzuvāda. ↑
57 The School Master of Todia. ↑
58 The School Master of Dhānk. ↑
59 The School Master of Kolki. ↑
60 The School Master of Songadh. ↑
61 The School Master of Limbdi. ↑
62 The School Master of Todia. ↑
63 The School Master of Dhānk. ↑
64 The School Masters of Dhānk and Vanod. ↑
65 The School Master of Kalavad and Mr. K. D. Desāi. ↑
66 The School Masters of Ganod and Dhānk. ↑
67 The School Master of Dadvi. ↑
68 The School Masters of Ganod and Kalavad and Mr. K. D. Desāi. ↑
69 The School Master of Todia. ↑
70 The School Master of Wala. ↑
71 The School Masters of Dadvi and Dhānk. ↑
72 The School Master of Jetpur. ↑
73 The School Master of Aman. ↑
Welcome to Our Bookstore - The Ultimate Destination for Book Lovers
Are you passionate about testbank and eager to explore new worlds of
knowledge? At our website, we offer a vast collection of books that
cater to every interest and age group. From classic literature to
specialized publications, self-help books, and children’s stories, we
have it all! Each book is a gateway to new adventures, helping you
expand your knowledge and nourish your soul
Experience Convenient and Enjoyable Book Shopping Our website is more
than just an online bookstore—it’s a bridge connecting readers to the
timeless values of culture and wisdom. With a sleek and user-friendly
interface and a smart search system, you can find your favorite books
quickly and easily. Enjoy special promotions, fast home delivery, and
a seamless shopping experience that saves you time and enhances your
love for reading.
Let us accompany you on the journey of exploring knowledge and
personal growth!

ebooksecure.com