Get ebook downloads in full at ebookmeta.

com

Machine Learning for Decision Sciences with Case

Studies in Python 1st Edition S. Sumathi

https://ebookmeta.com/product/machine-learning-for-decision-
sciences-with-case-studies-in-python-1st-edition-s-sumathi/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Innovations in Machine and Deep Learning Case Studies and

Applications Gilberto Rivera

https://ebookmeta.com/product/innovations-in-machine-and-deep-
learning-case-studies-and-applications-gilberto-rivera/

ebookmeta.com

Machine Learning with Python Cookbook, 2nd Edition Kyle

Gallatin

https://ebookmeta.com/product/machine-learning-with-python-
cookbook-2nd-edition-kyle-gallatin/

ebookmeta.com

Machine Learning with Python Cookbook 2nd Edition Chris

Albon

https://ebookmeta.com/product/machine-learning-with-python-
cookbook-2nd-edition-chris-albon/

ebookmeta.com

Exploring a Business Case for High Value Continuing

Professional Development Proceedings of a Workshop 1st
Edition And Medicine Engineering National Academies Of
Sciences
https://ebookmeta.com/product/exploring-a-business-case-for-high-
value-continuing-professional-development-proceedings-of-a-
workshop-1st-edition-and-medicine-engineering-national-academies-of-
sciences/
ebookmeta.com
Close Reading the Basics 1st Edition David Greenham

https://ebookmeta.com/product/close-reading-the-basics-1st-edition-
david-greenham/

ebookmeta.com

Black Light Rocked Black Light Series Book 1 1st Edition

Livia Grant

https://ebookmeta.com/product/black-light-rocked-black-light-series-
book-1-1st-edition-livia-grant-2/

ebookmeta.com

Urban Planning for Transitions 1st Edition Nicolas Douay

https://ebookmeta.com/product/urban-planning-for-transitions-1st-
edition-nicolas-douay/

ebookmeta.com

Collaboration for Career and Technical Education Teamwork

Beyond the Core Content Areas in a PLC at Work a Guide for
Collaborative Teaching in Career and Technical Education
1st Edition Wendy Custable
https://ebookmeta.com/product/collaboration-for-career-and-technical-
education-teamwork-beyond-the-core-content-areas-in-a-plc-at-work-a-
guide-for-collaborative-teaching-in-career-and-technical-
education-1st-edition-wendy-custabl/
ebookmeta.com

Economics, Anthropology and the Origin of Money as a

Bargaining Counter 1st Edition Patrick Spread

https://ebookmeta.com/product/economics-anthropology-and-the-origin-
of-money-as-a-bargaining-counter-1st-edition-patrick-spread/

ebookmeta.com
Custom Search - Discover More: : A Complete Guide to
Google Programmable Search Engines 1st Edition Irina
Shamaeva
https://ebookmeta.com/product/custom-search-discover-more-a-complete-
guide-to-google-programmable-search-engines-1st-edition-irina-
shamaeva/
ebookmeta.com
Machine Learning for Decision
Sciences with Case Studies
in Python
Machine Learning for Decision
Sciences with Case Studies
in Python

S. Sumathi
Suresh V. Rajappa
L. Ashok Kumar
Surekha Paneerselvam
First edition published 2022
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742

and by CRC Press

2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

© 2022 S. Sumathi, Suresh V. Rajappa, L Ashok Kumar and Surekha Paneerselvam

CRC Press is an imprint of Taylor & Francis Group, LLC

Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright
holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowl-
edged please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or
utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including pho-
tocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission
from the publishers.

For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]

Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for
identification and explanation without intent to infringe.

ISBN: 978-1-032-19356-4 (hbk)

ISBN: 978-1-032-19357-1 (pbk)
ISBN: 978-1-003-25880-3 (ebk)

DOI: 10.1201/9781003258803

Typeset in Times
by codeMantra
Contents
Preface.............................................................................................................................................. xv
Acknowledgment............................................................................................................................xvii
About the Authors............................................................................................................................xix
Introduction......................................................................................................................................xxi

Chapter 1 Introduction...................................................................................................................1
1.1 Introduction to Data Science..............................................................................1
1.1.1 Mathematics..........................................................................................1
1.1.2 Statistics................................................................................................1
1.2 Describing Structural Patterns...........................................................................2
1.2.1 Uses of Structural Patterns....................................................................2
1.3 Machine Learning and Statistics........................................................................3
1.4 Relation between Artificial Intelligence, Machine Learning, Neural
Networks, and Deep Learning...........................................................................4
1.5 Data Science Life Cycle.....................................................................................6
1.6 Key Role of Data Scientist..................................................................................8
1.6.1 Difference between Data Scientist and Machine
Learning Engineer................................................................................8
1.7 Real-World Examples.........................................................................................8
1.8 Use Cases............................................................................................................9
1.8.1 Financial and Insurance Industries.......................................................9
1.8.1.1 Fraud Mitigation....................................................................9
1.8.1.2 Personalized Pricing............................................................ 10
1.8.1.3 AML – Anti-Money Laundering......................................... 10
1.8.2 Utility Industries................................................................................. 11
1.8.2.1 Smart Meter and Smart Grid............................................... 11
1.8.2.2 Manage disaster and Outages.............................................. 11
1.8.2.3 Compliance.......................................................................... 11
1.8.3 Oil and Gas Industries......................................................................... 11
1.8.3.1 Manage Exponential Growth............................................... 11
1.8.3.2 3D Seismic Imaging and Kirchhoff.................................... 12
1.8.3.3 Rapidly Process and Display Seismic Data......................... 12
1.8.4 E-Commerce and Hi-Tech Industries.................................................. 12
1.8.4.1 Association and Complementary Products.......................... 12
1.8.4.2 Cross-Channel Analytics..................................................... 12
1.8.4.3 Event analytics..................................................................... 13
Summary..................................................................................................................... 13
Review Questions........................................................................................................ 14

Chapter 2 Overview of Python for Machine Learning................................................................ 15

2.1 Introduction...................................................................................................... 15
2.1.1 The Flow of Program Execution in Python........................................ 15
2.2 Python for Machine Learning.......................................................................... 15
2.2.1 Why Is Python Good for ML?............................................................. 16
2.3 Setting up Python............................................................................................. 16

v
vi Contents

2.3.1 Python on Windows............................................................................ 16

2.3.2 Python on Linux.................................................................................. 17
2.3.2.1 Ubuntu................................................................................. 17
2.4 Python Basics................................................................................................... 17
2.4.1 Python Operators................................................................................ 18
2.4.1.1 Arithmetic Operators........................................................... 18
2.4.1.2 Comparison Operators......................................................... 18
2.4.1.3 Assignment Operators......................................................... 18
2.4.1.4 Logical Operators................................................................ 18
2.4.1.5 Membership Operators........................................................ 19
2.4.2 Python Code Samples on Basic Operators.......................................... 19
2.4.2.1 Arithmetic Operators........................................................... 19
2.4.2.2 Comparison Operators......................................................... 21
2.4.2.3 Logical Operators................................................................ 22
2.4.2.4 Membership Operators........................................................ 23
2.4.3 Flow Control........................................................................................24
2.4.3.1 If & elif Statement...............................................................24
2.4.3.2 Loop Statement....................................................................25
2.4.3.3 Loop Control Statements.....................................................26
2.4.4 Python Code Samples on Flow Control Statements........................... 26
2.4.4.1 Conditional Statements........................................................26
2.4.4.2 Python if...else Statement.................................................... 27
2.4.4.3 Python if…elif…else Statement..........................................28
2.4.4.4 The For Loop....................................................................... 29
2.4.4.5 The range() Function........................................................... 29
2.4.4.6 For Loop with else............................................................... 31
2.4.4.7 While Loop.......................................................................... 31
2.4.4.8 While Loop with else.......................................................... 32
2.4.4.9 Python Break and Continue................................................. 32
2.4.4.10 Python Break Statement...................................................... 32
2.4.4.11 Python Continue Statement................................................. 33
2.4.5 Review of Basic Data Structures and Implementation in Python.......34
2.4.5.1 Array Data Structure...........................................................34
2.4.5.2 Implementation of Arrays in Python................................... 35
2.4.5.3 Linked List........................................................................... 36
2.4.5.4 Implementation of Linked List in Python........................... 36
2.4.5.5 Stacks and Queues............................................................... 38
2.4.5.6 Queues.................................................................................40
2.4.5.7 Implementation of Queue in Python.................................... 41
2.4.5.8 Searching............................................................................. 42
2.4.5.9 Implementation of Searching in Python..............................44
2.4.5.10 Sorting.................................................................................46
2.4.5.11 Implementation of Bubble Sort in Python........................... 47
2.4.5.12 Insertion Sort....................................................................... 47
2.4.5.13 Implementation of Insertion Sort in Python........................ 49
2.4.5.14 Selection Sort....................................................................... 51
2.4.5.15 Implementation of Selection Sort in Python....................... 52
2.4.5.16 Merge Sort........................................................................... 52
2.4.5.17 Implementation of Merge Sort in Python............................ 53
2.4.5.18 Shell Sort............................................................................. 54
2.4.5.19 Quicksort............................................................................. 55
Contents vii

2.4.5.20 Data Structures in Python with Sample Codes................... 55

2.4.5.21 Python Code Samples for Data Structures in Python......... 58
2.4.6 Functions in Python............................................................................ 68
2.4.6.1 Python Code Samples for Functions................................... 68
2.4.6.2 Returning Values from Functions........................................ 68
2.4.6.3 Scope of Variables............................................................... 69
2.4.6.4 Function Arguments............................................................ 70
2.4.7 File Handling....................................................................................... 74
2.4.8 Exception Handling............................................................................. 74
2.4.9 Debugging in Python.......................................................................... 75
2.4.9.1 Packages.............................................................................. 75
2.5 Numpy Basics................................................................................................... 75
2.5.1 Introduction to Numpy........................................................................ 76
2.5.1.1 Array Creation..................................................................... 76
2.5.1.2 Array Slicing....................................................................... 77
2.5.2 Numerical Operations......................................................................... 77
2.5.3 Python Code Samples for Numpy Package......................................... 78
2.5.3.1 Array Creation..................................................................... 78
2.5.3.2 Class and Attributes of ndarray—.ndim.............................. 82
2.5.3.3 Class and Attributes of ndarray—.shape............................. 82
2.5.3.4 Class and Attributes of ndarray—ndarray.size,
ndarray.Itemsize, ndarray.resize.......................................... 83
2.5.3.5 Class and Attributes of ndarray—.dtype............................. 83
2.5.3.6 Basic Operations..................................................................84
2.5.3.7 Accessing Array Elements: Indexing.................................. 85
2.5.3.8 Shape Manipulation............................................................. 88
2.5.3.9 Universal Functions (ufunc) in Numpy................................90
2.5.3.10 Broadcasting........................................................................90
2.5.3.11 Args and Kwargs................................................................. 91
2.6 Matplotlib Basics..............................................................................................92
2.6.1 Creating Graphs with Matplotlib........................................................ 93
2.7 Pandas Basics...................................................................................................94
2.7.1 Getting Started with Pandas...............................................................94
2.7.2 Data Frames........................................................................................ 95
2.7.3 Key Operations on Data Frames......................................................... 95
2.7.3.1 Data Frame from List.......................................................... 95
2.7.3.2 Rows and Columns in Data Frame......................................96
2.8 Computational Complexity...............................................................................97
2.9 Real-world Examples........................................................................................97
2.9.1 Implementation using Pandas.............................................................. 98
2.9.2 Implementation using Numpy............................................................. 98
2.9.3 Implementation using Matplotlib........................................................ 98
Summary.....................................................................................................................99
Review Questions...................................................................................................... 100
Exercises for Practice................................................................................................ 101

Chapter 3 Data Analytics Life Cycle for Machine Learning.................................................... 103

3.1 Introduction.................................................................................................... 103
3.2 Data Analytics Life Cycle.............................................................................. 104
3.2.1 Phase 1 – Data Discovery.................................................................. 104
viii Contents

3.2.2 Phase 2 – Data Preparation and Exploratory Data Analysis............. 107

3.2.2.1 Exploratory Data Analysis................................................. 110
3.2.3 Phase 3 – Model Planning................................................................. 136
3.2.4 Phase 4 – Model Building................................................................. 139
3.2.5 Phase 5 – Communicating Results.................................................... 140
3.2.6 Phase 6 – Optimize and Operationalize the Models......................... 140
Summary................................................................................................................... 142
Review Questions...................................................................................................... 143

Chapter 4 Unsupervised Learning............................................................................................. 145

4.1 Introduction.................................................................................................... 145
4.2 Unsupervised Learning.................................................................................. 145
4.2.1 Clustering.......................................................................................... 147
4.3 Evaluation Metrics for Clustering.................................................................. 147
4.3.1 Distance Measures............................................................................ 148
4.3.1.1 Minkowski Metric............................................................. 149
4.3.2 Similarity Measures.......................................................................... 149
4.4 Clustering Algorithms.................................................................................... 150
4.4.1 Hierarchical and Partitional Clustering Approaches........................ 150
4.4.2 Agglomerative and Divisive Clustering Approaches........................ 150
4.4.3 Hard and Fuzzy Clustering Approaches........................................... 150
4.4.4 Monothetic and Polythetic Clustering Approaches........................... 151
4.4.5 Deterministic and Probabilistic Clustering Approaches................... 151
4.5 k-Means Clustering......................................................................................... 151
4.5.1 Geometric Intuition, Centroids......................................................... 151
4.5.2 The Algorithm................................................................................... 152
4.5.3 Choosing k......................................................................................... 152
4.5.4 Space and Time Complexity............................................................. 153
4.5.5 Advantages and Disadvantages of k-Means Clustering.................... 153
4.5.5.1 Advantages......................................................................... 153
4.5.5.2 Disadvantages.................................................................... 153
4.5.6 k-Means Clustering in Practice Using Python.................................. 154
4.5.6.1 Illustration of the k-Means Algorithm Using Python........ 154
4.5.7 Fuzzy k-Means Clustering Algorithm............................................... 157
4.5.7.1 The Algorithm................................................................... 158
4.5.8 Advantages and Disadvantages of Fuzzy k-Means Clustering......... 158
4.6 Hierarchical Clustering.................................................................................. 159
4.6.1 Agglomerative Hierarchical Clustering............................................ 159
4.6.2 Divisive Hierarchical Clustering....................................................... 161
4.6.3 Techniques to Merge Cluster............................................................. 161
4.6.4 Space and Time Complexity............................................................. 163
4.6.5 Limitations of Hierarchical Clustering............................................. 163
4.6.6 Hierarchical Clustering in Practice Using Python............................ 163
4.6.6.1 DATA_SET....................................................................... 164
4.7 Mixture of Gaussian Clustering..................................................................... 165
4.7.1 Expectation Maximization................................................................ 166
4.7.2 Mixture of Gaussian Clustering in Practice Using Python............... 168
4.8 Density-Based Clustering Algorithm............................................................. 169
4.8.1 DBSCAN (Density-Based Spatial Clustering of
Applications with Noise)................................................................... 169
Contents ix

4.8.2 Space and Time Complexity............................................................. 171

4.8.3 Advantages and Disadvantages of DBSCAN.................................... 171
4.8.3.1 Advantages......................................................................... 171
4.8.3.2 Disadvantages.................................................................... 171
4.8.4 DBSCAN in Practice Using Python.................................................. 172
Summary................................................................................................................... 174
Review Questions...................................................................................................... 174

Chapter 5 Supervised Learning: Regression............................................................................. 177

5.1 Introduction.................................................................................................... 177
5.2 Supervised Learning – Real-Life Scenario.................................................... 177
5.3 Types of Supervised Learning........................................................................ 178
5.3.1 Supervised Learning – Classification............................................... 178
5.3.1.1 Classification – Predictive Modeling................................. 179
5.3.2 Supervised Learning – Regression................................................... 179
5.3.2.1 Regression Predictive Modeling........................................ 180
5.3.3 Classification vs. Regression............................................................. 180
5.3.4 Conversion between Classification and Regression Problems.......... 181
5.4 Linear Regression........................................................................................... 181
5.4.1 Types of Linear Regression............................................................... 182
5.4.1.1 Simple Linear Regression.................................................. 183
5.4.1.2 Multiple Linear Regression............................................... 184
5.4.2 Geometric Intuition........................................................................... 186
5.4.3 Mathematical Formulation................................................................ 187
5.4.4 Solving Optimization Problem.......................................................... 201
5.4.4.1 Maxima and Minima......................................................... 201
5.4.4.2 Gradient Descent...............................................................202
5.4.4.3 LMS (Least Mean Square) Update Rule...........................205
5.4.4.4 SGD Algorithm..................................................................205
5.4.5 Real-World Applications...................................................................206
5.4.5.1 Predictive Analysis............................................................206
5.4.5.2 Medical Outcome Prediction.............................................208
5.4.5.3 Wind Speed Prediction......................................................208
5.4.5.4 Environmental Effects Monitoring....................................209
5.4.6 Linear Regression in Practice Using Python....................................209
5.4.6.1 Simple Linear Regression Using Python...........................209
5.4.6.2 Multiple Linear Regression Using Python........................ 212
Summary................................................................................................................... 215
Review Questions...................................................................................................... 215

Chapter 6 Supervised Learning: Classification......................................................................... 219

6.1 Introduction.................................................................................................... 219
6.2 Use Cases of Classification............................................................................. 219
6.3 Logistic Regression........................................................................................ 219
6.3.1 Geometric Intuition........................................................................... 220
6.3.2 Variants of Logistic Regression........................................................ 222
6.3.2.1 Simple Logistic Regression............................................... 222
6.3.2.2 Multiple Logistic Regression............................................. 223
6.3.2.3 Binary Logistic Regression............................................... 223
x Contents

6.3.2.4 Multiclass Logistic Regression..........................................224

6.3.2.5 Nominal Logistic Regression............................................224
6.3.2.6 Ordinal Logistic Regression.............................................. 226
6.3.3 Optimization Problem....................................................................... 226
6.3.4 Regularization................................................................................... 226
6.3.5 Real-World Applications................................................................... 227
6.3.5.1 Medical Diagnosis............................................................. 227
6.3.5.2 Text Classification.............................................................. 227
6.3.5.3 Marketing........................................................................... 227
6.3.6 Logistic Regression in Practice using Python................................... 228
6.3.6.1 Variable Descriptions........................................................ 231
6.3.6.2 Checking for Missing Values............................................. 231
6.3.6.3 Converting Categorical Variables to a
Dummy Indicator............................................................... 234
6.4 Decision Tree Classifier.................................................................................. 236
6.4.1 Important Terminology in the Decision Tree.................................... 236
6.4.2 Example for Decision Tree................................................................ 237
6.4.3 Sample Decision Tree........................................................................ 238
6.4.4 Decision Tree Formation................................................................... 238
6.4.5 Algorithms Used for Decision Trees.................................................240
6.4.5.1 ID3 Algorithm...................................................................240
6.4.5.2 C 4.5 Algorithm................................................................. 241
6.4.5.3 CART Algorithm............................................................... 241
6.4.6 Overfitting and Underfitting.............................................................. 241
6.4.6.1 Overfitting.......................................................................... 241
6.4.6.2 Underfitting........................................................................ 242
6.4.6.3 Pruning to Avoid Overfitting............................................. 243
6.4.7 Advantages and Disadvantages.........................................................244
6.4.7.1 Advantages.........................................................................244
6.4.7.2 Disadvantages....................................................................244
6.4.8 Decision Tree Examples.................................................................... 245
6.4.9 Regression Using Decision Tree....................................................... 262
6.4.10 Real-World Examples........................................................................266
6.4.10.1 Predicting Library Book....................................................266
6.4.10.2 Identification of Tumor...................................................... 267
6.4.10.3 Classification of Telescope Image..................................... 269
6.4.10.4 Business Management....................................................... 269
6.4.10.5 Fault Diagnosis.................................................................. 271
6.4.10.6 Healthcare Management.................................................... 271
6.4.10.7 Decision Tree in Data Mining........................................... 271
6.4.11 Decision Trees in Practice Using Python.......................................... 273
6.5 Random Forest Classifier............................................................................... 279
6.5.1 Random Forest and Their Construction............................................280
6.5.2 Sampling of the Dataset in Random Forest...................................... 281
6.5.2.1 Creation of Subset Data..................................................... 285
6.5.3 Pseudocode for Random Forest......................................................... 286
6.5.3.1 Pseudocode for Prediction in Random Forest................... 287
6.5.4 Regression Using Random Forest..................................................... 287
6.5.5 Classification Using Random Forest................................................. 288
6.5.5.1 Random Forest Problem for Classification – Examples.... 293
6.5.6 Features and Properties of Random Forest....................................... 295
Contents xi

6.5.6.1 Features.............................................................................. 295

6.5.6.2 Properties........................................................................... 296
6.5.7 Advantages and Disadvantages of Random Forest........................... 296
6.5.7.1 Advantages......................................................................... 296
6.5.7.2 Disadvantages.................................................................... 296
6.5.8 Calculation of Error Using Bias and Variance.................................. 296
6.5.8.1 Bias.................................................................................... 296
6.5.8.2 Variance............................................................................. 296
6.5.8.3 Properties of Bias and Variance........................................ 297
6.5.9 Time Complexity............................................................................... 297
6.5.10 Extremely Randomized Tree............................................................. 297
6.5.11 Real-World Examples........................................................................ 298
6.5.11.1 Machine Fault Diagnosis................................................... 298
6.5.11.2 Medical Field..................................................................... 298
6.5.11.3 Banking............................................................................. 299
6.5.11.4 E-Commerce......................................................................300
6.5.11.5 Security..............................................................................300
6.5.12 Random Forest in Practice Using Python.........................................300
6.6 Support Vector Machines...............................................................................306
6.6.1 Geometric Intuition...........................................................................307
6.6.2 Mathematical Formulation................................................................ 310
6.6.2.1 Maximize Margin with Noise........................................... 312
6.6.2.2 Slack Variable ξi ................................................................ 312
6.6.3 Loss Minimization............................................................................ 315
6.6.4 Dual Formulation.............................................................................. 317
6.6.5 The Kernel Trick............................................................................... 320
6.6.6 Polynomial Kernel............................................................................. 320
6.6.6.1 Mercer’s Theorem.............................................................. 322
6.6.6.2 Radial Basis Function (RBF) Kernel................................. 322
6.6.6.3 Other Domain-Specific Kernel.......................................... 323
6.6.6.4 Sigmoid Kernel.................................................................. 323
6.6.6.5 Exponential Kernel............................................................ 323
6.6.6.6 ANOVA Kernel.................................................................. 323
6.6.6.7 Rational Quadratic Kernel................................................. 323
6.6.6.8 Multiquadratic Kernel........................................................ 323
6.6.6.9 Inverse Multiquadratic Kernel........................................... 323
6.6.6.10 Circular Kernel.................................................................. 324
6.6.6.11 Bayesian Kernel................................................................. 324
6.6.6.12 Chi-Square Kernel............................................................. 324
6.6.6.13 Histogram Intersection Kernel.......................................... 324
6.6.6.14 Generalized Histogram Intersection Kernel...................... 324
6.6.7 nu SVM............................................................................................. 324
6.6.8 SVM Regression................................................................................ 325
6.6.9 One-Class SVM................................................................................ 326
6.6.10 Multiclass SVM................................................................................. 326
6.6.10.1 One against All.................................................................. 326
6.6.10.2 One against One................................................................ 327
6.6.10.3 Directed Acyclic Graph SVM........................................... 327
6.6.11 SVM Examples................................................................................. 328
6.6.12 Real-World Applications................................................................... 341
6.6.12.1 Classification of Cognitive Impairment............................. 341
xii Contents

6.6.12.2 Preprocessing..................................................................... 342

6.6.12.3 Feature Extraction............................................................. 342
6.6.12.4 SVM Classification............................................................ 342
6.6.12.5 Procedure........................................................................... 342
6.6.12.6 Performance Analysis........................................................ 343
6.6.12.7 Text Categorization............................................................ 343
6.6.12.8 Handwritten Optical Character Recognition.....................344
6.6.12.9 Natural Language Processing............................................344
6.6.12.10 Cancer Prediction.............................................................. 345
6.6.12.11 Stock Market Forecasting.................................................. 345
6.6.12.12 Protein Structure Prediction..............................................346
6.6.12.13 Face Detection Using SVM...............................................346
6.6.13 Advantages and Disadvantages of SVM........................................... 347
6.7 SVM Classification in Practice Using Python................................................ 347
6.7.1 Support Vectors................................................................................. 347
6.7.2 What Is a Hyperplane?......................................................................348
Summary................................................................................................................... 349
Review Questions...................................................................................................... 349

Chapter 7 Feature Engineering.................................................................................................. 351

7.1 Introduction.................................................................................................... 351
7.2 Feature Selection............................................................................................ 352
7.2.1 Wrapper Methods.............................................................................. 353
7.2.1.1 Forward Selection.............................................................. 353
7.2.1.2 Backward Elimination....................................................... 353
7.2.1.3 Exhaustive Feature Selection............................................. 354
7.2.2 Featured Methods.............................................................................. 354
7.3 Factor Analysis............................................................................................... 355
7.3.1 Types of Factor Analysis................................................................... 355
7.3.2 Working of Factor Analysis.............................................................. 355
7.3.3 Terminologies.................................................................................... 356
7.3.3.1 Definition of Factor............................................................ 356
7.3.3.2 Factor Loading................................................................... 356
7.3.3.3 Eigenvalues........................................................................ 356
7.3.3.4 Communalities................................................................... 356
7.3.3.5 Factor Rotation.................................................................. 356
7.3.3.6 Selecting the Number of Factors....................................... 356
7.4 Principal Component Analysis....................................................................... 357
7.4.1 Center the Data.................................................................................. 357
7.4.2 Normalize the Data........................................................................... 357
7.4.3 Estimate the Eigen decomposition.................................................... 357
7.4.4 Project the Data................................................................................. 357
7.5 Eigenvalues and PCA..................................................................................... 359
7.5.1 Usage of eigendecomposition in PCA............................................... 359
7.6 Feature Reduction........................................................................................... 361
7.6.1 Factor Analysis Vs. Principal Component Analysis......................... 362
7.7 PCA Transformation in Practice Using Python............................................. 362
7.8 Linear Discriminant Analysis........................................................................364
7.8.1 Mathematical Operations in LDA..................................................... 365
7.9 LDA Transformation in Practice Using Python............................................. 368
Contents xiii

7.9.1 Implementation of Scatter within the Class (Sw).............................. 368

7.9.2 Implementation of Scatter between Class (Sb).................................. 369
Summary................................................................................................................... 371
Review Questions...................................................................................................... 371

Chapter 8 Reinforcement Engineering...................................................................................... 373

8.1 Introduction.................................................................................................... 373
8.2 Reinforcement Learning................................................................................. 373
8.2.1 Examples of Reinforcement Learning.............................................. 375
8.3 How RL Differs from Other ML Algorithms?............................................... 376
8.3.1 Supervised Learning......................................................................... 376
8.4 Elements of Reinforcement Learning............................................................ 376
8.4.1 Policy................................................................................................. 376
8.4.2 Reward Signal................................................................................... 377
8.4.3 Value Function.................................................................................. 377
8.4.3.1 Examples of Rewards........................................................ 377
8.4.4 Model of the Environment................................................................ 378
8.4.5 The Reinforcement Learning Algorithm.......................................... 378
8.4.6 Methods to Implement Reinforcement Learning in ML................... 379
8.5 Markov Decision Process............................................................................... 379
8.5.1 Preliminaries..................................................................................... 379
8.5.2 Value Functions................................................................................. 380
8.6 Dynamic Programming.................................................................................. 381
8.6.1 Policy Evaluation............................................................................... 382
8.6.2 Policy Improvement.......................................................................... 383
8.6.3 Policy Iteration.................................................................................. 385
8.6.4 Efficiency of Dynamic Programming............................................... 385
8.6.5 Dynamic Programming in Practice using Python............................ 386
Summary................................................................................................................... 387
Review Questions...................................................................................................... 387

Chapter 9 Case Studies for Decision Sciences Using Python.................................................... 389

9.1 Use Case 1 − Retail Price Optimization Using Price Elasticity of
Demand Method............................................................................................. 389
9.1.1 Background....................................................................................... 389
9.1.2 Understanding the Data..................................................................... 390
9.1.3 Conclusion.........................................................................................400
9.2 Use Case 2 − Market Basket Analysis (MBA)................................................ 401
9.2.1 Introduction....................................................................................... 401
9.2.2 Understating the Data........................................................................ 401
9.2.3 Conclusion......................................................................................... 412
9.3 Use Case 3 − Sales Prediction of a Retailer.................................................... 412
9.3.1 Background....................................................................................... 412
9.3.2 Understanding the Data..................................................................... 413
9.3.3 Conclusion......................................................................................... 418
9.4 Use Case 4 − Predicting the Cost of Insurance Claims for a
Property and Causalty (P&C) Insurance Company....................................... 419
9.4.1 Background....................................................................................... 419
9.4.2 Understanding the Data..................................................................... 419
xiv Contents

9.5 Use Case 5 − E-Commerce Product Ranking and Sentiment Analysis.......... 430
9.5.1 Background....................................................................................... 430
9.5.2 Understanding the Data..................................................................... 431
Summary................................................................................................................... 441
Review Questions...................................................................................................... 442

Appendix: Python Cheat Sheet for Machine Learning............................................................. 443

Bibliography..................................................................................................................................449
Index............................................................................................................................................... 453
Other documents randomly have
different content
armor from sixteen to twenty-four inches thick, and measures 11,400
tons.
The Egyptian forts constituted two distinct systems of defence.
The first consisted of those which protected the new port and
eastern town; and the second those which covered the entrances to
the outer western harbor. Seymour divided his fleet so as to
simultaneously bombard the whole. His ironclads and wooden gun-
boats were fitted, in addition to heavy guns, with torpedoes, and
Nordenfelt and Gatling machine guns.
The Invincible (flag-ship), Monarch and Penelope, with the
Téméraire outside, took up a position at the entrance of the western
harbor, about opposite Fort Meks, and about twelve hundred yards
from another important work, Fort Marsa-el-Kanat.
They attacked these forts, on the shore of the mainland, while the
Superb, Sultan, and Alexandra attacked and totally destroyed the
lighthouse fort, and another near it on the peninsula. The Inflexible
took up a position between the two divisions, and with her enormous
guns, assisted in the work of both.
The gun-boats attacked the “Marabout” batteries, at the entrance
of the harbor, running close in, and soon silencing them. One of the
gun-boats afterwards covered a landing party, which blew up the
heavy guns in Fort Meks.
The Egyptian artillerists surprised the English by the determination
with which they fought their guns; but they were all silenced by four
o’clock in the afternoon—rather a long time, it would seem, for
vessels carrying guns of such power, if they were properly pointed.
By this time four of the forts had been blown up, and the Khedive’s
palace and harem was in flames. The English fire ceased about half-
past five in the afternoon.
The casualties of the English were five killed and twenty-eight
wounded. This is rather a high number, considering the character of
the vessels employed. The Egyptians seemed to have fired only
solid shot, and these, in some cases, entered the ships, and caused
most of the injuries by splinters.
The Egyptian loss was great, but will probably never be known. It
is said that the gunners in the forts were mostly blacks—Soudanese
—who are as remarkable for their bravery and bull-dog tenacity, as
they are for the very dark color of their skin.
The English are reported to have burst the guns in the ruined forts
with dynamite.
Fort Napoleon, a very strong work, somewhat inland, and dating
from the French occupation, and Fort Gabarrie, had not been
thoroughly bombarded on the first day, and still held out; and
arrangements were made to have the Invincible, Monarch and
Penelope go in the next day to attack them. In the meantime the
Invincible had silenced some outlying batteries, and had sent on
shore a party which had burst nine large guns. During the night the
Egyptians had repaired an outside battery, but when the Inflexible
and the Téméraire opened on it, it did not reply, and was found to be
abandoned. On the morning after the bombardment the Khedive’s
palace was still burning, and there were other fires in the town.
The wind had risen, and a swell was coming in, which prevented
accurate firing, and at one P. M. all fire ceased, on both sides, having
not, of course, been so sustained and continuous as on the
preceding day. A white flag was now shown in the town, and a gun-
boat was sent up the inner harbor to the Arsenal, with a flag of truce
flying. The Arsenal is the official residence of the Ministers of War
and Marine, but the gun-boat found no one there, in authority, and no
one, in consequence, able to tell the meaning of the white flag flying
in the city. The officer sent up, therefore returned to Seymour. Night
now drew on, and the fires in the town were evidently extending.
At daybreak, next morning, the whole of the forts were found to be
abandoned, and the English Admiral telegraphed to the fleet not to
open fire. Half the city appeared to be in flames, with a dense cloud
of smoke hanging over it. The conflagration was, in fact,
tremendous, and involved the whole European quarter and the
Grand Square. The Egyptian army had retreated.
Towards morning a body of about one hundred Europeans fought
their way down to the beach, and were taken off by armed boats
from the fleet. They had spent a terrible night, defending themselves
in the building of the Ottoman Bank, and other buildings adjacent.
During the night hundreds of people were massacred, mostly
Christians, and for the two succeeding days Alexandria was the
scene of horrors hardly equalled by Paris during the Commune.
An uncontrolled soldiery, released convicts and the scum of the
population were let loose, and murder and pillage went on.
Petroleum was used to fire many buildings in the best part of this fine
and flourishing city.
The Khedive was helpless, and really in great danger of his life, in
his palace at Ramleh.
Seymour and his officers and men looked on at “this sad spectacle
of awful and unexpected catastrophe, which they had no power to
prevent.”
It is hard to tell whether to blame most the want of statesmanlike
forethought, or the want of military perception in the English naval
Commander, who precipitated this dreadful state of affairs, without
having the means, in the shape of troops, to land and seize the town.
To disinterested observers it was an indefensible act, to bombard
a city with such a mixed population, many of whom were prone to
rapine and murder, without having the power promptly to put things
in order.
 Section of the Alexandra,
showing working of Guns.

BOMBARDMENT OF ALEXANDRIA, 1882.

The English Government seems to have thought Admiral

Seymour’s conduct good, for they have just made him a Baron.
The sailors and marines of the fleet were landed as soon as it was
ascertained that Arabi’s forces had retreated. These were joined by
detachments from the German and American men-of-war, for the
purpose of protecting their consulates, in the first place, and then
they assisted in extinguishing fires, seizing marauders, and rescuing
many terror-stricken people from the most imminent danger;
patrolling the streets, and assisting, in every way, to restore order.
As regards the bombardment itself, it may be said that the
Egyptian batteries were served steadily and rapidly; and their aim
was good. The officers appear to have set a good example to their
men, appearing often upon the parapet, to watch the effect of their
shot. All the batteries facing the sea were destroyed by the heavy
guns opposed to them, and their guns dismounted. An explosion of a
magazine, in one fort, is said to have killed all of the garrison. The
faces of the batteries were pulverized, and large holes were made in
the masonry of the lighthouse, and the large stone fort adjoining was
reduced to ruin, and all its guns dismounted. The loss of life in the
garrisons of these forts must have been frightful. The harem palace
was damaged extensively by shell and by fire.
The Arab quarter behind Fort Pharos caught all the shell which
missed the batteries; and here all was chaos and destruction.
Of the English ships, the Penelope was struck five times, and had
eight men wounded, and one gun disabled. The Invincible was
struck many times, and six shots penetrated. She had six wounded,
and some spars shot away. The Monarch, which kept under way,
was not hit. The Superb had her funnel injured, and plates damaged.
The Alexandra suffered slight damage in her hull. The Sultan had
her main-mast and funnel shot through, and her hull pierced several
times in the unarmored part. Two of the 18-ton guns of the Alexandra
were disabled by shot, which passed in at the port-holes.
 THE ALEXANDRA. 1877.
(Ironclad, English Navy. Her appearance after being launched.)
 THE WAR BETWEEN CHINA AND JAPAN.

The Opening of Japan to Foreign Nations.

e may well be proud of the fact that our country

was the first to open the empire of Japan to the
free intercourse which, in common with the rest
of the world, we now enjoy with that formerly
mysterious country. In former years the Dutch,
Portuguese, English, and others had a limited
intercourse with the Japanese, but were cut off
from that, and, with the exception of the Dutch,
entirely expelled from the country two centuries
and a half ago. The Dutch alone, of all Christian
nations, were allowed to remain for the purposes
of traffic, “and they purchased the privilege at the
price of national humiliation and personal
imprisonment, for which all the profits of gainful barter offer but an
inadequate compensation.”
This self-isolated empire, Japan, has experienced more radical
and startling changes within the last generation than any people or
nation of which history treats. It seemed as if, once freed from the
trammels which had so long confined it, the empire was determined,
at one bound, to place itself abreast of other nations which had
attained a high civilization and enlightenment by slow and painful
steps. With a wealth of the best models before them, and the
intelligence to be able to throw off their prejudices and avail
themselves of those models, the feat was easier, but still remains
wonderful, the more we consider it.
Situated at the eastern extremity of Asia, between 31° and 49°
north latitude, the empire consists of a large group of islands, many
of them small and surrounded by a sea which is not very easy to
navigate at all times.
There are three very large islands—Niphon, or Nippon, seven
hundred miles long, but so narrow that its breadth in the centre is not
more than fifty miles; Kiusiu, about two hundred miles long and fifty
miles wide; and Yesso, formerly sometimes called Xicoco, eighty-five
or ninety miles long and fifty wide.
There are many mountains, some of them volcanic, and the
country is subject to earthquakes, often of a serious character.
The number of inhabitants is given as about forty millions; but it is
said by late observers that, dense as the population appears to be in
certain regions, the country could support many more.
The Japanese appear to be a mixture of the Malay and Mongolian
races, like the Chinese, from whom there seems little doubt they
derived their civilization, ages ago.
The first knowledge of Japan which the Western world had was
given by the Venetian traveller, Marco Polo, at the end of the
thirteenth century. When he returned from a long sojourn in Asia he
was hardly believed when he spoke of a large island off the coast of
Cathay, or China, which he called Zipangu. That island is the modern
Nippon.
There is no doubt that Marco Polo’s written story and
accompanying maps had much to do with the determination of
Columbus to find the farthest east by sailing west. Although he was
not able to find and open Japan, he did discover a country which has
performed a part of his contemplated work—a nation which, if it did
not discover Zipangu, was to become the instrument of bringing it
into free and full communication with the rest of the world.
It is to the Portuguese that we owe the first real knowledge of
Japan. When Mendez Pinto, on a voyage to China in 1542, was
driven by a storm to Japan and landed there, the event was
considered so important by the authorities of that isolated country
that they not only entered it in their archives, but preserved portraits
of persons who seemed most strange to them in complexion,
features, dress, and language.
The authorities and the visitors were so much pleased with each
other that an arrangement was entered into by which a Portuguese
ship was to be yearly despatched—probably from Macao—laden
with articles of trade. The returns were made in gold, silver, and
copper, of which latter metal there was abundance in Japan.
Then came the establishment of a mission under Francis Xavier,
afterwards canonized by the Catholic Church, and a man of
wonderful ability and with all the requisites for a Christian missionary
of his time. He and his assistants did not meddle with Japanese
affairs or politics, soon made friends, and many converts; but St.
Francis Xavier died in 1552, and his successors were not so wise or
so Christian-like as he had been. They differed among themselves
and meddled with matters which did not concern them. The
Franciscans and Dominicans quarrelled with the Jesuits, but they
obtained, among them, a very large number of converts, among
whom were numbered some of the princes or feudal lords.
The Dutch came next in establishing a footing in Japan, about
1598. One of their first vessels had an English pilot named William
Adams, who has left us a narration of his long residence there—a
romantic story, but which can only be alluded to here. He arose to
high distinction, and, among other things, instructed the Japanese in
the art of ship-building and mathematics.
An English factory was established at Hirado in 1613, but the
enterprise was soon abandoned.
All went well with the Portuguese until about the year 1617, when
a revolution occurred in Japan which placed in power those who
were hostile to both foreign traders and to missionaries. This
revolution had a fatal effect upon the Portuguese influence,
especially as they had, as has been said, showed imprudence in
mingling in the politics of the country, while their ambassador
exhibited great pride and haughtiness, in marked contrast with the
Dutch. The latter, attending strictly to their mercantile transactions
and moved by hatred and jealousy of their rivals, the Portuguese,
took good care to give the assurance that they themselves were of a
different creed from the Jesuits.
When, therefore, in 1637, the Portuguese—traders, missionaries,
and all—were banished from the country, after many persecutions
and much bloodshed, the Dutch were allowed to keep up an
intercourse, but under severe restrictions.
 Once rid of the Portuguese, there then began a persecution of the
native Christians, which continued for many years, during which it is
said that several millions suffered for their faith. The number sounds
large, but all accounts agree in regard to it.
Then the Dutch fell under suspicion, for several good reasons, and
were only permitted to remain at all on condition of residing at one
spot, where they could be guarded and every motion observed. In
1641 they were ordered to remove to Dezima, a small island near
Nagasaki, which Kæmpfer said “was more like a prison than a
factory.” But, unwilling to quit the prospect of profitable trade, they
willingly underwent this imprisonment and agreed to forego any of
the outward signs of Christianity, such as leaving off divine service,
etc.
The island Dezima is shaped like a fan, and is very small, only
about six hundred by two hundred and fifty feet, and mostly of
artificial construction. It was connected with the town of Nagasaki by
a bridge, on which there was always a strong guard. The whole
island was surrounded by a high fence with iron spikes. No stone
houses were permitted to be built, and the interpreters, clerks, and
servants were spies, whom the Dutch were obliged to pay. The few
ships allowed to come annually were searched and their arms and
powder taken away. “A more annoying and thorough system of
imprisonment and espionage was never devised.”
Although subject to this oppression and contumely, the Dutch
continued their trade—one or two ships going from Batavia every
year—until Japan was thrown open to the world by means of the
action of the American expedition of 1853. But before speaking of
that expedition we must allude to the idea which has been
extensively entertained that there were formerly, and until quite
recently, two Emperors at the same time in Japan. This erroneous
idea was conceived in this way. About the year 1200, of our era, the
then Emperor created a supreme general, called Shógun. Each
Shógun owed allegiance to the Emperor, and was invested by him,
but his position as supreme head of the military organization, which
gave him immense influence with the powerful nobles or feudal
lords, made him almost the equal of the sovereign. Several years
after Japan had been opened to the world and treaties made with
many nations, in 1868, the Shógun’s power was shattered during a
war which might be denominated a revolution. Military domination
was swept away and the Mikado reinstated in his position of
supreme authority. In that year the powerful Tokugawa clan, and
others who supported the Shógun, were overcome by the great
clans of Satsuma, Choshin, and Tosa, and a powerful northern
opposition was put down by the Mikado’s forces.
It is a curious fact that the Stonewall Jackson, an ironclad, took
part in this war. She had been built in France for the Confederates,
taken to Havana, and then claimed by our government as a spoil of
war. She was sold by the United States to Japan, and taken out
there and delivered by one of our naval officers.
And now to relate some of the events in Japan in which our
country was most concerned.
In 1831 our first effort at intercourse began. A Japanese junk
which had been blown off their coast had drifted about the Pacific for
a long time, and at length went ashore near the mouth of the
Columbia River. Her crew were kindly treated and sent to China,
whence they were sent, on board an American merchant ship, the
Morrison, to Japan. People were not then aware of a Japanese law
which prohibited the return of any Japanese who had left their
country. At any rate, it was an errand of mercy. When the Morrison
entered Jeddo Bay the Japanese, finding she was unarmed, fired at
her with shotted guns, and she was forced to go to Kagosima.
Meeting the same reception there, she returned to Macao with the
shipwrecked Japanese on board.
Complaints having multiplied that American seamen wrecked on
the coast of Japan had been harshly treated by the authorities of that
country—which was very true, for the writer was a shipmate of one
who had been so treated, and often talked with him of his adventures
—our government was anxious to make a treaty which would provide
that such unfortunates should be kindly treated, and also that
American vessels in distress should be able to enter Japanese ports
for needed supplies. So Commodore Biddle, with the Columbus, of
90 guns, and the sloop-of-war Vincennes, of 20 guns, entered the
Bay of Jeddo, in 1846. The ships were at once surrounded by guard
boats—four hundred of them. The ships remained ten days, but no
one belonging to them landed, and an application for license to trade
was met by the answer: “No trade can be allowed with any foreign
nation but Holland.”
The next attempt was in 1849, when the United States sloop-of-
war Preble, Commander Glynn, was sent to inquire as to the
detention in Japan of sixteen American seamen who had been
shipwrecked on the coast of the islands. As the Preble approached
Nagasaki harbor she was surrounded by boats and warned off. But
the ship stood in with a fair breeze, and anchored in spite of them.
Troops were hurriedly gathered and heavy batteries erected on the
elevated shores, all bearing upon the ship. But Commander Glynn
persisted, in spite of threats and subterfuge, demanding the
prisoners and saying that the government meant to protect its
citizens and means would be at hand to enforce its demands.
Afterwards he sent word that the men must be delivered to him in
two days’ time; and then the Japanese, finding him in earnest, gave
them up. They had been very cruelly treated. Other attempts than
those we have mentioned were made at different times, without
success, by other nations, the English and the Russians especially.
Commodore Perry’s successful expedition left the United States in
November, 1852, several vessels intending to join it being already in
Chinese ports.
It was well known that our government contemplated such an
expedition, and it had been the subject of much comment in several
European countries more immediately concerned. The general
opinion was that the mission would, like the many others which had
been attempted by various powers, prove fruitless, from the
prejudices and obstinacy of the Japanese. But they did not reckon
upon the great tact, skill, and firmness which were to be displayed in
the accomplishment of this difficult task. The President’s letter was
presented on July 14th, 1853, when the squadron left with a promise
to return next season for an answer. On March 31st, 1854, a treaty
of peace and amity, providing especially for the protection of
American sailors, was signed.
In June, 1857, a new treaty was made at Simoda, by Townsend
Harris, American Consul-General to Japan, who succeeded in the
next year in reaching Jeddo in spite of opposition, where he
negotiated a third treaty, covering many more points than the first
ones.
Other nations soon followed us in making treaties, until Japan was
in full intercourse with the world at large.
It is impossible, here, to give more than a sketch of the many
interesting incidents of Perry’s first visit, but we will endeavor to give
a few points.
On the 7th of July, 1853, the Susquehanna and the Mississippi,
paddle-wheel frigates, with the Plymouth and the Saratoga, sloops-
of-war, entered the Bay of Yedo, the sailing sloops-of-war being in
tow of the steamers, and the crews of the junks which were
overhauled showed every evidence of surprise at the sight of the first
steamships they had ever seen, taking to their oars and sweeps and
hastily getting out of their course. At 5 o’clock in the afternoon the
squadron anchored off the city of Uraga, and from their position
distinctly saw the sacred mountain, Fusiyama, although it was at a
distance of about sixty miles. Previous to anchoring a number of
guard boats were observed coming off, and, contrary to the practice
permitted during the visit of the Columbus, the Commodore
determined to exclude casual visitors, so that they were not
permitted even to make fast their boats to the ships, much less to
come on board—a proceeding which seemed to anger the Japanese
officials, but had a good effect in the end. Soon after an official came
to warn the ships off, and he made signs to have the gangway ladder
of the flag-ship lowered for him. But the interpreter told him that the
Commodore was a very high officer of his government, and that he
would receive no one but a functionary of the highest rank, and was
asked why the Governor himself did not come off. He replied that he
was forbidden by law to do so, and asked that he (the speaker, who
was Vice-Governor,) should be received. After some delay this was
done, but he only saw the Commodore’s aide, who told him that the
Commodore’s intentions were perfectly friendly, and that he brought
a letter from the President of the United States addressed to the
Emperor. The Japanese official insisted that the ships must go to
Nagasaki and there deliver the letter, as that was the only place,
under their law, for the transaction of business with foreigners. He
was told that the Commodore would not go to Nagasaki, but
expected to be duly and properly received where he was, near Yedo,
and intimated that force might be used to deliver the message with
which he was charged. He was prepared to meet the Japanese on
their own ground and imitate their own policy. The result was that the
squadron was left free from all annoyance, an event unprecedented
in the intercourse of Japan with foreign ships for two centuries.
There were a good many forts and batteries to be seen on the
shore, however, and every precaution was taken against a sudden
attack, especially as bodies of soldiers could be seen moving about.
But the next day the Governor of the city appeared and came on
board. Being an official of the third rank, however, the Commodore
would not personally meet him. The Governor still insisted upon the
ships leaving there and going to Nagasaki, and was again told that
they would deliver the letter there, as the nearest point to the capital.
At a later interview he was informed that unless an answer was
given in three days, and the business which had brought the
squadron there was arranged at the present visit, the Commodore
would be obliged to return with a larger force, and, as Uraga was an
unsafe anchorage, he should go much nearer to Yedo.
It would take many pages to give all the arguments for delay
offered by the Japanese; but the firmness of Perry, who had not
been personally seen—as too exalted a person—at last gained the
day. The Emperor consented to have a meeting of high officers
deputized by him and Commodore Perry in a house built for the
purpose, on shore, where the letters could be formally exchanged.
All the officers of the squadron who could be spared accompanied
the Commodore, in full uniform, and a large force of marines and
sailors, under arms, formed a guard of honor. The United States flag
and the Commodore’s pennant were borne in front by two stalwart
seamen, and two boys, appropriately dressed, bore, in an envelope
of scarlet cloth, the President’s letter and the Commodore’s
credentials.
After long ceremonial conversations, everything was settled
pleasantly, and an answer promised upon the return of the squadron
the next spring.
On the 12th of the following February, Perry returned for his
answer. The Japanese were quite friendly, and the squadron, of
three steamers and four sailing men-of-war, anchored about twelve
miles beyond the town of Uraga and about twenty miles from the
capital city, Yedo. Even then the Japanese tried to change the place
of meeting, but without success, for the Americans persisted in
having it at that place, which is now known as Yokohama. Here a
fine building was erected as a “Treaty House,” and, on the 8th of
March, 1854, Perry landed in state for the second time, and on that
and the following days conferences were held and handsome
presents interchanged. Among arms, implements, wines, and other
things, was a small locomotive and tender, with a passenger car, and
enough rails to form a track. The Japanese Government sent to the
President a great quantity of things peculiar to the country, and all
ended in a good understanding and the granting of the demands of
the American Government.
Thus, not within the lifetime, but within the naval lifetime of the
writer, has a nation emerged from complete isolation and become so
powerful as to challenge and successfully meet in battle a
neighboring nation, some of the provinces of which contain as many
inhabitants as the whole of Japan.
No matter what may be the result of the war which is now going
on, it is certain that none of the great Western nations which have
hitherto controlled, more or less directly, the course of events in the
extreme East, will in future venture to take political steps without
reckoning Japan as a first-class power. Her resources, military and
naval, are present, while those of the Western nations must be
transported halfway round the world to reach them.
Before war was actually proclaimed the ships of China and Japan
had two or three conflicts on the Korean coast, one of which involved
quite a battle, and the destruction of a small Chinese cruiser which
was protecting the landing of Chinese troops, from transports, on the
coast of Korea. The second was the sinking of the Kow-Shing, which
steamer, in spite of her Chinese name, was an English vessel, and
one of the fastest and best employed in the Chinese coast trade.
When the news of the sinking of the Kow-Shing, which took place
on the 25th of July, first appeared, there was great bluster in the
English papers about holding Japan responsible; but, when the true
facts came to light all this talk quickly died away, as it was clearly
seen that the Japanese were within their own right in preventing the
landing of their enemy’s troops in Korea. Of their merciless treatment
of the Chinese when struggling in the water a difference of opinion
may probably be held.
The “Kow-Shing incident” was as follows:—This vessel, of about
1400 tons, had a crew of Chinese, but the captain, the three mates,
and three engineers were Englishmen. She was chartered by the
Chinese government, by the month, for military purposes. Toward
the end of July she took on board twelve hundred Chinese troops,
with two generals, and their body-guards, of about one hundred and
fifty men.
War was not formally declared, but two other Chinese troop ships
trying to land men, and under escort of men-of-war, accomplished
their purpose, but an action succeeded between the convoying
vessels and some Japanese cruisers, in which one of the Chinese
ships suffered great loss, and was set on fire. Her commander ran
her on shore, where she shortly blew up.
The Japanese succeeded in intercepting the Kow-Shing, and
determined to force her to return without landing the troops she had
on board.
But one of the Chinese cruisers, the Tsi-Yuen, accompanying the
two which had landed troops, observing the Japanese cruiser
Naniwa taking note of the operation, is said to have approached the
Naniwa with the Japanese flag flying and suddenly opened fire upon
her, as evidence of which an officer of the Kow-Shing was shown a
shell, which happily did not explode, in the ward-room of the Naniwa.
“What happened afterwards was probably done, at least in part, as
retaliation for this act of fighting under false colors.”
On July 25th at 8 o’clock in the morning, the Kow-Shing, with the
Chinese troops on board, sighted the Naniwa, which signalled her to
stop and then to anchor; she did so, and then signalled “Can we
proceed?” As an answer to this the Japanese cruiser sent a boat,
with an armed crew and two officers, who proceeded to the captain’s
cabin, where they examined the ship’s papers. They were told that
the Kow-Shing was a British steamer, with the British Consul’s
clearance, flying the British flag, and that war had not been declared
when she left port.
 Major Von Hanneken, the German officer in Chinese employ, told
the Chinese generals what had passed, and the latter said they had
rather die where they were, and said that if the British officers
attempted to leave the Kow-Shing they would be killed by their body-
guard. The English captain tried his hardest to show them how
useless it was to resist the Naniwa, but without success. By this time
the boat had returned to the Naniwa, and the latter signalled, “weigh,
cut or slip, wait for nothing” meaning that the English captain was to
carry his ship back to the place whence she had come, and not
attempt to land the generals and their troops in Korea. If they had
obeyed the order there would have been no loss of property or life.
But the Chinese would not allow the captain to move, and threatened
death again if he did so. The Naniwa then steamed abeam of the
Kow-Shing, on the port side, about 500 yards off. Then she blew her
whistle, ran a red flag up to her foremast head, and discharged a
torpedo, which however fell short. Immediately afterward, seeing that
the torpedo had missed, the Naniwa fired a broadside which hulled
the Kow-Shing, which keeled over to starboard and immediately
began to sink.
The English officers at once jumped overboard, and began to
swim for the land, through swarms of Chinamen, dead or drowning.
Bullets were striking on every side. They came from the Chinese
soldiers who were herded on the only part of the Kow-Shing left
above water. Then the Englishmen swam toward the Naniwa, and
after being a long time in the water were picked up by her boats. By
this time only the Kow-Shing’s masts were visible, and two of her
boats, while crowds of Chinese in the water were swimming about.
The officer of the Japanese cutter which had picked up the
Englishmen said he was ordered to sink the boats. He did fire at
them and then returned to the Naniwa without picking up any of the
Chinese. The next day the Naniwa joined the rest of the Japanese
fleet, and the Englishmen were sent by despatch boat to Japan,
where they were set at liberty a few days afterwards.
The Chinese and Japanese have for ages been in communication,
mercantile and otherwise, but there has always, so far as history
goes, been an underlying hostility in the feelings of the two nations.
These feelings have been aggravated by collisions at various
periods in regard to sovereignty, and the commercial intercourse with
the Loo-choo Islands, as well as in regard to Formosa, a very large
and immensely valuable possession for the nation which may be
fortunate enough to hold it. At present the greater part of the island is
in the possession of native clans, and the Chinese control the
country for only a short distance inland, upon the southwestern
portion mostly.
Then again, China and Japan have been at difference for a long
time in regard to the Korean territory, over which China has always
claimed a jurisdiction, which however she has not practically
exercised, except by intriguing in the state affairs of the country and
demanding acts of vassalage. When Japan, whose interests in her
neighbors are very important, protested against Chinese intrigue and
influence, she was received with ill-disguised contempt. Upon war
being declared by Japan, the Emperor of China and his advisers, not
recognizing the forward state of preparation of the Japanese, is said
to have ordered his military and naval commanders to “exterminate
the Japanese vermin.” How far the effort at “extermination” went, the
whole world now knows.
Japan solemnly declared, in a diplomatic note, that her whole
object in the war was to settle and secure once for all the separation
and independence of Korea. Of course, if successful, she would
demand compensation for the immense outlay incurred in her
campaigns by land and sea; and, while not approaching the sum
paid by France to Germany at the close of the Franco-German war, it
will be very large indeed, and one which will hamper the Chinese
government for a generation to come, as their fiscal methods do not
readily lend themselves to such an emergency.

The Naval Battle of the Yalu, Sept. 17, 1894.

Since the advent of modern battle-ships of the new type, armed

with high-powered rifled ordnance, naval officers of all nations had
been eagerly looking for an occasion when the use of such ships
and guns would be an object lesson to them, and various theories in
regard to naval warfare would be put to the test of actual practice.
While most people were looking to movements in other and widely
distant parts of the world—some predicting a naval battle in the
North Sea, while others looked for a battle of giants in the
Mediterranean—the problem was in part solved for them by a
pitched battle in the far Orient, between the Japanese and Chinese
fleets, and which will be known in history as the battle of the Yalu.
The rival fleets may be said to have illustrated each a different
principle. That represented by the Chinese was the principle of the
school which puts material above personnel, for their fleet contained
the heaviest ships and the largest guns, although these were not so
numerous as those of the Japanese. They had also the most
extensive torpedo equipment.
The Japanese represented the school which believes in lighter,
more active ships, and in “the man behind the gun”—that is, the
greater rapidity and accuracy of fire and ability in manœuvring—
much the same as Farragut’s conviction that the best protection for a
ship was a rapid and accurate fire from her battery.
 BATTLE OF THE YALU—SINKING OF THE CHIH-YUEN.

Before proceeding to describe the battle it would be well to give

some account of the strength of the contending fleets. By this we
mean the available naval strength of each nation at the outbreak of
the conflict.
The Chinese navy owes its existence principally to the fostering
care of the Imperial Viceroy, Li Hung-Chang, now in disgrace. He
employed Captain Lang, an Englishman, and other Europeans to
drill the ships’ companies. But Captain Lang was forced to leave that
service some time before the war began, and Captain Von
Henneken, a German, who constructed the forts at and near the
naval port of Wei-hai-wei, appears to have taken his place as adviser
to Admiral Ting—as much as a military man can advise upon naval
matters. The Chinese had five heavy ironclads—Ting-Yuen, Chen-
Yuen, King-Yuen, Lai-Yuen, and Ping-Yuen—with armor from
fourteen to eight inches thick, and armed with Krupp guns, from
twelve-inch to eight-inch calibre, mounted in barbette. They had also