Experience Seamless Full Ebook Downloads for Every Genre at textbookfull.

com

Deep Learning Vol 2 From Basics to Practice Andrew

Glassner

https://textbookfull.com/product/deep-learning-vol-2-from-
basics-to-practice-andrew-glassner/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Deep Learning Vol 1 From Basics to Practice Andrew

Glassner

https://textbookfull.com/product/deep-learning-vol-1-from-basics-to-
practice-andrew-glassner/

textboxfull.com

Deep Learning Vol 1 From Basics to Practice Andrew

Glassner

https://textbookfull.com/product/deep-learning-vol-1-from-basics-to-
practice-andrew-glassner-2/

textboxfull.com

Programming Machine Learning From Coding to Deep Learning

1st Edition Paolo Perrotta

https://textbookfull.com/product/programming-machine-learning-from-
coding-to-deep-learning-1st-edition-paolo-perrotta/

textboxfull.com

From Deep Learning to Rational Machines [converted]

Cameron J. Buckner

https://textbookfull.com/product/from-deep-learning-to-rational-
machines-converted-cameron-j-buckner/

textboxfull.com
From Deep Learning to Rational Machines 1st Edition
Cameron J. Buckner

https://textbookfull.com/product/from-deep-learning-to-rational-
machines-1st-edition-cameron-j-buckner/

textboxfull.com

Deep Learning Applications, Volume 2 M. Arif Wani

https://textbookfull.com/product/deep-learning-applications-
volume-2-m-arif-wani/

textboxfull.com

COVID 19 From Basics To Clinical Practice 1st Edition

Wenhong Zhang

https://textbookfull.com/product/covid-19-from-basics-to-clinical-
practice-1st-edition-wenhong-zhang/

textboxfull.com

Introduction to Deep Learning From Logical Calculus to

Artificial Intelligence 1st Edition Sandro Skansi

https://textbookfull.com/product/introduction-to-deep-learning-from-
logical-calculus-to-artificial-intelligence-1st-edition-sandro-skansi/

textboxfull.com

Processing for Visual Artists: How to Create Expressive

Images and Interactive Art 1st Edition Andrew Glassner

https://textbookfull.com/product/processing-for-visual-artists-how-to-
create-expressive-images-and-interactive-art-1st-edition-andrew-
glassner/
textboxfull.com
 Volume 2

DEEP LEARNING:
From Basics
to Practice

Andrew Glassner
Deep Learning:
From Basics to Practice
Volume 2
Copyright (c) 2018 by Andrew Glassner

www.glassner.com / @AndrewGlassner
All rights reserved. No part of this book, except as noted below, may be reproduced,
stored in a retrieval system, or transmitted in any form or by any means, without
the prior written permission of the author, except in the case of brief quotations
embedded in critical articles or reviews.

The above reservation of rights does not apply to the program files associated with
this book (available on GitHub), or to the images and figures (also available on
GitHub), which are released under the MIT license. Any images or figures that are
not original to the author retain their original copyrights and protections, as noted
in the book and on the web pages where the images are provided.

All software in this book, or in its associated repositories, is provided “as is,” with-
out warranty of any kind, express or implied, including but not limited to the
warranties of merchantability, fitness for a particular pupose, and noninfringe-
ment. In no event shall the authors or copyright holders be liable for any claim,
damages or other liability, whether in an action of contract, tort, or otherwise,
arising from, out of or in connection with the software or the use or other dealings
in the software.

First published February 20, 2018

Version 1.0.1 March 3, 2018

Version 1.1 March 22, 2018

Published by The Imaginary Institute, Seattle, WA.

http://www.imaginary-institute.com

Contact: [email protected]
 For Niko,
who’s always there
with a smile
and a wag.
Contents of Both Volumes

Volume 1
Preface..................................................................... i
Chapter 1: An Introduction.................................... 1
1.1 Why This Chapter Is Here................................ 3
1.1.1 Extracting Meaning from Data............................. 4
1.1.2 Expert Systems...................................................... 6

1.2 Learning from Labeled Data........................... 9

1.2.1 A Learning Strategy............................................... 10
1.2.2 A Computerized Learning Strategy.................... 12
1.2.3 Generalization....................................................... 16
1.2.4 A Closer Look at Learning.................................... 18

1.3 Supervised Learning......................................... 21

1.3.1 Classification.......................................................... 21
1.3.2 Regression.............................................................. 22

1.4 Unsupervised Learning.................................... 25

1.4.1 Clustering............................................................... 25
1.4.2 Noise Reduction.................................................... 26
1.4.3 Dimensionality Reduction................................... 28

1.5 Generators......................................................... 32
1.6 Reinforcement Learning.................................. 34
1.7 Deep Learning................................................... 37
1.8 What’s Coming Next........................................ 43
References............................................................... 44
Image credits.................................................................. 45
Chapter 2: Randomness and Basic Statistics...... 46
2.1 Why This Chapter Is Here................................ 48
2.2 Random Variables............................................ 49
2.2.1 Random Numbers in Practice............................. 57

2.3 Some Common Distributions......................... 59

2.3.1 The Uniform Distribution.................................... 60
2.3.2 The Normal Distribution..................................... 61
2.3.3 The Bernoulli Distribution.................................. 67
2.3.4 The Multinoulli Distribution............................... 69
2.3.5 Expected Value..................................................... 70

2.4 Dependence..................................................... 70
2.4.1 i.i.d. Variables......................................................... 71

2.5 Sampling and Replacement............................ 71

2.5.1 Selection With Replacement............................... 73
2.5.2 Selection Without Replacement........................ 74
2.5.3 Making Selections................................................ 75

2.6 Bootstrapping.................................................. 76
2.7 High-Dimensional Spaces............................... 82
2.8 Covariance and Correlation............................ 85
2.8.1 Covariance............................................................. 86
2.8.2 Correlation............................................................ 88

2.9 Anscombe’s Quartet........................................ 93

References............................................................... 95
Chapter 3: Probability............................................ 97
3.1 Why This Chapter Is Here................................ 99
3.2 Dart Throwing.................................................. 100
3.3 Simple Probability........................................... 103
3.4 Conditional Probability................................... 104
3.5 Joint Probability............................................... 109
3.6 Marginal Probability........................................ 114
3.7 Measuring Correctness................................... 115
3.7.1 Classifying Samples............................................... 116
3.7.2 The Confusion Matrix.......................................... 119
3.7.3 Interpreting the Confusion Matrix.................... 121
3.7.4 When Misclassification Is Okay.......................... 126
3.7.5 Accuracy................................................................. 129
3.7.6 Precision................................................................ 130
3.7.7 Recall...................................................................... 132
3.7.8 About Precision and Recall................................. 134
3.7.9 Other Measures.................................................... 137
3.7.10 Using Precision and Recall Together................ 141
3.7.11 f1 Score.................................................................. 143

3.8 Applying the Confusion Matrix..................... 144

References............................................................... 151
Chapter 4: Bayes Rule............................................ 153
4.1 Why This Chapter Is Here............................... 155
4.2 Frequentist and Bayesian Probability .......... 156
4.2.1 The Frequentist Approach................................... 156
4.2.2 The Bayesian Approach....................................... 157
4.2.3 Discussion............................................................. 158

4.3 Coin Flipping ................................................... 159

4.4 Is This a Fair Coin?........................................... 161
4.4.1 Bayes’ Rule............................................................. 173
4.4.2 Notes on Bayes’ Rule........................................... 175

4.5 Finding Life Out There................................... 178

4.6 Repeating Bayes’ Rule..................................... 183
4.6.1 The Posterior-Prior Loop..................................... 184
4.6.2 Example: Which Coin Do We Have?.................. 186

4.7 Multiple Hypotheses....................................... 194

References............................................................... 203
Chapter 5: Curves and Surfaces............................ 205
5.1 Why This Chapter Is Here................................ 207
5.2 Introduction..................................................... 207
5.3 The Derivative.................................................. 210
5.4 The Gradient.................................................... 222
References............................................................... 229
Chapter 6: Information Theory............................. 231
6.1 Why This Chapter Is Here............................... 233
6.1.1 Information: One Word, Two Meanings............. 233

6.2 Surprise and Context...................................... 234

6.2.1 Surprise.................................................................. 234
6.2.2 Context.................................................................. 236

6.3 The Bit as Unit................................................. 237

6.4 Measuring Information.................................. 238
6.5 The Size of an Event........................................ 240
6.6 Adaptive Codes................................................ 241
6.7 Entropy ............................................................ 250
6.8 Cross-Entropy.................................................. 253
6.8.1 Two Adaptive Codes............................................. 253
6.8.2 Mixing Up the Codes.......................................... 257

6.9 KL Divergence.................................................. 260

References............................................................... 262
Chapter 7: Classification........................................ 265
7.1 Why This Chapter Is Here................................ 267
7.2 2D Classification............................................... 268
7.2.1 2D Binary Classification........................................ 269

7.3 2D Multi-class classification........................... 275

7.4 Multiclass Binary Categorizing...................... 277
7.4.1 One-Versus-Rest .................................................. 278
7.4.2 One-Versus-One .................................................. 280

7.5 Clustering.......................................................... 286

7.6 The Curse of Dimensionality.......................... 290
7.6.1 High Dimensional Weirdness............................... 299

References............................................................... 307
Chapter 8: Training and Testing............................ 309
8.1 Why This Chapter Is Here............................... 311
8.2 Training............................................................. 312
8.2.1 Testing the Performance..................................... 314

8.3 Test Data........................................................... 318

8.4 Validation Data................................................ 323
8.5 Cross-Validation.............................................. 328
8.5.1 k-Fold Cross-Validation........................................ 331

8.6 Using the Results of Testing.......................... 334

References............................................................... 335
Image Credits.................................................................. 336

Chapter 9: Overfitting and Underfitting............. 337

9.1 Why This Chapter Is Here................................ 339
9.2 Overfitting and Underfitting......................... 340
9.2.1 Overfitting............................................................. 340
9.2.2 Underfitting.......................................................... 342

9.3 Overfitting Data.............................................. 342

9.4 Early Stopping.................................................. 348
9.5 Regularization.................................................. 350
9.6 Bias and Variance............................................ 352
9.6.1 Matching the Underlying Data........................... 353
9.6.2 High Bias, Low Variance...................................... 357
9.6.3 Low Bias, High Variance...................................... 359
9.6.4 Comparing Curves............................................... 360

9.7 Fitting a Line with Bayes’ Rule....................... 363

References............................................................... 372
Chapter 10: Neurons............................................... 374
10.1 Why This Chapter Is Here.............................. 376
10.2 Real Neurons.................................................. 376
10.3 Artificial Neurons........................................... 379
10.3.1 The Perceptron.................................................... 379
10.3.2 Perceptron History............................................. 381
10.3.3 Modern Artificial Neurons................................ 382

10.4 Summing Up................................................... 390

References............................................................... 390
Chapter 11: Learning and Reasoning..................... 393
11.1 Why This Chapter Is Here............................... 395
11.2 The Steps of Learning..................................... 396
11.2.1 Representation..................................................... 396
11.2.2 Evaluation............................................................. 400
11.2.3 Optimization........................................................ 400

11.3 Deduction and Induction............................... 402

11.4 Deduction........................................................ 403
11.4.1 Categorical Syllogistic Fallacies.......................... 410
11.5 Induction.......................................................... 415
11.5.1 Inductive Terms in Machine Learning............... 419
11.5.2 Inductive Fallacies............................................... 420

11.6 Combined Reasoning..................................... 422

11.6.1 Sherlock Holmes, “Master of Deduction”.......... 424

11.7 Operant Conditioning..................................... 425

References............................................................... 428
Chapter 12: Data Preparation................................ 431
12.1 Why This Chapter Is Here.............................. 433
12.2 Transforming Data.......................................... 433
12.3 Types of Data.................................................. 436
12.3.1 One-Hot Encoding............................................... 438

12.4 Basic Data Cleaning....................................... 440

12.4.1 Data Cleaning....................................................... 441
12.4.2 Data Cleaning in Practice.................................. 442

12.5 Normalizing and Standardizing..................... 443

12.5.1 Normalization...................................................... 444
12.5.2 Standardization................................................... 446
12.5.3 Remembering the Transformation................... 447
12.5.4 Types of Transformations.................................. 448

12.6 Feature Selection........................................... 450

12.7 Dimensionality Reduction............................. 451
12.7.1 Principal Component Analysis (PCA)................ 452
12.7.2 Standardization and PCA for Images................ 459

12.8 Transformations............................................. 468

12.9 Slice Processing.............................................. 475
12.9.1 Samplewise Processing....................................... 476
12.9.2 Featurewise Processing...................................... 477
12.9.3 Elementwise Processing.................................... 479

12.10 Cross-Validation Transforms....................... 480

References............................................................... 486
Image Credits.................................................................. 486

Chapter 13: Classifiers............................................ 488

13.1 Why This Chapter Is Here.............................. 490
13.2 Types of Classifiers......................................... 491
13.3 k-Nearest Neighbors (KNN).......................... 493
13.4 Support Vector Machines (SVMs)................ 502
13.5 Decision Trees................................................. 512
13.5.1 Building Trees....................................................... 519
13.5.2 Splitting Nodes.................................................... 525
13.5.3 Controlling Overfitting...................................... 528

13.6 Naïve Bayes..................................................... 529

13.7 Discussion........................................................ 536
References............................................................... 538
Chapter 14: Ensembles........................................... 539
14.1 Why This Chapter Is Here.............................. 541
14.2 Ensembles....................................................... 542
14.3 Voting............................................................... 543
14.4 Bagging............................................................ 544
14.5 Random Forests.............................................. 547
14.6 ExtraTrees....................................................... 549
14.7 Boosting........................................................... 549
References............................................................... 561
Chapter 15: Scikit-learn.......................................... 563
15.1 Why This Chapter Is Here.............................. 566
15.2 Introduction.................................................... 567
15.3 Python Conventions....................................... 569
15.4 Estimators....................................................... 574
15.4.1 Creation................................................................ 575
15.4.2 Learning with fit()............................................... 576
15.4.3 Predicting with predict()................................... 578
15.4.4 decision_function(), predict_proba().............. 581

15.5 Clustering........................................................ 582

15.6 Transformations............................................. 587
15.6.1 Inverse Transformations..................................... 594

15.7 Data Refinement............................................ 598

15.8 Ensembles....................................................... 601
15.9 Automation..................................................... 605
15.9.1 Cross-validation................................................... 606
15.9.2 Hyperparameter Searching............................... 610
15.9.3 Exhaustive Grid Search...................................... 614
15.9.4 Random Grid Search.......................................... 625
15.9.5 Pipelines............................................................... 626
15.9.6 The Decision Boundary...................................... 641
15.9.7 Pipelined Transformations ................................ 643
15.10 Datasets......................................................... 647
15.11 Utilities............................................................ 650
15.12 Wrapping Up.................................................. 652
References............................................................... 653
Chapter 16: Feed-Forward Networks................... 655
16.1 Why This Chapter Is Here.............................. 657
16.2 Neural Network Graphs................................ 658
16.3 Synchronous and Asynchronous Flow......... 661
16.3.1 The Graph in Practice......................................... 664

16.4 Weight Initialization...................................... 664

16.4.1 Initialization......................................................... 667

References............................................................... 670
Chapter 17: Activation Functions.......................... 672
17.1 Why This Chapter Is Here............................... 674
17.2 What Activation Functions Do...................... 674
17.2.1 The Form of Activation Functions..................... 679

17.3 Basic Activation Functions............................ 679

17.3.1 Linear Functions................................................... 680
17.3.2 The Stair-Step Function...................................... 681

17.4 Step Functions................................................ 682

17.5 Piecewise Linear Functions............................ 685
17.6 Smooth Functions........................................... 690
17.7 Activation Function Gallery............................ 698
17.8 Softmax............................................................ 699
References............................................................... 702
Chapter 18: Backpropagation................................ 703
18.1 Why This Chapter Is Here.............................. 706
18.1.1 A Word On Subtlety............................................. 708

18.2 A Very Slow Way to Learn............................. 709

18.2.1 A Slow Way to Learn........................................... 712
18.2.2 A Faster Way to Learn........................................ 716

18.3 No Activation Functions for Now................ 718

18.4 Neuron Outputs and Network Error........... 719
18.4.1 Errors Change Proportionally............................ 720

18.5 A Tiny Neural Network.................................. 726

18.6 Step 1: Deltas for the Output Neurons....... 732
18.7 Step 2: Using Deltas to Change Weights..... 745
18.8 Step 3: Other Neuron Deltas........................ 750
18.9 Backprop in Action........................................ 758
18.10 Using Activation Functions......................... 765
18.11 The Learning Rate......................................... 774
18.11.1 Exploring the Learning Rate............................. 777
18.12 Discussion...................................................... 787
18.12.1 Backprop In One Place...................................... 787
18.12.2 What Backprop Doesn’t Do............................. 789
18.12.3 What Backprop Does Do.................................. 789
18.12.4 Keeping Neurons Happy.................................. 790
18.12.5 Mini-Batches...................................................... 795
18.12.6 Parallel Updates................................................ 796
18.12.7 Why Backprop Is Attractive............................. 797
18.12.8 Backprop Is Not Guaranteed .......................... 797
18.12.9 A Little History.................................................. 798
18.12.10 Digging into the Math..................................... 800

References............................................................... 802
Chapter 19: Optimizers.......................................... 805
19.1 Why This Chapter Is Here.............................. 807
19.2 Error as Geometry.......................................... 807
19.2.1 Minima, Maxima, Plateaus, and Saddles........... 808
19.2.2 Error as A 2D Curve............................................ 814

19.3 Adjusting the Learning Rate......................... 817

19.3.1 Constant-Sized Updates..................................... 819
19.3.2 Changing the Learning Rate Over Time.......... 829
19.3.3 Decay Schedules................................................. 832

19.4 Updating Strategies....................................... 836

19.4.1 Batch Gradient Descent..................................... 837
19.4.2 Stochastic Gradient Descent (SGD)................. 841
19.4.3 Mini-Batch Gradient Descent........................... 844

19.5 Gradient Descent Variations......................... 846

19.5.1 Momentum.......................................................... 847
19.5.2 Nesterov Momentum......................................... 856
19.5.3 Adagrad................................................................ 862
19.5.4 Adadelta and RMSprop...................................... 864
19.5.5 Adam.................................................................... 866
19.6 Choosing An Optimizer................................. 868
References............................................................... 870

Volume 2
Chapter 20: Deep Learning................................... 872
20.1 Why This Chapter Is Here............................. 874
20.2 Deep Learning Overview.............................. 874
20.2.1 Tensors................................................................. 878

20.3 Input and Output Layers.............................. 879

20.3.1 Input Layer........................................................... 879
20.3.2 Output Layer...................................................... 880

20.4 Deep Learning Layer Survey........................ 881

20.4.1 Fully-Connected Layer....................................... 882
20.4.2 Activation Functions.......................................... 883
20.4.3 Dropout............................................................... 884
20.4.4 Batch Normalization......................................... 887
20.4.5 Convolution........................................................ 890
20.4.6 Pooling Layers.................................................... 892
20.4.7 Recurrent Layers................................................ 894
20.4.8 Other Utility Layers.......................................... 896

20.5 Layer and Symbol Summary........................ 898

20.6 Some Examples............................................. 899
20.7 Building A Deep Learner............................... 910
20.7.1 Getting Started.................................................... 912

20.8 Interpreting Results...................................... 913

20.8.1 Satisfactory Explainability................................. 920
References............................................................... 923
Image credits:................................................................. 925

Chapter 21: Convolutional Neural Networks....... 927

21.1 Why This Chapter Is Here.............................. 930
21.2 Introduction.................................................... 931
21.2.1 The Two Meanings of “Depth”............................ 932
21.2.2 Sum of Scaled Values.......................................... 933
21.2.3 Weight Sharing.................................................... 938
21.2.4 Local Receptive Field......................................... 940
21.2.5 The Kernel............................................................ 943

21.3 Convolution..................................................... 944

21.3.1 Filters.................................................................... 948
21.3.2 A Fly’s-Eye View.................................................. 953
21.3.3 Hierarchies of Filters.......................................... 955
21.3.4 Padding................................................................ 963
21.3.5 Stride.................................................................... 966

21.4 High-Dimensional Convolution.................... 971

21.4.1 Filters with Multiple Channels.......................... 975
24.4.2 Striding for Hierarchies..................................... 977

24.5 1D Convolution............................................... 979

24.6 1×1 Convolutions............................................ 980
24.7 A Convolution Layer...................................... 983
24.7.1 Initializing the Filter Weights............................ 984

24.8 Transposed Convolution............................... 985

24.9 An Example Convnet..................................... 991
24.9.1 VGG16 .................................................................. 996
21.9.2 Looking at the Filters, Part 1.............................. 1001
21.9.3 Looking at the Filters, Part 2............................. 1008
21.10 Adversaries.................................................... 1012
References............................................................... 1017
Image credits.................................................................. 1022

Chapter 22: Recurrent Neural Networks............. 1023

22.1 Why This Chapter Is Here.............................. 1025
22.2 Introduction................................................... 1027
22.3 State................................................................ 1030
22.3.1 Using State........................................................... 1032

22.4 Structure of an RNN Cell.............................. 1037

22.4.1 A Cell with More State....................................... 1042
22.4.2 Interpreting the State Values .......................... 1045

22.5 Organizing Inputs.......................................... 1046

22.6 Training an RNN............................................. 1051
22.7 LSTM and GRU............................................... 1054
22.7.1 Gates..................................................................... 1055
22.7.2 LSTM.................................................................... 1060

22.8 RNN Structures............................................. 1066

22.8.1 Single or Many Inputs and Outputs................. 1066
22.8.2 Deep RNN........................................................... 1070
22.8.3 Bidirectional RNN.............................................. 1072
22.8.4 Deep Bidirectional RNN.................................... 1074

22.9 An Example..................................................... 1076

References............................................................... 1084
Chapter 23: Keras Part 1......................................... 1090
23.1 Why This Chapter Is Here.............................. 1093
23.1.1 The Structure of This Chapter............................ 1094
23.1.2 Notebooks............................................................ 1094
23.1.3 Python Warnings................................................. 1094

23.2 Libraries and Debugging............................... 1095

23.2.1 Versions and Programming Style...................... 1097
23.2.2 Python Programming and Debugging............. 1098

23.3 Overview......................................................... 1100

23.3.1 What’s a Model?.................................................. 1101
23.3.2 Tensors and Arrays............................................. 1102
23.3.3 Setting Up Keras................................................. 1102
23.3.4 Shapes of Tensors Holding Images.................. 1104
23.3.5 GPUs and Other Accelerators.......................... 1108

23.4 Getting Started.............................................. 1109

23.4.1 Hello, World......................................................... 1110

23.5 Preparing the Data........................................ 1114

23.5.1 Reshaping............................................................. 1115
23.5.2 Loading the Data................................................ 1126
23.5.3 Looking at the Data........................................... 1129
23.5.4 Train-test Splitting............................................. 1136
23.5.5 Fixing the Data Type.......................................... 1138
23.5.6 Normalizing the Data........................................ 1139
23.5.7 Fixing the Labels................................................. 1142
23.5.8 Pre-Processing All in One Place....................... 1148

23.6 Making the Model......................................... 1150

23.6.1 Turning Grids into Lists...................................... 1152
23.6.2 Creating the Model............................................ 1154
23.6.3 Compiling the Model......................................... 1163
23.6.4 Model Creation Summary................................ 1167
23.7 Training The Model........................................ 1169
23.8 Training and Using A Model......................... 1172
23.8.1 Looking at the Output....................................... 1174
23.8.2 Prediction............................................................ 1180
23.8.3 Analysis of Training History.............................. 1186

23.9 Saving and Loading........................................ 1190

23.9.1 Saving Everything in One File........................... 1190
23.9.2 Saving Just the Weights.................................... 1191
23.9.3 Saving Just the Architecture............................ 1192
23.9.4 Using Pre-Trained Models................................ 1193
23.9.5 Saving the Pre-Processing Steps...................... 1194

23.10 Callbacks....................................................... 1195

23.10.1 Checkpoints....................................................... 1196
23.10.2 Learning Rate.................................................... 1200
23.10.3 Early Stopping................................................... 1201

References............................................................... 1205
Image Credits.................................................................. 1208

Chapter 24: Keras Part 2........................................ 1209

24.1 Why This Chapter Is Here.............................. 1212
24.2 Improving the Model.................................... 1212
24.2.1 Counting Up Hyperparameters........................ 1213
24.2.2 Changing One Hyperparameter....................... 1214
24.2.3 Other Ways to Improve..................................... 1218
24.2.4 Adding Another Dense Layer........................... 1219
24.2.5 Less Is More........................................................ 1221
24.2.6 Adding Dropout................................................. 1224
24.2.7 Observations....................................................... 1230
24.3 Using Scikit-Learn......................................... 1231
24.3.1 Keras Wrappers................................................... 1232
24.3.2 Cross-Validation................................................. 1237
24.3.3 Cross-Validation with Normalization.............. 1243
24.3.4 Hyperparameter Searching ............................. 1247

24.4 Convolution Networks.................................. 1259

24.4.1 Utility Layers....................................................... 1260
24.4.2 Preparing the Data for A CNN......................... 1263
24.4.3 Convolution Layers............................................ 1268
24.4.4 Using Convolution for MNIST.......................... 1276
24.4.5 Patterns............................................................... 1290
24.4.6 Image Data Augmentation............................... 1293
24.4.7 Synthetic Data.................................................... 1298
24.4.8 Parameter Searching for Convnets................. 1300

24.5 RNNs............................................................... 1301

24.5.1 Generating Sequence Data................................ 1302
24.5.2 RNN Data Preparation...................................... 1306
24.5.3 Building and Training an RNN ......................... 1314
24.5.4 Analyzing RNN Performance............................ 1320
24.5.5 A More Complex Dataset.................................. 1330
24.5.6 Deep RNNs......................................................... 1334
24.5.7 The Value of More Data.................................... 1338
24.5.8 Returning Sequences........................................ 1343
24.5.9 Stateful RNNs..................................................... 1349
24.5.10 Time-Distributed Layers.................................. 1352
24.5.11 Generating Text................................................. 1357

24.6 The Functional API........................................ 1366

24.6.1 Input Layers......................................................... 1370
24.6.2 Making A Functional Model............................. 1371

References............................................................... 1378
Image Credits.................................................................. 1379
Chapter 25: Autoencoders..................................... 1380
25.1 Why This Chapter Is Here.............................. 1382
25.2 Introduction................................................... 1383
25.2.1 Lossless and Lossy Encoding............................. 1384
25.2.2 Domain Encoding............................................... 1386
25.2.3 Blending Representations................................. 1388

25.3 The Simplest Autoencoder........................... 1393

25.4 A Better Autoencoder................................... 1400
25.5 Exploring the Autoencoder.......................... 1405
25.5.1 A Closer Look at the Latent Variables.............. 1405
25.5.2 The Parameter Space......................................... 1409
25.5.3 Blending Latent Variables................................. 1415
25.5.4 Predicting from Novel Input............................ 1418

25.6 Discussion....................................................... 1419

25.7 Convolutional Autoencoders........................ 1420
25.7.1 Blending Latent Variables.................................. 1424
25.7.2 Predicting from Novel Input............................. 1426

26.8 Denoising........................................................ 1427

25.9 Variational Autoencoders............................. 1430
25.9.1 Distribution of Latent Variables........................ 1432
25.9.2 Variational Autoencoder Structure ................. 1433

29.10 Exploring the VAE........................................ 1442

References............................................................... 1455
Image credits.................................................................. 1457
Chapter 26: Reinforcement Learning................... 1458
26.1 Why This Chapter Is Here.............................. 1461
26.2 Goals................................................................ 1462
26.2.1 Learning A New Game........................................ 1463

26.3 The Structure of RL....................................... 1469

26.3.1 Step 1: The Agent Selects an Action................. 1471
26.3.2 Step 2: The Environment Responds................ 1473
26.3.3 Step 3: The Agent Updates Itself..................... 1475
26.3.4 Variations on The Simple Version.................... 1476
26.3.5 Back to the Big Picture..................................... 1478
26.3.6 Saving Experience.............................................. 1480
26.3.7 Rewards............................................................... 1481

26.4 Flippers........................................................... 1490

26.5 L-learning........................................................ 1492
26.5.1 Handling Unpredictability................................. 1505

26.6 Q-learning...................................................... 1509

26.6.1 Q-values and Updates........................................ 1510
26.6.2 Q-Learning Policy.............................................. 1514
26.6.3 Putting It All Together....................................... 1518
26.6.4 The Elephant in the Room................................ 1519
26.6.5 Q-learning in Action.......................................... 1521

26.7 SARSA.............................................................. 1532

26.7.1 SARSA in Action................................................... 1535
26.7.2 Comparing Q-learning and SARSA................... 1543

26.8 The Big Picture.............................................. 1548

26.9 Experience Replay......................................... 1550
26.10 Two Applications.......................................... 1551
References............................................................... 1554
Chapter 27: Generative Adversarial Networks.... 1558
27.1 Why This Chapter Is Here.............................. 1560
27.2 A Metaphor: Forging Money........................ 1562
27.2.1 Learning from Experience.................................. 1566
27.2.2 Forging with Neural Networks.......................... 1569
27.2.3 A Learning Round............................................... 1572

27.3 Why Antagonistic?......................................... 1574

27.4 Implementing GANs...................................... 1575
27.4.1 The Discriminator................................................ 1576
27.4.2 The Generator..................................................... 1577
27.4.3 Training the GAN................................................ 1578
27.4.4 Playing the Game............................................... 1581

27.5 GANs in Action............................................... 1582

27.6 DCGANs.......................................................... 1591
27.6.1 Rules of Thumb.................................................... 1595

27.7 Challenges....................................................... 1596

27.7.1 Using Big Samples................................................ 1597
27.7.2 Modal Collapse.................................................... 1598

References............................................................... 1600
Chapter 28: Creative Applications........................ 1603
28.1 Why This Chapter Is Here............................. 1605
28.2 Visualizing Filters.......................................... 1605
28.2.1 Picking A Network.............................................. 1605
28.2.2 Visualizing One Filter........................................ 1607
28.2.3 Visualizing One Layer........................................ 1610

23.3 Deep Dreaming.............................................. 1613

28.4 Neural Style Transfer.................................... 1620
28.4.1 Capturing Style in a Matrix............................... 1621
28.4.2 The Big Picture................................................... 1623
28.4.3 Content Loss...................................................... 1624
28.4.4 Style Loss............................................................ 1628
28.4.5 Performing Style Transfer................................ 1633
28.4.6 Discussion........................................................... 1640

28.5 Generating More of This Book.................... 1642

References............................................................... 1644
Image Credits.................................................................. 1646

Chapter 29: Datasets.............................................. 1648

29.1 Public Datasets............................................... 1650
29.2 MNIST and Fashion-MNIST.......................... 1651
29.3 Built-in Library Datasets............................... 1652
29.3.1 scikit-learn........................................................... 1652
29.3.2 Keras.................................................................... 1653

29.4 Curated Dataset Collections........................ 1654

29.5 Some Newer Datasets................................... 1655
Chapter 30: Glossary.............................................. 1658
About The Glossary................................................ 1660
0-9.................................................................................... 1660
Greek Letters.................................................................. 1661
A........................................................................................ 1662
B........................................................................................ 1665
C........................................................................................ 1670
D....................................................................................... 1678
E........................................................................................ 1685
F........................................................................................ 1688
G....................................................................................... 1694
H....................................................................................... 1697
I......................................................................................... 1699
J......................................................................................... 1703
K........................................................................................ 1703
L........................................................................................ 1704
M...................................................................................... 1708
N....................................................................................... 1713
O....................................................................................... 1717
P........................................................................................ 1719
Q....................................................................................... 1725
R........................................................................................ 1725
S........................................................................................ 1729
T........................................................................................ 1738
U....................................................................................... 1741
V........................................................................................ 1743
W....................................................................................... 1745
X........................................................................................ 1746
Z........................................................................................ 1746
 Preface

Welcome!

A few quick words of

introduction to the book, how
to get the notebooks and figures,
and thanks to the people who helped me.
 Preface

What You’ll Get from This Book

Hello!

If you’re interested in deep learning (DL) and machine learning (ML),

then there’s good stuff for you in this book.

My goal in this book is to give you the broad skills to be an effective

practitioner of machine learning and deep learning.

When you’ve read this book, you will be able to:

• Design and train your own deep networks.

• Use your networks to understand your data, or make new data.

• Assign descriptive categories to text, images, and other types of data.

• Predict the next value for a sequence of data.

• Investigate the structure of your data.

• Process your data for maximum efficiency.

• Use any programming language and DL library you like.

• Understand new papers and ideas, and put them into practice.

• Enjoy talking about deep learning with other people.

We’ll take a serious but friendly approach, supported by tons of illus-

trations. And we’ll do it all without any code, and without any math
beyond multiplication.
If that sounds good to you, welcome aboard!

ii
Exploring the Variety of Random
Documents with Different Content
the things in them which seem to me most interesting are not quite
those which struck M. Rigault.
419. This is referred to, in the extract from Leone Allacci prefixed
to the 1646 ed. of Tassoni, as libellus Patavii editus. Tassoni seems
to have replied under a pseudonym and pretty savagely (magis
aculeatis dentibus).
420. Eight volumes, in 16 parts, of a not small quarto (Venice,
1643). This is one of the many books for the opportunity of studying
which, without burdening shelves and lightening purse, I am
indebted to the Library of the Faculty of Advocates.
421. Venice, 1612-13.
422. See Professors Gayley and Scott’s invaluable book so often
cited, p. 447, a passage based on Blankenburg’s older Zusätze (3
vols., Leipsic, 1796-98).
423. Venice, 1613.
424. Blankenburg is the sinner here.
425. Padua, 1681, pp. 1025-1088.
426. Henry Cary, Earl of Monmouth, translated the book in 1656.
427. Spain can boast, however, perhaps the very best History of
its criticism as a whole that any European language has—if not as
yet the only good one—in the Historia de las Ideas Estéticas en
España of Don Marcelino Menéndez y Pelayo (Ed. 2, 9 vols.,
Madrid, 1890-1896). This is fortunate for me, inasmuch as I do not
pretend to any extensive familiarity with Spanish literature beyond
the early poets, and indeed do not read the language with very great
facility. Besides Señor Menéndez I have relied chiefly on the texts
and comments recently furnished (v. inf.) by M. Morel-Fatio (who will,
I hope, continue in so good a road), on Ticknor, on the short but
valuable notices of this period in Mr Spingarn, op. cit., on those in my
friend Mr Hannay’s The Later Renaissance (Edinburgh and London,
1898), and on Mr James Fitzmaurice Kelly’s History of Spanish
Literature (London, 1898). I am particularly obliged to Mr Kelly for a
copy of the recent (undated) Spanish translation of his book, with a
few corrections, and a preface by Señor Menéndez himself. The
Spanish critic combines, with a just praise of the book, a mild
remonstrance as to the small space which Mr Fitzmaurice Kelly has
given to this very critical subject—a fact in which I own I myself had
felt some comfort. The silence of the specialist is the shield of the
expatiator. I have not failed, wherever I could, to verify all the critical
deliverances in the text, and examine almost all, if not all, the books
mentioned; but I do not know the circumference of them as I do
elsewhere. And as I began this History on the principle of going to
the sources, I think myself bound to warn the reader of any case in
which I have been obliged to modify that principle.
428. This was written before M. Morel-Fatio had expressed the
same view in his Les Défenseurs de la Comedia (Bulletin
Hispanique, ubi cit. inf.) See also on the point Mr Ker’s Essays of
Dryden, i. lxvi, and the references in his index to Dryden’s mention of
Spanish plays. Of course the main interest of the matter lies in the
much stronger resemblances that exist between the great English
and Spanish dramas than between any other two national branches
of the European theatre.
429. They say now that he was not only not (as used to be said)
the premier and only Marquis of Spain, but not a Marquis at all. Non
moror: non sum invidus—especially as the next monographer will
probably restore the Marquisate.
430. I have duly looked this up in what appears to be the only
accessible place (a place valuable for other documents), the
Orígenes de La Lengua Española, Madrid, 1737, of Mayáns y
Siscar. It is merely a tissue of troubadours’ names, scholastic
citations, and minute details of pronunciation and versification. Señor
Menéndez has reprinted part of it in the Appendix to his second
volume.
431. Poesias Castellanas anteriores al siglo xv. (Paris, 1842) pp.
13-17.
432. Quem nova concepit olla servabit odorem. It may be
observed that, on the principles of Low Latin scansion from
Commodian downwards, the first four words will do well enough.
433. I have used the somewhat later Grenville copy in the British
Museum Salamanca, 1509, fol.; and Señor Menéndez' reprint in the
Appendix to his second volume, which also contains one or two
other early documents.
434. The plural was used in the version of Mayans y Siscar
(Origenes, v. supra), which was long the only one accessible. In
1860 a better text appeared at Madrid with the singular, which
Ticknor and Mr Kelly approve. For any one who professes no
Spanish scholarship to set himself against these authorities may
seem absurd. But in the book itself sub finem the author writes
“habiendo considerado estas tres lenguas,” and the changes are
rung on Latin, Tuscan, and Spanish throughout.
435. After a conversation with Navagero which he has reported,
and which is, in its way, also a critical document.
436. Señor Menéndez refers to two Poetics anterior to Rengifo,
neither of which I have seen. The first, by Miguel Sanchez de Lima
(sometimes called de Viana), Alcala, 1580, has a slight interest in
the wording of its title, “El Arte Poética en romance castillano.” The
second—which from its date (1593) would seem to be a little later
than Rengifo, though the historian mentions it first—is Hierónymo de
Mondragon’s Arte para componer en metro castillano (Saragossa,
1593).
437. The above paragraph was written from notes taken while
reading Rengifo at the British Museum. In subsequently reading
Señor Menéndez on him I was surprised to find the learned historian
protesting against the Labyrinth, and other such things, as having
been foisted in cir. 1700-1720, and referring to the editions of 1592
and 1606 as alone genuine. But the British Museum copy is that of
1606! Let it by us even be said to Rengifo’s credit that, like Sidney,
he felt the charm of old romance. See M. y P., p. 320.
438. “The inexorable Cascales,” as Señor Menéndez calls him in a
passage which I had not read when I wrote the text. Of Cascales, as
of Pinciano (v. infra), the Señor thinks far more highly than I do. Both
seem to me (though Cascales more than Pinciano) to be simply
uncompromising Aristotelians who borrowed from the Italians; but,
like most borrowers and imitators, hardened and emphasised what
they borrowed. Both were forced to allow little “easements” in regard
to the drama; but only such as are consistent with Aristotle’s text,
though not with some glosses on him. And Pinciano simply
translates the Aristotelian definition of tragedy, while Cascales,
doubtless with reference to the different heresies of Castelvetro and
Giraldi, is quite Athanasian in his doctrine that poetic verities are
absolutely unchangeable, and independent of custom and time.
 439. The book appeared in 1616; but I have had to use the reprint
of 1779. I have not seen Mesa’s Compendio de la Poética (Madrid,
1607) or Carillo’s Libro de Erudicion poética.
440. Madrid, 1596.
441. The Filosofía Antigua is extremely rare, and does not appear
to be in the British Museum either under “Pinciano” or under Lopez,
his real name. Fortunately there is a recent reprint (Valladolid, 1894),
ed. by Professor Don Pedro Muñoz Peña, which I duly possess. It
may be observed that bibliographers and librarians are particularly
hard on the laity in the Spanish department. It is surely needless to
make one hunt in vain for an author of world-wide reputation under
his world-name till one runs him to earth as Gómez de Quevedo [y]
Villegas.
442. Señor Peña, himself a professor (catedrático) of Valladolid in
Rhetoric and Poetry, explains that this surname was taken by
distinguished alumni of that University, and derived from the Roman
city (Pincia) supposed to have existed on the site. Few definite dates
or facts seem to be known about Alfonso Lopez, except that he was
physician to Mary of Austria, daughter of Charles V. and widow of
Maximilian II. during her life at Madrid from 1576 to 1603, and that
he wrote, besides the Filosofía and other things, a poem on Pelayo,
languide nec eleganter, one regrets to hear.
443. As where Fadrique substitutes, for the stately old image of
the honey on the edge of a bitter cup, the familiar come quien dora
una píldora, “as one who gilds a pill,” ed. cit., p. 120.
444. Ejemplar Poético, first printed, and, I think, still only to be
found, in the Parnaso Español, Madrid, 1774, vol. viii.
445. See Spingarn, p. 146, who gives the passage.
446. The Later Renaissance, p. 39.
447. Cf. Spingarn, p. 233.
448. (With a much longer title), Medina del Campo, 1602. The
quaint title is connected with a quainter fancy, that the poet is noble
as such—a “Knight of the Swan.” Señor Menéndez makes some use
of Carvallo, but admits that he is pedagogo adocenado, “a common
dominie.”
449. Nueva Idea de la Tragedia Antigua, &c. Madrid, 1633.
450. La misma lobreguez y el mismo desconsuelo, M. y P., iii. 364.
 451. In the passage quoted by M. y P., iii. 366, 367.
452. Madrid, 1624. Noted by Señor Menéndez (who has given the
whole passage, iii. 457-60) as a specially rare book. Fortunately the
British Museum, according to a wise habit of its own in such cases
(cf. Capriano), has two copies, and M. Morel-Fatio has included the
piece which concerns us in an invaluable collection (also including
Lope’s Arte Nuevo and other things) of Spanish critical documents,
which he is issuing in the Bulletin Hispanique of the Faculty of
Letters of Bordeaux, and republishing separately (Paris, Fontemoing;
Bordeaux, Feret, 1901-1902). The man who gives a text attains merit
which mere commentators and historians can never hope to have
imputed to them.
453. Also reprinted by M. Morel-Fatio in the issue noticed above.
454. Los casos de la honra son mejores, Porque mueven con
fuerça a toda gente—A.N., 327, 328. At least one of Lope’s
innumerable works, the Laurel de Apolo, written late in his life
(1630), is busied with the poets of his time in the fashion of Caporali
and Cervantes, but, it would seem, in a spirit of wholly uncritical
panegyric.
455. V. sup., pp. 49, 50 note. It is fair to say that Lope quotes
Robortello.
456. Omnes dilaberentur. Señor Menéndez (iii. 444) gives all the
important parts, both in Latin and Spanish. R. de[l] Turia, infra, has
been reprinted, but the marrow of him also will be found in the
Historia, as well as much else: for instance, an interesting Invectiva y
Apologia, by Francesco de la Barreda in 1622, which is dignified by
the words: “There was no greater dramatic-poetic written in the
seventeenth century”—a large statement. But Barreda is certainly a
staunch anti-Unitarian, and has well reached the important doctrine
that “Art is merely a careful observation of classified [graduados]
examples.” The whole dispute, in which the more or less great
names of the Argensolas, Artieda, Cristóbal de Mesa, and others,
figure, together with the subsequent one on culteranism, will be
found exhaustively treated in the tenth chapter of the Historia, and
more summarily, but still usefully, in Ticknor. Since most of the text
was written M. Morel-Fatio, in his Défenseurs de la Comedia (v. sup.,
p. 343), has subjoined Turia to Tirso, and a certain Carlos Boyl to
both, adding a notice of the Frenchman Ogier (v. sup., pp. 256, 257),
who is already familiar to readers of these pages. Boyl, one of the
Valencian group above referred to, wrote in “romance” form rules of
the comedia nueva.
457. Let it be remembered that the curious passage on which
Pope dwells (Ess. Crit., 267 sq.) is not Cervantic, but from the
spurious and intrusive work of the mysterious Avellaneda.
458. Enthusiastically Englished, with much apparatus, by the late
James Y. Gibson (London, 1883). It is closely modelled on the
Viaggio di Parnasso of Cesare Caporali (1531-1601).
459. Or of their Spanish followers, such as Pinciano and Cascales.
This opinion, formed independently from reading of Don Quixote,
agrees with one of much more importance, that of Señor Menéndez
himself. Nay, Mr Fitzmaurice Kelly (op. cit., p. 237) roundly
pronounces Cervantes “the least critical of men.”
460. In his Works. Bibl. de Ribadeneyra.
461. In his Works. 2 vols., Barcelona, 1748.
462. Later Renaissance, p. 172.
463. Hist. Spanish Lit., p. 340.
464. I am very well aware that culteranismo and conceptismo are
perhaps not identical, and have been asserted to be quite different.
But both alike belong to the “better-bread-than-is-made-of-wheat”
division of writing.
 CHAPTER III.

GERMAN AND DUTCH CRITICISM.

THE HINDMOST OF ALL—ORIGINS—STURM—FABRICIUS—VERSION A.—
VERSION B.—JAC. PONTANUS—HEINSIUS: THE ‘DE TRAGŒDIÆ
CONSTITUTIONE’—VOSS—HIS ‘RHETORIC’—HIS ‘POETICS’—OPITZ—
THE ‘BUCH DER DEUTSCHEN POETEREI.’

It is not necessary to add much to what has been said in the first
chapter of the last Book on the subject of Erasmus, in order to
indicate the reasons why the growth of criticism in Germany, High
and Low,[465] was far more tardy, and for a long time far scantier, than
even in England; and why, when it came, it displayed a one-eyed
character which is not visible in any other of the great European
The hindmost countries.[466] Want of unity, religious and political
of all. troubles, Grobianism and its opposite or companion
Pedantry—all had to do with this; but the principal hindrance was the
non-existence of any considerable German vernacular literature, and
the consequent inveteracy of the habit of writing in Latin. So long as
this lasted the Germans and Dutch might be and were
commentators, scholars, grammarians—but they could hardly be
critics, because they still lacked the comparative stimulus. And it is
not a little noteworthy that the earlier development of Criticism in the
Low Countries as compared with Germany, during our present
period, at least coincided with a greater development of Dutch
vernacular literature, though this is a matter which lies out of our
direct route.
There may easily be differences of opinion as to the persons, not
mere Humanists, who shall be selected as representing the
beginning of German criticism in modern times, in so far at least as
Origins. the section of Poetics is concerned. The choice may
lie between the famous Johann Sturm, who touches
on literary matters in his letters, who wrote on Rhetoric, and whose
pupil, late in his life, drew up a commentary on the Epistle to the
Pisos in 1576; Georgius Fabricius, of Chemnitz, the first form of
whose De Re Poetica appeared in 1565; and Jacobus Pontanus,
whose real name was Spanmüller, whose book on the subject was
published thirty years later, but who, as he was then a man of over
fifty, and had long been a professor, had probably dealt with the
subject, if only in lectures, much earlier.[467]
Sturm’s interests were more in pædagogy than in poetry, and he
does not rank high as a critic: though there is no doubt that he
Sturm. helped to spread devotion to books. It is not in his
favour that, in the teeth of both external and internal
evidence, he fights[468] for the name De Arte Poetica, on the special
ground that the work of Horace is an Ars Perfecta (which, put its
merits as high as you please, it most certainly is not), and that it has
all the six parts of poetry—fable, character, dianoia, lexis, melopœia,
and “sight.” For the rest he has few general remarks, and is almost
wholly commentatorial. His Rhetorical writing yields little really
critical: nor in his Letters have I yet found half so much criticism as is
extant in that single letter of Ascham to him, which has been noticed
above.[469]
The other two were both men of very wide influence as teachers of
Poetics: and both underwent the process—complimentary but
disfiguring, and specially usual in the late sixteenth and seventeenth
centuries—of having their work watered out, or boiled down, by
Fabricius. others. I do not know, and I have not considered it
tanti to spend much time or labour in the attempt to
discover, the exact process by which the small four books of the first
edition of Fabricius’ De Re Poetica[470] became the fat volume of
seven, which presents itself under the same title thirty years later.[471]
Version A. It seemed better to give this time and labour to the
reading of the books themselves. Version A (as we
may call that of 1565) is an early example of the kind of gradus
which was particularly popular among the northern nations, though,
as we have seen from the work of Mazzone da Miglionico,[472] it was
by no means unknown in Italy. In his first book Fabricius discusses
quantity, metre, and diction in general, with plentiful examples. Book
II. is an elaborate table of locutions from the Latin poets, listed under
heads as thus:—
Amor tangit. Matrimonium
 promittere.
” versat. ” ” inire.
” torquet. ” ” fallere.
” dat vulnus. ” ” odisse.
” mordet.
” torret.
Book III. provides the dull-witted versifier with store of clichés of the
same kind, but a little more elaborate; there being, for instance,
dozens of phrases for embracing. And IV. is a sort of common place-
book of short copies of verses on everything in Heaven and Earth.
Version B (which is dated long after Fabricius’ death in 1571) is
Version B. not only much enlarged but differently arranged.
Book I. deals as before with Quantity and Feet;
Book II. with the subject of A, Book III.; and the rest follow the same
schemes,—III. B with tags on Ages, Seasons, Heavenly Bodies, &c.;
IV. with epithets suitable to proper names; V. with ditto to common;
VI. with a pot-pourri of poetical faults and beauties, &c.; while VII.
gives a sort of appendix on prosody and diction generally.
There is no need to say much on the inevitable critical result, the
obvious critical value or valuelessness, of this. There is in A a
reference to Scaliger’s Poetic, which had appeared a little before. As
a matter of fact Scaliger and Fabricius between them provided the
average late sixteenth-century man—sometimes even when he was
a professed critic or poet, constantly when he was merely a person
of ordinary culture—with a sort of joint poetical Thesaurus,—Scaliger
doing the historical, critical, and (of its kind) philosophical business
for him, and Fabricius keeping a general marine-store of materials,
with precepts for their use.
The Institutiones Poeticæ of Spanmüller [Pontanus] appeared first
Jac. in 1594. Its author is quoted, among other prophets
Pontanus. of criticism, by the Spaniard Juan de La Cueva a
dozen years later, and, independently of its original form, the book
acquired, early in the seventeenth century, a large currency by being
arranged (concinnata) in the Sacrarum Profanarumque Phraseum
Thesaurus of J. Buehler,[473] where it serves as theoretic handbook to
another Gradus. Indeed Pontanus’ own work has all the
characteristics of a decoction or abstract of Scaliger himself. And
once more the same reflection applies. It is impossible not to see
how powerful and (beyond mere school-work, in which they were no
doubt invaluable[474]) how maleficent must have been the influence of
such works on the critical temper of the generations influenced by
them. La Bruyère’s Tout est dit—an ironical fling in its author’s mouth
partly, no doubt, though perhaps not quite so even there—tended to
become matter of breviary. Everything had been said and done; all
the Kinds found out; all the phrases set down; all the poetry raised
from shaft and vein and seam. You simply rearranged it like a child’s
house of wooden bricks, according to patterns provided on the lid.
The “Causes of Corrupted Arts” into which Vives inquired, “The Lost
System of Speaking” which Sturm deplored, were all to be found,
and found sufficiently, in the Ancients.
The solid qualities of the German race have not commonly
distinguished themselves in pure criticism, and to this day Lessing
and Goethe are rather captains without companies, and with at best
a staff of Schlegels, and suchlike, for lieutenants and ancients.
Germans were, however, to do something better, in this century of
erudition, than the mere preparation of fourth-form handbooks.
Daniel Heinsius and Gerard Voss may be regarded with some
reason as the Jachin and the Boaz of the temple of seventeenth-
century Poetics. The De Tragœdiæ Constitutione of the first, which
appeared at Leyden in 1611,[475] is the succinctest and best argued
statement of the neo- and to a great extent pseudo-Aristotelian view
of Drama. The new Institutes of Oratory,[476] and the much later
Poetical Institutes[477] of the second, construct, with a great deal of
learning and a very considerable amount of good sense, an entire
neo-classical Rhetoric and Poetic. To both we must give some
attention.
The De Tragœdiæ Constitutione is beyond all doubt a very
Heinsius. The remarkable book. It is quite short; only some 250
De Tragœdiæ very small pages of very large print, so that there
Constitutione. are scarcely more than a hundred words in a page.
But Heinsius writes as one having authority; and we can read but
little of him before it becomes perfectly clear why that authority was
accepted, for the rest of the century at least, with more docility and
less cavil than that of almost any other critic. He takes the Poetics—
as many, indeed most men for more than half a century, had taken
them—for gospel. But he neither translates them on the one hand,
nor wanders in the wilderness of scrappy and desultory commentary
on the other. Not merely does he confine himself to that part of the
book which concerns his actual subject, but he renders this part in a
fashion which may best be described as a very rare, and very
masterly, kind of lecturing. He neither slavishly keeps nor prudishly
avoids the actual words of his author; his paraphrases are brief but
lucid; he adds to Aristotle what he thinks necessary[478] in the way of
illustrations from the Greek tragedians, citation from Horace,
examples (by no means always laudatory) from Seneca, and the
like; but in such a fashion as never to overload, or water down, the
milk of the Aristotelian word. That he always gives that milk quite
“sincere” we cannot say; he emphasises the “single revolution of the
sun” more than he has any right to do, though he does not do the
same for the still more pestilent and apocryphal Unity of Place. He
may sometimes, or often in the disputable places (as of “purgation”
and so forth), miss the full meaning of Aristotle according to the view
of some judges, or impute a wrong one according to others. But
nobody, let it be repeated, can read him impartially without seeing
that he has soaked himself with the spirit of his author, has equipped
himself pretty thoroughly with the literature of his subject, and, as a
result, is speaking, as we said, with authority. There is no clearer or
more workmanlike exposition of the neo-classic, and not too neo-
classic, dramatic ideal than his.
Heinsius, like his successor Hédelin in France, and like Hédelin’s
successors Rymer and Dennis in England, was rash enough to
forget that though a critic is (thank Heaven!) not bound to write good
Voss. poetry, he is bound not to publish bad. And he
ventured on a tragedy, Herodes Infanticida, and
other things which did not meet much quarter even from those who
agreed with him in critical principles. Voss was wiser, and confined
himself to the pure erudition and comment of which the two books
referred to above are far from being the worst examples. Indeed his
unboastful scholarship, his immense reading, and his untiring
industry would seem to have fitted him quite exceptionally for the
duty; and he has actually given us in these two books, or rather
collections of books, the completest Rhetoric and Poetic of modern,
if not of any, times. Only two things more were needed to put these
books in a place even more unique; but Nature refused the one to
Voss personally, and the other was a thing almost unreasonable to
require from a Dutch savant of the seventeenth century. The first
was positive critical genius; and the second was an impartial
appreciation of ancient and modern literature.
His Rhetoric. The Rhetoric, which the author put out in its first
form in 1606, revising and enlarging it for at least
thirty years, till it forms a quarto of a thousand closely printed pages,
has some seventy more of minute index, but lacks the Table of
Contents, or displayed syllabus of section headings, for which we
have so often had to be thankful in Italian work. Voss evidently had
the practice of the Roman Law constantly before him, and he thus
follows the method of the Latin treatises in a way which makes it for
the most part superfluous for us to follow him, though he has plenty
of modern instances and applications. From the Fourth Book
onwards, however, he deals with Elocution and Style, chiefly of
course by the way of Figures, yet, according to his lights, in the most
careful and exhaustive fashion. But what is at once noteworthy and
rather tell-tale is his unqualified admiration for the Scaligers,—father
and son. “That divine man,” “that man, ad unguem factus,” that
“emperor of the literary world,” that “prince of the senate of criticism”;
without some phrase of this kind he seems unable to name them.
And in fact the whole book is rather a huge commentatorial digest of
what they and others, from Aristotle downwards, have said than
anything more.
The Poetical Institutions are somewhat more original, and they
His Poetics. had much greater influence. The book consists
really of three separate works, a brief De Arte
Poetica of less than ninety pages, of which Grotius, in a
commendatory epigram prefixed to some editions, says—

“non magnus dat tibi cuncta liber;”

of the Institutiones proper in about four hundred; and of a De

Imitatione which is rather shorter than the Ars. The first, as reason
and its title both import, is a purely general tractate, which, after
pointing out that Poetry has much in common with Oratory, and that
therefore much which concerns it has been said in the earlier book,
discusses all the old generalities about the origin, nature, moral
character, and so forth, of poetry, with expositions of most of the
cruces and technical catchwords from Ψιλὸς λόγος down to furor
poeticus. Voss is here also very generally Scaligerian; he adheres to
the “natural” origin of poetry, love-songs, cradle-songs, &c., as
against the religious and the deliberately “imitative”; gives very wide
scope of subject to the poet, and defends him handsomely against
his enemies and detractors from Plato downwards, but is properly
indignant with naughty poets.[479]
The Institutions deal more directly with the question of Poetic Art,
and proceed by a series of section-headings in the form of
Propositions, which are then explained, commented, and defended.
The first of the Three Books deals with the matter common to all
kinds of poetry; the Second with the Drama; the Third with the Epic
and the minor Kinds. All this is old stuff rehandled. There is
somewhat more originality in the De Imitatione, which does not
exactly correspond to any of the older books, or parts of books, on
that subject. Voss generally supposes the question, “How is the poet
to set about his work?” “How is he to apply all these rules that we
have given him?” and before very long we see that he is really
thinking of the wrong Imitation no less than Vida was. He devotes
himself (no doubt under the happier inspiration of Quintilian) to
discussing how we are to imitate, how to read. But he very soon slips
into the inquiry, practical indeed but a little undignified, “How are we
to escape plagiarism?” to which one is tempted to reply, “By not
imitating in this sense at all.” That is not his opinion. He thinks, if we
may vary a well-known proverb, that the safe way is to take all your
eggs out of one basket. But you are never to imitate bad words and
thoughts; you must plan your work carefully beforehand, correct
carefully, invite criticism, but distinguish between what is good and
what is not. It is all very just in this way; but that way has led us far
from furor poeticus. We feel at the end of Voss’s laborious and
erudite book that we are indeed in the century of the Gradus. And
here, as in his other volume, we also feel that he has, for good or for
evil, caught up and uttered the gospel of Neo-Classicism.
So far we have dealt only with Latin authors. The work of Heinsius
is mentioned, both in the text by the author, and by the introducer,
Augustinus Iskra, of the Buch von der Deutschen Poeterei[480] of
Opitz. Martin Opitz. This interesting and agreeable little book,
though not exactly (as it has sometimes been incorrectly
called) the first[481] piece of German poetic in the vernacular, is
entitled, with the usual reserves, to the place of origin in modern
German Poetics. It cannot be called prolix, for it only occupies sixty
pages in the recent reprint; but it is equally modest and business-
like, and helps to redeem from the utter absurdity of most of such
appellations (though it still remains absurd) the title of German
Dryden which somebody or other has given to Opitz. Augustine Iskra
does not exaggerate when he says—

“Altius scandes patria canendo

Barbyto, quam si Latium peritæ
Atticæ jungas, Syriæque Peithus
Noveris artem.”

And it is the peculiar glory of the Silesian poet that he not only sang
himself on the lyre of his country, but did his best to enable others to
do so. The spirit of genuine patriotism breathes in his dedication of
the booklet to the magistrates of his native town, Buntzlau; and that
of a modest scholarship (an adjective and substantive which make
such an agreeable couple that it is pity they should live so much
apart) in the opening of the book itself. He has not the least idea, he
says, that you can make a poet by rules and laws; nor has he any
intention of doing over again the work which Aristotle, Horace, Vida,
The Buch der and Scaliger have done. But he arrays himself (to
Deustchen speak ecclesiastically) in a “decent tippet” of the old
Poeterei. stuff about Linus and Orpheus, with the Strabo
passage all complete, and a train of citations as to the nobility of the
poet’s office and the like. He comes in his fourth chapter to business.
He actually quotes Walther von der Vogelweide; and I do not think
that he can be fairly charged with that ὕβρις towards the ancient
poetry of his country which too frequently marks others in other
countries. But he is evidently set on the work of Reform—of
substituting “smoothness of numbers” for the “wild sweetness” of the
folk-song. Wherein no doubt he was wrong. Not that way did the
counsels of perfection lie for the Higher Dutch; and they have always
had to come back to the woodnotes and the wood-Muses to find
poetic luck. But Opitz was entitled—was in his day almost bound—to
think differently. The interesting thing—much more interesting to us
than the details to which it led him, such as the patronage of the
Alexandrine, the alternation of masculine and feminine rhymes, &c.
—is the particular source to which he turned for inspiration and
guidance. He knew, as has been said, the Italian critics, at least
those in Latin, and he probably knew the Italian poets (he cites
Petrarch). But it is to France, and specially to Ronsard, that he fondly
turns. Now it need hardly be said that in 1624 the influence of the
Pléiade in its own country, though not quite dead, was moribund; the
correctness of Malherbe, on the one hand, was doing its best
deliberately to throttle it, and the Italianated and Spaniolated
extravagances which were fashionable were choking it in another
way. This is no doubt not the only instance of a literary influence
which is dead or dying in its own country showing full vitality in
another, but it is one of the most remarkable. For, beyond question,
the French influence—in successive forms, but still French—reigned
in Germany for some hundred and fifty years; and it was Opitz who
first brought it to bear.
His details, as has been said, are less interesting: yet they do not
lack interest. He begins by stickling for pure High German: and
certainly no one who, for his sins, has been condemned to read
much of late fifteenth- and early sixteenth-century German—one of
the ugliest and most mongrel speeches in history, and quite
astounding after the musical sweetness of the best
Mittelhochdeutsch—will owe him a grudge for this. He protests
against the mingle-mangle of foreign words which was flooding the
language, and even against the famous -iren by which, to the
present day to some extent, Germans give a sort of spurious
naturalisation to such foreigners. He would have limits set (though
he does not forbid it altogether) to that odd custom of declining
classical names in German speech, which is also maintained to
some extent, but which sometimes made a mere Macaronic of
sixteenth-century German. On the other hand, it is curious to find
him urging on Germans, who by right were, and by practice long
have been, among the busiest and most successful of word-
compounders, the sonderliche anmuthigkeit of compounds: and
actually quoting the French as, next the Greeks, the masters of such
things.[482] Of course the historical student, even if citations from
Ronsard were not on the same page, would know at once whence
this comes. Still, there still remains the oddity of alleging the
undoubtedly awkward and exotic-sounding chasse-nue, ébranle-
rocher, and irrite-mer as warrants and patterns for words like
wolkentreiber, felsenstürmer, and meeraufreitzer, which simply seem
to us natural-born, and to require no warranty but their own sound
and appearance.
But Opitz (of whom if any critic speaks disrespectfully, I fear that it
argues him uncritical) wrote not merely on the eve, but in the actual
stormy morning, of the Thirty Years’ War: and Germany had
something else to do for a long time besides listening to him. When
matters settled down again, the advice to attend to the French was
rather unfortunately “carried over” to a state of things in which
French influence was still less the influence for Germany. But this
imitation, whether right or wrong, found no important critical
expression, and it would be losing labour and space to devote either
to German criticism in the last half of the seventeenth century.
It is more remarkable that the real activity and accomplishment of
Dutch during the early part of the century did not lead to some
development of vernacular criticism. But to the best of my
information[483] it did not. The Dutch and the Germans, however, of
course still continued to write in Latin, to edit, to comment, to carry
on that division of critical work which, according to the laying out of
our subject, lies, except at particular seasons and for special ends,
beyond the scope of this book. Moreover, both Holland and some of
the German Free-towns, but especially the former country, became
the adopted, as they were almost the natural, homes of those
beginners of judicial criticism, who have been noticed in part at the
conclusion of the French chapter of this Book. Bayle’s Nouvelles de
la République de Lettres were Hollandish by domicile, as was the
Bibliothèque Universelle of Le Clerc, while at Leipsic the Acta
Eruditorum maintained the same principle of critical annals for nearly
a century. Bayle, as has been said before, was too much of a
partisan, and perhaps of a wit, for anything of his to have a judicial,
however much in some senses of the word it might have a critical,
character: but the less mercurial talents of Jean Le Clerc, which
have been characterised under the head of the Ana (v. sup., p. 276),
were very well suited to the conduct of a critical record.

465. I do not know any general-special books on the subject of this

chapter, except those of Blankenburg, and Gayley and Scott, cit.
sup.
466. Of course Olmucensis (v. supra, p. 27) and Cornelius Agrippa
(p. 28) in strictness belong to the subject, as does Erasmus himself.
But the last is too cosmopolitan, and the two first too unimportant, to
make the abstraction of them from this place a great wrong to the
Teutsche Nation. Ulrich von Hutten wrote on versification, but not
importantly.
467. The Disputationes de Tragœdia of Schosser (1559) are
earlier than any of these; but they seem to be pure commentary on
Aristotle. I have not been able to see them.
468. Commentarii in Art. Poet. Horat. (Strasburg, 1576). The
compiler was Johann Lobart. Sturm’s Rhetorical works are rather
numerous, and range from the De amissa dicendi ratione (ibid.,
1538) onwards.
469. P. 155.
470. De Re Poetica, Lib. iv. (Antwerp, 1565).
471. Ibid., Lib. vii. (Leipsic, 1595).
472. V. sup., p. 107.
473. S. l. 1633, and continually reprinted.
474. Let me not be supposed for one moment to depreciate Latin
verse-making. I hardly know (speaking from actual experience as a
school-master) a single study which is better for boys; and the
intelligent use of the gradus is a better discipline in observation,
critical selection, and method, than smatterings of a hundred so-
called “sciences.” But there is a time to put away childish things as
well as a time to use them.
475. The copy of this which belongs to the University of Edinburgh
has the additional interest of having belonged to, and of having been
given by, Drummond of Hawthornden, and so of having been, not
improbably, in the hands of Ben Jonson.
476. Commentariorum Rhetoricorum sive Oratoriarum
Institutionum Libri Sex, 8vo, Dordrecht, 1609. But this was greatly
enlarged in the 4to of Leyden, 1643, which I use.
477. De Artis Poeticæ Natura ac Constitutione, 4to, Amsterdam,
1647.
478. He has no room for much historical illustration, but what he
says is generally sound, though it is odd that in mentioning the
Christus Patiens (which, of course, he attributes to Gregory
Nazianzen) he should not have noticed its cento character, and
though his remarks on Muretus and Buchanan smack a little of the
rival author of Herodes Infanticida.
479. Our whole history has shown us the obsession of the pius
poeta, the vir bonus; but I think the uncompromising submission to it
of the later seventeenth and eighteenth century is as much due to
the influence of Voss as to that of any single mediate person.
480. Printed at Brieg and published at Breslau in 1624; reprinted
as the first number of Niemeyer’s Neudrucke des xviten und xviiten
Jahrhunderts, at Halle in 1886. The title of Prosodia Germanica,
which the later editions bore, does not seem to be the author’s own.
481. For instance, the very interesting Grundlicher Bericht des
Deutschen Meistergesangs of Adam Puschmann, edited by Herr
Jonas for the same collection as No. 73 (Halle, 1888), is more than
half a century older than Opitz’s book, having appeared at Görlitz in
1571. But Puschmann, a pupil of Hans Sachs himself, and active in
the Masterschool, is only looking back on that school, the rules and
regulations of which he lays down in the most approved fashion.
“The face” of Opitz “meets the morning’s breath.”
482. Buch der Poet., ed. cit., p. 29.
483. I must here repeat, with additional emphasis, the caution and
apology which I put in as to Spanish. I do not know anything of this
language. I have been content to apply to Low Dutch the precept of
a great High Dutchman, Entbehren sollst du. But for our purpose I
believe it will be generally admitted that the renunciation is not fatal,
important Dutch critics having, almost to a man, written in Latin.
 CHAPTER IV.

DRYDEN AND HIS CONTEMPORARIES.

DEAD WATER IN ENGLISH CRITICISM—MILTON—COWLEY—THE
PREFATORY MATTER OF ‘GONDIBERT’—THE “HEROIC POEM”—
DAVENANT’S ‘EXAMEN’—HOBBES’S ANSWER—DRYDEN—HIS
ADVANTAGES—THE EARLY PREFACES—THE ‘ESSAY OF DRAMATIC
POESY’—ITS SETTING AND OVERTURE—CRITES FOR THE ANCIENTS
—EUGENIUS FOR THE “LAST AGE”—LISIDEIUS FOR THE FRENCH—
DRYDEN FOR ENGLAND AND LIBERTY—‘CODA’ ON RHYMED PLAYS,
AND CONCLUSION—CONSPICUOUS MERITS OF THE PIECE—THE
MIDDLE PREFACES—THE ‘ESSAY ON SATIRE’ AND THE ‘DEDICATION
OF THE ÆNEIS’—THE PARALLEL OF POETRY AND PAINTING—THE
‘PREFACE TO THE FABLES’—DRYDEN’S GENERAL CRITICAL POSITION
—HIS SPECIAL CRITICAL METHOD—DRYDEN AND BOILEAU—RYMER—
THE ‘PREFACE TO RAPIN’—THE ‘TRAGEDIES OF THE LAST AGE’—THE
‘SHORT VIEW OF TRAGEDY’—THE RULE OF TOM THE SECOND—
SPRAT—EDWARD PHILLIPS—HIS ‘THEATRUM POETARUM’—
WINSTANLEY’S ‘LIVES’—LANGBAINE’S ‘DRAMATIC POETS’—TEMPLE—
BENTLEY—COLLIER’S ‘SHORT VIEW’—SIR T. P. BLOUNT—
PERIODICALS: THE ‘ATHENIAN MERCURY,’ ETC.

The middle third, if not the whole first half, of the seventeenth
century in England was too much occupied with civil and religious
broils to devote attention to such a subject literary criticism. Between
Dead water in the probable date of Jonson’s Timber (1625-37) and
English the certain one of Dryden’s Essay of Dramatic
Criticism. Poesy (1668) we have practically nothing
substantive save the interesting prefatory matter to Gondibert
Milton (1650). Milton, the greatest man of letters wholly of
the time, must indeed during this time have
conceived, or at least matured, that cross-grained prejudice against
rhyme, which is more surprising in him than even in Campion, and
which was itself even more open to Daniel’s strictures. For not only
is Milton himself in his own practice a greater and more triumphant
vindicator of rhyme than Campion, but Daniel’s strongest and
soundest argument, “Why condemn this thing in order to establish
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com