Read Anytime Anywhere Easy Ebook Downloads at ebookmeta.

com

Python for Data Analysis Data Wrangling with

Pandas NumPy and IPython 1st Edition Wes Mckinney

https://ebookmeta.com/product/python-for-data-analysis-data-
wrangling-with-pandas-numpy-and-ipython-1st-edition-wes-
mckinney/

OR CLICK HERE

DOWLOAD EBOOK

Visit and Get More Ebook Downloads Instantly at https://ebookmeta.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Python Data Analysis Numpy, Matplotlib and Pandas Bernd

Klein

https://ebookmeta.com/product/python-data-analysis-numpy-matplotlib-
and-pandas-bernd-klein/

ebookmeta.com

Python for Data Analysis, 3rd Edition (Second Early

Release) Wes Mckinney

https://ebookmeta.com/product/python-for-data-analysis-3rd-edition-
second-early-release-wes-mckinney/

ebookmeta.com

Data Analysis with Python Introducing NumPy Pandas

Matplotlib and Essential Elements of Python Programming
1st Edition Rituraj Dixit
https://ebookmeta.com/product/data-analysis-with-python-introducing-
numpy-pandas-matplotlib-and-essential-elements-of-python-
programming-1st-edition-rituraj-dixit/
ebookmeta.com

Illiberal Europe Eastern Europe from the Fall of the

Berlin Wall to the War in Ukraine 2nd Edition Léon Marc

https://ebookmeta.com/product/illiberal-europe-eastern-europe-from-
the-fall-of-the-berlin-wall-to-the-war-in-ukraine-2nd-edition-leon-
marc/
ebookmeta.com
Mapping the Field of Adult and Continuing Education An
International Compendium 1st Edition Alan B. Knox

https://ebookmeta.com/product/mapping-the-field-of-adult-and-
continuing-education-an-international-compendium-1st-edition-alan-b-
knox/
ebookmeta.com

What Really Happens in Vegas True Stories of the People

Who Make Vegas Vegas 1st Edition Patterson

https://ebookmeta.com/product/what-really-happens-in-vegas-true-
stories-of-the-people-who-make-vegas-vegas-1st-edition-patterson/

ebookmeta.com

The Healthy, Happy Gut Cookbook: Simple, Non-Restrictive

Recipes to Treat IBS, Bloating, Constipation and Other
Digestive Issues the Natural Way Dr. Heather Finley
https://ebookmeta.com/product/the-healthy-happy-gut-cookbook-simple-
non-restrictive-recipes-to-treat-ibs-bloating-constipation-and-other-
digestive-issues-the-natural-way-dr-heather-finley/
ebookmeta.com

Marketing Research Delivering Customer Insight 4th Edition

Alan Wilson

https://ebookmeta.com/product/marketing-research-delivering-customer-
insight-4th-edition-alan-wilson/

ebookmeta.com

Mastering Financial Pattern Recognition Finding and Back

Testing Candlestick Patterns with Python 1st Edition
Sofien Kaabar
https://ebookmeta.com/product/mastering-financial-pattern-recognition-
finding-and-back-testing-candlestick-patterns-with-python-1st-edition-
sofien-kaabar-2/
ebookmeta.com
Competition Cauldrons Conspiracy Moonflower Mystery 5 1st
Edition Beverly Rearick

https://ebookmeta.com/product/competition-cauldrons-conspiracy-
moonflower-mystery-5-1st-edition-beverly-rearick/

ebookmeta.com
 Python for Data Analysis
Download from Wow! eBook <www.wowebook.com>

Wes McKinney

Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo

Python for Data Analysis
by Wes McKinney

Copyright © 2013 Wes McKinney. All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions
are also available for most titles (http://my.safaribooksonline.com). For more information, contact our
corporate/institutional sales department: 800-998-9938 or [email protected].

Editors: Julie Steele and Meghan Blanchette Indexer: BIM Publishing Services
Production Editor: Melanie Yarbrough Cover Designer: Karen Montgomery
Copyeditor: Teresa Exley Interior Designer: David Futato
Proofreader: BIM Publishing Services Illustrator: Rebecca Demarest

October 2012: First Edition.

Revision History for the First Edition:

2012-10-05 First release
See http://oreilly.com/catalog/errata.csp?isbn=9781449319793 for release details.

Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of
O’Reilly Media, Inc. Python for Data Analysis, the cover image of a golden-tailed tree shrew, and related
trade dress are trademarks of O’Reilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as
trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a
trademark claim, the designations have been printed in caps or initial caps.

While every precaution has been taken in the preparation of this book, the publisher and author assume
no responsibility for errors or omissions, or for damages resulting from the use of the information con-
tained herein.

ISBN: 978-1-449-31979-3

[LSI]

1349356084
 Table of Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

1. Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
What Is This Book About? 1
Why Python for Data Analysis? 2
Python as Glue 2
Solving the “Two-Language” Problem 2
Why Not Python? 3
Essential Python Libraries 3
NumPy 4
pandas 4
matplotlib 5
IPython 5
SciPy 6
Installation and Setup 6
Windows 7
Apple OS X 9
GNU/Linux 10
Python 2 and Python 3 11
Integrated Development Environments (IDEs) 11
Community and Conferences 12
Navigating This Book 12
Code Examples 13
Data for Examples 13
Import Conventions 13
Jargon 13
Acknowledgements 14

2. Introductory Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.usa.gov data from bit.ly 17
Counting Time Zones in Pure Python 19

iii
 Counting Time Zones with pandas 21
MovieLens 1M Data Set 26
Measuring rating disagreement 30
US Baby Names 1880-2010 32
Analyzing Naming Trends 36
Conclusions and The Path Ahead 43

3. IPython: An Interactive Computing and Development Environment . . . . . . . . . . . . 45

IPython Basics 46
Tab Completion 47
Introspection 48
The %run Command 49
Executing Code from the Clipboard 50
Keyboard Shortcuts 52
Exceptions and Tracebacks 53
Magic Commands 54
Qt-based Rich GUI Console 55
Matplotlib Integration and Pylab Mode 56
Using the Command History 58
Searching and Reusing the Command History 58
Input and Output Variables 58
Logging the Input and Output 59
Interacting with the Operating System 60
Shell Commands and Aliases 60
Directory Bookmark System 62
Software Development Tools 62
Interactive Debugger 62
Timing Code: %time and %timeit 67
Basic Profiling: %prun and %run -p 68
Profiling a Function Line-by-Line 70
IPython HTML Notebook 72
Tips for Productive Code Development Using IPython 72
Reloading Module Dependencies 74
Code Design Tips 74
Advanced IPython Features 76
Making Your Own Classes IPython-friendly 76
Profiles and Configuration 77
Credits 78

4. NumPy Basics: Arrays and Vectorized Computation . . . . . . . . . . . . . . . . . . . . . . . . . . 79

The NumPy ndarray: A Multidimensional Array Object 80
Creating ndarrays 81
Data Types for ndarrays 83

iv | Table of Contents
 Operations between Arrays and Scalars 85
Basic Indexing and Slicing 86
Boolean Indexing 89
Fancy Indexing 92
Transposing Arrays and Swapping Axes 93
Universal Functions: Fast Element-wise Array Functions 95
Data Processing Using Arrays 97
Expressing Conditional Logic as Array Operations 98
Mathematical and Statistical Methods 100
Methods for Boolean Arrays 101
Sorting 101
Unique and Other Set Logic 102
File Input and Output with Arrays 103
Storing Arrays on Disk in Binary Format 103
Saving and Loading Text Files 104
Linear Algebra 105
Random Number Generation 106
Example: Random Walks 108
Simulating Many Random Walks at Once 109

5. Getting Started with pandas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Introduction to pandas Data Structures 112
Series 112
DataFrame 115
Index Objects 120
Essential Functionality 122
Reindexing 122
Dropping entries from an axis 125
Indexing, selection, and filtering 125
Arithmetic and data alignment 128
Function application and mapping 132
Sorting and ranking 133
Axis indexes with duplicate values 136
Summarizing and Computing Descriptive Statistics 137
Correlation and Covariance 139
Unique Values, Value Counts, and Membership 141
Handling Missing Data 142
Filtering Out Missing Data 143
Filling in Missing Data 145
Hierarchical Indexing 147
Reordering and Sorting Levels 149
Summary Statistics by Level 150
Using a DataFrame’s Columns 150

Table of Contents | v
 Other pandas Topics 151
Integer Indexing 151
Panel Data 152

6. Data Loading, Storage, and File Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Reading and Writing Data in Text Format 155
Reading Text Files in Pieces 160
Writing Data Out to Text Format 162
Manually Working with Delimited Formats 163
JSON Data 165
XML and HTML: Web Scraping 166
Binary Data Formats 171
Using HDF5 Format 171
Reading Microsoft Excel Files 172
Interacting with HTML and Web APIs 173
Interacting with Databases 174
Storing and Loading Data in MongoDB 176

7. Data Wrangling: Clean, Transform, Merge, Reshape . . . . . . . . . . . . . . . . . . . . . . . . 177

Combining and Merging Data Sets 177
Database-style DataFrame Merges 178
Merging on Index 182
Concatenating Along an Axis 185
Combining Data with Overlap 188
Reshaping and Pivoting 189
Reshaping with Hierarchical Indexing 190
Pivoting “long” to “wide” Format 192
Data Transformation 194
Removing Duplicates 194
Transforming Data Using a Function or Mapping 195
Replacing Values 196
Renaming Axis Indexes 197
Discretization and Binning 199
Detecting and Filtering Outliers 201
Permutation and Random Sampling 202
Computing Indicator/Dummy Variables 203
String Manipulation 205
String Object Methods 206
Regular expressions 207
Vectorized string functions in pandas 210
Example: USDA Food Database 212

vi | Table of Contents
8. Plotting and Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
A Brief matplotlib API Primer 219
Figures and Subplots 220
Colors, Markers, and Line Styles 224
Ticks, Labels, and Legends 225
Annotations and Drawing on a Subplot 228
Saving Plots to File 231
matplotlib Configuration 231
Plotting Functions in pandas 232
Line Plots 232
Bar Plots 235
Histograms and Density Plots 238
Scatter Plots 239
Plotting Maps: Visualizing Haiti Earthquake Crisis Data 241
Python Visualization Tool Ecosystem 247
Chaco 248
mayavi 248
Other Packages 248
The Future of Visualization Tools? 249

9. Data Aggregation and Group Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

GroupBy Mechanics 252
Iterating Over Groups 255
Selecting a Column or Subset of Columns 256
Grouping with Dicts and Series 257
Grouping with Functions 258
Grouping by Index Levels 259
Data Aggregation 259
Column-wise and Multiple Function Application 262
Returning Aggregated Data in “unindexed” Form 264
Group-wise Operations and Transformations 264
Apply: General split-apply-combine 266
Quantile and Bucket Analysis 268
Example: Filling Missing Values with Group-specific Values 270
Example: Random Sampling and Permutation 271
Example: Group Weighted Average and Correlation 273
Example: Group-wise Linear Regression 274
Pivot Tables and Cross-Tabulation 275
Cross-Tabulations: Crosstab 277
Example: 2012 Federal Election Commission Database 278
Donation Statistics by Occupation and Employer 280
Bucketing Donation Amounts 283
Donation Statistics by State 285

Table of Contents | vii

 10. Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Date and Time Data Types and Tools 290
Converting between string and datetime 291
Time Series Basics 293
Indexing, Selection, Subsetting 294
Time Series with Duplicate Indices 296
Date Ranges, Frequencies, and Shifting 297
Generating Date Ranges 298
Frequencies and Date Offsets 299
Shifting (Leading and Lagging) Data 301
Time Zone Handling 303
Localization and Conversion 304
Operations with Time Zone−aware Timestamp Objects 305
Operations between Different Time Zones 306
Periods and Period Arithmetic 307
Period Frequency Conversion 308
Quarterly Period Frequencies 309
Converting Timestamps to Periods (and Back) 311
Creating a PeriodIndex from Arrays 312
Resampling and Frequency Conversion 312
Downsampling 314
Upsampling and Interpolation 316
Resampling with Periods 318
Time Series Plotting 319
Moving Window Functions 320
Exponentially-weighted functions 324
Binary Moving Window Functions 324
User-Defined Moving Window Functions 326
Performance and Memory Usage Notes 327

11. Financial and Economic Data Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

Data Munging Topics 329
Time Series and Cross-Section Alignment 330
Operations with Time Series of Different Frequencies 332
Time of Day and “as of” Data Selection 334
Splicing Together Data Sources 336
Return Indexes and Cumulative Returns 338
Group Transforms and Analysis 340
Group Factor Exposures 342
Decile and Quartile Analysis 343
More Example Applications 345
Signal Frontier Analysis 345
Future Contract Rolling 347

viii | Table of Contents

 Rolling Correlation and Linear Regression 350

12. Advanced NumPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

ndarray Object Internals 353
NumPy dtype Hierarchy 354
Advanced Array Manipulation 355
Reshaping Arrays 355
C versus Fortran Order 356
Concatenating and Splitting Arrays 357
Repeating Elements: Tile and Repeat 360
Fancy Indexing Equivalents: Take and Put 361
Broadcasting 362
Broadcasting Over Other Axes 364
Setting Array Values by Broadcasting 367
Advanced ufunc Usage 367
ufunc Instance Methods 368
Custom ufuncs 370
Structured and Record Arrays 370
Nested dtypes and Multidimensional Fields 371
Why Use Structured Arrays? 372
Structured Array Manipulations: numpy.lib.recfunctions 372
More About Sorting 373
Indirect Sorts: argsort and lexsort 374
Alternate Sort Algorithms 375
numpy.searchsorted: Finding elements in a Sorted Array 376
NumPy Matrix Class 377
Advanced Array Input and Output 379
Memory-mapped Files 379
HDF5 and Other Array Storage Options 380
Performance Tips 380
The Importance of Contiguous Memory 381
Other Speed Options: Cython, f2py, C 382

Appendix: Python Language Essentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433

Table of Contents | ix
 Preface

The scientific Python ecosystem of open source libraries has grown substantially over
the last 10 years. By late 2011, I had long felt that the lack of centralized learning
resources for data analysis and statistical applications was a stumbling block for new
Python programmers engaged in such work. Key projects for data analysis (especially
NumPy, IPython, matplotlib, and pandas) had also matured enough that a book written
about them would likely not go out-of-date very quickly. Thus, I mustered the nerve
to embark on this writing project. This is the book that I wish existed when I started
using Python for data analysis in 2007. I hope you find it useful and are able to apply
these tools productively in your work.

Conventions Used in This Book

The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width
Used for program listings, as well as within paragraphs to refer to program elements
such as variable or function names, databases, data types, environment variables,
statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values deter-
mined by context.

This icon signifies a tip, suggestion, or general note.

xi
 This icon indicates a warning or caution.

Using Code Examples

This book is here to help you get your job done. In general, you may use the code in
this book in your programs and documentation. You do not need to contact us for
permission unless you’re reproducing a significant portion of the code. For example,
writing a program that uses several chunks of code from this book does not require
permission. Selling or distributing a CD-ROM of examples from O’Reilly books does
require permission. Answering a question by citing this book and quoting example
code does not require permission. Incorporating a significant amount of example code
from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title,
author, publisher, and ISBN. For example: “Python for Data Analysis by William Wes-
ley McKinney (O’Reilly). Copyright 2012 William McKinney, 978-1-449-31979-3.”
If you feel your use of code examples falls outside fair use or the permission given above,
feel free to contact us at [email protected].

Safari® Books Online

Safari Books Online (www.safaribooksonline.com) is an on-demand digital
library that delivers expert content in both book and video form from the
world’s leading authors in technology and business.
Technology professionals, software developers, web designers, and business and cre-
ative professionals use Safari Books Online as their primary resource for research,
problem solving, learning, and certification training.
Safari Books Online offers a range of product mixes and pricing programs for organi-
zations, government agencies, and individuals. Subscribers have access to thousands
of books, training videos, and prepublication manuscripts in one fully searchable da-
tabase from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley
Professional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John
Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT
Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Tech-
nology, and dozens more. For more information about Safari Books Online, please visit
us online.

xii | Preface
How to Contact Us
Please address comments and questions concerning this book to the publisher:
O’Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at http://oreil.ly/python_for_data_analysis.
To comment or ask technical questions about this book, send email to
[email protected].
For more information about our books, courses, conferences, and news, see our website
at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia

Preface | xiii
 CHAPTER 1
Preliminaries

What Is This Book About?

This book is concerned with the nuts and bolts of manipulating, processing, cleaning,
and crunching data in Python. It is also a practical, modern introduction to scientific
computing in Python, tailored for data-intensive applications. This is a book about the
parts of the Python language and libraries you’ll need to effectively solve a broad set of
data analysis problems. This book is not an exposition on analytical methods using
Python as the implementation language.
When I say “data”, what am I referring to exactly? The primary focus is on structured
data, a deliberately vague term that encompasses many different common forms of
data, such as
• Multidimensional arrays (matrices)
• Tabular or spreadsheet-like data in which each column may be a different type
(string, numeric, date, or otherwise). This includes most kinds of data commonly
stored in relational databases or tab- or comma-delimited text files
• Multiple tables of data interrelated by key columns (what would be primary or
foreign keys for a SQL user)
• Evenly or unevenly spaced time series
This is by no means a complete list. Even though it may not always be obvious, a large
percentage of data sets can be transformed into a structured form that is more suitable
for analysis and modeling. If not, it may be possible to extract features from a data set
into a structured form. As an example, a collection of news articles could be processed
into a word frequency table which could then be used to perform sentiment analysis.
Most users of spreadsheet programs like Microsoft Excel, perhaps the most widely used
data analysis tool in the world, will not be strangers to these kinds of data.

1
Why Python for Data Analysis?
For many people (myself among them), the Python language is easy to fall in love with.
Since its first appearance in 1991, Python has become one of the most popular dynamic,
programming languages, along with Perl, Ruby, and others. Python and Ruby have
become especially popular in recent years for building websites using their numerous
web frameworks, like Rails (Ruby) and Django (Python). Such languages are often
called scripting languages as they can be used to write quick-and-dirty small programs,
or scripts. I don’t like the term “scripting language” as it carries a connotation that they
cannot be used for building mission-critical software. Among interpreted languages
Python is distinguished by its large and active scientific computing community. Adop-
tion of Python for scientific computing in both industry applications and academic
research has increased significantly since the early 2000s.
For data analysis and interactive, exploratory computing and data visualization, Python
will inevitably draw comparisons with the many other domain-specific open source
and commercial programming languages and tools in wide use, such as R, MATLAB,
SAS, Stata, and others. In recent years, Python’s improved library support (primarily
pandas) has made it a strong alternative for data manipulation tasks. Combined with
Python’s strength in general purpose programming, it is an excellent choice as a single
language for building data-centric applications.

Python as Glue
Part of Python’s success as a scientific computing platform is the ease of integrating C,
C++, and FORTRAN code. Most modern computing environments share a similar set
of legacy FORTRAN and C libraries for doing linear algebra, optimization, integration,
fast fourier transforms, and other such algorithms. The same story has held true for
many companies and national labs that have used Python to glue together 30 years’
worth of legacy software.
Most programs consist of small portions of code where most of the time is spent, with
large amounts of “glue code” that doesn’t run often. In many cases, the execution time
of the glue code is insignificant; effort is most fruitfully invested in optimizing the
computational bottlenecks, sometimes by moving the code to a lower-level language
like C.
In the last few years, the Cython project (http://cython.org) has become one of the
preferred ways of both creating fast compiled extensions for Python and also interfacing
with C and C++ code.

Solving the “Two-Language” Problem

In many organizations, it is common to research, prototype, and test new ideas using
a more domain-specific computing language like MATLAB or R then later port those

2 | Chapter 1: Preliminaries
ideas to be part of a larger production system written in, say, Java, C#, or C++. What
people are increasingly finding is that Python is a suitable language not only for doing
research and prototyping but also building the production systems, too. I believe that
more and more companies will go down this path as there are often significant organ-
izational benefits to having both scientists and technologists using the same set of pro-
grammatic tools.

Why Not Python?

While Python is an excellent environment for building computationally-intensive sci-
entific applications and building most kinds of general purpose systems, there are a
number of uses for which Python may be less suitable.
As Python is an interpreted programming language, in general most Python code will
run substantially slower than code written in a compiled language like Java or C++. As
programmer time is typically more valuable than CPU time, many are happy to make
this tradeoff. However, in an application with very low latency requirements (for ex-
ample, a high frequency trading system), the time spent programming in a lower-level,
lower-productivity language like C++ to achieve the maximum possible performance
might be time well spent.
Python is not an ideal language for highly concurrent, multithreaded applications, par-
ticularly applications with many CPU-bound threads. The reason for this is that it has
what is known as the global interpreter lock (GIL), a mechanism which prevents the
interpreter from executing more than one Python bytecode instruction at a time. The
technical reasons for why the GIL exists are beyond the scope of this book, but as of
this writing it does not seem likely that the GIL will disappear anytime soon. While it
is true that in many big data processing applications, a cluster of computers may be
required to process a data set in a reasonable amount of time, there are still situations
where a single-process, multithreaded system is desirable.
This is not to say that Python cannot execute truly multithreaded, parallel code; that
code just cannot be executed in a single Python process. As an example, the Cython
project features easy integration with OpenMP, a C framework for parallel computing,
in order to to parallelize loops and thus significantly speed up numerical algorithms.

Essential Python Libraries

For those who are less familiar with the scientific Python ecosystem and the libraries
used throughout the book, I present the following overview of each library.

Essential Python Libraries | 3

NumPy
NumPy, short for Numerical Python, is the foundational package for scientific com-
puting in Python. The majority of this book will be based on NumPy and libraries built
on top of NumPy. It provides, among other things:
• A fast and efficient multidimensional array object ndarray
• Functions for performing element-wise computations with arrays or mathematical
operations between arrays
• Tools for reading and writing array-based data sets to disk
• Linear algebra operations, Fourier transform, and random number generation
• Tools for integrating connecting C, C++, and Fortran code to Python
Beyond the fast array-processing capabilities that NumPy adds to Python, one of its
primary purposes with regards to data analysis is as the primary container for data to
be passed between algorithms. For numerical data, NumPy arrays are a much more
efficient way of storing and manipulating data than the other built-in Python data
structures. Also, libraries written in a lower-level language, such as C or Fortran, can
operate on the data stored in a NumPy array without copying any data.

pandas
pandas provides rich data structures and functions designed to make working with
structured data fast, easy, and expressive. It is, as you will see, one of the critical in-
gredients enabling Python to be a powerful and productive data analysis environment.
The primary object in pandas that will be used in this book is the DataFrame, a two-
dimensional tabular, column-oriented data structure with both row and column labels:
>>> frame
total_bill tip sex smoker day time size
1 16.99 1.01 Female No Sun Dinner 2
2 10.34 1.66 Male No Sun Dinner 3
3 21.01 3.5 Male No Sun Dinner 3
4 23.68 3.31 Male No Sun Dinner 2
5 24.59 3.61 Female No Sun Dinner 4
6 25.29 4.71 Male No Sun Dinner 4
7 8.77 2 Male No Sun Dinner 2
8 26.88 3.12 Male No Sun Dinner 4
9 15.04 1.96 Male No Sun Dinner 2
10 14.78 3.23 Male No Sun Dinner 2

pandas combines the high performance array-computing features of NumPy with the
flexible data manipulation capabilities of spreadsheets and relational databases (such
as SQL). It provides sophisticated indexing functionality to make it easy to reshape,
slice and dice, perform aggregations, and select subsets of data. pandas is the primary
tool that we will use in this book.

4 | Chapter 1: Preliminaries
For financial users, pandas features rich, high-performance time series functionality
and tools well-suited for working with financial data. In fact, I initially designed pandas
as an ideal tool for financial data analysis applications.
For users of the R language for statistical computing, the DataFrame name will be
familiar, as the object was named after the similar R data.frame object. They are not
the same, however; the functionality provided by data.frame in R is essentially a strict
subset of that provided by the pandas DataFrame. While this is a book about Python, I
will occasionally draw comparisons with R as it is one of the most widely-used open
source data analysis environments and will be familiar to many readers.
The pandas name itself is derived from panel data, an econometrics term for multidi-
mensional structured data sets, and Python data analysis itself.

matplotlib
matplotlib is the most popular Python library for producing plots and other 2D data
visualizations. It was originally created by John D. Hunter (JDH) and is now maintained
by a large team of developers. It is well-suited for creating plots suitable for publication.
It integrates well with IPython (see below), thus providing a comfortable interactive
environment for plotting and exploring data. The plots are also interactive; you can
zoom in on a section of the plot and pan around the plot using the toolbar in the plot
window.

IPython
IPython is the component in the standard scientific Python toolset that ties everything
together. It provides a robust and productive environment for interactive and explor-
atory computing. It is an enhanced Python shell designed to accelerate the writing,
testing, and debugging of Python code. It is particularly useful for interactively working
with data and visualizing data with matplotlib. IPython is usually involved with the
majority of my Python work, including running, debugging, and testing code.
Aside from the standard terminal-based IPython shell, the project also provides
• A Mathematica-like HTML notebook for connecting to IPython through a web
browser (more on this later).
• A Qt framework-based GUI console with inline plotting, multiline editing, and
syntax highlighting
• An infrastructure for interactive parallel and distributed computing
I will devote a chapter to IPython and how to get the most out of its features. I strongly
recommend using it while working through this book.

Essential Python Libraries | 5

SciPy
SciPy is a collection of packages addressing a number of different standard problem
domains in scientific computing. Here is a sampling of the packages included:
• scipy.integrate: numerical integration routines and differential equation solvers
• scipy.linalg: linear algebra routines and matrix decompositions extending be-
yond those provided in numpy.linalg.
• scipy.optimize: function optimizers (minimizers) and root finding algorithms
• scipy.signal: signal processing tools
• scipy.sparse: sparse matrices and sparse linear system solvers
• scipy.special: wrapper around SPECFUN, a Fortran library implementing many
common mathematical functions, such as the gamma function
• scipy.stats: standard continuous and discrete probability distributions (density
functions, samplers, continuous distribution functions), various statistical tests,
and more descriptive statistics
• scipy.weave: tool for using inline C++ code to accelerate array computations
Together NumPy and SciPy form a reasonably complete computational replacement
for much of MATLAB along with some of its add-on toolboxes.

Installation and Setup

Since everyone uses Python for different applications, there is no single solution for
setting up Python and required add-on packages. Many readers will not have a complete
scientific Python environment suitable for following along with this book, so here I will
give detailed instructions to get set up on each operating system. I recommend using
one of the following base Python distributions:
• Enthought Python Distribution: a scientific-oriented Python distribution from En-
thought (http://www.enthought.com). This includes EPDFree, a free base scientific
distribution (with NumPy, SciPy, matplotlib, Chaco, and IPython) and EPD Full,
a comprehensive suite of more than 100 scientific packages across many domains.
EPD Full is free for academic use but has an annual subscription for non-academic
users.
• Python(x,y) (http://pythonxy.googlecode.com): A free scientific-oriented Python
distribution for Windows.
I will be using EPDFree for the installation guides, though you are welcome to take
another approach depending on your needs. At the time of this writing, EPD includes
Python 2.7, though this might change at some point in the future. After installing, you
will have the following packages installed and importable:

6 | Chapter 1: Preliminaries
 • Scientific Python base: NumPy, SciPy, matplotlib, and IPython. These are all in-
cluded in EPDFree.
• IPython Notebook dependencies: tornado and pyzmq. These are included in EPD-
Free.
• pandas (version 0.8.2 or higher).
At some point while reading you may wish to install one or more of the following
packages: statsmodels, PyTables, PyQt (or equivalently, PySide), xlrd, lxml, basemap,
pymongo, and requests. These are used in various examples. Installing these optional
libraries is not necessary, and I would would suggest waiting until you need them. For
example, installing PyQt or PyTables from source on OS X or Linux can be rather
arduous. For now, it’s most important to get up and running with the bare minimum:
EPDFree and pandas.
For information on each Python package and links to binary installers or other help,
see the Python Package Index (PyPI, http://pypi.python.org). This is also an excellent
resource for finding new Python packages.

To avoid confusion and to keep things simple, I am avoiding discussion

of more complex environment management tools like pip and virtua-
lenv. There are many excellent guides available for these tools on the
Internet.

Some users may be interested in alternate Python implementations, such

as IronPython, Jython, or PyPy. To make use of the tools presented in
this book, it is (currently) necessary to use the standard C-based Python
interpreter, known as CPython.

Windows
To get started on Windows, download the EPDFree installer from http://www.en
thought.com, which should be an MSI installer named like epd_free-7.3-1-win-
x86.msi. Run the installer and accept the default installation location C:\Python27. If
you had previously installed Python in this location, you may want to delete it manually
first (or using Add/Remove Programs).
Next, you need to verify that Python has been successfully added to the system path
and that there are no conflicts with any prior-installed Python versions. First, open a
command prompt by going to the Start Menu and starting the Command Prompt ap-
plication, also known as cmd.exe. Try starting the Python interpreter by typing
python. You should see a message that matches the version of EPDFree you installed:
C:\Users\Wes>python
Python 2.7.3 |EPD_free 7.3-1 (32-bit)| (default, Apr 12 2012, 14:30:37) on win32
Type "credits", "demo" or "enthought" for more information.
>>>

Installation and Setup | 7

If you see a message for a different version of EPD or it doesn’t work at all, you will
need to clean up your Windows environment variables. On Windows 7 you can start
typing “environment variables” in the programs search field and select Edit environ
ment variables for your account. On Windows XP, you will have to go to Control
Panel > System > Advanced > Environment Variables. On the window that pops up,
you are looking for the Path variable. It needs to contain the following two directory
paths, separated by semicolons:
C:\Python27;C:\Python27\Scripts

If you installed other versions of Python, be sure to delete any other Python-related
directories from both the system and user Path variables. After making a path alterna-
tion, you have to restart the command prompt for the changes to take effect.
Once you can launch Python successfully from the command prompt, you need to
install pandas. The easiest way is to download the appropriate binary installer from
http://pypi.python.org/pypi/pandas. For EPDFree, this should be pandas-0.9.0.win32-
py2.7.exe. After you run this, let’s launch IPython and check that things are installed
correctly by importing pandas and making a simple matplotlib plot:
C:\Users\Wes>ipython --pylab
Python 2.7.3 |EPD_free 7.3-1 (32-bit)|
Type "copyright", "credits" or "license" for more information.

IPython 0.12.1 -- An enhanced Interactive Python.

? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

Welcome to pylab, a matplotlib-based Python environment [backend: WXAgg].

For more information, type 'help(pylab)'.

In [1]: import pandas

In [2]: plot(arange(10))

If successful, there should be no error messages and a plot window will appear. You
can also check that the IPython HTML notebook can be successfully run by typing:
$ ipython notebook --pylab=inline

If you use the IPython notebook application on Windows and normally

use Internet Explorer, you will likely need to install and run Mozilla
Firefox or Google Chrome instead.

EPDFree on Windows contains only 32-bit executables. If you want or need a 64-bit
setup on Windows, using EPD Full is the most painless way to accomplish that. If you
would rather install from scratch and not pay for an EPD subscription, Christoph
Gohlke at the University of California, Irvine, publishes unofficial binary installers for

8 | Chapter 1: Preliminaries
 all of the book’s necessary packages (http://www.lfd.uci.edu/~gohlke/pythonlibs/) for 32-
and 64-bit Windows.

Apple OS X
To get started on OS X, you must first install Xcode, which includes Apple’s suite of
software development tools. The necessary component for our purposes is the gcc C
and C++ compiler suite. The Xcode installer can be found on the OS X install DVD
that came with your computer or downloaded from Apple directly.
Once you’ve installed Xcode, launch the terminal (Terminal.app) by navigating to
Applications > Utilities. Type gcc and press enter. You should hopefully see some-
thing like:
$ gcc
i686-apple-darwin10-gcc-4.2.1: no input files
Download from Wow! eBook <www.wowebook.com>

Now you need to install EPDFree. Download the installer which should be a disk image
named something like epd_free-7.3-1-macosx-i386.dmg. Double-click the .dmg file to
mount it, then double-click the .mpkg file inside to run the installer.
When the installer runs, it automatically appends the EPDFree executable path to
your .bash_profile file. This is located at /Users/your_uname/.bash_profile:
# Setting PATH for EPD_free-7.3-1
PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}"
export PATH

Should you encounter any problems in the following steps, you’ll want to inspect
your .bash_profile and potentially add the above directory to your path.
Now, it’s time to install pandas. Execute this command in the terminal:
$ sudo easy_install pandas
Searching for pandas
Reading http://pypi.python.org/simple/pandas/
Reading http://pandas.pydata.org
Reading http://pandas.sourceforge.net
Best match: pandas 0.9.0
Downloading http://pypi.python.org/packages/source/p/pandas/pandas-0.9.0.zip
Processing pandas-0.9.0.zip
Writing /tmp/easy_install-H5mIX6/pandas-0.9.0/setup.cfg
Running pandas-0.9.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-H5mIX6/
pandas-0.9.0/egg-dist-tmp-RhLG0z
Adding pandas 0.9.0 to easy-install.pth file

Installed /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/
site-packages/pandas-0.9.0-py2.7-macosx-10.5-i386.egg
Processing dependencies for pandas
Finished processing dependencies for pandas

To verify everything is working, launch IPython in Pylab mode and test importing pan-
das then making a plot interactively:

Installation and Setup | 9

 $ ipython --pylab
22:29 ~/VirtualBox VMs/WindowsXP $ ipython
Python 2.7.3 |EPD_free 7.3-1 (32-bit)| (default, Apr 12 2012, 11:28:34)
Type "copyright", "credits" or "license" for more information.

IPython 0.12.1 -- An enhanced Interactive Python.

? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

Welcome to pylab, a matplotlib-based Python environment [backend: WXAgg].

For more information, type 'help(pylab)'.

In [1]: import pandas

In [2]: plot(arange(10))

If this succeeds, a plot window with a straight line should pop up.

GNU/Linux

Some, but not all, Linux distributions include sufficiently up-to-date

versions of all the required Python packages and can be installed using
the built-in package management tool like apt. I detail setup using EPD-
Free as it's easily reproducible across distributions.

Linux details will vary a bit depending on your Linux flavor, but here I give details for
Debian-based GNU/Linux systems like Ubuntu and Mint. Setup is similar to OS X with
the exception of how EPDFree is installed. The installer is a shell script that must be
executed in the terminal. Depending on whether you have a 32-bit or 64-bit system,
you will either need to install the x86 (32-bit) or x86_64 (64-bit) installer. You will then
have a file named something similar to epd_free-7.3-1-rh5-x86_64.sh. To install it,
execute this script with bash:
$ bash epd_free-7.3-1-rh5-x86_64.sh

After accepting the license, you will be presented with a choice of where to put the
EPDFree files. I recommend installing the files in your home directory, say /home/wesm/
epd (substituting your own username for wesm).
Once the installer has finished, you need to add EPDFree’s bin directory to your
$PATH variable. If you are using the bash shell (the default in Ubuntu, for example), this
means adding the following path addition in your .bashrc:
export PATH=/home/wesm/epd/bin:$PATH

Obviously, substitute the installation directory you used for /home/wesm/epd/. After
doing this you can either start a new terminal process or execute your .bashrc again
with source ~/.bashrc.

10 | Chapter 1: Preliminaries
You need a C compiler such as gcc to move forward; many Linux distributions include
gcc, but others may not. On Debian systems, you can install gcc by executing:
sudo apt-get install gcc

If you type gcc on the command line it should say something like:
$ gcc
gcc: no input files

Now, time to install pandas:

$ easy_install pandas

If you installed EPDFree as root, you may need to add sudo to the command and enter
the sudo or root password. To verify things are working, perform the same checks as
in the OS X section.

Python 2 and Python 3

The Python community is currently undergoing a drawn-out transition from the Python
2 series of interpreters to the Python 3 series. Until the appearance of Python 3.0, all
Python code was backwards compatible. The community decided that in order to move
the language forward, certain backwards incompatible changes were necessary.
I am writing this book with Python 2.7 as its basis, as the majority of the scientific
Python community has not yet transitioned to Python 3. The good news is that, with
a few exceptions, you should have no trouble following along with the book if you
happen to be using Python 3.2.

Integrated Development Environments (IDEs)

When asked about my standard development environment, I almost always say “IPy-
thon plus a text editor”. I typically write a program and iteratively test and debug each
piece of it in IPython. It is also useful to be able to play around with data interactively
and visually verify that a particular set of data manipulations are doing the right thing.
Libraries like pandas and NumPy are designed to be easy-to-use in the shell.
However, some will still prefer to work in an IDE instead of a text editor. They do
provide many nice “code intelligence” features like completion or quickly pulling up
the documentation associated with functions and classes. Here are some that you can
explore:
• Eclipse with PyDev Plugin
• Python Tools for Visual Studio (for Windows users)
• PyCharm
• Spyder
• Komodo IDE

Installation and Setup | 11

Community and Conferences
Outside of an Internet search, the scientific Python mailing lists are generally helpful
and responsive to questions. Some ones to take a look at are:
• pydata: a Google Group list for questions related to Python for data analysis and
pandas
• pystatsmodels: for statsmodels or pandas-related questions
• numpy-discussion: for NumPy-related questions
• scipy-user: for general SciPy or scientific Python questions
I deliberately did not post URLs for these in case they change. They can be easily located
via Internet search.
Each year many conferences are held all over the world for Python programmers. PyCon
and EuroPython are the two main general Python conferences in the United States and
Europe, respectively. SciPy and EuroSciPy are scientific-oriented Python conferences
where you will likely find many “birds of a feather” if you become more involved with
using Python for data analysis after reading this book.

Navigating This Book

If you have never programmed in Python before, you may actually want to start at the
end of the book, where I have placed a condensed tutorial on Python syntax, language
features, and built-in data structures like tuples, lists, and dicts. These things are con-
sidered prerequisite knowledge for the remainder of the book.
The book starts by introducing you to the IPython environment. Next, I give a short
introduction to the key features of NumPy, leaving more advanced NumPy use for
another chapter at the end of the book. Then, I introduce pandas and devote the rest
of the book to data analysis topics applying pandas, NumPy, and matplotlib (for vis-
ualization). I have structured the material in the most incremental way possible, though
there is occasionally some minor cross-over between chapters.
Data files and related material for each chapter are hosted as a git repository on GitHub:
http://github.com/pydata/pydata-book

I encourage you to download the data and use it to replicate the book’s code examples
and experiment with the tools presented in each chapter. I will happily accept contri-
butions, scripts, IPython notebooks, or any other materials you wish to contribute to
the book's repository for all to enjoy.

12 | Chapter 1: Preliminaries
Code Examples
Most of the code examples in the book are shown with input and output as it would
appear executed in the IPython shell.
In [5]: code
Out[5]: output

At times, for clarity, multiple code examples will be shown side by side. These should
be read left to right and executed separately.
In [5]: code In [6]: code2
Out[5]: output Out[6]: output2

Data for Examples

Data sets for the examples in each chapter are hosted in a repository on GitHub: http:
//github.com/pydata/pydata-book. You can download this data either by using the git
revision control command-line program or by downloading a zip file of the repository
from the website.
I have made every effort to ensure that it contains everything necessary to reproduce
the examples, but I may have made some mistakes or omissions. If so, please send me
an e-mail: [email protected].

Import Conventions
The Python community has adopted a number of naming conventions for commonly-
used modules:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

This means that when you see np.arange, this is a reference to the arange function in
NumPy. This is done as it’s considered bad practice in Python software development
to import everything (from numpy import *) from a large package like NumPy.

Jargon
I’ll use some terms common both to programming and data science that you may not
be familiar with. Thus, here are some brief definitions:
Munge/Munging/Wrangling
Describes the overall process of manipulating unstructured and/or messy data into
a structured or clean form. The word has snuck its way into the jargon of many
modern day data hackers. Munge rhymes with “lunge”.

Navigating This Book | 13

Pseudocode
A description of an algorithm or process that takes a code-like form while likely
not being actual valid source code.
Syntactic sugar
Programming syntax which does not add new features, but makes something more
convenient or easier to type.

Acknowledgements
It would have been difficult for me to write this book without the support of a large
number of people.
On the O’Reilly staff, I’m very grateful for my editors Meghan Blanchette and Julie
Steele who guided me through the process. Mike Loukides also worked with me in the
proposal stages and helped make the book a reality.
I received a wealth of technical review from a large cast of characters. In particular,
Martin Blais and Hugh White were incredibly helpful in improving the book’s exam-
ples, clarity, and organization from cover to cover. James Long, Drew Conway, Fer-
nando Pérez, Brian Granger, Thomas Kluyver, Adam Klein, Josh Klein, Chang She, and
Stéfan van der Walt each reviewed one or more chapters, providing pointed feedback
from many different perspectives.
I got many great ideas for examples and data sets from friends and colleagues in the
data community, among them: Mike Dewar, Jeff Hammerbacher, James Johndrow,
Kristian Lum, Adam Klein, Hilary Mason, Chang She, and Ashley Williams.
I am of course indebted to the many leaders in the open source scientific Python com-
munity who’ve built the foundation for my development work and gave encouragement
while I was writing this book: the IPython core team (Fernando Pérez, Brian Granger,
Min Ragan-Kelly, Thomas Kluyver, and others), John Hunter, Skipper Seabold, Travis
Oliphant, Peter Wang, Eric Jones, Robert Kern, Josef Perktold, Francesc Alted, Chris
Fonnesbeck, and too many others to mention. Several other people provided a great
deal of support, ideas, and encouragement along the way: Drew Conway, Sean Taylor,
Giuseppe Paleologo, Jared Lander, David Epstein, John Krowas, Joshua Bloom, Den
Pilsworth, John Myles-White, and many others I’ve forgotten.
I’d also like to thank a number of people from my formative years. First, my former
AQR colleagues who’ve cheered me on in my pandas work over the years: Alex Reyf-
man, Michael Wong, Tim Sargen, Oktay Kurbanov, Matthew Tschantz, Roni Israelov,
Michael Katz, Chris Uga, Prasad Ramanan, Ted Square, and Hoon Kim. Lastly, my
academic advisors Haynes Miller (MIT) and Mike West (Duke).
On the personal side, Casey Dinkin provided invaluable day-to-day support during the
writing process, tolerating my highs and lows as I hacked together the final draft on

14 | Chapter 1: Preliminaries
top of an already overcommitted schedule. Lastly, my parents, Bill and Kim, taught me
to always follow my dreams and to never settle for less.

Acknowledgements | 15
 CHAPTER 2
Introductory Examples

This book teaches you the Python tools to work productively with data. While readers
may have many different end goals for their work, the tasks required generally fall into
a number of different broad groups:
Interacting with the outside world
Reading and writing with a variety of file formats and databases.
Preparation
Cleaning, munging, combining, normalizing, reshaping, slicing and dicing, and
transforming data for analysis.
Transformation
Applying mathematical and statistical operations to groups of data sets to derive
new data sets. For example, aggregating a large table by group variables.
Modeling and computation
Connecting your data to statistical models, machine learning algorithms, or other
computational tools
Presentation
Creating interactive or static graphical visualizations or textual summaries
In this chapter I will show you a few data sets and some things we can do with them.
These examples are just intended to pique your interest and thus will only be explained
at a high level. Don’t worry if you have no experience with any of these tools; they will
be discussed in great detail throughout the rest of the book. In the code examples you’ll
see input and output prompts like In [15]:; these are from the IPython shell.

1.usa.gov data from bit.ly

In 2011, URL shortening service bit.ly partnered with the United States government
website usa.gov to provide a feed of anonymous data gathered from users who shorten
links ending with .gov or .mil. As of this writing, in addition to providing a live feed,
hourly snapshots are available as downloadable text files.1

17
In the case of the hourly snapshots, each line in each file contains a common form of
web data known as JSON, which stands for JavaScript Object Notation. For example,
if we read just the first line of a file you may see something like
In [15]: path = 'ch02/usagov_bitly_data2012-03-16-1331923249.txt'

In [16]: open(path).readline()
Out[16]: '{ "a": "Mozilla\\/5.0 (Windows NT 6.1; WOW64) AppleWebKit\\/535.11
(KHTML, like Gecko) Chrome\\/17.0.963.78 Safari\\/535.11", "c": "US", "nk": 1,
"tz": "America\\/New_York", "gr": "MA", "g": "A6qOVH", "h": "wfLQtf", "l":
"orofrog", "al": "en-US,en;q=0.8", "hh": "1.usa.gov", "r":
"http:\\/\\/www.facebook.com\\/l\\/7AQEFzjSi\\/1.usa.gov\\/wfLQtf", "u":
"http:\\/\\/www.ncbi.nlm.nih.gov\\/pubmed\\/22415991", "t": 1331923247, "hc":
1331822918, "cy": "Danvers", "ll": [ 42.576698, -70.954903 ] }\n'

Python has numerous built-in and 3rd party modules for converting a JSON string into
a Python dictionary object. Here I’ll use the json module and its loads function invoked
on each line in the sample file I downloaded:
import json
path = 'ch02/usagov_bitly_data2012-03-16-1331923249.txt'
records = [json.loads(line) for line in open(path)]

If you’ve never programmed in Python before, the last expression here is called a list
comprehension, which is a concise way of applying an operation (like json.loads) to a
collection of strings or other objects. Conveniently, iterating over an open file handle
gives you a sequence of its lines. The resulting object records is now a list of Python
dicts:
In [18]: records[0]
Out[18]:
{u'a': u'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like
Gecko) Chrome/17.0.963.78 Safari/535.11',
u'al': u'en-US,en;q=0.8',
u'c': u'US',
u'cy': u'Danvers',
u'g': u'A6qOVH',
u'gr': u'MA',
u'h': u'wfLQtf',
u'hc': 1331822918,
u'hh': u'1.usa.gov',
u'l': u'orofrog',
u'll': [42.576698, -70.954903],
u'nk': 1,
u'r': u'http://www.facebook.com/l/7AQEFzjSi/1.usa.gov/wfLQtf',
u't': 1331923247,
u'tz': u'America/New_York',
u'u': u'http://www.ncbi.nlm.nih.gov/pubmed/22415991'}

1. http://www.usa.gov/About/developer-resources/1usagov.shtml

18 | Chapter 2: Introductory Examples

Note that Python indices start at 0 and not 1 like some other languages (like R). It’s
now easy to access individual values within records by passing a string for the key you
wish to access:
In [19]: records[0]['tz']
Out[19]: u'America/New_York'

The u here in front of the quotation stands for unicode, a standard form of string en-
coding. Note that IPython shows the time zone string object representation here rather
than its print equivalent:
In [20]: print records[0]['tz']
America/New_York

Counting Time Zones in Pure Python

Suppose we were interested in the most often-occurring time zones in the data set (the
tz field). There are many ways we could do this. First, let’s extract a list of time zones
again using a list comprehension:
In [25]: time_zones = [rec['tz'] for rec in records]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
/home/wesm/book_scripts/whetting/<ipython> in <module>()
----> 1 time_zones = [rec['tz'] for rec in records]

KeyError: 'tz'

Oops! Turns out that not all of the records have a time zone field. This is easy to handle
as we can add the check if 'tz' in rec at the end of the list comprehension:
In [26]: time_zones = [rec['tz'] for rec in records if 'tz' in rec]

In [27]: time_zones[:10]
Out[27]:
[u'America/New_York',
u'America/Denver',
u'America/New_York',
u'America/Sao_Paulo',
u'America/New_York',
u'America/New_York',
u'Europe/Warsaw',
u'',
u'',
u'']

Just looking at the first 10 time zones we see that some of them are unknown (empty).
You can filter these out also but I’ll leave them in for now. Now, to produce counts by
time zone I’ll show two approaches: the harder way (using just the Python standard
library) and the easier way (using pandas). One way to do the counting is to use a dict
to store counts while we iterate through the time zones:
def get_counts(sequence):
counts = {}

1.usa.gov data from bit.ly | 19

 for x in sequence:
if x in counts:
counts[x] += 1
else:
counts[x] = 1
return counts

If you know a bit more about the Python standard library, you might prefer to write
the same thing more briefly:
from collections import defaultdict

def get_counts2(sequence):
counts = defaultdict(int) # values will initialize to 0
for x in sequence:
counts[x] += 1
return counts

I put this logic in a function just to make it more reusable. To use it on the time zones,
Download from Wow! eBook <www.wowebook.com>

just pass the time_zones list:

In [31]: counts = get_counts(time_zones)

In [32]: counts['America/New_York']
Out[32]: 1251

In [33]: len(time_zones)
Out[33]: 3440

If we wanted the top 10 time zones and their counts, we have to do a little bit of dic-
tionary acrobatics:
def top_counts(count_dict, n=10):
value_key_pairs = [(count, tz) for tz, count in count_dict.items()]
value_key_pairs.sort()
return value_key_pairs[-n:]

We have then:
In [35]: top_counts(counts)
Out[35]:
[(33, u'America/Sao_Paulo'),
(35, u'Europe/Madrid'),
(36, u'Pacific/Honolulu'),
(37, u'Asia/Tokyo'),
(74, u'Europe/London'),
(191, u'America/Denver'),
(382, u'America/Los_Angeles'),
(400, u'America/Chicago'),
(521, u''),
(1251, u'America/New_York')]

20 | Chapter 2: Introductory Examples

If you search the Python standard library, you may find the collections.Counter class,
which makes this task a lot easier:
In [49]: from collections import Counter

In [50]: counts = Counter(time_zones)

In [51]: counts.most_common(10)
Out[51]:
[(u'America/New_York', 1251),
(u'', 521),
(u'America/Chicago', 400),
(u'America/Los_Angeles', 382),
(u'America/Denver', 191),
(u'Europe/London', 74),
(u'Asia/Tokyo', 37),
(u'Pacific/Honolulu', 36),
(u'Europe/Madrid', 35),
(u'America/Sao_Paulo', 33)]

Counting Time Zones with pandas

The main pandas data structure is the DataFrame, which you can think of as repre-
senting a table or spreadsheet of data. Creating a DataFrame from the original set of
records is simple:
In [289]: from pandas import DataFrame, Series

In [290]: import pandas as pd

In [291]: frame = DataFrame(records)

In [292]: frame
Out[292]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 3560 entries, 0 to 3559
Data columns:
_heartbeat_ 120 non-null values
a 3440 non-null values
al 3094 non-null values
c 2919 non-null values
cy 2919 non-null values
g 3440 non-null values
gr 2919 non-null values
h 3440 non-null values
hc 3440 non-null values
hh 3440 non-null values
kw 93 non-null values
l 3440 non-null values
ll 2919 non-null values
nk 3440 non-null values
r 3440 non-null values
t 3440 non-null values
tz 3440 non-null values

1.usa.gov data from bit.ly | 21

 u 3440 non-null values
dtypes: float64(4), object(14)

In [293]: frame['tz'][:10]
Out[293]:
0 America/New_York
1 America/Denver
2 America/New_York
3 America/Sao_Paulo
4 America/New_York
5 America/New_York
6 Europe/Warsaw
7
8
9
Name: tz

The output shown for the frame is the summary view, shown for large DataFrame ob-
jects. The Series object returned by frame['tz'] has a method value_counts that gives
us what we’re looking for:
In [294]: tz_counts = frame['tz'].value_counts()

In [295]: tz_counts[:10]
Out[295]:
America/New_York 1251
521
America/Chicago 400
America/Los_Angeles 382
America/Denver 191
Europe/London 74
Asia/Tokyo 37
Pacific/Honolulu 36
Europe/Madrid 35
America/Sao_Paulo 33

Then, we might want to make a plot of this data using plotting library, matplotlib. You
can do a bit of munging to fill in a substitute value for unknown and missing time zone
data in the records. The fillna function can replace missing (NA) values and unknown
(empty strings) values can be replaced by boolean array indexing:
In [296]: clean_tz = frame['tz'].fillna('Missing')

In [297]: clean_tz[clean_tz == ''] = 'Unknown'

In [298]: tz_counts = clean_tz.value_counts()

In [299]: tz_counts[:10]
Out[299]:
America/New_York 1251
Unknown 521
America/Chicago 400
America/Los_Angeles 382
America/Denver 191
Missing 120

22 | Chapter 2: Introductory Examples

 Europe/London 74
Asia/Tokyo 37
Pacific/Honolulu 36
Europe/Madrid 35

Making a horizontal bar plot can be accomplished using the plot method on the
counts objects:
In [301]: tz_counts[:10].plot(kind='barh', rot=0)

See Figure 2-1 for the resulting figure. We’ll explore more tools for working with this
kind of data. For example, the a field contains information about the browser, device,
or application used to perform the URL shortening:
In [302]: frame['a'][1]
Out[302]: u'GoogleMaps/RochesterNY'

In [303]: frame['a'][50]
Out[303]: u'Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20100101 Firefox/10.0.2'

In [304]: frame['a'][51]
Out[304]: u'Mozilla/5.0 (Linux; U; Android 2.2.2; en-us; LG-P925/V10e Build/FRG83G) AppleWebKit/533.1 (K

Figure 2-1. Top time zones in the 1.usa.gov sample data

Parsing all of the interesting information in these “agent” strings may seem like a
daunting task. Luckily, once you have mastered Python’s built-in string functions and
regular expression capabilities, it is really not so bad. For example, we could split off
the first token in the string (corresponding roughly to the browser capability) and make
another summary of the user behavior:
In [305]: results = Series([x.split()[0] for x in frame.a.dropna()])

In [306]: results[:5]
Out[306]:
0 Mozilla/5.0
1 GoogleMaps/RochesterNY
2 Mozilla/4.0
3 Mozilla/5.0
4 Mozilla/5.0

1.usa.gov data from bit.ly | 23

 In [307]: results.value_counts()[:8]
Out[307]:
Mozilla/5.0 2594
Mozilla/4.0 601
GoogleMaps/RochesterNY 121
Opera/9.80 34
TEST_INTERNET_AGENT 24
GoogleProducer 21
Mozilla/6.0 5
BlackBerry8520/5.0.0.681 4

Now, suppose you wanted to decompose the top time zones into Windows and non-
Windows users. As a simplification, let’s say that a user is on Windows if the string
'Windows' is in the agent string. Since some of the agents are missing, I’ll exclude these
from the data:
In [308]: cframe = frame[frame.a.notnull()]

We want to then compute a value whether each row is Windows or not:

In [309]: operating_system = np.where(cframe['a'].str.contains('Windows'),
.....: 'Windows', 'Not Windows')

In [310]: operating_system[:5]
Out[310]:
0 Windows
1 Not Windows
2 Windows
3 Not Windows
4 Windows
Name: a

Then, you can group the data by its time zone column and this new list of operating
systems:
In [311]: by_tz_os = cframe.groupby(['tz', operating_system])

The group counts, analogous to the value_counts function above, can be computed
using size. This result is then reshaped into a table with unstack:
In [312]: agg_counts = by_tz_os.size().unstack().fillna(0)

In [313]: agg_counts[:10]
Out[313]:
a Not Windows Windows
tz
245 276
Africa/Cairo 0 3
Africa/Casablanca 0 1
Africa/Ceuta 0 2
Africa/Johannesburg 0 1
Africa/Lusaka 0 1
America/Anchorage 4 1
America/Argentina/Buenos_Aires 1 0

24 | Chapter 2: Introductory Examples

Other documents randomly have
different content
 “I did make a long speech though. Did you realize I was speaking
for two hours? You were not there all that time?”
“Yes, I was. Uncle John went to get something to eat, but I never
budged.”
“Claudia, how sweet of you.” He came a little nearer to her and his
nostrils dilated a little. No man is unmoved by the subtle flattery of a
beautiful woman, and Claudia was looking her best that night.
“But,” said Claudia, with an abrupt change of voice, “I wish the
man, the prisoner, had been more worth it. An awful poor thing,
wasn’t he? Even if he didn’t murder the boy, he was only a wisp of
straw, wasn’t he?”
“If men and women were all fine strong characters, my services
wouldn’t be required, would they?”
Claudia looked thoughtful, and the brown eyes seemed to grow
larger and brighter, as though some lamp were burning behind
them. “No, I suppose you live on people’s weaknesses and lack of
morals and stamina. Oh! dear, I don’t like to think that.”
“Well, don’t think it. Don’t let’s talk about my work. Tell me what
you have been doing since I saw you last week?”
She was leaning a little forward, her elbow on her knee, and he
could see the rise and fall of her bosom, the soft curves outlined by
the clinging chiffon. And though he sat outwardly unmoved,
something tingled within him and strained like a dog in a leash.
Claudia sat up with a shrug of her shoulders. It was a little trick of
hers that suited her dark eyes. “I have been gloriously doing nothing
in particular, the same things as I did last year, meeting much the
same people and talking much the same talk. I spent two afternoons
helping at the Duchess’s bazaar, and I smiled a continuous
persuasive smile from ear to ear all the time, and I told a great
many lies trying to sell things that were of an unutterable
hideousness, and that nobody could want to buy. There was such a
funny man came up to me. I tried to sell him a poker-work photo
frame. ‘Isn’t it charming?’ I murmured. ‘Madam,’ he said, with a little
twisty smile that began in his eyes and came down to his lips, ‘if you
will frankly tell me what you think of it, I will purchase it. Your tone
lacks conviction.’ ‘Sir,’ I replied, ‘frankly I think it one of the ugliest
things I have ever seen and nothing would induce me to have it in
my room.’ ‘How much?’ said he. And he bought it. I should like to
meet him again. I am sure we should be friends.”
“I wonder what he did with it?” laughed Gilbert. “Perhaps he put
his worst enemy into it.”
“If I ever see him again I shall ask him.... Have you heard about
Pat? She has run away from Germany and come home. She says
that speaking the Teutonic language all day was spoiling the shape
of her mouth, and there was something in the air or the water that
she was sure was making her figure spread! Isn’t she too quaint?
She announces that she has learnt quite enough for the present, and
she insists that mother shall bring her out.”
“Why, she’s quite a child, surely!”
“Oh, no! Patsy is—let me see—nearly eighteen. Mother is so
annoyed. You see I keep out of her way, but Pat is noisy about the
house. She finds Pat absolutely antagonistic to—well to the spooks
and the thought waves. She had hoped Pat would stay over in
Germany for six months and acquire a philosophic language. Pat
informed mother yesterday that she knew her type of good looks
went off early, and she advised mother to get her safely off-hand
before she began to fade.” Claudia laughed heartily at the
remembrance. “She’s awfully pretty. You don’t remember her?”
“I remember a small child with forget-me-not eyes and flaxen hair,
who was always sitting down heavily on choice seedlings in the
flower-beds and then crying because she had ‘hurted them.’”
“Yes, that was Patsy. But she’ll get married quite easily. She’s
really sweet. She’s got little tricks with her eyes, quite natural, not
affected—and her eyebrows go up in a funny way that makes her
look like an intelligent cock robin. By the way, have robins got
eyebrows? They seem eyebrows all over, don’t they? Oh! Pat will
make a hit when she comes out.”
Gilbert looked at her curiously. Did Claudia not think about getting
married? He hazarded a question in a bantering, rather intimate way.
“And when are you going off?”
“It sounds like a firework, doesn’t it? I don’t mind telling you in a
burst of confidence that Aunt Lucretia thinks the squib is a little
damp. It hasn’t gone bang yet! But Pat will make a brilliant firework.
Mind you don’t get burnt.”
The music had struck up again, and Claudia took up her
programme with a faint sigh.
Gilbert put his hand over the little white-gloved one that held the
card. “Who are you dancing with? Never mind who it is. Throw him
over. Yes,” he said firmly, as she protested, “I know it isn’t your usual
habit. But—well, isn’t it a special night somehow? It’s my birthday
for one thing and——”
“Oh, is it? Many happy returns. You got my photograph this
morning?”
“Yes, it’s on my mantelpiece now.... Never mind the wretched
programme.”
“But what shall I say?” she protested laughingly, for, womanlike,
she loved a high-handed man who insists on getting his own way.
“Say—say you prefer to dance with me.... Isn’t it true, Claudia?
Say it is.”
One hand was quite lost in his. His compelling eyes were on her
face. Something for an instant caught her by the throat and made
her shut her eyes as she said almost under her breath, “Yes, it’s
true.”
They made their way back to the ball-room. More than one man
stopped to congratulate Gilbert, and a good many women smiled up
at him invitingly.
 As far as Claudia knew, Gilbert never flirted. She had never heard
his name coupled even lightly with that of any woman. And he was
thirty-two! It was almost unique in her set, where sexual
philandering is one of the most amusing games for passing the time.
She did not realize that it was precisely for lack of time that he had
not paid much attention to women. The Law had been his only love.
Claudia was a little tired and contemptuous of the hurrying, bee-like
gentleman who sips from many flowers with no distinct preference
for any bloom. Many such had buzzed around her, but she had kept
fast closed the petals of her heart. But Gilbert Currey was different;
yes, he was certainly different.
A pale-faced, vapid youth, the heir to a famous dukedom, was just
inside the door.
“Quick, that’s my real partner. He’ll grab me.”
“He won’t,” said Gilbert firmly. He caught her to him a little
fiercely, with all the primitive man in him awake. His mother’s
warning about the bad stock from which Claudia sprang was
forgotten. He had decided that Claudia was his. He, and he only, was
going to grab her and carry her off to his Wigwam. His wife would
never want to be a Circe. Geoffrey Iverson had never been worth
much as a husband. Like most men, Gilbert had his fair share of
conceit.
He guided her skilfully round the room, interposing himself and his
arm between her and possible collisions, for the room was
inconveniently crowded. She happily forgot the rest of the world and
gave herself up to the sensuous music. But some of the gay spirit
with which she had danced earlier in the evening had gone from her,
a slight languor, more than a little pleasant, had stolen into her
veins. The music seemed a lullaby to send her brain to sleep. She
liked to feel the pressure of Gilbert’s arm and know that it enclosed
her safely. She had danced with him before on one or two other
occasions; but to-night his arm seemed to caress her. There was a
curious charm in it and she abandoned herself to it. She had never
before danced with anyone who had given her this sensation.
 And Gilbert felt the blood rushing through his veins as he would
have thought impossible an hour ago. The knowledge of her liking,
her nearness to him, seemed to make a little hammer pound away in
his head, so that he had to set his teeth not to let himself get giddy.
And Gilbert, when roused, had a good deal of the masculine animal
in him, only he was so seldom roused. When he was a youth at
Oxford his very clear and reasonable brain had warned him of a
possible danger to his working powers in the delights of the flesh,
and he had made himself not think about them by grinding away at
his books. His work and his intellect had become an almost
invulnerable armour. But to-night passion took him by the throat and
he could think of nothing but the lissome pretty body in his arms.
And his intellect, not quite drugged, approved of this diversion. His
mother had said it was time to marry. Why not combine pleasure
and duty? His reason quite approved of this proceeding.
“Claudia,” he said breathlessly, coming to a standstill, “it’s
confoundedly hot in here. Don’t you feel it. Shall we—shall we try for
some fresh air?”
She nodded, she did not want to speak. A beautiful dream had
been roughly broken into. She had been happy in her unsubstantial
dream; he—had not.
Gilbert was lucky enough to find an untenanted cosy corner in a
convenient angle that cut them off from the rest of the world.
“Claudia, will you?” His arm was round the back of the couch
ready to take her in his arms.
“Will I—what?” faltered the girl. She knew what he would ask, but
she had not imagined being proposed to thus. She had thought the
man she could love would lead up gradually with protestations, with
promises, with entreaties. Why did there seem no time for this? Why
did something hurry her into his arms, something irresistibly
compelling, stronger than herself?
“Will you marry me?” She tried to raise her eyes to his, and
perhaps he caught a glimpse of what was in them for the next
instant she felt his lips on hers, and the world rocked and then stood
still.

Afterwards, she wished that it had been more as her imagination

had planned. Though every pulse in her body still throbbed with his
kisses, she yet vaguely regretted that Prince Charming had not come
in the guise she had imagined. But that it was the real Prince
Charming—in somewhat of a hurry and a little inarticulate—she did
not doubt for a moment.
“But nothing is just as one imagines it will be,” said Claudia to
herself and the pillow that night. And having discovered that truth,
Nature kindly pulled down the blinds and she went to sleep.

CHAPTER IV
A TOY MOTHER

Later the same day Claudia awakened to the sound of snortings

and snufflings. Exasperated puffings sounded in her ear. For an
instant she dreamed that she was being pursued down a long road
by an angry motor-car bent on her destruction. It came nearer and
nearer—now it was quite close—she put out her hand in vague
dreamlike fashion to push it away. The motor-car retreated
somewhere at her touch, but returned in a few seconds to make a
fresh onslaught. Then something soft and velvety—obviously not a
motor-car—rubbed up against her cheek. Claudia came back from
the world of Nowhere in Particular to her own room in Grosvenor
Square.
To whom could those snortings and snufflings belong save Billie,
her beloved dachshund!
 Claudia yawned lazily. Billie gave another tug to the brown plait on
her pillow. That was always Billie’s way of engaging her attention in
the morning. Extraordinary, how long these superior mortals took to
awake in the morning when they were always so bright and fond of
pulling his ears at night!
The outline of the map of the world was still blurred for his
mistress when she vaguely remembered that something very
pleasant had happened to her. What was it that made her open her
eyes that sense of bien être?
“Oh!... Oh! Billie!” She turned on her elbow and kissed Billie’s silky
brown coat with unusual fervour. He was the most delightful thing in
dachshunds, with a coat like sealskin, only softer and warmer, and
the most pathetic and companionable eyes in the world. He was
exclusively devoted to Claudia, who, in return, gave him a big corner
of her heart. To the rest of the family he was a little elusive and
aloof, rather bored with their desultory attentions, occasionally very
busy with his own thoughts and affairs. Only Claudia’s hand gave
him real joy. Sometimes out of politeness he allowed Pat to think he
liked her petting, but that was because she was only a young thing
and Claudia was fond of her.
“Billie,” she said, with a rippling laugh of sheer happiness, “you
don’t know it, but I’m different from what I was yesterday morning.
I’m engaged!”
Billie regarded her seriously. He seemed to be digesting the news
and wondering what difference it would make to him.
“Yes,” continued his mistress, giving him a hug. “I’m engaged. I’ve
promised to marry someone very, very nice. Congratulate me, Billie.”
Billie rose to the situation and barked joyously.
“Thank you, sir. I am sure they were most sincere congratulations.
Heigh ho! we shall have to tell mother.... What do you say to
breakfast, eh?”
 She put her hand on the bell, and Billie blinked happily. He always
waited to take his breakfast with Claudia, and really she was very
late during the season.
“Billie, don’t rootle about in the bed like that. Be more respectful,
because I’m much more important to-day than I was yesterday.”
Then she lay back among the pillows and thought happily of Gilbert.
She longed to see him or hear from him. She hoped he would
telephone or perhaps send her some flowers on his way to his
chambers. She was certain he must be thinking of her just as she
was thinking of him.
She had a curious and not unpleasant feeling that last night she
had settled her whole life. She was like someone who had been
standing at the cross-roads awaiting an indication which turning to
take. Last night she had taken what she was sure was the right
turning. Now the road of her life seemed to stretch before her like a
glorious golden riband.
Yet, oddly enough, at the back of her mind was a sense of loss.
She had lost the right of making her choice, she had lost a certain
excited feeling that life was a great adventure. The adventure had
taken definite shape now like a fluid that has been poured into a
mould. Some of the delightful indecision, one of the biggest
“perhapses” of life had gone. She had always taken it for granted
that she would marry without making it her business to do so. She
had looked with soft, speculative eyes at the men she met. Perhaps
it will be this one—perhaps I shall sit next to him at dinner to-night
—perhaps he will be one of my partners at the dance to-morrow! A
girl who knows that she is attractive to men always has this feeling
consciously or unconsciously. Now this feeling had merged into
something else, the happy glow of knowledge. Love had come.
It seemed to Claudia that it had come rather suddenly, although
she had known Gilbert for many years. It was only the last month he
had seemed a “possible.” She remembered the exact moment that
the label had fixed itself upon him. She had been at a big dinner-
party, given by the wife of the Home Secretary, and the man who
took her in had talked all through the fish and the entrée about him.
That was before the Driver case, when he had definitely proved his
metal, but her dinner companion had been brought into contact with
him over some business and been greatly impressed with his ability.
Claudia had heard vaguely of Gilbert’s distinguished career at
Oxford, but the thumb-nail sketch which her companion drew of him
in his chambers arrested her attention. Then later that very evening
she had met him at a reception which her aunt, Lady Pitsea, gave.
Claudia had an almost Greek appreciation and love for physical
fitness, and had Gilbert not been a most personable man, her
interest in his mental achievements might have evaporated. But
because he was strong and came of healthy stock, the night-oil that
he had burned had so far left no mark upon him. There was no
doubt that he had personality, that he would never be overlooked
wherever he went. Claudia could never have married a handsome
man without brains, but it is doubtful if she could have loved
anything lacking in physical fitness. She demanded a certain amount
of beauty and colour in her life, just as she demanded a certain
amount of fresh air and food.
Until the reception they had not met for a couple of years, and he
showed unmistakably that he admired her. After that he seemed to
dwarf the other men with whom she ate and danced and talked.
That she did not meet him often at social gatherings—he was too
busy to go—whetted her appetite for his company. Sometimes he
would come in to some gathering with a little line of fatigue between
his brows. It had been an agreeable pastime to smooth it out by her
conversation and gaiety.
She realized this morning, as she stirred her coffee, that actually
they had talked very little. Not that he was a silent companion, but
they had always talked in crowded places of other people and
current events. Necessarily their talk had been largely on the surface
—a large surface, but yet only the surface of the things that matter.
She had never, since childhood’s days, been with him for many
consecutive hours. She had never, since those days, been alone with
him in the country, tramping side by side, or sitting for long, lazy
hours under the green trees. Claudia knew that such times bare the
man or woman of mannerisms and conventionalities, and expose the
real ego. Two or three times before she had thought she liked men,
but always on further and closer knowledge she had found them
disappointing. Then she had been annoyed with herself for even that
faint stirring of interest. In some unaccountable way she had felt
humiliated when her brain failed to approve of them. But Gilbert
could not disappoint her. How could such an admittedly clever man
disappoint any woman? She was glad he was going to have a career,
she saw herself helping him, entering into his thoughts and aims,
working and loving side by side. She was glad she had not fallen in
love with a nonentity or an idle, rich man.
She reflected that she would have hated to feel apologetic for her
husband. And yet she had seen girls of her own age, whom she
knew to be clever and even brilliant, marry men, and not for money
or position, who seemed to be absolutely devoid of the grey matter
we call brain. She had heard them rave rapturously over
commonplace males that bored her in twenty minutes, and she
knew that Love is a freakish thing. Fate might have played a joke on
her. “I wonder what it is exactly—this sex attraction?” she murmured
to the sleeping dachshund, and pigeonholed the question for future
investigation, when her mind was quite clear and at rest, for Gilbert
had urged a speedy marriage.
Gilbert’s love-making had been almost inarticulate. She wished he
had said something memorable, something she could enshrine in her
heart and when she was an old woman bring forth with a happy
smile—“Do you remember you said——” But Gilbert had hardly even
said the conventional Ich liebe dich. Ah! but his heart, beating
violently against her own, had said it. Claudia did not know that in
the crucial moment love and passion are indistinguishable, so she
had no doubt that his soul had spoken to hers.
Billie raised his head from the eiderdown and looked questioningly
at the door. Someone was approaching. A rap with something sharp
and hard followed.
“Can I come in, Claudia? Johnson said you were having your
breakfast.”
Claudia called out permission to enter, and a fair young Amazon,
riding-crop in hand, stalked into the room. It was Patricia Iverson,
generally called Pat, the youngest of the three children of Circe. Pat
was unusually tall, and in her long riding-habit she looked even taller
than usual. She was flushed with exertion, her fine, fair skin showed
almost startlingly against the black of her hat and habit.
“Bill, where are your manners? Why don’t you wag your tail? All
right, I shall wag it for you! What’s the good of being a dog with a
usable tail, if you don’t wave it when a lady enters the room? Oh! it
was spiffing in the Park this morning.”
“I am sure it was. I feel ashamed to be in bed, but I was so late
again this morning. Past four. Aren’t we fools to dance the night
away and spend the mornings in bed?”
“Yes,” said Patricia, disposing her long limbs in an easy chair. “But
I shall do it when I get the chance.”
“You ache to be dissipated?”
“Rather, because after dissipation you can appreciate virtue and—
a rest. Claudia, why are you smiling like a Cheshire cat this morning?
I hate people to smile like that unless they tell me the reason. It’s
like hearing the music of a dance you can’t go to.”
Claudia wondered if she would break the news to Pat. It was
strange, but there was nobody to whom she felt compelled to impart
her news. There was no one would quite understand and be glad
with her in her gladness. Pat was so young, and then you never
knew how she would take things. Sometimes she was as hard as
nails, and Claudia naturally felt she would like a sympathetic ear.
“I’ve been riding with Mr. Paton,” continued Patricia, pulling Billie’s
ear, a proceeding which he bore with the patience of an early
Christian martyr. “We had such a jolly gallop. He’s awfully nice, isn’t
he?”
“Very nice,” agreed Claudia heartily. She felt that the whole world
of men and women were nice this morning, but she could honestly
give Paton an emphatic adjective. “He’s a great friend of—of Gilbert
Currey’s.”
“He says such quaint things sometimes, and he isn’t a bit like
most men you meet. Do you know what we were talking about this
morning? We were discussing animals, and how far they feel human
emotion, and how much brain they’ve got. He’d been reading some
German book on the subject. He’s fond of animals. Oh! he sent you
a message.”
“Yes?” Claudia was wondering what the bond of sympathy was
between the two men.
“He told me to tell you that he’s ordered that book you wanted
from the publisher. And I am to convey an invitation for us both to
have tea with him to-day in Kensington Gardens. We don’t need
Jujubes.”
Jujubes was a disrespectful name applied to Miss Morrow who had
once been with them as governess, and had slid into position of
amiable General Utility. She could be used as a chaperon, walking-
stick, or sedative. Hence Pat’s nickname for her.
“I promised to go to some theatricals at Stretton House,” said
Claudia, grabbing her diary, “and, let me see—yes, I ought to go
with Aunt Carrie to call on some people.”
But her words were regretful. She would have loved to sit in the
Park and have tea under the trees, where the birds come hopping
round your chairs for crumbs, and everything around is green and
fragrant. It would have accorded so much better with her mood than
paying formal calls on people to whom she couldn’t tell the great
and important thing that had happened to her.
 “Don’t be a pig, Claudia. I’m not allowed to do much, and you
might say yes. Mr. Paton won’t want me without you.”
“Oh, yes, he would. Take Jujubes.”
“Pooh! he looks upon me as a flapper. Wait till mother gives me
some proper dresses and I begin to fill out. I look like the Bones of
the Holy Innocents now, but you wait till I get some curves. They
are beginning to come.”
She nodded her head knowingly, as she looked down at herself.
Claudia suddenly decided she would throw over Aunt Carrie. This
was a special day in her life, and she felt she ought to do just what
she wanted most. If only Gilbert could tea with them! She thought of
telephoning, and then some instinct warned her that Gilbert would
think it trivial. Gilbert not being available, Claudia found the idea of a
quiet sunny afternoon with Colin Paton quite pleasing. One never
had to be politely talkative and interested in him. One talked or one
didn’t talk, just as one pleased. Sometimes one found oneself talking
particularly well, helped by the right word or the appreciative smile.
Claudia thought of him in a sort of revolving roundabout with
Gilbert, as she took her bath, and tried to find the right word to
express him. The best she could get was “companionable,” although
she felt that was a little tepid.
When she was dressed she sent a message to her mother. She
must tell her the news. Sometimes Claudia did not see her for days
together, and they were in no sense mother and daughter, but
Claudia felt it was the proper thing to inform her at once. It had
always seemed to her friends that Mrs. Iverson was a mother merely
for the three weeks she had to remain an invalid. After that she
shook off her maternity.
The maid came back with the answer that Mrs. Iverson was
having her face massaged, but that Claudia could come to her.
Her mother’s bedroom and dressing-room suggested a hothouse
with a quantity of mirrors. Circe had always been something of an
exotic, and lately she had grown more so, or what Pat called
“stuffier.” There was an insidious Eastern perfume that always trailed
after Sybil Iverson, and the room Claudia entered was heavy with it.
The hangings and huge divan were Oriental in colouring and
material. The sun was excluded from the room by pink curtains
closely shrouding the windows, and electric lamps with becoming
shades were burning. Her mother was in the dressing-room,
prostrate under the hands of the masseur, who had a great
reputation among women, especially those who were on the
borderland of youth and middle age. He was ridiculously expensive,
but his hands were magical.
Mrs. Iverson lazily opened her closed eyelids and regarded
Claudia. Her eyes were still very beautiful. “You wanted to see me
dear?”
Claudia hesitated. “Yes, but——” If it had been Pat she would have
said cryptically “P and P”—private and particular.
“Well, Jules has nearly finished.” Mrs. Iverson was still beautiful,
but with a great effort. In her youth when the famous portrait had
been painted, she had been almost as fair as Patricia, but now her
hair was tinted auburn and her complexion was enamelled to match.
Her eyes—still marvellous—were of a deep shade of blue, like a
violet under the rays of the midday sun. Her mouth was much fuller
than Patricia’s; and told its own tale. Mrs. Iverson had always been
unutterably bored with her children, but she seemed to like or rather
dislike Claudia the least. Patricia annoyed her, because she was
reminded of her own lost freshness, and Jack she found stupid. She
really rather liked to talk to Claudia for a quarter of an hour or so.
Claudia was neither gauche nor ignorant. And her brown eyes, with
their purposeful gaze—well, some memories are pleasanter than
others, even to a Circe.
Claudia picked up the Occult Review, and tried to be interested in
it till her mother should be free.
At length Jules departed. Mrs. Iverson inspected the result in the
hand-mirror.
 “He’s a marvel. I hope he’ll still be alive when you want him.... I
like the cut of your skirt, Claudia. Who made it? Ah! I thought so.
She can cut skirts. Don’t you find her ruinous?”
It was a polite interrogation, as though to a stranger.
“Yes, I thought her more of a robber than usual,” continued her
mother. “I’m glad you haven’t got such long legs as Patricia. When
she comes toward me with her arm waving she reminds me of a
sign-post on a country road. It’s a pity. Men don’t like too long
women. You and I are just the right height. I think this modern girl
by the yard is a mistake. None of the famous women such as Jeanne
du Barry and Ninon de Lenclos were very tall. Patricia will make
most men look ridiculous.”
“Perhaps Pat doesn’t want to be a Ninon de Lenclos,” suggested
Claudia, with a twinkle.
“Nonsense, every woman wants to be a Ninon de Lenclos, if she
could have the chance. Don’t be taken in by this talk of ‘I wouldn’t.’
It’s a case of ‘I couldn’t.’ Most women have to be virtuous, because
they can’t be anything else, and they make the best of it. What’s
that American saying, ‘Virtue must be its own reward—any other
would be a tip.’ Do you know what Ninon said herself, ‘Love is a
passion, not a virtue: and a passion does not turn into a virtue
because it happens to last—it merely becomes a longer passion.’ ...
But what did you want to see me about?”
It should have been a propitious opening, this discussion of love,
but somehow it was not.
“I think—I think I ought to tell you something.”
“Don’t unless you want to,” said her mother quickly. “I don’t think
you ought to tell me anything. If you think it will interest me—tell
me, but don’t use me as a mother, please.”
“Would it interest you to know that I am engaged?” It was out.
Claudia breathed more freely. Then she blushed as her mother
looked at her with unusual attention.
 “Yes, that quite interests me. I have wondered once or twice what
sort of a man you would choose. Who is it?”
“Gilbert Currey, mother.”
“Gilbert C——yes, the M.P.’s son. Does something, doesn’t he? A
barrister? I remember his mother Marian Darby. She never liked me,
and I returned the compliment; but we were once great friends.
What made you choose her son?”
“Mother! I—I fell in love with him. Why do people marry?”
Circe smiled at her young daughter, who met her eyes quite
squarely but was obviously uncomfortable.
“For hundreds of reasons, my dear. You’ll find out some of them
later on. Of course, one must marry”—she retouched an eyebrow
with a little brush—“just as one must have a birth certificate and a
license for the motor.... I don’t think I’ve noticed him since he was a
boy. I remember him at Wynnstay. I used to see him in a canoe on
the river, deep in his books. Is he still strenuous and booky?”
“People say he is going to have a big career.” It was difficult to talk
to her mother.
“Really? And you want to be part of that career? Well, I daresay it
is all right. Better tell your father. I should think you might have
done better from a worldly point of view, though the Curreys are
rich, and Gilbert will succeed to the baronetcy.... You’ve really made
up your mind? Your aunt was telling me the other day that you are
considered one of the most attractive girls in Society to-day. She
mentioned a Russian prince of great renown—I forget his name——”
“He is fifty and has been married twice already.”
“Men grow more appreciative, not less so, as they get older. And
Russians are sometimes fascinating. I remember one—Russians can
be very wild and romantic.”
“I don’t want a wild and romantic husband.” Claudia laughed
outright.
 “No?—perhaps you are right. There is plenty of time, and I
daresay a Russian would not make a comfortable husband. Well,
child, I am glad if you are glad. I must meet my future son-in-law.”
She made a little grimace. “It adds at least five years to my age, but
I suppose I can’t ask you to consider me. I think he had better come
to dinner one night. Look in my engagement book and find a night.
Thursday—yes, that will do. Write down your name and Gilbert’s,
and then I shall remember all about it. One or two of the family
might be asked.” She gave her daughter a smile of dismissal. It was
very sweet, if a trifle automatic, and it showed to advantage her
perfect and natural teeth. Mrs. Iverson never kissed her children, but
then she thought kissing between women ridiculous. The only thing
she ever kissed of the female sex was a little toy terrier.
When Claudia went downstairs, relieved that the news had been
broken, she found the book had arrived that Colin Paton had
promised to obtain for her. She cut the string and dipped into it. It
was a volume of essays that he had mentioned to her and that she
had expressed a desire to read. Colin Paton never forgot things.
She looked from the book to the telephone and wished that
Gilbert had found time to ring her up and just say, “Hallo! Here am I
and there are you!” It would have seemed to make last night more
real, more sure. Like a puff of wind it crossed her mind that the
sender of the book would have somehow got in touch with the
woman he had asked to be his wife the night before. Pat liked him.
Perhaps he would marry Pat, she thought idly.
She was too keenly, too tinglingly alive for delicate essays that
morning. Later on she would enjoy them. She put them down and
picked up an illustrated paper.
The first thing that met her gaze was a portrait of Gilbert and a
paragraph recording his right to such a distinction.
There was no one in the library, and she raised it impulsively to
her lips. It was not a satisfactory kiss, for the paper smelt of
something nasty and oily. Still the portrait seemed to bring Gilbert
into the room with her. And this man was hers, this man at whom all
the Bar was looking, was hers, hers, hers!
Because she was only twenty-one, thoroughly healthy and full of
life, she danced round the room holding the paper to her breast. Her
eyes were alight with happiness; her soft lips were curved with the
joy of love and life.
Then having danced her little Te Deum to the music of her heart,
she waltzed out of the door with a cheery shout for Billie. She would
take him for a walk and give him joy, too.

CHAPTER V
GREEN BAY-LEAVES

Lady Currey was not at all pleased with her son’s engagement,
and she said so. She came to town for this purpose, and made
Gilbert give her lunch while she strongly disapproved, from the hors
d’œuvres to the coffee. She had the soulless good looks which Time,
as if contemptuous, neglects to touch. And because she could afford
to do so, she purposely dressed in a middle-aged, sober fashion
which she considered dignified. She had a great sense of her own
importance, and the modern grandmother of fifty in ninon and
picture-hats was to her extreme anathema. She and Circe were
much the same age. Sybil Daunton-Pole had flashed into society like
a brilliant comet, a trail of admirers behind her, when Gilbert’s
mother, the amiable daughter of the then Home Secretary, had been
one of the small and unremarked stars that dot the social firmament.
Lady Currey had brought her husband a considerable sum of
money, but the only thing for which she needed money was to
gratify her craze for old china. If she had any heart or soul it was
given to her specimens of priceless Ming and old Chelsea. She spent
hours every day dusting her cabinets. Her only idea of travel was the
opportunity it gave her for visiting museums and picking up bargains
in rare porcelain.
For Gilbert she had a pleasant feeling of proprietorship—much the
same as she felt for the wonderful famille rose-jar of the Kien-Lung
period which she had herself unearthed in a visit to the East. Gilbert
was an only child, and he had been little or no trouble. This was the
first time he had disappointed her. When other mothers complained
of their sons, of escapades at Eton and Oxford, or premature and
undesirable love affairs, of monumental debts and lack of family
pride, Lady Currey’s lips always took on an added shade of
complacency as she thought of Gilbert and the even and admirable
tenour of his way. It was entirely becoming that Gilbert should be so
satisfactory and in some way reflected well on herself, just as did the
discovery of the famille rose-jar. Lady Currey liked everything around
her to be comme il faut, not the elastic comme il faut of fashion, but
rather the correctness of the copybook and the ten commandments.
Curiously enough, engrossed in herself and her china, she had never
until quite recently speculated, as do most mothers, on her son’s
probable choice of a wife. When she had thought of it, she had
dismissed the idea with the assurance that Gilbert would choose
wisely and soberly and to his advantage. It was not in her to feel
any jealousy of the woman Gilbert should love.
“I am grieved,” she said, sitting very upright—she rarely used the
back of a chair—“I am grieved to think that you intend to marry into
the Iverson family. The Iversons are not a family of which I—or any
right-thinking people—approve.”
“But, mother,” said Gilbert, rather taken aback, for he had become
used to her invariable approval, “I am not marrying the family. I am
marrying Claudia.”
“Ah! that’s what you think—the usual reply. For Geoffrey Iverson I
have no particular dislike—he has been the cat’s-paw of a clever and
unscrupulous woman. His family is a very good one. She would have
spoilt any man who had the misfortune to be married to her. Why,
Sybil Iverson is notorious!”
 “Claudia is quite unlike her in every way. Why, she is not even like
her in appearance.”
Lady Currey lifted her thin, fair eyebrows. It was unbecoming that
she should tell him the scandalous rumours that floated about
respecting Claudia’s parentage: Such things could only be told by a
father to a son. She vehemently disapproved of any plain speaking
between the sexes. Such a crime could never be laid to her charge;
not even in the marital chamber had she ever discussed any such
thing.
“She is the daughter of her mother, Gilbert, and the mother—I say
it deliberately—is a bad woman, a woman who has trailed the glory
and purity of the flower of womanhood in the dust.” Lady Currey
occasionally indulged in such flights of rhetoric. She had rehearsed
this in the train.
“I don’t think the two women see much of one another.” Gilbert
was a little nettled. “Claudia told me herself that she hardly knew
her mother at all in her young days. She was left entirely to her
governesses. She can hardly have imbibed any—any idea from her
mother.”
The pathos of such an admission did not strike Lady Currey, it only
helped to justify her present attitude.
“It is, of course, very painful for me to have to mention such
matters to you, but why has she seen so little of her mother?
Because Sybil was—I blush to say it—so surrounded by lovers that
she neglected her maternal duties. I say again, she is notorious for
her lax life and morals. Don’t you believe in heredity, Gilbert? Think
of the blood that runs in that girl’s veins.”
Gilbert frowned. “Heredity is a curious thing. Not worth worrying
over, I think. I don’t profess to understand it.”
“I have studied the question.” She had read one book that was
quite out of date. “I firmly believe in heredity. The vices or the
virtues of the father and mother are surely transmitted to the
children.” It was pleasing to think that only virtues could be
transmitted to Gilbert, but it was all the more annoying that those
inherited virtues should be linked with the vices of Sybil Iverson’s
child.
Gilbert was becoming annoyed, and made no reply. After all, his
mother was only a woman, and women never could argue. It jarred
on his manhood that she should take him to task, and his voice was
a little cold as he inquired what she would take to drink.
“You know I always take one glass of claret.” The tone somehow
implied that a woman like Sybil Iverson might reprehensibly vary her
drink with lunch, but she had regular habits. Then she returned to
the attack.
“Claudia is not the woman that we—your dear father and I—would
have chosen for you.”
“Doesn’t every mother say that about her son’s choice?”
His mother sighed and waited while Gilbert ordered the wine.
“What sort of bringing-up has she had? What sort of a wife and
mother will such a girl make? Her mother’s only god was pleasure,
her only commandment ‘Enjoy the fleeting hour.’ Do you mean to tell
me that the daughter of such a woman has proper ideas about life?
Would you care to be the complaisant husband of a Circe?”
But here Gilbert put his foot down. His mother must be made to
see that he knew quite well what he was about, that he had not run
haphazard into this engagement. Not on any account would he let
her see that curious mixture of surprise and annoyance at the back
of his mind when he thought of the proposal scene. He had an
undefined feeling that he had been hurried into it, though how he
had been hurried, by whom or by what, he did not seek to explain
even to himself. To Gilbert’s cast of mind vague feelings were best
ignored as symptoms of a weak and illogical brain, much the same
as vague symptoms may denote an illness of the body. Still the
feeling was there, behind many stacks of docketed and pigeonholed
pieces of information. Yet he had almost made up his mind to
propose to Claudia—oh! yes—only—that particular night?
 “Mother, I cannot hear you say such ridiculous things about
Claudia. You do not know her. You might as well say that the
children of murderers will all grow up murderers.”
“You might commit murder in a sudden fit of passion, but such a
warped, degraded nature as Sybil Iverson’s is another story. Besides
—the sons of a murderer have probably seen him hanged or
punished—the law steps in; but who punishes a woman like Sybil
Iverson? Society, nowadays, is too lax to such creatures, and
virtuous women have to mix with them and take them by the hand,
or else be dubbed ridiculous or old-fashioned. Well,” with a sudden
little gust of passion like a disturbance in a tea-cup, “thank God, I
am old-fashioned and absurd. I can say my prayers every night and
lie down in peace.... No, Gilbert, you know I only take one glass of
claret.”
“They say Mrs. Iverson has given up her wicked, siren-like ways
and gone in for spiritualism.” He wished his mother realized that she
was keeping him from his work and would hurry up with her lunch.
The leisurely ways of the country were not those of town. But Lady
Currey was doing her duty.
“Such women never give up their wicked ways, they take them to
the grave with them.” Both Gilbert and his mother had very little
sense of humour, with the distinction that Gilbert knew when things
were ridiculous. “I know Sybil’s mother died of a broken heart.” This
was quite untrue, she had died of fatty degeneration of the liver.
“But there, the Psalms say that the wicked flourish like green bay-
trees, and if they did in King David’s time there is no doubt they do
now. But their punishment awaits them, Gilbert; always remember
that.”
Gilbert nodded absently. Life after death was one of the vague
things, like psychology, that he did not consider as practical politics.
But he did not tell his mother this. If she liked to imagine him
striving for a golden harp with humility of soul, she might.
 “I confess I am disappointed in you, Gilbert. I had looked forward
to your choosing some nice girl I could take to my heart, someone
like Maud Curtice, for example.”
Maud Curtice was a colourless girl who agreed with Lady Currey in
being shocked at the modern scanty fashion of dressing—she was
painfully thin and had ungainly hands and feet—and who devoted
herself to the mothers of eligible sons. She also had a large income.
“Wait till you know Claudia, mother. You are sure to like her.”
“I have heard she is very handsome and a great favourite in
Society,” returned his mother gloomily. “It is a bad report to my way
of thinking. That’s how her mother started.”
Just then, to his great relief, Gilbert caught sight of Colin Paton
wending his way out of the restaurant. He hailed him with joy, and
Paton came to a standstill beside their table.
Lady Currey approved of Colin Paton. His manners were respectful
and he showed an intelligent interest in china. She never noticed the
quizzical gravity with which he received her views on life, nor the
humorous twinkle in his eyes at her criticisms. She thought him “a
very nice young man.”
“Colin, old man, come and have some coffee with us.”
“Just had some. I hope you are quite well, Lady Currey?”
Gilbert made a business of looking at his watch and starting with
alarm. “By jove, I didn’t know it was so late. I must just swallow my
coffee and run. May I leave the mater with you to finish her coffee
at her leisure?”
Colin caught the appeal in Gilbert’s eyes and guessed the cause.
“Certainly, if Lady Currey will accept me as a poor substitute for
you.”
Lady Currey smiled a gracious assent. “I hope your dear mother is
better, Mr. Paton?”
 “Yes, thank you.... Busy as usual, Gilbert? I hear the proverbial
busy bee is quite out of it.”
“Well, I am tearingly busy. Don’t get a minute to myself.”
Paton slipped into his chair. “And yet you’ve found time to get
engaged, I hear? I wrote my congratulation this morning.”
“Thanks, old chap. Oh! getting engaged doesn’t take very long.”
Gilbert laughed pleasantly and displayed his firm white teeth.
“Doesn’t it?” returned Paton, smiling. “I think it would take me no
end of a time. But there, we shall soon be born in the morning,
married at midday, and buried in the evening!” He saw Lady Currey
looking at him rather doubtfully. “A man like your son, Lady Currey,
takes a woman and the world by storm. Veni, vidi, vici is not for me.
Women have to know me quite a long time before they remember
me.”
“I am sure you have a great many friends,” she said
encouragingly.
“Yes, that’s why I expect I shall never get a wife.... Really must
go, Gilbert? I had tea with Claudia and the long-legged Patricia
yesterday. We wished you could have been with us.”
“Teas are not in my line. I suppose I shall see you again soon?”
“Well, I’m going away, you know.”
Gilbert turned back in surprise.
“What, at the beginning of the season!” exclaimed Lady Currey.
“Going out to the Argentine for a while. A friend of mine is going
out on a political mission and wants an assistant. I’ve decided to
accompany him. Never been there, and it must be an interesting
country.”
Gilbert raised his eyebrows. Why on earth didn’t Paton stop in one
place and make a name for himself? He had often advised him to do
so.
 “Sudden isn’t it? I thought you said the other night that you were
remaining in town until the end of July.”
Paton nodded. “I’ve changed my mind. I think I want a change. I
shall only be away six months or so, perhaps a year.”
Gilbert’s thoughts had raced ahead. “Then if we’re married at the
end of July, as is probable, you’ll be away? That’s too bad. I had
relied on you for being best man.”
“You’ll be married so soon? No, I am afraid I can’t assist to give
you away.”
Gilbert again expressed his regrets, which were quite genuine, and
left his mother with Paton. Colin did not make the mistake of rushing
in where angels fear to tread, but waited for Lady Currey’s
comments.
“What do you think of this engagement, Mr. Paton? I know I can
speak to you quite frankly. I think it is a great mistake. Weren’t you
surprised?”
“Yes,” returned Paton truthfully, “I was very surprised. Gilbert did
not confide his hopes in me. I didn’t see any wooing going on, and
he never talked about her to me. He must have made the running
quickly.” Then he added, half to himself, “He can’t have seen a great
deal of her.”
“Of course not, or he wouldn’t have done it. Gilbert, for once in his
life, has lost his head over a pretty woman. Why, you are much more
of a friend than Gilbert.”
A slight shadow crossed her companion’s face and he dropped his
eyelids. “Well, I thought I was. But then friend—oh! it’s the veni,
vidi, vici trick. She’s a charming girl, Lady Currey, with all sorts of
possibilities.”
Lady Currey pursed up her thin lips that had never bestowed or
received a kiss of passion. “She is handsome, certainly. But is she
the wife for Gilbert? I have lived long enough to know that looks are
a poor foundation for matrimony.”
 “She has quite a good deal of character,” said her companion
quietly, without any annoying enthusiasm. “I am sure she will
develop into a splendid woman with the man she loves. She isn’t the
usual pretty society doll, you know.”
“Does it strike you that Gilbert wants a woman of character?”
asked his mother with unexpected acuteness. “Clever men are
usually better mated to stupid wives. Look at Carlyle and Jane
Welsh! Much too clever for one another.” Then irrelevantly, “There
are too many clever girls nowadays. I don’t believe they make any
the better wives and mothers for being so clever. I am sure I never
wanted such a daughter-in-law.”
Paton found himself at a loss for conversation. He knew he could
do Claudia no good by praising her warmly to her future mother-in-
law, he might even make matters worse. Yet to hear Claudia belittled
made something leap within him into fierce flame. It seemed disloyal
to listen to Lady Currey’s sneers. Yet he knew that Claudia must
storm the citadel of Lady Currey’s heart herself. As an advance agent
his labours would be wasted. But Paton, looking across the table into
the light, offended eyes of the woman, was sorry for the girl. It was
rather odd. His mother, a confirmed invalid, and Lady Currey had
been close friends in their youth. Yet his mother had warmly liked
Claudia when she had once met her for a few minutes. He was
startled to find that his current of thought had communicated itself
to Lady Currey.
“Your mother always did like pretty things—I know she admires
Claudia—but she was always unduly swayed by good looks, even at
school. I know how deceptive they are. A man told me the other day
that his wife had left him and been through the Divorce Court, and
he attributed it entirely to her good looks. ‘A very pretty woman is
difficult to live with,’ he said; ‘she gets a great deal of adulation and
flattery in Society, and naturally the husband at home falls rather
flat.’ There is a lot of truth in that, Mr. Paton.”
“Perhaps he was the typical English husband who, as soon as he
has won a wife, forgets to be her lover,” replied Paton. “You are very
careful and precious of your rare china, Lady Currey.”
His vis-à-vis stared. She wondered that Paton, who was usually so
smooth in conversation, should make such a sudden jump. But it
served to divert her mind from Claudia.
“I had such luck last week. I was walking along the High Street in
Moulton and I caught sight of a pair of vases. I thought that powder
blue could be nothing less than Chinese. They had blue and white
reserves on them. You know what that means. I got them for a mere
song, and they’re beauties. Since I last saw you I have bought....”
Still talking china, Paton saw her into a taxi.
He strolled away from the restaurant. It was warm and sunny, and
the pedestrians seemed all in a good humour. Paton often wandered
for hours through the streets of London, finding in that wonderful
panorama food for eyes and brain and heart. He loved the feeling
that he was part of the crowd, and his mind was stored with many
observations and memories. The romance of the streets was no idle
journalistic phrase to him. He felt it around him on all sides, plucking
at him with alluring fingers leading him into the land of dreams.
Often at night he would give himself wholly up to its enchantment,
wandering along mile after mile through quaint byways and on misty
commons, through silent Suburbia and the noisy, restless East-end
slums. London was to him a book of unending pages with countless
illustrations.
This afternoon he mingled with the crowd, but he did not heed it,
so that he did not see a woman in a motor energetically waving her
hand to him and directing the chauffeur to stop.
“Mr. Paton—oh! Mr. Paton, what a day-dream!”
It was Claudia herself, looking altogether charming in light
summer attire. There were waving, greeny-blue ostrich feathers in
her Leghorn hat and around her neck. The softness of the feathers
and the peculiar shade of blue accentuated the creamy tint of her
skin and the brightness of her eyes. Her happiness shone through
the envelope of the flesh like a flame through clear glass. A heavy-
eyed woman of the lower classes who was passing marked her and
muttered, “She has a good time, I’ll be bound,” then, wrapped in her
own bad one, passed on.
Paton went up to the car and held out his hand.
“Mr. Paton, you’re just the man I want. Do come and see some
pictures with me. Jujubes hates pictures, don’t you, Jujubes?” She
turned to the faded, amiable woman beside her in the car.
“I don’t hate them, but they all look so alike,” said Jujubes mildly.
“When you’ve seen one, it seems to me you’ve seen the lot.”
“There, listen to this awful heathen who rejoices in her darkness!
Leave me not to her tender mercies. Jujubes can do some shopping
for me.” She looked entreatingly at him with her fresh young mouth
smiling at herself, Jujubes, Paton and the whole world.
He hesitated for the fraction of a second. Then he said cheerily:
“Of course I’ll come, if only out of kindness to Miss Jujubes. And I
shan’t be seeing any more English pictures for a long time, I
suppose.” Then he told her of his intended visit to the Argentine.
“Oh!” said Claudia blankly. “Oh! I wish you weren’t going away. I
shall miss you so much—we shall all miss you.” She said it quite
naturally as the thought came to her mind. One could always do that
with Colin Paton.
“Thank you,” he said smilingly, as he helped Jujubes to alight. “It’s
very good of you to say so.” He seated himself beside Claudia.
“Don’t. You needn’t be formal and polite. Why are you going? Is it
the wanderlust again? Or is it to help you in your career?”
Gilbert had taught her to think of careers.
“Oh! I shall never have a career,” said Paton lightly, aware of the
soft, dark eyes on his face questioning him. But he did not meet
them. Somehow they held a look in them to-day that he could not
bear. “I don’t concentrate, you know. I’m just ‘a blooming amateur.’
Gilbert was reading me a solemn lecture the other day, but—I go on
the same old way. I’m glad, however, that Gilbert is getting on so
well. But then, he does concentrate.”
“He works very hard,” said Claudia thoughtfully, “I had no idea
how hard. He does too much, I think.” Then she looked at the rather
fine lines of the face beside her. “But I don’t believe you are afraid of
hard work. I remember how hard you worked when you were on
that Hospital Committee.”
“No, I don’t think it’s that,” said Paton quietly. “Let’s say it’s lack of
ambition and driving power.”
Was there something in his tone that sent a vague shadow of
distrust over Claudia’s expression, or was it the echo of some secret
misgiving in herself?
“Does that mean you think ambition—the ordinary get-to-the-top-
of-the-tree ambition—rather commonplace?”
“Not a bit,” he said heartily. “After all, we live on a commonplace
earth. Gilbert is right and I am wrong, and when Gilbert is Lord Chief
Justice and I’m an obscure old bore of a bachelor, I shall, no doubt,
fully realize my wrongness. But do ask me to dinner sometimes.”
“But you mustn’t remain a bachelor,” said Claudia, with all the
enthusiasm of the newly-engaged woman, “because your life will be
incomplete. That sounds like sex conceit, but you said it yourself to
me, and then I began to believe it. And now——” she completed the
sentence with a charming blush.
“Can you imagine any modern woman wanting a man without
worldly ambition, a man she will never be proud of, a man who is
nothing and does nothing?” The tone was light enough, and the girl,
engrossed in her own happiness, did not detect an unusual note of
bitterness. For Colin Paton was never bitter. He could be sarcastic
and even scathing when roused, but he never indulged in the refuge
of cowardly souls.
Claudia took him quite seriously, for happiness, just as sorrow,
may temporarily obscure a sense of humour. “I forbid you to say
such things of yourself,” she said, with an engaging air of
motherliness. “You’re awfully clever—awfully clever. Why, you are
one of the best-read and best-informed men in London.” Suddenly
she realized how often she had turned to him for information or
advice. And she could never remember an occasion on which he had
failed her, or an opinion that her critical faculty on reflection deemed
unsound.
“No market value, dear lady.”
She paused a moment thoughtfully. “Is that true?” she said slowly.
“Gilbert said that the other day when I asked him if he had read
something. He says he has no time for books, it’s as much as he can
do to read the newspapers.... Somehow it seems all wrong.” She
looked away with a puzzled expression at the trees of the Park.
He cast a quick glance at her profile and the beautiful lines of her
throat. He seemed about to say something with unusual impetuosity,
and then he resolutely locked his lips. He allowed her to go on
speaking.
“Ambition gets in the way of—of a lot of other things, doesn’t it? It
seems a voracious dragon, swallowing up everything: friends, books,
pictures—all the beautiful, graceful things of life. Isn’t it a pity?”
“I think so; but then I’m in the minority.”
“And that’s why you are not ambitious,” she flashed out with
sudden insight. “Yes, I see. I wonder if you are right.” Her voice was
a little wistful.
“No,” he said, with resolute reassurance. “No. I’m wrong, and
Gilbert is right. Wife of the Lord Chief Justice—what greater honour
could you wish?”
“Now you are making fun of me,” she replied, with a tiny frown,
“and I was quite serious. It’s difficult to explain. But—well, I hate the
usual sort of man who does nothing except wear his clothes well,
don’t you? Look at Jack. He sets off his uniform beautifully, but he
just footles his life away. There doesn’t seem anything between that
and great strenuosity—except you. I can’t place you. Somehow you
always make me see things in a different perspective from anyone
else. I wonder why it is. Sometimes you make things seem better
and sometimes you make them seem worse.”
He drew in his breath a little and his hand in its thin suède
covering clenched itself on his knee. “Claudia, you mustn’t let me
make things seem worse or any different from—what they are. I’d
be content if my mission in life were to make things better, not
worse, for you. Not that you want that now,” he added hastily,
pulling himself in. “I know, from things you have left unsaid, that
your home life hasn’t been all you wanted and ought to have had,
but now—now you are going to be very happy. Gilbert is a splendid
fellow.”
She turned to him, her face glowing, her eyes deep and dark with
emotion.
“Yes, I think I am going to be very happy. Somehow you have
always understood. I have never had to tell you things. You see,
nobody ever wanted me very much, and I—I wanted somebody to
want me and to rely on me and care for my companionship. It is so
wonderful to think that our interests are one, that what interests me
interests him, that I can tell him my good news and bad news and
be always sure that I don’t bore him. I’ve always had to bottle up
things. I’ve had one or two girl friends, but it isn’t the same. And
even then they get engaged and married and you fall in the
background. But when I’ve got a husband of my own it will be
different, won’t it?”
He hesitated the fraction of a second. “Yes, Claudia, it will be
different. You know how glad I am that you have found happiness,
don’t you? I wanted that so much for my—friend.”
“And isn’t it nice that I am marrying your friend?” she exclaimed
joyfully. “Because you might not have liked my husband, or my
husband might not have liked you. Oh, I know,” sagely. “I have
heard from my friends who got married, that it is sometimes very