Download the full version of the textbook now at textbookfull.

com

Python Programming and Numerical Methods: A

Guide for Engineers and Scientist 1st Edition
Qingkai Kong

https://textbookfull.com/product/python-
programming-and-numerical-methods-a-guide-for-
engineers-and-scientist-1st-edition-qingkai-kong/

Explore and download more textbook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Numerical Methods for Engineers and Scientists Using

MATLAB Second Edition Esfandiari

https://textbookfull.com/product/numerical-methods-for-engineers-and-
scientists-using-matlab-second-edition-esfandiari/

textbookfull.com

Programming For Computations - Python: A Gentle

Introduction To Numerical Simulations With Python 3.6
Svein Linge
https://textbookfull.com/product/programming-for-computations-python-
a-gentle-introduction-to-numerical-simulations-with-python-3-6-svein-
linge/
textbookfull.com

Numerical Methods for Engineers 8ed 2021 8th Edition

Chapra S

https://textbookfull.com/product/numerical-methods-for-
engineers-8ed-2021-8th-edition-chapra-s/

textbookfull.com

Henry Miller and Modernism: The Years in Paris, 1930–1939

Finn Jensen

https://textbookfull.com/product/henry-miller-and-modernism-the-years-
in-paris-1930-1939-finn-jensen/

textbookfull.com
Beauty and the Alpha 1st Edition Liliana Carlisle

https://textbookfull.com/product/beauty-and-the-alpha-1st-edition-
liliana-carlisle/

textbookfull.com

The SAGE Handbook Of Qualitative Research Ethics Ron

Iphofen

https://textbookfull.com/product/the-sage-handbook-of-qualitative-
research-ethics-ron-iphofen/

textbookfull.com

Antisemitism Before and Since the Holocaust: Altered

Contexts and Recent Perspectives 1st Edition Anthony
Mcelligott
https://textbookfull.com/product/antisemitism-before-and-since-the-
holocaust-altered-contexts-and-recent-perspectives-1st-edition-
anthony-mcelligott/
textbookfull.com

Alice in Wonderland Syndrome Jan Dirk Blom

https://textbookfull.com/product/alice-in-wonderland-syndrome-jan-
dirk-blom/

textbookfull.com

Practical Point-of-Care Medical Ultrasound 1st Edition

James M. Daniels

https://textbookfull.com/product/practical-point-of-care-medical-
ultrasound-1st-edition-james-m-daniels/

textbookfull.com
Engineering Applications of Discrete Element Method:
Operation Analysis and Optimization Design of Coal and
Agricultural Machinery Xuewen Wang
https://textbookfull.com/product/engineering-applications-of-discrete-
element-method-operation-analysis-and-optimization-design-of-coal-and-
agricultural-machinery-xuewen-wang/
textbookfull.com
 Python Programming and
Numerical Methods
A Guide for Engineers and Scientists

Qingkai Kong
Timmy Siauw
Alexandre M. Bayen
 Python Programming and
Numerical Methods
A Guide for Engineers and Scientists
Academic Press is an imprint of Elsevier
125 London Wall, London EC2Y 5AS, United Kingdom
525 B Street, Suite 1650, San Diego, CA 92101, United States
50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States
The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom
Copyright © 2021 Elsevier Inc. All rights reserved.

MATLAB® is a trademark of The MathWorks, Inc. and is used with permission.

The MathWorks does not warrant the accuracy of the text or exercises in this book.
This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by
The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying, recording, or any information storage and retrieval system, without permission in writing from the
publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our
arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found
at our website: www.elsevier.com/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may
be noted herein).
Notices
Knowledge and best practice in this field are constantly changing. As new research and experience broaden our
understanding, changes in research methods, professional practices, or medical treatment may become necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any
information, methods, compounds, or experiments described herein. In using such information or methods they should be
mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any
injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or
operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN: 978-0-12-819549-9

For information on all Academic Press publications

visit our website at https://www.elsevier.com/books-and-journals

Publisher: Katey Birtcher

Acquisitions Editor: Steve Merken
Editorial Project Manager: Alice Grant
Production Project Manager: Kamesh Ramajogi
Designer: Miles Hitchen
Typeset by VTeX
To Fanyu, Fanqi, and Fan without whom this book would have been finished
¨ To Xianzhong, and Ping!
earlier 
To the students of UC Berkeley’s E7 class: past, present, and future.
To my daughter Myriam,
who is discovering the world of programming.
Contents

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

PART 1 INTRODUCTION TO PYTHON PROGRAMMING

CHAPTER 1 Python Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Getting Started With Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.1 Setting Up Your Working Environment . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.2 Three Ways to Run Python Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Python as a Calculator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3 Managing Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.3.1 Managing Packages Using Package Managers . . . . . . . . . . . . . . . . . . . 15
1.3.2 Install Packages From Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4 Introduction to Jupyter Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4.1 Starting the Jupyter Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4.2 Within the Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.4.3 How Do I Close a Notebook? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.4.4 Shutting Down the Jupyter Notebook Server . . . . . . . . . . . . . . . . . . . . 20
1.5 Logical Expressions and Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
CHAPTER 2 Variables and Basic Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1 Variables and Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Data Structure – String . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3 Data Structure – List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.4 Data Structure – Tuple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.5 Data Structure – Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.6 Data Structure – Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.7 Introducing NumPy Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.8 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.8.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.8.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
CHAPTER 3 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.1 Function Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.1.1 Built-In Functions in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.1.2 Define Your Own Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.2 Local Variables and Global Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
vii
viii Contents

3.3 Nested Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.4 Lambda Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.5 Functions as Arguments to Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
CHAPTER 4 Branching Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.1 If-Else Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.2 Ternary Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.3 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.3.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.3.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
CHAPTER 5 Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.1 For-Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2 While Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.3 Comprehensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.3.1 List Comprehension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.3.2 Dictionary Comprehension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.4 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.4.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.4.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
CHAPTER 6 Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.1 Recursive Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.2 Divide-and-Conquer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.2.1 Tower of Hanoi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.2.2 Quicksort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6.3 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.3.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.3.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
CHAPTER 7 Object-Oriented Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.1 Introduction to OOP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.2 Class and Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.2.1 Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.2.2 Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7.2.3 Class vs Instance Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.3 Inheritance, Encapsulation, and Polymorphism . . . . . . . . . . . . . . . . . . . . . . . 127
7.3.1 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
7.3.2 Encapsulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.3.3 Polymorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.4 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.4.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.4.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
 Contents ix

CHAPTER 8 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

8.1 Complexity and Big-O Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
8.2 Complexity Matters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
8.3 The Profiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.3.1 Using the Magic Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.3.2 Use Python Profiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
8.3.3 Use Line Profiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
8.4 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.4.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.4.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
CHAPTER 9 Representation of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
9.1 Base-N and Binary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
9.2 Floating Point Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
9.3 Round-Off Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
9.3.1 Representation Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
9.3.2 Round-Off Error by Floating-Point Arithmetic . . . . . . . . . . . . . . . . . . 152
9.3.3 Accumulation of Round-Off Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
9.4 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.4.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.4.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
CHAPTER 10 Errors, Good Programming Practices, and Debugging . . . . . . . . . . . . . . . . 157
10.1 Error Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
10.2 Avoiding Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
10.2.1 Plan Your Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
10.2.2 Test Everything Often . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
10.2.3 Keep Your Code Clean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
10.3 Try/Except . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
10.4 Type Checking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
10.5 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
10.5.1 Activating Debugger After Running Into an Exception . . . . . . . . . . . 168
10.5.2 Activating Debugger Before Running the Code . . . . . . . . . . . . . . . . 171
10.5.3 Add a Breakpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
10.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
10.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
10.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
CHAPTER 11 Reading and Writing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
11.1 TXT Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
11.1.1 Writing to a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
11.1.2 Appending a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
11.1.3 Reading a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
11.1.4 Dealing With Numbers and Arrays . . . . . . . . . . . . . . . . . . . . . . . . . 178
11.2 CSV Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
11.2.1 Writing and Opening a CSV File . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
x Contents

11.2.2 Reading a CSV File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

11.2.3 Beyond NumPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
11.3 Pickle Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
11.3.1 Writing to a Pickle File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
11.3.2 Reading a Pickle File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
11.3.3 Reading in Python 2 Pickle File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
11.4 JSON Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
11.4.1 JSON Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
11.4.2 Writing a JSON File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
11.4.3 Reading a JSON File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
11.5 HDF5 Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
11.5.1 Reading an HDF5 File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
11.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
11.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
11.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
CHAPTER 12 Visualization and Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
12.1 2D Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
12.2 3D Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
12.3 Working With Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
12.4 Animations and Movies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
12.5 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
12.5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
12.5.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
CHAPTER 13 Parallelize Your Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
13.1 Parallel Computing Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
13.1.1 Process and Thread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
13.1.2 Python’s GIL Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
13.1.3 Disadvantages of Using Parallel Computing . . . . . . . . . . . . . . . . . . . 227
13.2 Multiprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
13.2.1 Visualize the Execution Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
13.3 Using Joblib . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
13.4 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
13.4.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
13.4.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

PART 2 INTRODUCTION TO NUMERICAL METHODS

CHAPTER 14 Linear Algebra and Systems of Linear Equations . . . . . . . . . . . . . . . . . . . 235
14.1 Basics of Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
14.1.1 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
14.1.2 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
14.1.3 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
14.2 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
14.3 Systems of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
 Contents xi

14.4 Solutions to Systems of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

14.4.1 Gauss Elimination Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
14.4.2 Gauss–Jordan Elimination Method . . . . . . . . . . . . . . . . . . . . . . . . . 250
14.4.3 LU Decomposition Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
14.4.4 Iterative Methods – Gauss–Seidel Method . . . . . . . . . . . . . . . . . . . . 253
14.5 Solving Systems of Linear Equations in Python . . . . . . . . . . . . . . . . . . . . . . 256
14.6 Matrix Inversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
14.7 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
14.7.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
14.7.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
CHAPTER 15 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
15.1 Eigenvalues and Eigenvectors Problem Statement . . . . . . . . . . . . . . . . . . . . 265
15.1.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
15.1.2 The Motivation Behind Eigenvalues and Eigenvectors . . . . . . . . . . . 265
15.1.3 The Characteristic Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
15.2 The Power Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
15.2.1 Finding the Largest Eigenvalue . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
15.2.2 The Inverse Power Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
15.2.3 The Shifted Power Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
15.3 The QR Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
15.4 Eigenvalues and Eigenvectors in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
15.5 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
15.5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
15.5.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
CHAPTER 16 Least Squares Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
16.1 Least Squares Regression Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . 279
16.2 Least Squares Regression Derivation (Linear Algebra) . . . . . . . . . . . . . . . . . 280
16.3 Least Squares Regression Derivation (Multivariate Calculus) . . . . . . . . . . . . 281
16.4 Least Squares Regression in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
16.4.1 Using the Direct Inverse Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
16.4.2 Using the Pseudo-Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
16.4.3 Using numpy.linalg.lstsq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
16.4.4 Using optimize.curve_fit From SciPy . . . . . . . . . . . . . . . . . . . . . 285
16.5 Least Squares Regression for Nonlinear Functions . . . . . . . . . . . . . . . . . . . . 286
16.5.1 Log Tricks for Exponential Functions . . . . . . . . . . . . . . . . . . . . . . . 286
16.5.2 Log Tricks for Power Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
16.5.3 Polynomial Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
16.5.4 Using optimize.curve_fit From SciPy . . . . . . . . . . . . . . . . . . . . . 289
16.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
16.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
16.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
CHAPTER 17 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
17.1 Interpolation Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
xii Contents

17.2 Linear Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

17.3 Cubic Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
17.4 Lagrange Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
17.4.1 Using the Lagrange Function From SciPy . . . . . . . . . . . . . . . . . . . . 304
17.5 Newton’s Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
17.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
17.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
17.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
CHAPTER 18 Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
18.1 Expressing Functions Using a Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . 315
18.2 Approximations Using Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
18.3 Discussion About Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
18.3.1 Truncation Errors for Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . 320
18.3.2 Estimating Truncation Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
18.3.3 Round-Off Errors for Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . 322
18.4 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
18.4.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
18.4.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
CHAPTER 19 Root Finding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
19.1 Root Finding Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
19.2 Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
19.3 Bisection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
19.4 Newton–Raphson Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
19.5 Root Finding in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
19.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
19.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
19.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
CHAPTER 20 Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
20.1 Numerical Differentiation Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 337
20.2 Using Finite Difference to Approximate Derivatives . . . . . . . . . . . . . . . . . . . 338
20.2.1 Using Finite Difference to Approximate Derivatives With Taylor
Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
20.3 Approximating of Higher Order Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . 344
20.4 Numerical Differentiation With Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
20.5 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
20.5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
20.5.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
CHAPTER 21 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
21.1 Numerical Integration Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 353
21.2 Riemann Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
21.3 Trapezoid Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
21.4 Simpson’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
21.5 Computing Integrals in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
 Contents xiii

21.6 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

21.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
21.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
CHAPTER 22 Ordinary Differential Equations (ODEs) Initial-Value Problems . . . . . . . . . . 371
22.1 ODE Initial Value Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
22.2 Reduction of Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
22.3 The Euler Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
22.4 Numerical Error and Instability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
22.5 Predictor–Corrector and Runge–Kutta Methods . . . . . . . . . . . . . . . . . . . . . . 382
22.5.1 Predictor–Corrector Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
22.5.2 Runge–Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
22.6 Python ODE Solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
22.7 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
22.7.1 Multistep Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
22.7.2 Stiffness ODE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
22.8 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
22.8.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
22.8.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
CHAPTER 23 Boundary-Value Problems for Ordinary Differential Equations (ODEs) . . . . . 399
23.1 ODE Boundary Value Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 399
23.2 The Shooting Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
23.3 The Finite Difference Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
23.4 Numerical Error and Instability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
23.5 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
23.5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
23.5.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
CHAPTER 24 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
24.1 The Basics of Waves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
24.1.1 Modeling a Wave Using Mathematical Tools . . . . . . . . . . . . . . . . . . 415
24.1.2 Characteristics of a Wave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
24.2 Discrete Fourier Transform (DFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
24.2.1 DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
24.2.2 The Inverse DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
24.2.3 The Limit of DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
24.3 Fast Fourier Transform (FFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
24.3.1 Symmetries in the DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
24.3.2 Tricks in FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
24.4 FFT in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
24.4.1 FFT in NumPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
24.4.2 FFT in SciPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
24.4.3 More Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
24.5 Summary and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
24.5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
xiv Contents

24.5.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

Appendix A Getting Started With Python in Windows . . . . . . . . . . . . . . . . . . . . . . . . . . 445
A.1 Getting Started With Python in Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
A.1.1 Setting Up Your Working Environment in Windows . . . . . . . . . . . . . . 445
A.1.2 Three Ways to Run Python Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
List of Figures

Fig. 1.1 The Miniconda download page; choose the installer based on your operating system. 4
Fig. 1.2 Screen shot of running the installer in a terminal. 5
Fig. 1.3 The default installation location of your file system. 5
Fig. 1.4 Quick way to check if Miniconda was installed successfully and the programs are run properly. 6
Fig. 1.5 Installation process for the packages that will be used in the rest of the book. 6
Fig. 1.6 Run “Hello World” in IPython shell by typing the command. “print” is a function that is
discussed later in the book that will print out anything within the parentheses. 7
Fig. 1.7 Example of a Python script file example using Visual Studio Code. Type in the commands you
want to execute and save the file with a proper name. 8
Fig. 1.8 To run the Python script from command line, type “python hello_world.py”. This line tells
Python to execute the commands that saved in this file. 8
Fig. 1.9 To launch a Jupyter notebook server, type jupyter notebook in the command line, which
will open a browser page as shown here. Click the “New” button on the top right and choose
“Python3”. This will create a Python notebook from which to run Python code. 9
Fig. 1.10 To run the Hello World example within Jupyter notebook, type the command in the code cell
(the grey boxes) and press Shift + Enter to execute it. 10
Fig. 1.11 An example demonstrating the interactive search for functions within IPython by typing TAB
after the dot. The grey box shown all available functions. 12
Fig. 1.12 The help document of pip after executing pip help. 16
Fig. 1.13 Using pip list to show all the packages installed on your machine. 17
Fig. 1.14 Using pip show to get detailed information about a installed package. 17
Fig. 1.15 The Jupyter notebook dashboard after launching the server. Red arrows (light grey arrows in
print version) are pointing to you the most common features in the dashboard. 19
Fig. 1.16 A quick view of a notebook. The Header of the notebook shows the name of the notebook. The
menu has various drop-down lists that let you access to all the functionalities of the notebook.
The tool bar provides you some shortcuts for the commonly used functionalities. 20
Fig. 1.17 Truth tables for the logical and/or. 22
Fig. 1.18 Truth tables for the logical XOR. 25
Fig. 2.1 String index for the example of "Hello World". 31
Fig. 2.2 Example of list index. 36
Fig. 5.1 Interrupt the kernel by pressing the little square. 99
Fig. 6.1 Recursion tree for factorial(3). 107
Fig. 6.2 Recursion tree for factorial(5). 108
Fig. 6.3 Illustration of the Tower of Hanoi: In eight steps, all disks are transported from pole 1 to pole 3,
one at a time, by moving only the disk at the top of the current stack, and placing only smaller
disks on top of larger disks. 111
Fig. 6.4 Breakdown of one iteration of the recursive solution of the Tower of Hanoi problem. 111
Fig. 6.5 Pascal’s triangle. 118
Fig. 8.1 Illustration of running time for complexity log(n), n, and n2 . 138
xv
xvi List of Figures

Fig. 8.2 The profiling result from prun. 141

Fig. 8.3 The line by line profiling result from line_profiler. 142
Fig. 9.1 Binary addition. 146
Fig. 9.2 Binary multiplication. 147
Fig. 9.3 Illustration of −12.0 that is represented in computer with 64-bit. Each square is one bit, with the
green square representing 1 and grey square as zero. 148
Fig. 11.1 The content in the text file we write. 177
Fig. 11.2 Append a line to the end of an existing file. 177
Fig. 11.3 The NumPy array we saved in the file. 179
Fig. 11.4 Open the csv file using Microsoft Excel. 181
Fig. 11.5 Open the csv file using a text editor. 181
Fig. 12.1 Plot of x versus y. 190
Fig. 12.2 Plot of the function f (x) = x 2 for −5 ≤ x ≤ 5. 191
Fig. 12.3 Plot of the function f (x) = x 2 for −5 ≤ x ≤ 5 using dashed line. 192
Fig. 12.4 Using different colors and markers. 192
Fig. 12.5 Adding title and axis labels to a figure. 193
Fig. 12.6 Using seaborn-poster style. 194
Fig. 12.7 Using a legend in the figure. 195
Fig. 12.8 Changing limits of the figure and turning on grid. 196
Fig. 12.9 Different type of plots. 198
Fig. 12.10 Saving a figure. 199
Fig. 12.11 3D axes. 200
Fig. 12.12 3D scatter plot. 202
Fig. 12.13 3D surface plot. 204
Fig. 12.14 3D wireframe and surface plot. 205
Fig. 12.15 Simple world map. 206
Fig. 12.16 World map with background. 207
Fig. 12.17 Simple US map. 208
Fig. 12.18 US map with various features. 209
Fig. 12.19 Bay area map with higher resolution. 210
Fig. 12.20 Add more entities on the map. 211
Fig. 13.1 Parallel computing vs. serial computing. 226
Fig. 13.2 The hardware on the author’s laptop with multicore processor. 226
Fig. 14.1 Graph for Problem 12. 263
Fig. 16.1 Illustration of the L2 projection of Y on the range of A. 281
Fig. 17.1 Illustration of the interpolation problem: estimate the value of a function in-between data
points. 295
Fig. 19.1 Illustration of the intermediate value theorem. If sign(f (a)) and sign(f (b)) are not equal, then
there exists a c in (a, b) such that f (c) = 0. 327
Fig. 19.2 Illustration of the bisection method. The sign of f (m) is checked to determine if the root is
contained in the interval (a, m) or (m, b). This new interval is used in the next iteration of the
bisection method; in the case depicted in the figure, the root is in the interval (m, b). 328
Fig. 19.3 Illustration of Newton step for a smooth function, g(x). 330
Fig. 20.1 Numerical grid used to approximate functions. 337
 List of Figures xvii

Fig. 20.2 Finite difference approximation of the derivative. 339

Fig. 20.3 Illustration of the forward difference, the backward difference, and the central difference. Note
the difference in slopes depending on the method used. 341
Fig. 21.1 Illustration of the integral. The integral from a to b of the function f is the area below the curve
(shaded in grey). 353
Fig. 21.2 Illustration of the trapezoid integral procedure. The area below the curve is approximated by a
sum of areas of trapezoids that approximate the function. 357
Fig. 21.3 Illustration of the Simpson integral formula. Discretization points are grouped by three, and a
parabola is fit between the three points. This can be done by a typical interpolation polynomial.
The area under the curve is approximated by the area under the parabola. 360
Fig. 21.4 Illustration of the accounting procedure to approximate the function f by the Simpson rule for
the entire interval [a, b]. 361
Fig. 22.1 Pendulum system. 372
Fig. 22.2 The illustration of the explicit Euler method. 376
Fig. 22.3 The illustration of the stiffness equation. 390
Fig. 23.1 Heat flow in a pin fin. The variable L is the length of the pin fin, which starts at x = 0 and
finishes at x = L. The temperatures at two ends are T0 and TL , with Ts being the surrounding
environment temperature. 400
Fig. 23.2 Target shooting analogy to the shooting method. 401
Fig. 23.3 Illustration of the finite difference method. 406
Fig. 24.1 Period and amplitude of a sine wave. 418
Fig. 24.2 Wavelength and amplitude of a sine wave. 418
Fig. 24.3 More general wave form. 422
Fig. 24.4 Illustration of Fourier transform with time and frequency domain signal. 422
Fig. 24.5 Signal generated before with 3 frequencies. 433
Fig. A.1 The Miniconda download page, choose the installer based on your operating system. 446
Fig. A.2 Screen shot of running the installer in Anaconda prompt. 446
Fig. A.3 The default installation location of your file system. 447
Fig. A.4 Open the Anaconda prompt from the start menu. 448
Fig. A.5 A quick way to check if your installed Miniconda runs properly. 449
Fig. A.6 Installation process for the packages that will be used in the rest of the book. 449
Fig. A.7 Run "Hello World" in IPython shell by typing the command, “print” is a function that we will
learn to print out anything within the parentheses. 450
Fig. A.8 A Python script file example using Visual Studio Code. You type in the commands you want to
execute and save the file with a proper name. 450
Fig. A.9 To run the Python script from command line, we can type “python hello_world.py”. This line
tells Python that we will execute the commands that were saved in this file. 451
Fig. A.10 To launch a Jupyter notebook server, type jupyter notebook in the command line, which
will open a browser page as shown here. Click “New” button at the top right corner, and choose
“Python3” which will create a Python notebook to run Python code. 451
Fig. A.11 Run the Hello World example within Jupyter notebook. Type the command in the code cell
(the grey boxes) and press Shift + Enter to execute it. 452
Preface

PURPOSE
Because programming has become an essential component of engineering, science, medicine, media,
business, finance, and many other fields, it is important for scientists and engineers to have a basic
foundation in computer programming to be competitive. This book introduces programming to students
from a wide range of backgrounds and gives them programming and mathematical tools that will be
useful throughout their careers.
For the most part, this book follows the standard material taught at the University of California,
Berkeley, in the class E7: Introduction to computer programming for scientists and engineers. This
class is taken by most science and engineering freshmen in the College of Engineering, and by un-
dergraduate students from other disciplines, including physics, biology, Earth, and cognitive sciences.
The course was originally taught in Matlab, but with the recent trend of the data science movement at
Berkeley, the Division of Data Sciences agreed on and supported the transform of this course into a
Python-oriented course to prepare students from different fields for further data science courses. The
course has two fundamental goals:

• Teach Python programming to science and engineering students who do not have prior exposure to
programming.
• Introduce a variety of numerical analysis tools that are useful for solving science and engineering
problems.

These two goals are reflected in the two parts of this book:

• Introduction to Programming for Scientists and Engineers

• Introduction to Numerical Methods

This book is written based on the book An Introduction to MATLAB® Programming and Numerical
Methods for Engineers by Timmy Siauw and Alexandre Bayen. The current book was first written in
Jupyter Notebook for interactive purposes, and then converted to LaTeX. Most of the codes showing
in this book are from the Jupyter Notebook code cells, which can be run directly in the notebook cell.
All the Jupyter Notebook codes can be found at pythonnumericalmethods.berkeley.edu.
Because this book covers such a wide range of topics, no topic is covered in great depth. Each
chapter has been designed to be covered in at most two lecture hours, even though there are entire
semester courses dedicated to these same chapters. Rather than an in-depth treatment, this book is
intended to give students a wide breadth of programming knowledge and mathematical vocabulary on
which they can expand.
We believe that just like learning a new foreign language, learning to program can be fun and
illuminating. We hope that as you journey through this book, you will agree.
xix
xx Preface

PREREQUISITES
This book is designed to introduce programming and numerical methods to students who have abso-
lutely no prior experience with computer programming. We hope this underlying concept is reflected
in the pace, tone, and content of the text. For the purpose of programming, we assume the reader has
the following prerequisite knowledge:
• Understanding of the computer monitor and keyboard/mouse input devices
• Understanding of the folder structure used to store files in most operating systems
For the mathematical portions of the text, we assume the reader has the following prerequisite
knowledge:
• High school level algebra and trigonometry
• Introductory, college-level calculus
That’s it! Anything in the text that assumes more than this level of knowledge is our mistake, and
we apologize in advance for any confusion of instances where concepts are unclear.

ORGANIZATION
Part 1 teaches the fundamental concepts of programming. Chapter 1 introduces the reader to Python
and Jupyter Notebook. Chapters 2 through 7 teach the fundamentals of programming. Proficiency in
the material from these chapters should provide enough background to enable you to program almost
anything you imagine. Chapter 8 provides the theory that characterizes computer programs based on
how fast they run, and Chapter 9 gives insights into how computers represent numbers and their effect
on arithmetic. Chapter 10 provides useful tips on good programming practices to limit mistakes from
popping up in computer code, and tells the user how to find them when they do. Chapter 11 explains
how to store data over the long term and how to make results from Python useful outside of Python
(i.e., for other programs). Chapter 12 introduces Python’s graphical features that allow you to produce
plots and charts, which is a really useful feature for engineers and scientists to visualize results. Fi-
nally, Chapter 13 introduces basics about the parallel programming in Python to take advantage of the
multicore design of today’s computers.
Part 2 gives an overview of a variety of numerical methods that are useful for engineers. Chapter 14
gives a crash course in linear algebra. Although theoretical in nature, linear algebra is the single most
critical concept for understanding many advanced engineering topics. Chapter 15 discusses eigenval-
ues and eigenvectors, which are important tools in engineering and science, and the ways we can utilize
them. Chapter 16 is about regression, a mathematical term that is simply a method of fitting theoretical
models to observed data. Chapter 17 is about inferring the value of a function between data points, a
framework known as “interpolation.” Chapter 18 introduces the idea of approximating functions with
polynomials, which can be useful for simplifying complicated functions. Chapter 19 teaches two algo-
rithms for finding roots of functions, that is, finding an x such that f (x) = 0, where f is a function.
Chapters 20 and 21 cover methods of approximating the derivative and integral of a function, respec-
tively. Chapters 22 and Chapter 23 introduce a mathematical model type called “ordinary differential
equations.” These two chapters focus on different problems, i.e., initial value problems and bound-
 How to Read This Book? xxi

ary value problems, and present several methods for finding their solutions. Chapter 24 introduces the
concepts of “discrete Fourier transform” and “fast Fourier transform” and their use in digital signal
processing.

HOW TO READ THIS BOOK?

Learning to program is all about practice, practice, and practice. Just like learning a new language,
there is no way to master computer programming without engaging with the material, internalizing it,
and putting it into constant use.
We suggest that as you go through the text, you should ideally have Jupyter Notebook open or
the interactive website in front of you, and run all of the numerous examples that are provided. Go
slowly. Taking the time to understand what Python is doing in every example will pay large dividends
compared to “powering through” the text like a novel.
In terms of the text itself, Chapters 1 through 5 should be read and understood first since they cover
the fundamentals of programming. Chapters 6 through 11 can be covered in any order. Chapter 12
and 13 on plotting and parallel programming are must-read chapters if you wish to improve your
problem-solving skills. In Part 2, Chapter 14 should be read first since subsequent chapters rely on
linear algebraic concepts. The remaining chapters can be read in any order. We do suggest reading
Chapters 17 and 18 before Chapters 19 and 20.
Throughout the text, there are words written in boldface. When you encounter one of them, you
should take the time to commit the word to memory and understand its meaning in the context of the
material being presented.
To keep the text from running on and overwhelming the reader, we have punctuated the material
with smaller blocks of text. Different blocks of text have different functions. For example,
TRY IT! This is the most common block in the text. It will usually have a short description of a
problem and/or an activity. We strongly recommend that you actually “try” all of these in Python.
TIP! This block gives some advice that we believe will make programming easier. Note that the
blocks do not contain any new material that is essential for understanding the key concepts of the text.
EXAMPLE: These sections are concrete examples of new concepts. They are designed to help you
think about new concepts; however, they do not necessarily need to be tried.
WARNING! Learning to program can have many pitfalls. These sections contain information that
will help you avoid confusion, developing bad habits, or misunderstanding key concepts.
WHAT IS HAPPENING? These sections follow Python in scrutinizing detail to help you under-
stand what goes on when Python executes programs.
CONSTRUCTION: In programming there are standard architectures that are reserved to perform
common and important tasks. These sections outline these architectures and how to use them.
There are two sections that end every chapter. The Summary section lists the main take-aways of
the chapter. These points should be intuitive to you by the end of the chapter. The Problems section
gives exercises that will reinforce concepts from the chapter.
As one final note, there are many ways of doing the same thing in Python. Although at first this
can seem like a useful feature, it can make learning Python confusing or overload the programming
novice with possibilities when the task is actually straightforward. This book presents a single way of
performing a task to provide structure for the new user’s learning experience and to keep that user on
xxii Preface

track and from being inundated by extraneous information. You may discover solutions that differ from
the text’s solutions but solve the problem just the same or even better! We encourage you to find these
alternative methods, and use your own judgment to given the tools we have provided herein to decide
which way is better.
We hope you enjoy the book!

WHY PYTHON?
Python is a high-level and general-purpose computer language that lends itself to many applications.
As it is beginner friendly, we hope that you will find it easy to learn and that it is fun to play with it.
The language itself is very flexible, which means that there are no hard rules on how to build features,
and you will find that there are several ways to solve the same problem. Perhaps its great strength is
that it has a great user community that supports it, with lots of packages to essentially plug in and go
with very little efforts. With the ongoing popular trend, Python suits the goals of data science today.
Python is free (open source), and most of the packages are also free for use. The idea of an open source
programming language makes a huge difference in the learning curve. Not only you can use these
packages for free, but also you can learn many advanced skills from the source code of these packages
developed by other users. We hope you can enjoy your learning of Python presented here and use it in
your work and life.

PYTHON AND PACKAGE VERSIONS

This book was written using Python 3. Here is a list of packages with their versions that used in this
book. As these packages are constantly under development, some features may be added, removed, or
changed in the versions on your computer:
• jupyter – 1.0.0
• IPython – 7.5.0
• NumPy – 1.16.4
• SciPy – 1.2.1
• h5py – 2.9.0
• matplotlib – 3.1.0
• cartopy – 0.17.0
• joblib – 0.13.2
Visit https://textbookfull.com
now to explore a rich
collection of eBooks, textbook
and enjoy exciting offers!
Acknowledgements

The first version of this book was written at a time when the standard generalist language taught in
engineering (and beyond) at UC Berkeley was Matlab. Its genesis goes back to the mid-2000s, which
precede the current era of data science, machine learning, in which Python emerged as a commonly
used language across the engineering profession. The first version was thus written as part of the E7
class at UC Berkeley, which introduces many students to programming and numerical analysis. It
would never have been written without the help of colleagues, teams of Graduate Student Instructors
(GSI), graders, and administrative staff members who helped us through the challenging process of
teaching E7 to several hundreds of students each semester at UC Berkeley. Furthermore, the first edi-
tion of this book would never have reached completion without the help of the students who had the
patience to read the book and give us their feedback. In the process of teaching E7 numerous times,
we have interacted with thousands of students, dozens of GSIs and graders, and a dozen colleagues
and administrators, and we apologize to those we will inevitably forget given the number of people
involved. We are extremely grateful for guidance from our colleagues Professors Panos Papadopou-
los, Roberto Horowitz, Michael Frenklach, Andy Packard, Tad Patzek, Jamie Rector, Raja Sengupta,
Mike Cassidy, and Samer Madanat. We owe thanks particularly to Professors Roberto Horowitz, Andy
Packard, Sanjay Govindjee, and Tad Patzek for sharing the material they used for the class, which
contributed to the material in this book. We also thank Professors Rob Harley and Sanjay Govindjee
for using a draft of this book during the semesters they taught E7 and giving us feedback that helped
improve the manuscript. The smooth running of the semester course gave the authors the time and
energy to produce this book. Managing the course was greatly facilitated by numerous administrative
staff members who bore much of the logistic load. We are particularly grateful to Joan Chamberlain,
Shelley Okimoto, Jenna Tower, and Donna Craig. Civil and Environmental Engineering Vice Chair
Bill Nazaroff deserves particular recognition for assigning the second author to teach the class in 2011.
Without this assignment the two authors of this book would not have had an opportunity to work to-
gether and write this book. E7 is notoriously the hardest class to teach at UC Berkeley in the College
of Engineering. However, it continued to run smoothly over the many semesters we learned to teach
this class, mainly due to the help of the talented GSIs we had the pleasure of working with. During the
years the coauthors taught the class, a series of legendary head GSIs have contributed to shaping the
class and making it a meaningful experience for students. In particular, Scott Payne, James Lew, Claire
Saint-Pierre, Kristen Parish, Brian McDonald, and Travis Walter have in their respective roles led a
team of dedicated GSI to exceed expectations. The GSI and grader team during the Spring of 2011
greatly influenced the material of this book. For their contribution during that critical semester, we
thank Jon Beard, Leah Anderson, Marc Lipoff, Sebastien Blandin, Sam Chiu, Rob Hansen, Jiangchuan
Huang, Brad Adams, Ryan Swick, Pranthik Samal, Matthieu Lewandowski, and Romain Bourcier. We
are also grateful to Claire Johnson and Katherine Mellis for finding errors in the text and helping us
incorporate edits into the manuscript. We are indebted to the E7 students for their patience with us
and their thorough reading of the material. Having seen thousands of them through the years, we are
sorry to only be able to mention a few for their extraordinary feedback and performance: Gursham-
njot Singh, Sabrina Nicolle Atienza, Yi Lu, Nicole Schauser, Harrison Lee, Don Mai, Robin Parrish,
xxiii
xxiv Acknowledgements

and Mara Minner. In 2018, as the UC Berkeley campus was already deeply engaged in the transition
leading to the birth of the Division of Computing, Data Science, and Society, numerous conversations
started on the need for UC Berkeley students to learn Python, which in the mean time had become
a commodity of choice for employment in most tech companies. Thus, this book started with the in-
tention of preparing engineering and science students with basic data science tools. The UC Berkeley
Division of Computing, Data Science, and Society played an active role in creating this book for a lower
division course to prepare students for further study. We thank Cathryn Carson and David Culler for
their support in writing this book and for the discussions on how to make it better. Their help happened
in parallel to the herculean efforts they led to build the Division of Computing, Data Science, and So-
ciety. It is one of the many expressions of their scientific generosity and dedication to building a rich
and innovative data science environment at UC Berkeley. Finally, we also appreciate the care and help
from Eric Van Dusen and Keeley Takimoto. About two thirds of the book are adapted from the original
Matlab version – An Introduction to MATLAB® Programming and Numerical Methods for Engineers
by the two last authors. We thank Jennifer Grannen, Brian Mickel, Nick Bourlier, and Austin Chang
for their help to convert some of the Matlab code to Python. We are (again!) grateful to Claire Johnson
for her help with the second version of the book, and to Jennifer Taggart for finding errors in the text
and helping us incorporate edits into the manuscript. We also thank the Berkeley Seismsology Lab for
the support of writing this book and the Python training over the years.

Qingkai Kong
Timmy Siauw
Alexandre M. Bayen
June 2020
 CHAPTER

PYTHON BASICS
1
CONTENTS
1.1 Getting Started With Python ......................................................................................... 3
1.1.1 Setting Up Your Working Environment ............................................................. 3
1.1.2 Three Ways to Run Python Code .................................................................... 7
1.2 Python as a Calculator ............................................................................................... 9
1.3 Managing Packages .................................................................................................. 15
1.3.1 Managing Packages Using Package Managers .................................................... 15
Install a Package ............................................................................................. 15
Upgrade a Package.......................................................................................... 16
Uninstall a Package.......................................................................................... 16
Other Useful Commands ................................................................................... 16
1.3.2 Install Packages From Source ....................................................................... 18
1.4 Introduction to Jupyter Notebook ................................................................................... 18
1.4.1 Starting the Jupyter Notebook....................................................................... 18
1.4.2 Within the Notebook .................................................................................. 19
1.4.3 How Do I Close a Notebook?......................................................................... 20
1.4.4 Shutting Down the Jupyter Notebook Server ...................................................... 20
1.5 Logical Expressions and Operators ................................................................................ 20
1.6 Summary and Problems .............................................................................................. 23
1.6.1 Summary ............................................................................................... 23
1.6.2 Problems ............................................................................................... 23

1.1 GETTING STARTED WITH PYTHON

1.1.1 SETTING UP YOUR WORKING ENVIRONMENT
The first step in using Python is to set up the working environment on the computer. This section
introduces the initial processes to get it started.
There are different ways to install Python and related packages, and we recommend using Ana-
conda1 or Miniconda2 to install and manage your packages. Depending on the operating system (OS)
you are using (i.e., Windows, Mac OS X, or Linux), you will need to download a specific installer for
your machine. Both Anaconda and Miniconda are aimed at providing easy ways to manage the Python
work environment in scientific computing and data sciences.

1 https://www.anaconda.com/download/.
2 https://conda.io/miniconda.html.

Python Programming and Numerical Methods. https://doi.org/10.1016/B978-0-12-819549-9.00010-5

Copyright © 2021 Elsevier Inc. All rights reserved.
3
4 CHAPTER 1 PYTHON BASICS

FIGURE 1.1
The Miniconda download page; choose the installer based on your operating system.

In this example, we will use Mac OS X to show you how to install Miniconda (the process of which
is very similar to installing on Linux). For Windows users, please skip the rest of this section and read
Appendix A on the installation instructions. The main differences between Anaconda and Miniconda
are as follows:
• Anaconda is a complete distribution framework that includes the Python interpreter, package man-
ager, and the commonly used packages in scientific computing.
• Miniconda is a “light” version of Anaconda that does not include the commonly used packages.
You need to install all the different packages yourself, but it does include the Python interpreter and
package manager.
The option we’ve chosen here is Miniconda, and we will install only those packages that we will
need. The Miniconda install process is described below:
Step 1. Download the Miniconda installer from the website.3 The download page is shown in
Fig. 1.1. Here you can choose a different installer based on your OS. In this example, we choose Mac
OS X and Python 3.7.
Step 2. Open a terminal (on a Mac, you can search for “terminal” in Spotlight search). Run the
installer from the terminal using the commands showing in Fig. 1.2. After you run the installer, follow
the guide to finish the installation.
Note that although you can change the installation location by giving it an alternative location on
your machine, the default is your home directory (Fig. 1.3).
After installation, you can check the installed packages by typing the commands shown in Fig. 1.4.

3 https://conda.io/miniconda.html.
 1.1 GETTING STARTED WITH PYTHON 5

FIGURE 1.2
Screen shot of running the installer in a terminal.

FIGURE 1.3
The default installation location of your file system.
Random documents with unrelated
content Scribd suggests to you:
The Project Gutenberg eBook of Will Somers,
the Boy Detective
 This ebook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this ebook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

Title: Will Somers, the Boy Detective

Author: Charles Morris

Release date: December 20, 2019 [eBook #60978]

Language: English

Credits: Produced by Demian Katz, Craig Kirkwood, and the

Online
Distributed Proofreading Team at http://www.pgdp.net
(Northern Illinois University Digital Library at
http://digital.lib.niu.edu/)

*** START OF THE PROJECT GUTENBERG EBOOK WILL SOMERS,

THE BOY DETECTIVE ***
 Transcriber’s Notes:
The Table of Contents was created by the transcriber and placed in the public domain.
Additional Transcriber’s Notes are at the end.
 CONTENTS
Chapter I. Looking For Work.
Chapter II. Store Life.
Chapter III. Mr. Leonard’s Visitor.
Chapter IV. Will Makes a New Acquaintance.
Chapter V. Under Lock and Key.
Chapter VI. Will’s First Sale.
Chapter VII. Trouble in the Basement.
Chapter VIII. Two Lovers.
Chapter IX. The Tell-tale Clew.
Chapter X. A Conference.
Chapter XI. In the Cellar.
Chapter XII. A Rejected Suit.
Chapter XIII. No Answer.
Chapter XIV. Will’s Revelation.
Chapter XV. Will Visits Mr. Somers.
Chapter XVI. The Old Companions.
Chapter XVII. Guardian and Ward.
Chapter XVIII. Will Prepares For Work.
Chapter XIX. A Prison Cell.
Chapter XX. Mr. Somers’s Story.
Chapter XXI. The Initials.
Chapter XXII. The Secret Out.
Chapter XXIII. Caught in the Act.
Chapter XXIV. The Lost Found.
Chapter XXV. Conclusion.
 Beadle’s
POCKET Library
Copyrighted, 1885, by Beadle and Adams. Entered at the Post Office at New York, N.
Y., as Second Class Mail Matter. March 18, 1885.

$2.50 Published Weekly by Beadle and Adams,

Vol. V. a Year. No. 98 William St., New York.

Price,
Five Cents. No. 62.

WILL SOMERS,
THE BOY DETECTIVE.
BY CHARLES MORRIS.
“SHALL I LAY YOU OUT A PIECE OF THIS? IT’S DOG CHEAP.
JUST LOOK AT THAT STUFF.”
 Will Somers,
THE BOY DETECTIVE.
BY CHARLES MORRIS.
 CHAPTER I.
LOOKING FOR WORK.
“Got any opening in these diggin’s for a feller of my size and good
looks?”
The speaker was a boy of some sixteen years of age, a well-built,
athletic lad, the sinewy development of his limbs showing through
more than one rent in his well-worn clothes.
His claim to good looks was indisputable. A bright black eye gave
character to a face of classical outline, straggling curls of dark hair
hanging low over his olive-hued cheeks and brow, while his nose
and mouth had all the fine curves of the Grecian type.
“What do you want?” asked the gentleman addressed, in a curt
tone.
“Well, I ain’t partik’lar,” drawled the boy. “I want a job. Most
anything will do. Say cashier, or head clerk.”
The merchant twisted himself around in his chair and looked at the
speaker. The latter bore his sharp look unabashed, standing in an
erect, easy attitude.
“Suppose I don’t want a cashier?”
“Maybe then you’d give me a job to make fires and run errands.”
“Who told you I wanted a boy?”
“A counter-jumper outside there. I axed him if there was room in
this row for a smart young man, and he said he guessed you wanted
a partner. So I jest stepped back to see if I wouldn’t suit.”
A frown came upon the merchant’s brow as he heard of this
impudent action of one of his clerks.
“Who told you this?” he sharply asked.
“Now look ye here, mister,” said the boy, impressively; “that’s not my
lay. I don’t tell tales out of school. I wouldn’t blow on a cat if I
caught her stealing a mouse in another man’s kitchen.”
“Get out of here then. I am busy and don’t want to be bothered.”
“See here now,” said the boy, leisurely seating himself in a chair.
“You’re not sayin’ nothing about that job. You’ve got a dozen men
out there in the store, and I don’t see a boy in the shanty. Now you
can’t run a place like this without a wide-awake boy, and I’m jest the
feller you want.”
“You have impudence enough to run it yourself,” said the merchant,
looking more closely at his importunate visitor.
“Wouldn’t be afeard to try,” said the boy, saucily, putting to his lips a
half-smoked cigar which he had all this time held in his hand, and
taking a long whiff. “I’ve a notion I could make dry-goods spin
amazing. Jest hand me the reins and I bet I put her through at two-
forty.”
The merchant laid aside the papers which he had been examining.
He pushed back his chair from the table and faced his visitor.
He was a hale, handsome man of some fifty years of age, somewhat
imperious in manner, but with a strong sense of humor in his face.
He seemed to think that he had met an original character.
“What is your name?” asked the merchant.
“Will Somers.”
“Where do you live?”
“In this here big town of Philadelphia, but in a little street that I
s’pose you never heered the name of. I make myself at home
anywhere, though.”
“So it seems,” said the merchant, glancing at the handsome
appointments of his private office, and then at the ragged dress of
the boy.
“It’s only my coat and pants that’s torn,” said the latter, with an air
of pride. “I’m all right inside, I bet there’s not a coon in these
diggin’s can jump further, run faster, or lift more than me. And I
never seen the day yet I was afeard of work! Now how about that
job, mister?”
“Leonard,” said the merchant.
“Mr. Leonard, I mean. I’ve been a-waiting to get holt of the north
end of your name.”
The merchant looked closely at his precocious visitor, who, to the
age of a boy, added the self-assertion and experience of a grown
man. The latter leaned back with easy assurance in his chair, and
seemed indeed “at home.”
“What have you been used to doing?” asked Mr. Leonard.
“What ain’t I been used to would be a bit more like it,” said Will,
resting his two elbows on the table. “Blackin’ boots, and sellin’
papers, and holdin’ hosses has been my big holts, but I’ve dipped
into ’most everything else ’cept preaching.”
“You have been a little vagabond, I suppose, all your life, and know
as much of the world as men ought to at twenty-five.”
“If there’s a feller inside of ten miles of here that says I ever done
anything mean, I can lick that feller; that’s me!” cried Will,
indignantly.
“Do you know Philadelphia well?”
“Does a cat know milk? Bet I do. Could navigate it with my eyes
shet.”
“Are your parents living?”
“Dunno ’bout my dad,” said Will. “’Spect I’m an orphan. Me and sis
was drapped in this here town when we was like young kittens. A
big white house, t’other side the Schuylkill, was our head-quarters.
Dad sloped. Never heered of him since.”
“The poor-house, eh?” said Mr. Leonard. “You have a sister?”
“Yes. She’s slipped, too. Was took out when I was a baby. Never
see’d her since. Hope the girl’s sound. Know I’ve had mighty hard
hoein’.”
There was a touch of feeling in Will’s voice which he sought to hide
by greater recklessness of manner. Evidently he had a secret
yearning for his lost sister.
Mr. Leonard was silent for several minutes before again speaking. He
seemed to be debating something within himself.
“So you want to learn something of business?” he at length said.
“You’ve hit that nail square on the head,” said Will, with energy. “I’m
gettin’ too big to shove the brush, or handle the extras. What’s
more, I’m not goin’ to be a poor critter all my life. I want a bizz
that’s got money in it. I’ve sot my eye on a brown-stone shanty up
Broad street. If it’s for sale ten years from now I’m in the market.”
Mr. Leonard laughed slightly at the boy’s tone of confidence.
“Fortunes ain’t made as quickly as you fancy, my lad,” he said.
“If I don’t hang my hat up in that shanty, you can count me out,”
said Will.
“The saucy young rascal has the making of a business man in him,”
said Mr. Leonard, to himself. “I would much rather have a boy that
aimed high than one that aimed low. He is a handsome lad, too, and
if better dressed would be quite presentable. I have half a notion to
try him, with all his impudence. He is a perfect specimen of the
street Arab, but he seems quick and intelligent.”
“How about that job?” asked Will, impatiently. “I’m bound to strike
one, somewhere, afore night. I’ve give you the refusal. The man that
gets me makes a ten-strike, and no braggin’.”
“If I should give you employment could I depend on you to do what
you were told?”
“What I was told?” said Will, rising impulsively to his feet. “I
wouldn’t give a smashed cent for the feller who couldn’t do more
than he was told.”
“That would never do,” replied Mr. Leonard. “I want a boy to do just
as he is told.”
“And what chance is there for genius, then, if a feller can’t spread a
little?” asked Will, earnestly. “The boy that only does what he’s told
won’t never get to Congress.”
“And they who act beyond their instructions sometimes get to the
State’s prison, my boy. If I give you a position you must learn to
never take a step without orders.”
“I can try,” said Will, with a comical leer, “but it’ll go mighty ag’in’ the
grain.”
Their conversation was interrupted at this point by the entrance of a
person into the office.
He seemed to be one of Mr. Leonard’s employes, and was a tall, well
built man, but dressed with a foppish vanity that at once attracted
the boy’s attention.
He looked with surprise at the merchant’s strange visitor, a look of
disdain coming upon his face, as he drew somewhat back, as if in
fear of contamination. Will glanced at him from head to foot, with a
steady, impudent stare.
“The Everhart is in,” he said. “The Danton shipment of silks on
board. I have just received notice.”
“That is good news, Wilson,” replied Mr. Leonard. “The market is just
ready for them. See to the custom-house charges at once. We must
have them in store as soon as possible.”
“I will attend to it,” he said with a somewhat pompous air.
With another look of supercilious wonder at Will he left the room.
“Who’s that cove?” asked the latter.
“That is Mr. Augustus Wilson, my principal bookkeeper.”
“He is a hoss, he is,” said Will, with a contemptuous puff. “A man of
his size dressed like a peacock, and biting off his words like a school-
girl. I bet he’s a dose.”
“If I should give you employment, Will, you must learn to curb your
tongue, and not be so insolent to the men in the store. They would
not stand impudence from a boy.”
“I’ll get along with them. Don’t you be afeard,” said Will, with a look
of confidence on his handsome face. “I’ve got along with folks all my
life, and never been kicked yet. But I’m doubtful if I won’t be callin’
that cove Gus. He’s a gay feller to Mister, he is.”
“You will not stay here long, my lad, if you do. I warn you of that.
He is my principal employe, and must be treated with respect.
Understand me. Impudence will not serve.”
“All right, Mr. Leonard. But I know I’ll have a fight every time I go to
say Mister. Gus will be coming up. When am I to take hold?”
“Come round this hour to-morrow and I will let you know my
decision,” said Mr. Leonard, turning again to his papers.
“That won’t gee,” said Will, positively. “If you want me you’d best
say so and be done with it. I’m bound to fetch work to-day.”
“Very well,” said the merchant, impatiently. “I will give you a trial.
Now don’t bother me any further.”
“If you’d said that half an hour ago I wouldn’t bothered you so long,”
said Will, saucily, as he strode out of the room.
 CHAPTER II.
STORE LIFE.

Mr. Leonard was the proprietor of a large wholesale dry-goods

house, on Market street. He dealt extensively in the richer qualities
of goods, and cases of costly silks, rich shawls, and other expensive
fabrics were constantly imported by him.
There were a dozen or more salesmen, porters, etc., employed
about the store, besides the numerous clerks in the counting-house,
and in the business hours of the day the long, wide store bustled
with activity till it seemed a very bee-hive of commerce.
A few days after his interview with Mr. Leonard found our young
hero duly installed as office-boy and general utility in the store. His
employer had found him a more creditable suit of clothes, and given
him some useful instruction as to personal cleanliness, politeness,
etc.
He had succeeded in greatly improving the outward appearance of
the boy, but his mental crookedness was not to be so easily
straightened. Will was essentially the same lad who had spent his
life till now in street avocations, with intervals of fishing, lounging,
swimming, fighting and saucing.
He took hold of business with a vim that promised well for his future
usefulness, doing the work given him so rapidly and well as to
greatly please his employer.
But there was no curbing his tongue, and more than one spat
occurred between him and the salesmen, before he had been a day
in the store. Before the end of the first week there was an outbreak
which threatened to end his engagement.
“Here, boy, take this roll of cloth down to the store and give it to Mr.
Johnson. Look alive, now, he is waiting for it.”
This imperative address was made by a nervous, quick-spoken
salesman, named Robert Brown.
Will was employed in opening a case of goods. He looked up with a
glance of disdain.
“I’ve took in another job,” he said. “Ain’t doin’ two things at once.
You know the way down. Tote it down yourself.”
“What do you mean, you young rascal?” cried the man, in a passion.
“If you stay in this place you will have to do what you are told or
you’ll be helped. Take this down at once.”
“What’s goin’ to happen if I don’t?” said Will, dropping the tool he
had been using.
“I’ll send you spinning down-stairs and out of the store door in a
hurry,” said the man, still fuming.
“Look here, Mr. Brown, or Bob Brown, if you like it better, maybe you
don’t know that you’re barking up the wrong tree,” said Will,
insolently. “Ordering ain’t in my line. Ask me like a gentleman and I’ll
stand on my head for you; but I’m not a feller that’s used to bein’
kicked by any man’s toe or tongue, either.”
“Then you won’t take it down?”
“I’ll see you so far t’other side of nowhere that a forty-horse team
couldn’t draw you back in a lifetime, afore I’ll take it an inch.”
Will returned to his former task of opening the case.
Mr. Brown’s face was purple with rage, and the veins stood out on
his forehead, as he listened to this unexampled rebellion.
“Why, you ragged young reprobate, who was only brought here by
charity!” he cried, hotly. “Hang me if I don’t kick you down-stairs
myself, and fling the goods after you!”
Suiting the action to the word he grasped Will with a nervous grip,
and sought to hustle him to the head of the stairs.
But if ever man caught a Tartar, Mr. Brown had done so in this
action.
Will lifted the iron tool in his hand with an impulse to strike his
assailant. With another impulse he threw it from him, and used his
sinewy limbs with a vigor which Mr. Brown had not dreamed of his
possessing.
In a moment he had torn loose from his grasp, and by an alert trip
had stretched his foe on his back on the floor.
“That’s the way I pile up my shingles!” cried Will exultingly. “Come to
time, old hoss. I’ve chawed up better men than you.”
The language of the street, which he had partly laid aside, came
back to Will in his excitement.
The furious salesman sprung to his feet and rushed at the boy with
clinched fists. Two other men, who had been engaged with them on
the third floor of the building, hurried up.
“Hold there, Bob!” cried one of these. “Don’t try that on a boy.”
“Let him alone,” said Will, as he deftly parried his blows. “He’s my
meat. I wasn’t brung up on free fights to back down from a counter-
hopper.”
But the man who had spoken pushed between and separated them,
just as Will planted his fist with a stinging blow on Brown’s left
cheek.
“Come, come, Bob!” said the peacemaker, “that’s no way to settle
disputes with a boy. If the fellow has been impudent report him to
Mr. Leonard, but never try your fists on a boy.”
Mr. Brown did report, and Will was sent for to Mr. Leonard’s office.
Our hero proved a very poor hand at giving evidence in his own
favor, but the men who had separated them described the whole
occurrence.
“Don’t let anything like this happen again,” said Mr. Leonard, after
lecturing Will, severely. “Mr. Brown placed himself in the wrong or I
would have to discharge you. Don’t misuse the confidence I have
placed in you.”
“All right,” said Will, independently. “But the man that tries to wipe
his feet on me is goin’ to touch ground with his nose, that’s all.”
This episode did not injure Will’s standing in the store, for Mr. Brown
was not a general favorite.
His good-humor and willingness to work soon gained him friends,
and faults were excused in him that would have proved fatal
otherwise to his position.
He had a fine voice, and sung ditties with wonderful vim. He could
dance like a negro minstrel, could tumble like an acrobat, and had
more tricks than a circus clown.
Nothing pleased him better than to get on one of the upper floors,
out of sight of customers, and treat the admiring clerks to a taste of
his quality.
His chief trouble, in such cases, sprung from the new clothes in
which Mr. Leonard had dressed him.
“It’s a gallus rig. I’ll give in,” he said, “but I ain’t been used to fancy
fixin’s. There goes the coat, and here goes the vest, and up go the
sleeves, and now I’m in trim for work. P’int out what you want done
and I’m in.”
“Nothing just now, Will. Let us have that ole Virginny break-down.”
“Yes, I see myself waltzing round on my ear for you, and no pianner
music, and not even a jews-harp. Don’t dance till I hear a tune.
Whistle up, somebody, my boot-soles is itching.”
One of the men whistled a quick tune, and Will’s feet rattled over the
floor in the most astonishing steps, relieving his feelings now and
then by a somerset, or a dance on his hands.
“Hi, lads! let the music out,” he cried. “That’s your style! Heel and
toe. Ain’t I a screamer? Just observe me.”
The next instant he was at the top of a high step-ladder, singing a
negro melody for dear life, and keeping time with hands and feet on
the boards.
“Hush! here comes Mr. Leonard,” cried a scout. “He has heard all
that rascally noise. Limber up and get to work quick.”
The men hastened to various avocations, somewhat to Will’s
surprise. He was as honest as the sun, and would never have
thought of such deceit. He had, all his life, fathered all his actions.
When Mr. Leonard appeared, Will was seated in his shirt-sleeves on
a dry-goods box, whistling in a low tone, and keeping time with his
heels.
Mr. Leonard looked inquiringly around, a look of displeasure on his
face.
“What has been going on here!” he asked, sharply. “I heard an
uproar all the way down to my office. You are all suddenly at work.
What have you been doing, boy?”
“Nothing. Only killing time,” said Will, indifferently. “We kinder run
out of work, and I wanted to learn these fellows a South street
wharf break-down. Want to see me do it?”
Will jumped from his box and struck an attitude.
“No. And I want less noise and commotion. If you wish to stay in my
employment you must learn to curb yourself a little. Let us have no
more such performances.”
“You suit me fu’st-rate so fur. Got no notion of dischargin’ you yet,”
said Will. “But what’s to be done? I can’t hold in. It’s out of the
question. There’d be something bu’st sure.”
“Well,” said Mr. Leonard, turning on his heel to hide a broad smile. “I
must give you some out-door work—send you on errands. Can I
trust you to do them correctly?”
“I dunno. Won’t make no promises. You can try me. That’s the way
to find out. I know everything and everybody out of doors; that’s
something.”
“Very well; I will try you.”
Thenceforth Will varied his store duties with out-door avocations, his
quick and intelligent performance of which gave much satisfaction.
 CHAPTER III.
MR. LEONARD’S VISITOR.

It was two or three days after the last-narrated events that a

slender, keen-eyed person stopped in front of Mr. Leonard’s store. He
was dressed in grayish clothes, and wore a wide-rimmed hat.
He glanced up at the lofty iron front, reaching five stories high, and
then briskly entered the store, threading his way back between the
open cases of goods which covered the long floor to the counting-
house.
Sending in his card he was at once admitted to Mr. Leonard’s private
office.
“Take a seat, Mr. Fitler,” said Mr. Leonard, pushing aside his papers.
Rising, he carefully closed the door and seated himself near the
visitor.
“You received my message, then?”
“Yes, sir. You have need of my services?”
“I wish your advice, at any rate. But first, does any of my men know
you?”
“I think not. I know none of them.”
“I have here an invoice of silks shipped me from Lyons, France, by
Danton & Co. There were two cases of these goods, valued, as you
see, at ten thousand dollars. The ship Everhart, which had them in
freight, reached Philadelphia two weeks ago. Her cargo was duly
discharged, and the goods deposited in the Government warehouse.”
“Why not brought at once to your store?” asked Mr. Fitler, drumming
with his fingers on the table.
“Such was my intention, and I directed my confidential clerk, Mr.
Wilson, to pay the duties, and see that they were brought here. He
did the first, and obtained the Government order for their delivery,
which was locked up in my presence, in the fire-proof there. He was
hindered from doing the second by news of the death of a near
relative in Harrisburg, whose funeral he was obliged to attend.”
“I perceive. What next?” said Mr. Fitler.
“A few words will conclude. On his return yesterday he went to the
safe for the custom-house delivery-order. It was gone.”
“Were you present?”
“Yes.”
“Had you seen it in his absence?”
“No. Why do you ask?”
“Only for a full statement of facts.”
“You will please understand that no possible suspicion attaches to
Mr. Wilson, even if his absence did not preclude it. He has been in
my service for ten years, and is incorruptible.”
“Certainly,” said Mr. Fitler, in an indifferent tone. “Somebody else has
taken it. But that is not all?”
“No. I have just learned that the order was presented at the bonded
warehouse last Tuesday by a gentlemanly-dressed person,
accompanied by a drayman. The necessary entries were made, and
the goods delivered.”
“And where are they now?”
“That is what I want to find out.”
“This is a bold robbery, Mr. Leonard,” said the visitor, with energy. “It
looks very much as if the black sheep was in your own store. Is
there any one whom you suspect?”
“There is none.”
“Who has access to your safe?”
“Any of my book-keepers. Three or four of them are obliged to enter
my office, in my absence, in the regular discharge of their duties.”
“What is the record of these three or four men?”
“The best. They are all steady, quiet business men, married and
living moderately. I know them all well.”
“These smooth-watered wells are sometimes very deep,” said Mr.
Fitler. “Have you any new hands?”
“Yes. One, whom I took on two weeks ago.”
“Ha!” said the visitor, interested. “Could he enter your office without
suspicion?”
“He could. I have employed him as a messenger.”
“Every confidence in him, too?”
“I think him thoroughly honest.”
“You have too much trust in human nature, Mr. Leonard,” said his
visitor, with a shrug. “If you were in my profession you would have
very little. I would like to see this new hand. Can you call him in on
some pretense?”
“You have seen him. He is the messenger I sent for you.”
“What? That boy? He in your employment? Well, that beats!” Mr.
Fitler leaned back in his chair and laughed heartily but silently.
“Do you know him?” asked Mr. Leonard, with some displeasure.
“Know him? I should think so. Who don’t know Willful Will? That is
his street name. Why, I thought you had picked him up at your door
and sent him after me. He did look strange at me. I never saw him
in a whole suit of clothes before. The idea of his settling down to
steady business! Just call him in; I would like to talk with him.”
“He is not in at present. I sent him, this morning, out to my
residence, near Germantown.”
“Is Mr. Wilson at hand?”
“Yes. I will call him.”
Mr. Leonard went to the door of the office and asked for Mr. Wilson.
The latter was in the counting-room, and immediately entered.
The keen eyes of the visitor were fixed on him as he came in, taking
in at a glance, as it seemed, every detail of his face, form and dress.
“Mr. Wilson, this is Mr. Fitler, a detective officer from the central
station,” said the merchant. “I have sent for him to investigate that
affair of the robbery.”
“I hope he can help us in it,” said Mr. Wilson, as he quietly seated
himself.
“The rascals were wide-awake, Mr. Wilson,” said the detective, “in
taking advantage of your trip to Harrisburg. Was the fact that you
had paid the duties and could not attend to taking the goods out of
bond talked about in the store?”
“It was no secret in the counting-room,” said Mr. Wilson.
“What is your opinion of the book-keepers, sir?”
“I have full confidence in them. They are only men, to be sure, and
may have talked outside.”
“Could the store have been entered at night?”
“No, no,” said Mr. Leonard. “Nothing has been tampered with. The
order was stolen in the daytime, while the safe was open.”
“How does the store-keeper at the bonded warehouse describe the
parties who took away the goods?”
“In a very vague fashion,” replied Mr. Wilson. “He could not have
noticed them closely. His description did not remind me of anybody I
knew.”
Mr. Fitler watched him as he spoke, seeming attracted by his foppish
dress and stilted manner of speaking.
“I will see the store-keeper myself,” he answered. “It is unlucky that
you were called away at such a time, Mr. Wilson. Was it a near
relative?”
“A first cousin,” he replied.
“Ah! I am somewhat acquainted in Harrisburg. What name, pray?”
“Miles Sartain,” answered Mr. Wilson, with composure.
“The name is not familiar. I thought I might have known him,” said
the officer carelessly.
“I will drop in myself to-morrow, and look round,” he continued. “You
may not know me, but don’t be surprised if a stranger makes himself
at home. I will see the store-keeper this afternoon, and will set the
police authorities to work to try and trace these missing goods.”
He bowed himself out of the office, leaving Mr. Wilson and his
employer in busy conversation.
 CHAPTER IV.
WILL MAKES A NEW ACQUAINTANCE.

Meanwhile Willful Will, as the officer had called him, was on his way
to Mr. Leonard’s country-seat. He bore a note addressed to a Miss
Jennie Arlington, a resident of the merchant’s house, which he was
directed to deliver into her own hands without delay, and wait for
any return message.
The mansion was a broad-fronted, brown-stone edifice, richly
ornamented, and surrounded with beautifully-kept grounds, in which
now a host of flowers were in bloom.
Seen from the front, through the vailing screen of leaves and
blossoms, it seemed an abode of wealth and taste very attractive to
any artistic eye. Even Will paused for five minutes, in a day-dream,
gazing in. The boy was not without the poetic instinct.
He was stirred from his reverie by the approach of a servant-woman
to the gate.
“Say, you there,” cried Will, as she turned back. “This Mr. Leonard’s?”
“Yes,” was the answer.
“All square, then. I want to see Jennie.”
“See who?” asked the woman, in surprise.
“Jennie. You know. Can’t go her last name. Slipped clean out of my
head.”
“Miss Arlington is not in. You can’t see her,” said the woman.
“Bet a hoss that I will,” he replied, positively. “Mr. Leonard didn’t
send me here on a fool’s errand. Where is she?”
“What do you want? I will give her your errand when she comes in.”
“Maybe so, when you get it,” said Will, mysteriously. “Hope you don’t
kalkerlate to pick up secrets that easy. There’s things it don’t do for
everybody to know. Where is the gal?”
“She is down there in the wood,” said the servant.
Breaking into a gay ditty, Will hastened off toward the piece of
woodland indicated by the woman.
The ground sloped downward from the back of the house into a
wide depression that led off toward the Schuylkill. This was occupied
by a piece of fine woodland, the trees growing straight and tall,
while the undergrowth had been mostly removed, leaving long,
green vistas.
The country was now arrayed in all its June-tide beauty, while
flowers lent a rich charm to the fields, and the fresh green of spring
arrayed every tree.
The old vagabond instinct rose in him as he sauntered on, now
chasing a gorgeous butterfly, now following the flight of some swift
bird, now stopping to listen to a trill of woodland music, now taking
a wild roll in the grass.
His cap was soon adorned with daisies and buttercups; violets
peeped from his button-holes, and he gathered a bouquet of yellow
dandelions as if he thought them choice flowers.
“If this ain’t gay I don’t know beans!” he cried, exultingly. “Wouldn’t
I like to live out here! Bet I’d go a-fishin’ every day, and a-swimmin’
every other. I’ll go a hoss there ain’t a tree in that woods I can’t
climb. Got a notion to shin up some of them just for fun.”
But a frisking ground-squirrel took his attention, and drove out of his
head the project of tree-climbing.
This fellow chased to his hole, other objects attracted Will’s delighted
eyes, and led him step by step into the woods.
Finally, some sounds behind a thin screen of bushes attracted his
notice. He drew carefully up and looked through. There on a flat