See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/245378079

Fundamentals of Engineering Numerical Analysis

Article in Canadian Journal of Civil Engineering · April 2002

DOI: 10.1139/l02-006

CITATIONS READS

275 13,765

1 author:

Parviz Moin
Stanford University
635 PUBLICATIONS 76,165 CITATIONS

SEE PROFILE

All content following this page was uploaded by Parviz Moin on 18 April 2016.

The user has requested enhancement of the downloaded file.

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

FUNDAMENTALS OF ENGINEERING NUMERICAL ANALYSIS

SECOND EDITION

Since the original publication of this book, available computer power has increased
greatly. Today, scientific computing is playing an ever more prominent role as a
tool in scientific discovery and engineering analysis. In this second edition, the
key addition is an introduction to the finite element method. This is a widely used
technique for solving partial differential equations (PDEs) in complex domains.
This text introduces numerical methods and shows how to develop, analyze, and use
them. Complete MATLAB programs for all the worked examples are now available
at www.cambridge.org/Moin, and more than 30 exercises have been added. This
thorough and practical book is intended as a first course in numerical analysis,
primarily for new graduate students in engineering and physical science. Along
with mastering the fundamentals of numerical methods, students will learn to write
their own computer programs using standard numerical methods.

Parviz Moin is the Franklin P. and Caroline M. Johnson Professor of Mechanical

Engineering at Stanford University. He is the founder of the Center for Turbu-
lence Research and the Stanford Institute for Computational and Mathematical
Engineering. He pioneered the use of high-fidelity numerical simulations and mas-
sively parallel computers for the study of turbulence physics. Professor Moin is a
Fellow of the American Physical Society, American Institute of Aeronautics and
Astronautics, and the American Academy of Arts and Sciences. He is a Member
of the National Academy of Engineering.

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

FUNDAMENTALS OF ENGINEERING
NUMERICAL ANALYSIS
SECOND EDITION

PARVIZ MOIN
Stanford University

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

CAMBRIDGE UNIVERSITY PRESS

Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore,
São Paulo, Delhi, Dubai, Tokyo, Mexico City

Cambridge University Press

32 Avenue of the Americas, New York, NY 10013-2473, USA
www.cambridge.org
Information on this title: www.cambridge.org/9780521711234

C Parviz Moin 2010

This publication is in copyright. Subject to statutory exception

and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the written
permission of Cambridge University Press.

First published 2010

Printed in the United States of America

A catalog record for this publication is available from the British Library.

Library of Congress Cataloging in Publication data

Moin, Parviz.
Fundamentals of engineering numerical analysis / Parviz Moin. – 2nd ed.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-521-88432-7 (hardback)
1. Engineering mathematics. 2. Numerical analysis.
I. Title. II. Title: Engineering numerical analysis.
TA335.M65 2010
620.001 518–dc22 2010009012

ISBN 978-0-521-88432-7 Hardback

ISBN 978-0-521-71123-4 Paperback

Additional resources for this publication at www.cambridge.org/Moin

Cambridge University Press has no responsibility for the persistence or

accuracy of URLs for external or third-party Internet Web sites referred to in
this publication and does not guarantee that any content on such Web sites is,
or will remain, accurate or appropriate.

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

Contents

Preface to the Second Edition page ix

Preface to the First Edition xi

1 INTERPOLATION 1
1.1 Lagrange Polynomial Interpolation 1
1.2 Cubic Spline Interpolation 4
Exercises 8
Further Reading 12

2 NUMERICAL DIFFERENTIATION – FINITE DIFFERENCES 13

2.1 Construction of Difference Formulas Using Taylor Series 13
2.2 A General Technique for Construction of Finite Difference
Schemes 15
2.3 An Alternative Measure for the Accuracy of Finite
Differences 17
2.4 Padé Approximations 20
2.5 Non-Uniform Grids 23
Exercises 25
Further Reading 29

3 NUMERICAL INTEGRATION 30
3.1 Trapezoidal and Simpson’s Rules 30
3.2 Error Analysis 31
3.3 Trapezoidal Rule with End-Correction 34
3.4 Romberg Integration and Richardson Extrapolation 35
3.5 Adaptive Quadrature 37
3.6 Gauss Quadrature 40
Exercises 44
Further Reading 47

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

vi CONTENTS

4 NUMERICAL SOLUTION OF ORDINARY DIFFERENTIAL EQUATIONS 48

4.1 Initial Value Problems 48
4.2 Numerical Stability 50
4.3 Stability Analysis for the Euler Method 52
4.4 Implicit or Backward Euler 55
4.5 Numerical Accuracy Revisited 56
4.6 Trapezoidal Method 58
4.7 Linearization for Implicit Methods 62
4.8 Runge–Kutta Methods 64
4.9 Multi-Step Methods 70
4.10 System of First-Order Ordinary Differential Equations 74
4.11 Boundary Value Problems 78
4.11.1 Shooting Method 79
4.11.2 Direct Methods 82
Exercises 84
Further Reading 100

5 NUMERICAL SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS 101

5.1 Semi-Discretization 102
5.2 von Neumann Stability Analysis 109
5.3 Modified Wavenumber Analysis 111
5.4 Implicit Time Advancement 116
5.5 Accuracy via Modified Equation 119
5.6 Du Fort–Frankel Method: An Inconsistent Scheme 121
5.7 Multi-Dimensions 124
5.8 Implicit Methods in Higher Dimensions 126
5.9 Approximate Factorization 128
5.9.1 Stability of the Factored Scheme 133
5.9.2 Alternating Direction Implicit Methods 134
5.9.3 Mixed and Fractional Step Methods 136
5.10 Elliptic Partial Differential Equations 137
5.10.1 Iterative Solution Methods 140
5.10.2 The Point Jacobi Method 141
5.10.3 Gauss–Seidel Method 143
5.10.4 Successive Over Relaxation Scheme 144
5.10.5 Multigrid Acceleration 147
Exercises 154
Further Reading 166

6 DISCRETE TRANSFORM METHODS 167

6.1 Fourier Series 167
6.1.1 Discrete Fourier Series 168
6.1.2 Fast Fourier Transform 169
6.1.3 Fourier Transform of a Real Function 170
6.1.4 Discrete Fourier Series in Higher Dimensions 172

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

CONTENTS vii

6.1.5 Discrete Fourier Transform of a Product of Two

Functions 173
6.1.6 Discrete Sine and Cosine Transforms 175
6.2 Applications of Discrete Fourier Series 176
6.2.1 Direct Solution of Finite Differenced Elliptic Equations 176
6.2.2 Differentiation of a Periodic Function Using Fourier
Spectral Method 180
6.2.3 Numerical Solution of Linear, Constant Coefficient
Differential Equations with Periodic Boundary
Conditions 182
6.3 Matrix Operator for Fourier Spectral Numerical
Differentiation 185
6.4 Discrete Chebyshev Transform and Applications 188
6.4.1 Numerical Differentiation Using Chebyshev Polynomials 192
6.4.2 Quadrature Using Chebyshev Polynomials 195
6.4.3 Matrix Form of Chebyshev Collocation Derivative 196
6.5 Method of Weighted Residuals 200
6.6 The Finite Element Method 201
6.6.1 Application of the Finite Element Method to a Boundary
Value Problem 202
6.6.2 Comparison with Finite Difference Method 207
6.6.3 Comparison with a Padé Scheme 209
6.6.4 A Time-Dependent Problem 210
6.7 Application to Complex Domains 213
6.7.1 Constructing the Basis Functions 215
Exercises 221
Further Reading 226

A A REVIEW OF LINEAR ALGEBRA 227

A.1 Vectors, Matrices and Elementary Operations 227
A.2 System of Linear Algebraic Equations 230
A.2.1 Effects of Round-off Error 230
A.3 Operations Counts 231
A.4 Eigenvalues and Eigenvectors 232

Index 235

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

To Linda

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

Preface to the Second Edition

Since the original publication of this book ten years ago, the available computer
power has increased by more than 2 orders of magnitude due to massive par-
allelism of computer processors and heterogeneous computer clusters. Today,
scientific computing is playing an ever more prominent role as a tool in scientific
discovery and engineering analysis.
In the second edition an introduction to the finite element method has
been added. The finite element method is a widely used technique for solving
partial differential equations (PDEs) in complex domains. As in the first edition,
numerical solution of PDEs is treated in Chapter 5, and the development there is
based on finite differences for spatial derivatives. This development is followed
in Chapter 6 by an introduction to more advanced transform methods for solving
PDEs: spectral methods and, now, the finite element method. These methods
are compared to the finite difference methods in several places throughout
Chapter 6.
Hopefully, most of the errors that remained in the 2007 reprint of the book
have now been corrected. Several exercises have also been added to all the
chapters. In addition, complete MATLAB programs used for all the worked
examples are available at www.cambridge.org/Moin. Students should find this
new feature helpful in attempting the exercises, as similar computer programs
are used in many of them. Working out the exercises is critical to learning
numerical analysis, especially using this book. The intention for including this
feature is for students to spend less time writing and debugging computer
programs and more time digesting the underlying concepts.
I thank all the students and teaching assistants who have provided valuable
feedback to me on the teaching of numerical analysis and the contents of this
book. In particular, I am grateful to Dr. Ali Mani who took a special interest
in this book and made significant technical contributions to the new edition.
Special thanks are due to Nick Henderson for compiling the examples programs
and Drs. Erich Elsen and Lawrence Cheung for their due diligence and help in

ix

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

x PREFACE TO THE SECOND EDITION

the preparation of this edition. Prof. Jon Freund suggested the addition of the
finite element section and gave me a draft of his notes on the subject to get me
started.

Parviz Moin
Stanford, California
March 2010

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

Preface to the First Edition

With the advent of faster computers, numerical simulation of physical phenom-

ena is becoming more practical and more common. Computational prototyping
is becoming a significant part of the design process for engineering systems.
With ever-increasing computer performance the outlook is even brighter, and
computer simulations are expected to replace expensive physical testing of de-
sign prototypes.
This book is an outgrowth of my lecture notes for a course in computational
mathematics taught to first-year engineering graduate students at Stanford. The
course is the third in a sequence of three quarter-courses in computational
mathematics. The students are expected to have completed the first two courses
in the sequence: numerical linear algebra and elementary partial differential
equations. Although familiarity with linear algebra in some depth is essential,
mastery of the analytical tools for the solution of partial differential equations
(PDEs) is not; only familiarity with PDEs as governing equations for physical
systems is desirable. There is a long tradition at Stanford of emphasizing that
engineering students learn numerical analysis (as opposed to learning to run
canned computer codes). I believe it is important for students to be educated
about the fundamentals of numerical methods. My first lesson in numerics in-
cludes a warning to the students not to believe, at first glance, the numerical
output spewed out from a computer. They should know what factors affect ac-
curacy, stability, and convergence and be able to ask tough questions before
accepting the numerical output. In other words, the user of numerical methods
should not leave all the “thinking” to the computer program and the person
who wrote it. It is also important for computational physicists and engineers
to have first-hand experience solving real problems with the computer. They
should experience both the power of numerical methods for solving non-trivial
problems as well as the frustration of using inadequate methods. Frustrating
experiences with a numerical method almost always send a competent numer-
ical analyst to the drawing board and force him or her to ask good questions

xi

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

xii PREFACE TO THE FIRST EDITION

about the choice and parameters of the method, which should have been asked
before going to the computer in the first place. The exercises at the end of
each chapter are intended to give these important experiences with numerical
methods.
Along with mastering the fundamentals of numerical methods, the students
are expected to write their own programs to solve problems using standard
numerical methods. They are also encouraged to use standard (commercial)
software whenever possible. There are several software libraries with well-
documented programs for basic computational work. Recently, I have used the
Numerical Recipes by Press et al. (Cambridge) as an optional supplement to
my lectures. Numerical Recipes is based on a large software library that is well
documented and available on computer disks. Some of the examples in this
book refer to specific programs in Numerical Recipes.
Students should also have a simple (x, y) plotting package to display their
numerical results. Some students prefer to use MATLAB’s plotting software,
some use the plotting capability included with a spreadsheet package, and oth-
ers use more sophisticated commercial plotting packages. Standard well-written
numerical analysis programs are generally available for almost everything cov-
ered in the first four chapters, but this is not the case for partial differential
equations, discussed in Chapter 5. The main technical reason for this is the
large variety of partial differential equations, which requires essentially tailor-
made programs for each application.
No attempt has been made to provide complete coverage of the topics that
I have chosen to include in this book. This is not meant to be a reference book;
rather it contains the material for a first course in numerical analysis for future
practitioners. Most of the material is what I have found useful in my career
as a computational physicist/engineer. The coverage is succinct, and it is ex-
pected that all the material will be covered sequentially. The book is intended
for first-year graduate students in science and engineering or seniors with good
post-calculus mathematics backgrounds. The first five chapters can be cov-
ered in a one-quarter course, and Chapter 6 can be included in a one-semester
course.
Discrete data and numerical interpolation are introduced in Chapter 1, which
exposes the reader to the dangers of high-order polynomial interpolation. Cu-
bic splines are offered as a good working algorithm for interpolation. Chapter 2
(finite differences) and Chapter 3 (numerical integration) are the foundations
of discrete calculus. Here, I emphasize systematic procedures for construct-
ing finite difference schemes, including high-order Padé approximations. We
also examine alternative, and often more informative, measures of numeri-
cal accuracy. In addition to introducing the standard numerical integration
techniques and their error analysis, we show in Chapter 3 how knowledge of
the form of numerical errors can be used to construct more accurate numeri-
cal results (Richardson extrapolation) and to construct adaptive schemes that

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

PREFACE TO THE FIRST EDITION xiii

obtain the solution to the accuracy specified by the user. Usually, at this point
in my lectures, I seize the opportunity, offered by these examples, to stress the
value of a detailed knowledge of numerical error and its pay-offs even for the
most application-oriented students. Knowledge is quickly transferred to power
in constructing novel numerical methods.
Chapter 4 is on numerical solution of ordinary differential equations
(ODEs) – the heart of this first course in numerical analysis. A number of
new concepts such as stability and stiffness are introduced. The reader begins
to experience new tools in the arsenal for solving relatively complex problems
that would have been impossible to do analytically. Because so many inter-
esting applications are cast in ordinary differential equations, this chapter is
particularly interesting for engineers. Different classes of numerical methods
are introduced and analyzed even though there are several well-known powerful
numerical ODE solver packages available to solve any practical ODE without
having to know their inner workings. The reason for this extensive coverage of
a virtually solved problem is that the same algorithms are used for solution of
partial differential equations when canned programs for general PDEs are not
available and the user is forced to write his or her own programs. Thus, it is
essential to learn about the properties of numerical methods for ODEs in order
to develop good programs for PDEs.
Chapter 5 discusses numerical solution of partial differential equations and
relies heavily on the analysis of initial value problems introduced for ODEs.
In fact by using the modified wavenumber analysis, we can cast into ODEs
the discretized initial value problems in PDEs, and the knowledge of ODE
properties becomes very useful and no longer of just academic value. Once
again the knowledge of numerical errors is used to solve a difficult problem
of dealing with large matrices in multi-dimensional PDEs by the approximate
factorization technique. Dealing with large matrices is also a focus of numerical
techniques for elliptic partial differential equations, which are dealt with by
introducing the foundations of iterative solvers.
Demand for high accuracy is increasing as computational engineering ma-
tures. Today’s engineers and physicists are less interested in qualitative features
of numerical solutions and more concerned with numerical accuracy. A branch
of numerical analysis deals with spectral methods, which offer highly accu-
rate numerical methods for solution of partial differential equations. Chapter 6
covers aspects of Fourier analysis and introduces transform methods for partial
differential equations.
My early work in numerical analysis was influenced greatly by discus-
sions with Joel Ferziger and subsequently by the works of Harvard Lomax at
NASA–Ames. Thanks are due to all my teaching assistants who helped me
develop the course upon which this book is based; in particular, I thank Jon
Freund and Arthur Kravchenko who provided valuable assistance in prepara-
tion of this book. I am especially grateful to Albert Honein for his substantial

© in this web service Cambridge University Press www.cambridge.org

Cambridge University Press
978-0-521-88432-7 - Fundamentals of Engineering Numerical Analysis, Second Edition
Parviz Moin
Frontmatter
More information

xiv PREFACE TO THE FIRST EDITION

help in preparing this book in its final form and for his many contributions
as my teaching assistant in several courses in computational mathematics at
Stanford.

Parviz Moin
Stanford, California
July 2000

© in this web service Cambridge University Press www.cambridge.org

View publication stats