IIE Transactions (2005) 37, 1083–1089

Copyright

C “IIE”

ISSN: 0740-817X print / 1545-8830 online

Book reviews
Methods of Multivariate Analysis, Second Edition

Alvin C. Rencher
Wiley-Interscience, New York, NY, 2002, 708 + 22 pages, ISBN 0-471-41889-7

Data sets that are multivariate in nature are common in any real-life data analysis. Rencher does an excellent job of
modern engineering, science, and business. In some anal- writing appropriately for his target audience.
ysis problems, univariate statistical methods applied “one The book is suitable for classroom use as well as for self
variable at a time” (such as in hypothesis tests) might be study, as each chapter has a set of problems, and a fairly
tried, but are often inadequate. For other problems such substantial Appendix (88 pages) provides hints and answers
as classification and clustering, anyone who has taken only to the problems. Another Appendix contains an extensive
a couple of standard introductory statistics courses would set of tables, which “enable the reader to conduct an ex-
not be taught how to solve such problems at all. Powerful act test in many cases for which software packages provide
methodologies developed by statisticians, computer scien- only approximate tests” (p. xvi). The book is supported by
tists, and others for analyzing multivariate data can help an FTP site where 59 data sets used in the book are available
engineers and others address these modern and challenging for download, as well as SAS command files corresponding
problems, but such methods are usually covered in a more to the book’s worked examples. The text itself does not re-
advanced course. The textbook literature for such a course fer to any particular statistical software package and could
can be divided roughly into two categories: the classical sta- be used independently of SAS. Engineers may find the free,
tistical approach, often based on normal distribution prob- open source statistical software system called R, an imple-
ability models, and the modern machine learning/pattern mentation of the S language, more intuitive to use than SAS,
recognition/data mining literature. There is remarkably lit- since S resembles Matlab in many ways. Some coverage of
tle overlap between the two categories. The book under multivariate methods in S and R can be found in Venables
review here belongs firmly in the first category of classi- and Ripley (2002).
cal statistical methods, where perhaps its most prominent Prerequisites for this book include as few as two basic
competitor is Johnson and Wichern (2002). Prominent ex- applied statistics courses (a course on inference and one on
amples of the more modern genre include Duda et al. (2001) linear models—regression and ANOVA), although to get
and Hastie et al. (2001). All of these books have much to the full benefit of the chapter on multivariate analysis of
offer to the serious data analyst, and the book under review variance, a course on experimental design would also be
is a strong contribution to this literature. useful. Calculus is almost never used, but the reader should
Rencher’s Methods of Multivariate Analysis is a “meth- be very comfortable with matrix algebra. Chapter 2 is a
ods” book for working data analysts, including nonstatisti- succinct yet thorough review of the matrix algebra results
cians and students. Many of the proofs of theorems stated that are used in the book.
in the book have been relegated to a companion volume, The selection of topics in the book is standard for the
Rencher (1998). Therefore, the present book focuses on ex- classical genre. Preliminaries occupy the first three chap-
plaining how and why methods work and applying them to ters: after an introduction and the matrix algebra review
real data sets in worked examples. The text provides both of Chapter 2, Chapter 3 discusses the characterization and
mathematical and heuristic motivation for the methods, as display of multivariate data. This discussion includes some
well as insight and intuition to guide readers in their se- simple imputation methods for missing data. The next four
lection and use. The writing style is clear, straightforward, chapters cover classical normal-theory statistical inference:
and user friendly. As an example, for multivariate methods Chapter 4 on the multivariate normal distribution, Chap-
which are generalizations of a univariate method, Rencher ter 5 on tests on mean vectors, Chapter 6 on Multivari-
always begins with a brief review of the corresponding uni- ate Analysis Of Variance (MANOVA), and Chapter 7 on
variate method to introduce the topic. This is extremely tests related to covariance matrices. The chapter on the
useful to orient readers who are not (yet) professional statis- multivariate normal distribution includes sections on as-
ticians. Rencher also pays attention to the important issues sessing multivariate normality and outlier detection. The
of robustness of methods to outliers or violations of under- chapter on MANOVA includes a lengthy section on re-
lying assumptions, which are of paramount importance to peated measures designs, as well as discussions of profile

0740-817X

C 2005 “IIE”
1084 Book reviews

analysis and growth curves. There is also an interesting sec- In addition to two new chapters on clustering and graph-
tion discussing the fact that “There is a MANOVA coun- ical methods, the second edition features new problems and
terpart for every univariate ANOVA design. This applies “numerous corrections and revisions (almost every page) in
to fixed, random, and mixed models and to experimen- the first 13 chapters” (p. xvii). Also, the computer files have
tal structures that are crossed, nested, or a combination” been relabeled and made available on the FTP site rather
(p. 196). Rencher also goes into more depth than than on a diskette. For these reasons, owners of the origi-
Johnson and Wichern (2002) in the comparison of the four nal edition should strongly consider acquiring the second
MANOVA test statistics (Wilks’ lambda, Roy’s largest root, edition.
the Pillai statistic, and the Lawley-Hotelling statistic). For The author writes that he has “provided careful intuitive
instance, he discusses how the geometry of the mean vectors explanations of the concepts and [has] included many in-
of the samples determines which test has the greatest power. sights typically available only in journal articles or in the
This is an example of the type of insightful discussion that minds of practitioners” (p. xvi). Not only do I agree, but I
gives this book its special character, a point I will return to think this is the main strength of the book. As an example,
later. in discussing how to proceed after a T-squared test rejects
Following are two chapters on multivariate data with the hypothesis that two samples have the same mean vector,
class labels: Chapter 8 on discriminant analysis and Chap- Rencher provides a critical discussion of seven procedures
ter 9 on classification analysis. Rencher makes the distinc- that could be used to look for specific variables that collec-
tion that discriminant analysis is simply the description of tively differentiate between the two samples. The discussion
classed data, such as evaluating the importance of each vari- proceeds at an intuitive level to describe the strengths and
able in separating the classes, whereas classification analysis weaknesses of the various approaches, as well as his recom-
is intended for predicting the labels of unknown “test” data, mendation on the best procedure (interpreting the coeffi-
using a model fitted to a set of “training” data. Rencher re- cients of a standardized discriminant function as a means
stricts his coverage of classification methods to linear and of weighing each variable’s contribution to the separation of
quadratic discriminant functions, and to three nonpara- the samples). Rencher is interested in assessing the effect of
metric methods: nearest neighbors, an approach based on a variable in the presence of other variables in the model, since
kernel density estimates (known in other texts as Parzen this approach pays full respect to the multivariate nature of
windows), and an approach based on a multinomial prob- the data, in contrast to running univariate tests on each in-
ability model. However, there is far more to be said about dividual variable. The discussion reflects careful thinking,
classification analysis; for instance, the linear support vec- long experience, and extensive study of the published liter-
tor machine is a fairly robust alternative to linear discrim- ature, and the resulting wisdom is convincing. Discussions
inant analysis. Furthermore, decision tree analysis (i.e., re- like this one, which appear in many places in the book,
cursive partitioning) is a powerful way to explore data with have few analogs in competing texts such as Johnson and
nonlinearities and interactions between variables, and en- Wichern (2002).
sembles of such trees have a strong track record as predictive Throughout the book, Rencher discusses stepwise vari-
tools. These methods and others, such as neural networks, able selection methods, but fails to warn the reader that
receive no coverage in this book, and at best cursory cov- there is some controversy surrounding these methods.
erage in other books in the classical genre. I advise readers Harrell (2001) claims that in regression modeling, stepwise
to also consult a text with a more modern perspective such variable selection “violates every principle of statistical es-
as Duda et al. (2001), Hastie et al. (2001), or Webb (2002), timation and hypothesis testing,” essentially because it sur-
which also have much more thorough discussions of classi- reptitiously inflates the Type I error. Harrell (2001) gives a
fier performance evaluation. laundry list of seven reasons why stepwise selection is a bad
Next, Chapter 10 deals with multivariate regression idea. One wonders how these problems generalize to the
(where there are several dependent as well as independent multivariate case, but Rencher, and to be fair, all other mul-
variables), Chapter 11 is on canonical correlations, Chapter tivariate texts that I know of, do not address this question.
12 is on Principal Component Analysis (PCA), and Chap- Unfortunately, some topics are beyond the scope of the
ter 13 is on factor analysis (well known to be a controver- book, such as discrete multivariate data, logistic regression,
sial topic). New to the second edition are Chapter 14 on and spatial statistics. Industrial engineers should be aware
clustering and Chapter 15 on miscellaneous graphical pro- that multivariate quality control, which gets substantial
cedures: multidimensional scaling, correspondence analy- coverage in Johnson and Wichern (2002), is not addressed
sis, and biplots. For the material in Chapters 11 and 15, I at all by Rencher. Other topics of great practical interest,
actually prefer the presentation in Johnson and Wichern such as multivariate analysis of covariance and all discus-
(2002), which for me is somewhat easier to digest. In dis- sion of confidence interval estimation, are relegated to the
cussing factor analysis, Rencher very clearly outlines both companion text (Rencher, 1998). Finally, although aspects
its similarities and differences with PCA, and the reasons of the infamous Curse of Dimensionality rear their heads
for its controversial reputation. His guiding hand here will in this book, Rencher doesn’t discuss the Curse by name.
be much appreciated by novice readers. [Neither do Johnson and Wichern (2002).] I think that the
Book reviews 1085

Curse is actually one of the governing principles of mul- Hastie, T., Tibshirani, R. and Friedman, J. (2001) The Elements of Sta-
tivariate data analysis, and it deserves a more substantial tistical Learning: Data Mining, Inference, and Prediction, Springer-
Verlag, New York, NY.
discussion like the one found in Hastie et al. (2001).
Johnson, R.A. and Wichern, D.W. (2002) Applied Multivariate Statistical
In conclusion, this book is a systematic, well-written, Analysis, 5th edn., Prentice Hall, Upper Saddle River, NJ.
well-organized text on multivariate analysis packed with Rencher, A.C. (1998) Multivariate Statistical Inference and Applications,
intuition and insight for guiding the working data ana- Wiley, New York, NY.
lyst. There is much practical wisdom in this book that Venables, W.N. and Ripley, B.D. (2002) Modern Applied Statistics with S,
4th edn., Springer-Verlag, New York, NY.
is hard to find elsewhere. It belongs in serious data ana-
Webb, A. (2002) Statistical Pattern Recognition, 2nd edn., Wiley,
lysts’ libraries, and course instructors who use Johnson and Hoboken, NJ.
Wichern (2002) as their main text should consider using
Rencher as a supplemental text.
Reviewed by
Christopher Tong
References Biometrics Research RY33-300
Merck Research Laboratories
Duda, R.O., Hart, P.E. and Stork, D.G. (2001) Pattern Classification, 2nd 126 East Lincoln Avenue
edn., Wiley, New York, NY.
Harrell, F.E., Jr. (2001) Regression Modeling Strategies: With Applica-
Rahway, NJ 07065, USA
tions to Linear Models, Logistic Regression, and Survival Analysis,
Springer-Verlag, New York, NY. Contributed by the Book and Software Review Department

Introduction to Probability and Statistics for Scientists and Engineers

Walter A. Rosenkrantz
McGraw-Hill, 1997, 592 pages, ISBN 0-07-053988-X

The goal of Walter A. Rosenkrantz, as he states in the pref- ter summary. In this way, a student at first knows where they
ace to his book, is to present the basic ideas of probability are going and then, afterwards, where they have been. The
theory and statistics within the context of applications in textbook includes statistical tables and answers to the odd-
the engineering, physical, biological and computer sciences. numbered problems. The index is very detailed and specific.
The author further states that upon the successful comple- The examples in the text are drawn from all areas and run
tion of a course based on his book, students will be well the gamut from gambling to theoretical physics to quality
equipped to collect, model, display, and interpret data for control.
a variety of scientific and engineering applications. This, Chapter 1, “Data Analysis”, is a short chapter in descrip-
according to the author, will prepare the student for entry tive statistics which briefly covers the usual material. It is
into the workplace for scientists and engineers as well as good at the mechanics of elementary statistics but short on
give them a good foundation for advanced studies in prob- the uses of the graphs and statistics to understand and rep-
ability and statistics. resent a data distribution. Moreover, the basic statistics are
The textbook is a good technical exposition of the math- often not interpreted simply and in plain language. Some of
ematical theory of statistics and probability, in the format the uses are left to the students to arrive at on their own in
of formal mathematical theorem and proof. There is a fair the problems. For example, there are problems to compare
amount of mathematical rigor which makes this book per- two groups on the basis of their calculated sample statistics.
haps most suitable and appropriate at the advanced under- Without an example in the text, an instructor will have to
graduate or beginning graduate level for students with a demonstrate these comparisons.
good technical background. Chapter 2, “Probability Theory”, is a formal presenta-
The textbook is comprehensive of the subject, covering tion of probability in terms of set theory. Some concepts
in 14 chapters the usual range of topics in their usual order are often presented with little explanation and discussion,
for an introductory textbook. The exception to the custom- which may obfuscate the essential points. For example, rel-
ary ordering is to place analysis of variance and design of ative frequency is discussed as a special case of the classical
experiments after regression. The book is deceptively small, method of computing probabilities of equally likely events.
being under 600 pages. However, the equally likely and relative frequency concepts
Each chapter begins with an orientation section that in- for assigning probabilities are fundamentally different, as
cludes a organization plan. Each chapter ends with a chap- discussed by Bulmer (1979).