James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am.

A 1055

Estimating fractal dimension

James Theiler
Lincoln Laboratory, Massachusetts Institute of Technology, Lexington, Massachusetts02173-9108

Received September 13, 1989; accepted November 21, 1989

Fractals arise from a variety of sources and have been observed in nature and on computer screens. One of the
exceptional characteristics of fractals is that they can be described by a noninteger dimension. The geometry of
fractals and the mathematics of fractal dimension have provided useful tools for a variety of scientific disciplines,
among which is chaos. Chaotic dynamical systems exhibit trajectories in their phase space that converge to a
strange attractor. The fractal dimension of this attractor counts the effective number of degrees of freedom in the
dynamical system and thus quantifies its complexity. In recent years, numerical methods have been developed for
estimating the dimension directly from the observed behavior of the physical system. The purpose of this paper is
to survey briefly the kinds of fractals that appear in scientific research, to discuss the application of fractals to
nonlinear dynamical systems, and finally to review more comprehensively the state of the art in numerical methods
for estimating the fractal dimension of a strange attractor.

Confusion is a word we have invented for an order similarity also implies a scale-invariant property. There are
which is not understood. crinkles upon crinkles and therefore no natural or preferred
-Henry Miller, "Interlude," crinkle size. A set is strictly self-similar if it can be ex-
Tropic of Capricorn pressed as a union of sets, each of which is a reduced copy of
(is geometrically similar to) the full set. See Fig. 1(a).
Numerical coincidence is a common path to intellectu- However most fractal-looking objects in nature do not dis-
al perdition in our quest for meaning. We delight in play quite this precise form. In a coastline, for instance,
catalogsof disparateitems united by the same number, there is an irregular nesting of gulfs, bays, harbors, coves,
and often feel in our gut that some unity must underlie and inlets that are observed over a broad range of spatial
it all. scales. See Fig. 1(b). A magnified view of one part of the
-Stephen Jay Gould, "The Rule of Five," coastline will not precisely reproduce the full picture, but it
The Flamingo's Smile will have the same qualitative appearance. A coastline dis-
plays the kind of fractal behavior that is called statistical
self-similarity.
INTRODUCTION
A fractal is self-affine if it can be decomposed into subsets
Fractals are crinkly objects that defy conventional mea- that can be linearly mapped into the full figure. If this
sures, such as length and area, and are most often character- linear map involves only rotation, translation, and (isotro-
ized by their fractional dimension. This paper will arbi- pic) dilation, then the figure is self-similar. For a self-affine
trarily categorize fractals into two main types: solid objects map, the contraction in one direction may differ from the
and strange attractors. contraction in another direction. The class of self-affine
Of the first type, coastlines and clouds are typical exam- fractals therefore includes the class of self-similar fractals.
ples. Further examples include electrochemical deposi- (A visually impressive collection of self-affine fractals, and a
tion, 1 ,2 viscous fingering, 3 and dielectric breakdown 4 as well method of generating them that goes under the name "the
as porous rocks5 and spleenwort ferns.6 These are physical chaos game," can be found in Refs. 6 and 9.) The distinction
objects that exist in ordinary physical space. Percolation between self-similar and self-affine fractals is not always
clusters, 7 diffusion-limited aggregates, 8 and iterated func- made in practice. The concept is most useful in cases for
tion systems9 provide mathematical models of fractals that which there are preferred global directions: in fractal sur-
correspond to these physical objects. An overview of many faces,5 for example, or for fractal profiles [these are continu-
of these topics can be found in Ref. 10, and a recent bibliog- ous nowhere-differentiable functions f(x) for which the
raphy has been compiled in Ref. 11. graph (x, f(x)) is a set of fractal dimension1 31 4].
Strange attractors, by contrast, are conceptual objects
that exist in the state space of chaotic dynamical systems. Quantifying Fractals
The emphasis of this paper will be on strange-attractor di- Stanley1 0 has outlined the program of the practicing scien-
mensions, although some of what is discussed can also be tist who wants to study fractals:
applied to solid fractal objects.
If you are an experimentalist, you try to measure
Self-Similarity and Self-Affinity the fractal dimension of things in nature. If you
Fractal objects, as Kadanoff notes, "contain structures nest- are a theorist, you try to calculate the fractal di-
ed within one another like Chinese boxes or Russian mension of models chosen to describe experimental
dolls." 12 This self-similar structure is perhaps the main situations; if there is no agreement then you try
reason for the striking beauty of so many fractals. Self- another model.

0740-3232/90/061055-19$02.00 © 1990 Optical Society of America

1056 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

One usually measures the complexity of a physical system

by the number of degrees of freedom that the system pos-
sesses. However, it is useful to distinguish nominal degrees
of freedom from effective (or active) degrees of freedom.
Although there may be many nominal degrees of freedom
available, the physics of the system may organize the motion
into only a few effective degrees of freedom. This collective
behavior, which can be observed in the laminar flow of a
fluid or in the spiral arms of a galaxy, is often termed self-
organization. (Much of the interest in the principle of self-
organization has been stimulated by the writings of Haken,15
an early and enthusiastic advocate.) Self-organization is
interesting because the laws of thermodynamics seem to
forbid it: A self-organizing system decreases its own entro-
py. (For real physical systems, this apparent decrease in
entropy is achieved by dissipation to an external bath. In
this case the total entropy-of the system and of the bath-
(a) does increase, and the second law of thermodynamics is
satisfied.)
Self-organization arises in dissipative dynamical systems
whose posttransient behavior involves fewer degrees of free-
dom than are nominally available. The system is attracted
to a lower-dimensional phase space, and the dimension of
this reduced phase space represents the number of active
degrees of freedom in the self-organized system. A system
that is nominally complex may in fact relax to a state of
chaotic but low-dimensional motion. Distinguishing behav-
ior that is irregular but low dimensional from behavior that
is irregular because it is essentially stochastic (many effec-
tive degrees of freedom) is the motivation for quantifying
chaos. Estimating dimension from a time series is one way
to detect and quantify the self-organizational properties of
natural and artificial complex systems.
4,c150. 148/ A L A S A(
150
4,C.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
148' 146' 144' Strange Attractors
(b) Strange attractors arise from nonlinear dynamical systems.
Fig. 1. Self-similar fractals. (a) The Sierpinski gasket. Here, the Physically, a dynamical system is anything that moves.
inner triangles are small copies of the full figure. (b) Prince William (And if it does not move, then it is a dynamical system at a
Sound. The sound has been darkened to highlight the fractal ap- fixed point.) Mathematically, a dynamical system is de-
pearance of the coastline. fined by a state space RM (also called phase space) that
describes the instantaneous states available to the system
and an evolution operator 0 that tells how the state of the
system changes in time. (One usually thinks of 0 as the
This pithy advice applies to strange attractors as well as to
physics of the system.) An element X e RM of the state
solid fractal objects: Fractal dimension provides the bench-
space specifies the current state of the system, and M char-
mark against which theories are compared with experi-
acterizes the number of degrees of freedom in the system.
ments. In the case of strange attractors, however, there are
For a particle in space, X might represent the three coordi-
further reasons for wanting to know the dimension.
nates of the particle's position, and the three coordinates of
its momentum, for a total of M = 6 degrees of freedom. The
CHAOS AND STRANGE ATTRACTORS evolution operator is a family of functions t:RM - RM that
map the current state of the system into its future state at a
Why Quantify Chaos?: A Manifesto time t units later. The operator t satisfies ,0 (X) = X and
Chaos is the irregular behavior of simple equations, and ,t+JX) = Pt[0J(X)]. The dynamical system is nonlinear if,
irregular behavior is ubiquitous in nature. A primary moti- in general, ot(cXl + c2X2) d5ckt(X0) + c20t(X 2).
vation for studying chaos is this: Given an observation of The function kt can be defined either as a discrete map or
irregular behavior, is there a simple explanation that can in terms of a set of ordinary differential equations, although
account for it? And if so, how simple? There is a growing partial differential equations and differential delay equa-
consensus that a useful understanding of the physical world tions have also been studied (in these last two cases, M is
will require more than finally uncovering the fundamental infinite).
laws of physics. Simple systems, which obey simple laws,
can nonetheless exhibit exotic and unexpected behavior. Dissipative Dynamics
Nature is filled with surprises that turn out to be direct Although conservative dynamical systems can also exhibit
consequences of Newton's laws. chaos,' 6 only dissipative dynamical systems have strange
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1057

attractors. A system is dissipative if the volume of a fiducial 1 . .. . . . . . . . ...

chunk of phase space tends to zero as t - -. In other words,
if 3B is a bounded subset of RM with (Lebesque) volume
AL(), then
0

lim .L[bt(fB)] = 0. (1)

t en
Nr
Usually the dissipation is exponential (at least on the aver- E
age for a long time): L['kt(!)] AL(,)e-At, with the rate of -1
dissipation A.
Informally the attractor A4 of a dynamical system is the
subset of phase space toward which the system evolves. An
initial condition X0 that is sufficiently near the attractor will -2
evolve in time so that ot(Xo) comes arbitrarily close to the set
A4 as t - a. If the set A is a fractal, then the attractor is -0. 5 0 0.5 1.0 1.5
said to be strange. A more formal treatment of strange Re(z)
attractors in dynamical systems can be found in Ref. 17.
0.4
Natural Invariant Measure
Equation (1) implies that the phase-space volume of the
attractor FL(A) is zero. To quantify the dynamics on the
0.2
attractor requires first the introduction of a new measure As
that is concentrated on the attractor. A measure ,u is de-
fined on the set A if the subsets of the set A4 can be
associated with real values o(CB) that represent how much of 0
,A is contained in 3. The measure that is defined on a set
reflects the varying density over the set and can intuitively
be regarded as a mass. See Fig. 2. The reader is referred to
-0.2
Ref. 20 for a more rigorous discussion of measure in the L ~~ ~ ~ . ~ ~ '
context of fractal sets.
A useful measure t should be invariant under time evolu-
tion: The proper way to define this is to write -0.4
-1 0 1
(2) (b)
AM = x
Fig. 2. Two strange attractors. (a) The Ikeda map. The complex
where 0-t(13) X e RM:ot(X) e M}. In general, Eq. (2) is map, z,,+1 = a + Rz, expli[O - p/(l + Izl2)]I, which derives from a
not enough to define uniquely the measure for a dynamical model of the plane-wave interactivity field in an optical ring laser,18
system; for instance, a measure that is concentrated on an is iterated many times, and the points [Re(zn), Im(zn)] are plotted
unstable fixed point satisfies Eq. (2) but has little to do with for n ' 1000. Here, a = 1.0, R = 0.9, 0 = 0.4, p = 6. The fractal
the generic posttransient motion of the system. dimension of this attractor is approximately 1.7. (b) The H6non
2
map.' 9 This map, Xn+1 = 1.0 - axn + Yn; Yn+1 = bxn, with a = 1.4
The physically relevant measure for a dynamical attractor and b = 0.3, gives an attractor with fractal dimension of approxi-
counts how often and for how long a typical trajectory visits mately 1.3. Note how much thinner the attractor of lower dimen-
various parts of the set. A constructive definition is given sion appears.
by
13
M( ) = lim -1[,Jt(X0)]dt, (3)
T- T impossible, even if the physics (that is, t) is known com-
where X0 is a typical initial condition and I (X) is the indica- pletely, because errors in measurement of the initial state
tor function for M3: It is unity if X E 13 and is zero otherwise. propagate exponentially fast.
The natural invariant measure is given by Eq. (3) for almost Consider two nearly initial conditions, X0 and XO + , and
all X0 . An alternative definition that avoids the notion of evolve both forward in time. A Taylor-series expansion
typical trajectories by adding infinitesimal noise to the dy- gives
namics is mentioned in Ref. 21. /0t(Xo + E)= 0t(XO) + J(t) f + o(IE12 ), (4)

Sensitivity to Initial Conditions where J(t) is the Jacobian matrix given by the linearization
The hallmark of a chaotic system is its sensitivity to initial of kt about the point XO:
conditions. This sensitivity is usually quantified in terms of
the Lyapunov exponents and the Kolmogorov entropy. The
J(t) = ot(X0) - X(t) (5)
Lyapunov exponents measure the rate of exponential diver-
gence of nearby trajectories, and the Kolmogorov entropy The ij element of this matrix is
measures the rate of information flow in the dynamical sys-
tem. OXL(t)
Jij(t) = aX3 (O) (6)
Long-term predictions of chaotic systems are virtually
1058 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

g (t = X (t), X (t _ r, . . . , X (t _ (M - 1),r) ] T,
where Xi(t) is the ith component of the state vector X at time (9)
t. Thus an initial small separation is magnified to J(t)e.
The determinant of the J(t) describes the overall contrac- where the delay time r and the embedding dimension m are
tion of phase-space volume (dissipation), and the eigenva- parameters of the embedding procedure. (The superscript
lues describe the divergence of nearby trajectories. The T denotes the transpose.) Here 1(t) represents a more
Lyapunov exponents quantify the average rate of expansion comprehensive description of the state of the system at time
of these eigenvalues: t than does x(t) and can be thought of as a map of
7r(m):RM - Rm (10)
X = lim - log Inth eigenvalue of J(t)I. (7)
t- t from the full state X(t) to the reconstructed state R(t).
Conventionally the Lyapunov exponents are indexed in de- Takens30 and Mane 31 have provided often-cited proofs
scending order: X1 2 X2 > .... The largest Lyapunov that this procedure does (almost always) reconstruct the
exponent is the clearly most important since, if the differ- original state space of a dynamical system, as long as the
ence between a pair of initial conditions has a nonzero com- embedding dimension m > 2D + 1, where D is the fractal
ponent in the eigendirection associated with the largest ei- dimension of the attractor. That is, r(nm), restricted to A, is a
genvalue, then that rate of divergence will dominate. If the smooth one-to-one map from the original attractor to the
largest Lyapunov exponent is positive, then chaotic motion reconstructed attractor. It should be noted, however, that as
is ensured. (Otherwise, trajectories will collapse to a fixed long as m > D, the reconstructed set will almost always have
point, a limit cycle, or a limit torus.) The sum of all the the same dimension as the attractor. 2 '
exponents X, + . . . + AM = -A is negative for a dissipative The delay-time embedding procedure is a powerful tool.
dynamical system and defines the rate A of phase-space If one believes that the brain (say) is a deterministic system,
contractions. then it may be possible to study the brain by looking at the
If the state X(t) of the system is known at time t to an electrical output of a single neuron. This example is an
accuracy e, then the future can be predicted by X(t + At) = ambitious one, but the point is that the delay-time embed-
0At[X(t)] but to an accuracy that is usually worse than E. ding makes it possible for one to analyze the self-organizing
Thus to observe (or measure) the state of the system again at behavior of a complex dynamical system without knowing
time t + At to the original accuracy e is to learn information the full state at any given time.
that was previously unavailable. The system is in this sense Although almost any delay time r and embedding dimen-
creating information. (Some authors prefer to say that the sion m > D will in principle work (with unlimited, infinitely
system is losing information since the initial accuracy is lost; precise data), it is nontrivial to choose the embedding pa-
this choice is largely a matter of taste, since the numerical rameters in an optimal way. For instance, if the product (m
results are the same.) The average rate of information gain - 1)T is too large, then the components x(t) and x[t + (m -
(or loss) is quantified by the Kolmogorov entropy, which is 1)T] of the reconstructed vector will be effectively decorre-
given by the sum of the positive Lyapunov exponents. 2' lated, and this result will be an inflated estimate of dimen-
The numerical estimation of Lyapunov exponents and sion. By the same token, if (m - 1)r is too small, then the
Kolmogorov entropy from a time series is discussed in Refs. components x(t), ... , x[t + (m - 1)T] will all be nearly
22-24. A method for estimating Kolmogorov entropy that is equal, and the reconstructed attractor will look like one long
related to the correlation dimension is developed in Refs. 25 diagonal line. It is also inefficient to take too small, even if
and 26. The notion of a generalized entropy Kq is discussed m is large, since x(t) x(t + T) means that successive compo-
in Refs. 27 and 28. nents in the reconstructed vector become effectively redun-
dant.
Delay-Time Embedding In general, one wants r to be not too much less than, and
An experimentalist, confronted with a physical system, mea- (m - 1)r not too much greater than, some characteristic
sures at regular and discrete intervals of time the value of decorrelation time. The (linear) autocorrelation time is one
some state variable (voltage, say) and records the time series: such characteristic, although Fraser and Swinney 32 have in-
x(to), x(t1), x(tD), . . , with x(ti) e R and ti = to + iAt. [This troduced a more sophisticated rule based on the mutual
section discusses a stroboscopic time discretization of a con- information time. A topological rule, based on maintaining
tinuous flow. The method of Poincar6 section (see Ref. 17 nearest-neighborhood relations as embedding dimension is
or any textbook) is another way to time discretize a flow, but increased, has recently been suggested.3 3
it is a method that is rarely available to the experimentalist.] Typically, having chosen r, one performs the dimension
The measurement x(t) represents a projection analysis for increasing values of m and looks for a plateau in
the plot of D versus m. Some authors choose to increase m
7r:RM - R (8)
and decrease r in such a way as to preserve (m - 1)r as a
34 35
from the full state vector X(t) e RM. A time series is constant. '
manifestly one dimensional and, as such, provides an incom- As a preprocessing step, linear transforms of the time-
plete description of the system in its time evolution. On the delayed variables have been suggested. One class of such
other hand, many properties of the system can be inferred transforms is based on a principal-value decomposition of
from the time series. the autocorrelation function. 3 43 6 37 (Engineers will recog-
Packard et al.29 devised a delay scheme to reconstruct the nize this decomposition as a Karhunen-Loeve expansion.)
state space by embedding the time series into a higher- However, it has been argued that the information available
dimensional space. From time-delayed values of the scalar in the (linear) autocorrelation function is not necessarily
time series, vectors X e R- are created: relevant to optimal processing of a time series that arise from
James Theiler Vol. 7,No. 6/June 1990/J. Opt. Soc. Am. A 1059

nonlinear systems.3 839 A transformation called relevance The obvious relevant measure of bulk for a subset !8 of a
weighting was suggested by Farmer and Sidorowich.40 In dynamical attractor is its natural invariant measure A(B),
their scheme, although other notions of bulk are also possible and will be
X(t) = {x(t), e-hrx(t-T), ...
discussed. A good quantity for the size of a set is its radius
or its diameter, the latter of which is defined by
exp[- h(m - 1)r]x[t - (m - 1)Tr]I. (11)
a(3) supt [Ix - YD:X, YFE l, (14)
The idea behind this embedding procedure is that the state
X(t) depends most significantly on measurements that are where sup is the supremum, or maximum, and lX - Yj] is the
taken most recently in time. distance between X and Y. How this distance is calculated
depends on the norm of the embedding space. If Xi is the
ith component of the vector X e R-, then the L, norm gives
DEFINITIONS OF DIMENSION: FORMAL AND distance according to
INFORMAL
1/s
A completely rigorous definition of dimension was given in fix - Y =_ 1In X - YJS (15)
1919 by Hausdorff,4 1but it is a definition that does not lend
i=1
itself to numerical estimation. Much interest in the past
decade has focused on numerical estimates of dimension, where I-I is the absolute value. The most useful of these
and it is natural to consider more operational definitions of norms are L2, the Euclidean norm, which gives distances
dimension, i.e., those that can be more readily translated that are rotation invariant; L,, the taxicab norm, which is
into algorithms for estimating dimension from a finite sam- easy to compute; and L., the maximum norm, which is also
ple of points. easy to compute. It is not difficult to show that fractal
This section will describe various definitions of fractal dimension is invariant to choice of norm.
dimension, with some discussion of how they are related to
one another and to Hausdorff's original definition and how Pointwise Dimension
they can be extended to generalized dimension. An algo- The pointwise dimension is a local measure of the dimension
rithm based on box counting will also be presented as a way of the fractal set at a point on the attractor. Let .Bx(r)
to illustrate some of the notions. The next section will then denote the ball of radius r centered at the point X. (Wheth-
take a more thorough look at the various algorithms that er this ball is a hypersphere or a hypercube will depend on
have been suggested for the practical estimation of fractal the norm L,.) Define the pointwise mass function Bx(r) as
dimension. the measure
In what follows, three different ways of thinking about
dimension will be pursued. First, dimension will be defined Bx(r) = A[Bx(r)] (16)
intuitively as a scaling of bulk with size. Then, a more
formal definition, which involves coarse-grained volumes, The scaling of the mass function at X with the radius r
will be given. Finally, the notion of counting degrees of defines the pointwise dimension at X (Ref. 42):
freedom will be related to the information dimension. Each
approach yields equivalent definitions for dimension and for Dp(X) = lim log Bx(r) (17)
generalized dimension, but each attempts to provide a dif- r-O log r
ferent intuition for understanding what the dimension The pointwise dimension is a local quantity, but one can
means. define the average pointwise dimension to be a global quan-
tity:
Local Scaling Comparison of Bulk with Size
A geometrically intuitive notion of dimension is as an expo-
nent that expresses the scaling of an object's bulk with its
D = J D(X)dgu(X). (18)
size:

bulk sizedimension (12) Coarse-Grained Volume and Hausdorff Dimension

Although Hausdorff's rigorous definition of dimension does
Here bulk may correspond to a volume, a mass, or even a not immediately lead to numerical estimates, it does provide
measure of information content, and size is a linear distance. a framework from which dimension-estimation algorithms
For example, the area (bulk) of a plane figure scales quadrat- can be derived.
ically with its diameter (size), and so it is two dimensional. The Hausdorff dimension is purely a description of the
The definition of dimension is usually cast as an equation of geometry of the fractal set and makes no reference to the a
the form priori measure A that may be defined on the attractor. In
fact, the Hausdorff definition begins by defining its own
dimension = lim log bulk (13) measure r, which corresponds to uniform density over the
size-'0 log size
fractal set.
where the limit of small size is taken to ensure invariance Suppose that.4 is the fractal whose dimension one wishes
over smooth coordinate changes. This small-size limit also to calculate. Let (r, A) = 1, , ... , SkI be a finite
implies that dimension is a local quantity and that any covering of 4 into sets whose diameters are less than r.
global definition of fractal dimension will require some kind That is, A C Ui=1 ki, and the diameter of each set satisfies
of averaging. bi -- b(Si) < r. Then the function
1060 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

F(.4, D, r) = in f
e(r,A4)i
I (19) n(r) - r-DH (22)
or, more formally, that
where inf is the infimum (or minimum) over all cover ings
satisfying i < r, defines a kind of coarse-grained measuree for DH = M log[1/n(r)] (23)
r-.O log r
the set A. For example, if D = 1, then r(.4, D, r) gives3the
length of the set.4 as measured with a ruler of length r, and, Here the local notion of bulk is replaced with a global one:
as r - 0, r approaches the actual length of A. For rnost 1/n(r) is the average bulk of each nonempty box, since each
values of D, the r - 0 limit leads to a degenerate measiure: box contains, on average, 1/n(r) of the whole fractal.
either r - 0 orr - . That is not surprising: A figIure
with finite length will have zero area, and a finite arc,a is Generalized Dimension
covered by a curve of infinite length. Something theit is In computing the box-counting dimension, one either counts
similar to a coastline will have infinite length and zero area. or does not count a box according to whether there are some
One can think of r(.4, D) as the D-dimensional volumLeof points or no points in the box. No provision is made for
the set A. In fact, since r(.A, D, r) is a function that weighting the box count according to how many points are
decreases monotonically with D, there is a unique transition inside a box. In other words, the geometrical structure of
point DH that defines the Hausdorff dimension: the fractal set is analyzed but the underlying measure is
ignored.
co for D < DH
r(A, D) = im sup r(., D, r) = o for D<DH (20) The generalized dimension that was introduced in Ref. 43,
r-O 10for D >DH' and independently in Ref. 44, does take into account the
number of points in the box. Let !Bi denote the ith box, and
so that DH = inflD:r(.4, D) = Oj defines the Hausdorff let Pi = y(Zi)/,(.4) be the normalized measure of this box.
dimension. Equivalently, it is the probability for a randomly chosen
For instance, the fractal coastline may have a one-dimen- point on the attractor to be in fBi, and it is usually estimated
sional volume (length) of infinity and a two-dimensional by counting the number of points that are in the ith box and
volume (area) of zero, but there is a D between 1 and 2 at dividing by the total number of points.
which the D volume crosses over from to 0, and that value The generalized dimension is defined by
of D is the Hausdorff dimension of the coastline.
The coarse-grained measure defined in Eq. (19) typically
exhibits the scaling r(.A, D, r) - rD-DH, which gives another log E Piq
way to estimate dimension. For instance, taking D = 1, one Dq _lim l (24)
can estimate r(.A, D, r) for a coastline, say, by measuring its q - 1-o log r
length with rulers of ever-decreasing length r. The mea-
sured length will be ever increasing at a rate L(r) rl-DH, Writing the sum of Piq as a weighted average piq =
which gives DH. EE p1(piq-) = (piq-1), one can associate bulk with the gener-
Having defined the Hausdorff dimension DH, we give the alized average probability per box ((piq1))1/(q-1) and identi-
Hausdorff measure on the set A by r(.A, DH). The Haus- fyDq as a scaling of bulk with size. For q = 2 the generalized
dorff measure of a subset 2 of A is given by r(2, DH) = average is the ordinary arithmetic average, and for q = 3 it is
lim supr-.oinfe(r, B) i bil, and this measure is one way to a root mean square. It is not hard to show that the limit q
describe the bulk of the set S. This equation suggests that 1 leads to a geometric average. Finally, it is noted that q = 0
r(m, DH) - (B)DH, which echoes the form of Eq. (12) and corresponds to the plain box-counting dimension defined
more prominently displays the role of the Hausdorff dimen- above.
sion as a local scaling exponent. For a uniform fractal, with all Pi equal, one obtains a
generalized dimension Dq that does not vary with q. For a
Definition of Box-Counting Dimension nonuniform fractal, however, the variation of Dq with q
The difficulty with implementing the Hausdorff dimension quantifies the nonuniformity. For instance,
numerically is the infimum over all coverings that is taken in
Eq. (19). If this requirement is relaxed and instead one log(max Pi)
chooses a covering that is simply a fixed-size grid, one ob- D. = lim ' , (25)
r- O log r
tains an upper bound on the Hausdorff dimension that has
been variously referred to as the capacity, the box-counting
dimension, and the fractal dimension. The last term, how- log mn Pi)
ever, has come to be used in a generic sense for any dimen- D_ =lim logi (26)
r-o log r
sion that may be nonintegral. For most fractal sets of inter-
est, the capacity and the Hausdorff dimension are equal. 42 It is clear from Eq. (24) that Dq decreases with increasing q.
With the grid size r, Eq. (19) becomes From this fact and the above equations, it is clear that the
maximum dimension D_ is associated with the least-dense
points on the fractal and the minimum dimension D. corre-
r(.A, D, r) = = rD = n(r)rD, (21)
sponds to the most-dense points. This should not be sur-
i i
prising: The densest set possible is a point, which has di-
where n(r) is the number of nonempty grid boxes. The box- mension zero.
counting dimension is the value of D on the transition be- The notion of generalized dimension first arose out of a
tween r - 0 and r - O. Here r(., DH, r) - 1 implies that need to understand why various algorithms gave different
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1061

answers for dimension. A further motivation came from the

need to characterize more fully fractals with nonuniform
Z, Piq = J n(a, r)rqdad (33)
measure. These sets are sometimes called multifractals and
are characterized by an a priorimeasure , that differs from rf(a)rqadd r° (34)
the Hausdorff measure r. An extensive review of multifrac-
tals in a variety of physical contexts (this includes solid
fractal objects as well as strange attractors) can be found in where 0 = minafqa - f(a)) since the integral will be dominat-
Ref. 45. The point is that, rather than measure just one ed by the smallest exponent of r when r - 0. Comparing
dimension, one can compute the full spectrum of generalized this to the definition of generalized dimension in Eq. (24),
which gives P -q r(q-l)Dq, one obtains (q - )Dq =
dimensions from D_ to D.
The formalism of coarse-grained measure introduced in mina{qa - f(AO), which is the formula given in Eq. (32).
Hausdorff's definition of dimension can be generalized. In- From this formula, Eqs. (29)-(31) follow. See Fig. 3.
stead of Eq. (19), one can write This scaling n(a, r) r-f(a) suggests interpreting f(a) as
the Hausdorff dimension of the points with scaling index a
[compare with formula (22)]. Indeed, the scaling index a is
rq(.A, D, r) = qi(-q)D, (27) associated with the pointwise dimension Dp, and the set
= {X
SaX .:Dp(X) =al (35)
where Ai = A(.Bi) is the a priorimeasure of the ith element of
the covering. Properly the sum in the above equation is
preceeded by either an inf or a sup over all coverings, accord-
ing to whether q is less than or greater than one.
As before, the r - 0 limit gives rq(.4, D) - 0 for (1 - q)D 1.0
...............

< (1 - q)Dq and rq(.A, D) - for (1 - q)D > (1 - q)Dq. \a(q)

And the transition between r - 0 and r - defines the 0.8 L
generalized dimension:

Dq = q)D : lim rq(, D, r) = 01.

1 -1 q inf{(1 - r
(28) 0.6 -

f \""'q).. --...................
The advantage of Eqs. (27) and (28) is that they provide a 0.4
definition of generalized dimension without requiring fixed- f(q)/
size boxes.
0.2
Spectrum of Scaling Indices: fa)
Halsey et al.4 6 introduced a change of variables that provides
a new interpretation of generalized dimension. Letr = (q - 0 _
l)Dq and take a Legendre transformation from the variables -30 -15 0 15 30
(q, -r)into a new set of variables (a, /): (a) q

a- d f = aq -r (29) 1.0
O'
q

and 0.8

=-
Of r = aq-f. Do
q (30)
o~a 0.6

In case either r(q) or f(a) is not differentiable, a more robust

formulation is given by 0.4

f(a) = minfqa -r(q), (31)

0.2

r(q) = minfqa - f(e)j. (32)

0
0 0.2 0.4 0.6 0.8 1.0
The authors of Ref. 46 interpret f(a) as a "spectrum of (b) a
scaling indices." The variable a is the scaling index: !i = Fig. 3. Generalized dimension. (a) Dq as a function of q for a
bia defines the local scaling at the ith element of the cover. typical multifractal. Also shown are both f and a as a function of q
Then, the number n(a, r) of cover elements with scaling for the same multifractal. (b) f as a function of a. The curve is
index between a and a + Aa scales as n(a, r) - r-f(a)Aa. always convex upward, and the peak of the curve occurs at q = 0. At
this pointf is equal to the fractal dimension Do. Also, the f(a) curve
To see that this interpretation leads to Eqs. (29)-(32), is tangent to the curve f = a, and the point of tangency occurs at q =
consider the fixed-box-size sum , pq. The number of 1. In general, the left-hand branch corresponds to q > 0 and the
terms in this sum for which Pi = ra is given by n(a, r). Thus right-hand branch to q < 0.
1062 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

is the set of all points in A for which the pointwise dimen- where Pi is the probability measure of the ith box: Pi =
sion is a. The Hausdorff dimension of the set Sa is given by As(.B)/t(.). This relation leads directly to an expression
f(a). for the information dimension of the attractor:
The reader who is hoping for a dramatic picture of this
fractal set Sa will probably be disappointed. As Sakar47 and DI = lim -S(r) (37)
others have pointed out, Sa is not necessarily a closed set and r-O log 2 r

may even be dense in the original fractalA. Thus, although

the set Sa may have a lower Hausdorff dimension than A, it
is possible for the box-counting dimension to be the same.
Z Pi log 2 Pi

And a picture of Sa would look just like the picture of the = lim (38)
original set A4. r-O log 2 r
The f(a) formalism provides a tool for testing the notion of
Renyi5 5 has defined a generalized information measure:
universality, which states that a wide variety of dynamical
systems should behave in a similar way and should leave the
same characteristic signatures. Indeed, several research- Sq(r) = I log E piq, (39)
ers4850 have found physical systems whose f(a) curves pre-
cisely matched the f(a) associated with a theoretical model
of a circle map undergoing a transition from quasi-periodici- which reduces to Shannon's formula in the limit q - 1. The
ty to chaos. generalized information dimension associated with the
Discussions of f(a) from the viewpoint of a thermodynam- Renyi entropy is just the generalized dimension that has
ic formalism can be found in Refs. 51-54. Here the authors been defined above by another approach. Thus
attempt to work backward by constructing a dynamical sys-
tem that exhibits the desired f(a) structure. log >,Pi
Information Dimension Dq = lim =q()1 __lim (40)
r-O log r q-1 r-O log r
As an alternative to the scaling of mass with size, one can also
think of the dimension of a set in terms of how many real which is the same as Eq. (24).
numbers are needed to specify a point on that set. For
instance, the position of a point on a line can be labeled by a
single real number, the position on a plane by two Cartesian ALGORITHMS FOR ESTIMATING DIMENSION
coordinates, and the position in ordinary (three-dimension- It is emphasized that numerical techniques can only esti-
al) space by three coordinates. Here, dimension is some- mate the dimension of a fractal. Practical estimation of
thing that counts the number of degrees of freedom. For dimension begins with a finite description of the fractal
sets more complicated than lines, surfaces, and volumes, object. This description may be a digitized photograph with
however, this informal definition of dimension needs to be finite resolution, an aggregation with a finite number of
extended. aggregrates, or a finite sample of points from the trajectory
One way to extend this definition is to determine not how of a dynamical system. In any case, what is sought is the
many real numbers but how many bits of information are dimension not of the finite description but of the underlying
needed to specify a point to a given accuracy. On a line set.
segment of unit length, k bits are needed to specify the In the previous section an algorithm based on box count-
position of a point to within r = 2 -k. For a unit square, 2k ing was introduced. However, the box-counting algorithm
bits are needed to achieve the same accuracy (k bits for each has a number of practical limitations,5 6 particularly at a high
of the two coordinates specified). And similarly, 3k bits are embedding dimension, and so a variety of other algorithms
needed for a three-dimensional cube. In general, S(r) = -d have also been developed.
log2(r) bits of information are needed to specify the position The most popular way to compute dimension is to use the
of a unit d-dimensional hypercube to an accuracy of r. This correlation algorithm, which estimates dimension based on
example leads to a natural definition for the information the statistics of pairwise distances. The box-counting algo-
dimension of a set; it is given by the small r limit of -S(r)/ rithm and the correlation algorithm are both in the class of
log2(r), where S(r) is the information (in bits) needed to fixed-size algorithms because they are based on the scaling
specify a point on the set to an accuracy r. If S(r) is the of mass with size for fixed-sized balls (or grids). An alterna-
entropy, then 2S(r) is the total number of available states, tive approach uses fixed-mass balls, usually by looking at the
and 2-S(r) can then be interpreted as the average bulk of each statistics of distances to kth nearest neighbors. Both fixed-
state. This interpretation permits one to express the infor- size and fixed-mass algorithms can be applied to estimation
mation dimension as a scaling of bulk with size. of generalized dimension Dq, although fixed-size algorithms
Consider partitioning the fractal into boxes Ji of size r. do not work well for q < 1.57
To specify the position of a point to an accuracy r requires Also discussed are methods that directly involve the dy-
that one specify in which box the point is. The average namical properties of the strange attractor. The Kaplan-
information needed to specify one box is given by Shannon's Yorke conjecture, for example, relates dimension to the Lya-
formula: punov exponents. Recently interest has focused on trying
to determine the unstable periodic orbits (these compose the
S(r) = - E Pi log2 Pi, (36) "Cheshire set" 58) of the attractor. The most direct use of
the dynamics is to make predictions of the future of the time
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1063

series. Successful predictions provide a reliable indication Here, Bxj(r) can be approximated from a finite data set of
that the dynamics is deterministic and low dimensional. size N by
Finally the notion of intrinsic dimension is introduced.
This is an integer dimension that provides an upper bound Bxj(r) #fXi:i rFj and Xi-Xj r
on the fractal dimension of the attractor by looking for the N -
lowest-dimensional manifold that can (at least locally) con- N
fine the data. =N 1 (r -Xi-Xj), (43)
A number of these algorithms can be used to compute ,=1
i,,j
generalized dimension, but from the point of view of practi-
cal estimation it bears remarking that this application is not where 0 is the Heaviside step function: (x) is zero for x <0
always useful. A generalized dimension is useful for quanti- and one for x 2 0. The importance of excluding i = j has
fying the nonuniformity of the fractal or, in general, for been overlooked by some authors, although Grassberger has
characterizing its multifractal properties. And this use is stressed this point.6 3 In fact, the case is made in Ref. 64 for
important if one wants to compare an exact and predictive excluding all values of i for which i - jl < W with W > 1. It
theory with an experimental result. On the other hand, the is now straightforward to approximate C(r) with a finite data
goal of dimension estimation is often more qualitative in set:
nature. One wants to know only whether the number of
degrees of freedom is large or reasonably small. To answer
the question "Is it chaos or is it noise?" a robust estimate of
dimension is more important than a precise estimate. In
C(N, r) = k1 N

j=1
Bxj(r)

these cases the subtle distinction between information di-

mension at q = 1 and correlation dimension at q = 2, say, (44)
may not be so important as the more basic issues that arise N(N-1)
= E(r - XX - Xj).
from experimental noise, finite samples, or even computa-
tional efficiency. In words,

Average Pointwise Mass Algorithms C(N, r) = # of distances less than r (45)

# of distances altogether'
Recall the definition of the pointwise mass function in Eq.
(16): Bx(r) = u[x(r)], where .Jx(r) is the ball of radius r Thus the correlation algorithm provides an estimate of di-
about the point X. mension based purely on the statistics of pairwise distances.
The scaling of Bx(r) with r is what defines the pointwise Not only is this a particularly elegant formulation but it has
dimension at X. If this scaling is the same for all X, then the the substantial advantage that the function C(r) is approxi-
fractal is uniform, and it follows that Dq is a constant inde- mated even for r as small as the minimum interpoint dis-
pendent of q and has the value of the pointwise dimension. tance. For N points, C(N, r) has a dynamic range of O(N2).
For most fractals, however, pointwise dimension is not a Logarithmically speaking, this range is twice that available
global quantity, and some averaging is necessary for the to h(N, r) in the box-counting method. It is also twice the
quantity to be made global. range available in an estimate of the pointwise mass function
From the point of view of practical computation, it is Bx(r) for a single point X. This greater range is the one
possible to compute the pointwise dimension at a sample of advantage that the correlation integral has over the average
points on the attractor and then to average these values to pointwise dimension.
obtain an estimate of the dimension of the attractor. This
average is one that in principle gives the information dimen- Generalized Dimension from Averaged Pointwise Mass
sion D1. This approach has been advocated, for example, in A more general average than the direct arithmetic average
Ref. 59. used above is given by
However, it is probably more efficient to estimate dimen-
sion by averaging the mass function before the limit in Eq. Gq(r) = [(BX(r)q-l)]11(q-1) (46)
(17) is taken. It is in this way that the statistics of the
interpoint distances can be most effectively exploited. The scaling of this average with r gives the generalized di-
mension: Gq(r) - Dq or
Correlation Dimension
The most natural such averaging strategy was introduced by Dq = lim log Gq(r) (47)
Grassberger and Procaccia 60 '61 and independently by Ta- r- O log r
kens.62 Here, a direct arithmetic average of the pointwise
mass function gives what Grassberger and Procaccia call the Note that q = 2 gives the direct arithmetic average that
defines the correlation dimension and that the q = 1 average
correlation integral:
(which is associated with the information dimension),
C(r) = (Bx(r)). (41)
Gi(r) = lim Gq(r) = exp(logBx(r)), (48)
q-1
From this, the correlation dimension v is defined:
is a geometric average of the pointwise mass function.
=lim log C(r) (42) From a finite set of points, Gq(N, r) can be approximated
r- O log r by65
1064 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

,y/D(-y)
Gq(N, r) rk') - (k/M (53)
lN N _q-1) 11(q-1)
This dimension function D(,y) is related to the generalized
# [.EU 1 E O(r - Xi - xl) (49) dimension by the implicit formulas
,y = (q-l)Dq, D(y) = Dq. (54)
This formula is easiest to evaluate when q = 2, in which 70
case it reduces to the simple form given in Eq. (44). Grassberger derived Eqs. (54) independently and fur-
Algorithms based on this equation for the general case q z ther provided a small k correction to the scaling of (rY) with
2 have been pursued in Refs. 35, 65, and 66. This approach
does not work well for q < 1 since the term that is raised to [ (k y/D)]
the power of q - 1 can be zero for r much larger than the - [r"F(k)kY/D IkN 'yD (55)
smallest interpoint distance. The formula ought to work
well for large positive values of q, however. Further pursuit of the dimension function can be found in
Ref. 71. Further corrections and discussions of efficient
q-Point Correlation implementation are provided in Ref. 72; see also Ref. 73.
Grassberger44 suggested a q-point correlation integral de- Applications of nearest-neighbor methods for estimating in-
fined by counting q-tuples of points that have the property trinsic dimension can be traced back to Ref. 74 (see the
that every pair of points in the q-tuple is separated by a cross-disciplinary review in Ref. 75), but these do not give a
distance of less than r: fractal dimension and were not applied to chaotic systems.

Cq(N, r) =q #(Xi. Xiq):"X -XJ r Algorithms That Use Dynamical Information

Recurrence Time
for all n, m e il, . . . Q, (50) Let Tx,(r) be the r-recurrence time of an initial condition X0.
which has the scaling behavior Cq(r) r(q-l)Dq. Thus This is the minimum amount of time necessary for the tra-
jectory of X0 to come back to within r of X0 . The inverse
D = I lim lim log Cq(Nr) (51) recurrence time provides an estimate of the pointwise mass
q - 1 rm Nm logr function, and one expects
This equation can in principle be applied to all integers q > 1 r-'Bxo(r).
(
(56)
2, although its implementation is awkward for q > 3, since Tx,(r)
the number of q-tuples Nq grows so rapidly with N.
Thus the scaling of the average recurrence time with r can be
k-Nearest-Neighbor (Fixed-Mass) Algorithms related to the generalized dimension 44' 48:
In contrast to the average pointwise mass functions de- [(TX(r)1 q)]1(1-q1 _ [(BX(r)q-1)]11(rq) (57)
scribed above, in which the variation of mass inside fixed-
size balls is considered, the nearest-neighbor algorithms
Lyapunov Dimension
consider the scaling of sizes in fixed-mass balls.
The Kaplan-Yorke conjecture 7 67 7 relates the fractal dimen-
An early implementation of this notion in a chaotic-dy-
namics context was suggested in Ref. 67. (Guckenheimer sion of a strange attractor to the Lyapunov exponents. The
and Buzyna6 8 used a method like this one on experimental conjectured formula defines what has come to be called the
Lyapunov dimension:
data.) Here one computes (rk), which is the average dis-
tance to the kth nearest neighbor, as a function of k. Let EJ .
R(Xi, k) denote the distance between point Xi and its kth dL =ji +. (58)I
(58)
nearest neighbor. [By convention, the 0th nearest neighbor
is the point itself, so R(X, 0) = 0 for all X.] The average is where 1Xi, . . , ,,, are the Lyapunov exponents in decreas-
computed: ing order and j is given by
k
-= R(Xi k). (52) j= suPXk: i>01.
i=l

The scaling (rk) - klD defines the dimension. (Actually, The conjecture is that the Lyapunov dimension corresponds
what was computed in Ref. 67 was (rk2 ), since for Euclidean to the information dimension Dq with q = 1.
distances this equation is computationally more efficient. Farmer 7 8 has used this method to achieve dimension esti-
And the scaling was taken to be (rk2 ) - k2/D.) The authors mates of as many as 20. Matsumoto 7 9 studies a model of
conjectured that they had estimated the Hausdorff dimen- optical turbulence that leads to an attractor with an estimat-
sion, although this statement was corrected by Badii and ed Lyapunov dimension of approximately 45. Such num-
Politi69 and by Grassberger. 70 bers are far too large to be estimated by conventional tech-
Badii and Politi69 consider moments of the average dis- niques. Badii and Politi80 used Lyapunov scaling to obtain
tance to the kth nearest neighbor and recommend keeping k extremely precise measurements of generalized dimension
fixed and computing a dimension function from the scaling for constant Jacobian maps. In each of these cases, howev-
of average moments of rk with the total number of points N: er, the equations of motion themselves were required.
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1065

When the equations of motion are available, computation of the smallest embedding dimension for which good predic-
Lyapunov exponents, and therefore of Lyapunov dimension, tions can be made is a reliable upper bound on the number of
is much easier. degrees of freedom in the time series.

Unstable Periodic Orbits IMPLEMENTATION OF BOX-COUNTING

It has been suggested 8l that the natural invariant measure ALGORITHMS
(and from this, the fractal dimension) of a chaotic attractor
can be systematically approximated by sets of unstable peri- Some of the first numerical estimates of fractal dimension
odic orbits. This tool is a powerful analytical one and shows were obtained with the box-counting method. 95 In this sec-
promise computationally, as well. See Refs. 82-85 for fur- tion some of the practical issues that arise in the implemen-
ther discussion. tation of box-counting algorithms are discussed. The next
section will discuss some of the same issues for the more
Prediction Methods popular correlation dimension.
In discussing algorithms for predicting the future of time To compute the box-counting dimension, break up the
series, Farmer and Sidorowich40 suggested several ways in embedding space Rm into a grid of boxes of size r. Count the
which these prediction algorithms might be used to make number of boxes n(r) inside which at least one point from the
more accurate and reliable dimension estimates. One attractor lies. The scaling n(r) - r-Do defines the box-
promising possibility is to use bootstrapping: Here a short counting dimension, which is formally defined by the limit in
time series of length N is extrapolated far into the future, Eq. (23). There are some obstacles, however, in applying
and the longer time series is used in a conventional dimen- Eq. (23) directly to an experimentally observed fractal.
sion algorithm. This method is appropriate when the data
set is small and computation time is not the limiting factor. Finite Resolution
A second possibility is to use the scaling of prediction error For a solid fractal object, only a finite resolution is available,
with N, the number of points from which the prediction is so the limit r - 0 cannot be taken. A natural and direct
made. This scaling can be related to the dimension, and it approximation is just to apply Eq. (23) directly but with the
has the desirable property that if it is observed, that is, if the smallest r available. That is,
prediction error is noticably reduced with increasing N, then
the system can be reliably considered low dimensional. Do log[l/n(r)] (59)
log r
Intrinsic Dimension The problem with this approximation is that it converges
Unlike the algorithms discussed above, which seek to com- with logarithmic slowness in r. For instance, n(r) = nor-Do
pute dimension as accurately as possible from the given gives
data, the algorithms in this section seek only to bound the
log[(1/no)rD] Do log r + log(1/no)
fractal dimension with an integer. The estimate is coarser
but, it is hoped, more reliable than the previous estimates. log r log r
As Somorjai's review 75 points out, the notion of local in- slow to vanish as r 0
trinsic dimension was first discussed by Fukunaga and Ol- = Do + log(1/no) Do. (60)
sen.86 The approach was first applied to dynamical systems
log r
by Froehling et al.8 7 Here the authors attempt to find D
dimensional hyperplanes that confine the data in a local Instead, log n(r) versus log r is usually plotted. The nega-
region. tive slope of this curve will give Do for small r:
A variety of similar approaches have been considered.
Broomhead et al.8 8 use principal-value decomposition to Do - A[(log n(r)] (61)
make the fit to the D dimensional hyperplane. An approach A(log r)
more closely related to the correlation dimension is de- The criterion for choosing the best slope is not immediately
scribed in Ref. 89. Another approach based on topological obvious. It is clear that the greater the range (the larger the
considerations is described in Ref. 33. An algorithm that lever arm) the better, but this range is limited on one end by
averages the local intrinsic dimension is discussed in Ref. 90. finite resolution and on the other end by the fractal's own
Although intrinsic dimension is necessarily an integer quan- finite size.
tity, the averaging in this algorithm gives a noninteger result
that the authors conjecture to be related to the fractal di- Finite N
mension. Brock et al. have developed a statistic based on For a dynamical attractor, although the points may be
the delay-time embedding procedure and the correlation known to a high precision, only a finite sample of N points is
integral for distinguishing m-dimensional deterministic available. The natural approximation in this case is to esti-
time series from uncorrelated noise.9' An analog electronic mate n(r) with h(N, r), the number of boxes inside which at
estimation of intrinsic dimension is described in Ref. 92. An least one of the sample points lies. Here, although h(N, r)
anecdotal comparison of an intrinsic dimension estimator clearly underestimates n(r), it is expected to be a good ap-
and the correlation dimension estimator can be found in Ref. proximation for large N since for fixed r:
93.
The prediction algorithms of Farmer and Sidorowich 40 94 n(r) = lim h(N, r). (62)
can also be used to estimate intrinsic dimension. Basically, N-
1066 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

Returning to the formal definition in Eq. (23), we find that IMPLEMENTATION OF THE CORRELATION
ALGORITHM
Do = lim lim log[1/h(N, r)] (63) One reason for the popularity of the correlation algorithm is
r--O N-' log r that it is easy to implement. One computes the correlation
integral C(N, r) merely by counting distances. Another and
The order of the limits is crucial. For finite N, h(N, r) is more fundamental advantage is that the correlation algo-
bounded by N (there can be no more nonempty boxes than rithm treats all distances equally, so one can probe to dis-
there are things to put in the boxes), and r - 0 gives a tance scales that are limited only by the smallest interpoint
dimension of zero. In a formal sense, this result is not distance (which will be of the order of N-2/D). Most other
surprising: A finite set of points does have dimension zero. algorithms are limited by the average distance to the nearest
But what is wanted is the dimension of the underlying at- neighbor (which is of the order of N-'ID). That is, the
tractor, not the dimension of the finite sample of points. correlation integral C(N, r) has a dynamic range of O(N 2 );
The finite sample is a serious limitation because, particu- this compares, for instance, with the O(N) range of the func-
larly for multifractal attractors that are nonuniformly popu- tion (N, r) that appears in the box-counting algorithm.
lated with points, the limit in Eq. (62) converges slowly. Recall Eq. (45), which defines C(N, r) as the fraction of
Grassberger 96 has fitted the form distances less than r. Then, the correlation dimension is
given by
n(r) - h(N, r) -r-N-f (64)
v = lim lim log C(N, r) (65)
and used the fit to accelerate the convergence. A similar r-O N-- log r
approach appears in Ref. 97. Caswell and Yorke98 point out However, there are a variety of practical issues and potential
that one way to increase the speed of convergence is to use pitfalls that come with making an estimate from finite data.
spheres instead of boxes.
Extraction of the Slope
Application to Generalized Dimension Extracting dimension directly from the correlation integral
The box-counting algorithm is one of a class of algorithms according to Eq. (65) is extremely inefficient, since the con-
based on fixed size and as such is not so well suited as the so- vergence to v as r - 0 is logarithmically slow. See relation
called fixed-mass algorithms for estimation of generalized (60) for the equivalent situation in the box-counting algo-
dimension of Dq when q < 1. Since the box-counting dimen- rithm.
sion corresponds to q = 0, the box-counting algorithm is a Taking a slope solves this problem. Either one can take
poor way to estimate the box-counting dimension. The the local slope:
modification of the box-counting algorithm for generalized
dimension will achieve better performance if q > 1. v(r) - d log C(N, r)
Indeed, the current interest in multifractals and their f(a) dlogr
characterization has led to a revival of the box-counting - dC(N, r)/dr
algorithm for estimation of generalized dimension. For ex- (66)
C(N, r)/r '
ample, Chhabra and Jensen 99 use box counting to compute
f(q) and (q) directly for experimental data. Their ap- or one can fit a chord through two points on the curve:
proach avoids the problem of having to evaluate the deriva- v(r) = A log C(N,r)
tive in Eqs. (29) numerically by analytically differentiating A log r
Eq. (24) for the generalized dimension.
log C(N, r2) - log C(N, r)
Computational Efficiency log r 2 -log r,
Depending on the details of the computer program that
implements the box-counting algorithm, either one must However, to implement this strategy requires the choice of
keep track of a large number of empty boxes, which can be a two length scales. The larger scale is limited by the size of
severe memory burden (in the small r limit almost all the the attractor, and the smaller scale is limited by the smallest
boxes will be empty), or else one must maintain a list of interpoint distance. The expression usually converges in
nonempty boxes. The latter case can be computationally the limit as r and r2 both go to zero, and convergence is
expensive since, for each point, the program has to check guaranteed as long as the ratio rl/r 2 also goes to zero.
whether the point belongs in any of the nonempty boxes on The direct difference in Eq. (67) fails to take full advan-
the list. An efficient compromise for low-dimensional em- tage of the information available in C(N, r) for values of r
bedding spaces is discussed in Ref. 96. between r, and r2. It is natural to attempt to fit a slope by
One can always compute n(N, r) in O(N log N) time by some kind of least-squares method, but this approach is
first discretizing the input points to the nearest integer mul- problematic. An unweighted, least-squares method is par-
tiple of r (this effectively converts the point's position to the ticularly poor, since the estimate of C(r) by C(N, r) is usually
position of the appropriate box), then sorting the discretized much better for large r than for small r. Weighted fits can
list (which is the same as sorting the list of boxes-a lexico- compensate for this effect, but there is still a problem be-
graphic or dictionary ordering is useful here), and finally cause successive values of C(N, r) are not independent.
counting the number of unique elements in the sorted list Since C(N, r + Ar) is equal to C(N, r) plus the fraction of the
(the number of boxes). distances between r and r + Ar, it is a mistake to assume that
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1067

C(N, r + Ar) is independent of C(N, r). An estimate of large: of the same order as the size of the attractor. Since it
dimension will depend on whether C(N, r) is sampled at is the short distances that are most important, and since
logarithmic intervals r = (0.001, 0.01, 0.1, etc.) or at uniform there are so few of them, it would be advantageous to orga-
intervals r = (0.001, 0.002, 0.003, etc.). Furthermore, the nize the points on the attractor so that only the short dis-
error estimate that the least-squares fit naturally provides tances are computed. The algorithm in Ref. 106 provides a
will have little to do with the actual error in the dimension way to compute the O(N) shortest distances with only O(N
estimate. log N) effort by placing the points into boxes of size ro. An
Takens' 00 has shown how this intermediate distance infor- alternative approach is to use multidimensional trees,' 0 7
mation can be incorporated in a consistent way. Using the such as were introduced by Fatrmer and Sidorowich 40 for
theory of best estimators, Takens derived an estimator for v nonlinear prediction. Such an organization of points would
that makes optimal use of the information in the interpoint also be useful for the k nearest-neighbor algorithms that
distances rij. In fact, the method uses all distances less than were discussed above. Hunt and Sullivan' 08 have also ar-
an upper bound ro, so that a lower cutoff distance need not gued for the importance of efficient data organization.
be specified: Franaszek' 09 has suggested an algorithm that computes
the correlation integral for a range of embedding dimensions
(r°) = 1 (68) simultaneously. A massively parallel analog optical compu-
tation of the correlation integral was performed by Lee and
where the angle brackets here refer to an average over all Moon."10
distances rij that are less than ro. In terms of the correlation
integral, this can be written'l' as Comparison with Filtered Noise
It is a common and well-advised tactic to compare the corre-
lation dimension computed for a given time series against a
d(ro)
= C(rd) (69)
set of test data whose properties are known in advance. For
J [C(r)/r]dr example, if the given time series has a significantly larger
C(N, r) at small r than does white noise, then one can rule out
the null hypothesis that the original time series is white
o [dC(r)/dr]dr
noise. However, one cannot rule out the possibility that the
O * (70) original time series is simply autocorrelated or colored noise.
f [C(r)/r]dr A more stringent test is to create a time series with the same
Fourier spectra as the original time series (for instance, one
The slightly more unwieldy form that is given in Eq. (70) is could take a Fourier transform, randomize the phases, and
meant to be compared with Eq. (66) for the local slope. then invert the transform). If the C(N, r) obtained from this
The Takens estimator not only provides an efficient time series is significantly different from that of the original
means to squeeze an optimal estimate out of the correlation time series, then a stronger statement can be made: Not
integral but also provides an estimate of the statistical error, only is the original time series not white noise, it could not
namely, av = v/NN, where N* is the number of distances less have been produced by any linear filter of white noise. A
than ro. However this error estimate makes the assumption simple version of this more stringent test was applied by
that all those N* distances are statistically independent. It Grassberger"' in rejecting earlier claims of low-dimensional
is argued in Ref. 102 that this assumption is valid only if N* chaos in a climatic time series.
< N because there cannot be N2 statistically independent Osborne and Provenzale"' recently made the striking ob-
quantities derived from only N independent points. This servation that stochastic systems with 1/pa power-law spec-
argument implies, for instance, that even for small dimen- tra can exhibit a finite correlation dimension when a > 1.
sions one cannot expect to get a precision of better than 1% This observation is not an artifact of finite N approximation
with fewer than 10,000 points. (Accuracy is an entirely but is a fundamental property of power-law correlation.
separate issue. The variety of sources of systematic error is This observation makes it all the more imperative that sto-
vast. Some of these errors will be discussed in later sec- chastic analogs of the original time series be created and
tions.) tested.
In Ref. 103 the Takens estimator is compared with the
local slope for a typical strange attractor. Ellner' 0 4 extends Sources of Error
the Takens estimator to a generalized dimension Dq, though There are two fundamental sources of error in the estimation
in a way that works only for integer q > 2 and efficiently only of dimension from a time series: statistical imprecision and
for q = 2. He also introduces a so-called local estimator systematic bias. The first arises directly from the finite
based on the same principle. In Ref. 105 it is pointed out sampling of the data (and as such is reasonably tractable);
that even though the Takens estimator uses all distances less the second comes from a wide variety of sources. Indeed, it
than ro, and therefore has in effect an arbitrarily long lever is this wide variety that has led to so much difficulty in the
arm, it is still sensitive to oscillations in the correlation field of verifying results as reliable.
integral. Computing fractal dimension is a tricky business. In the
words of Brandstater and Swinney,"13
Computation
Although there are O(N 2) interpoint distances to be consid- It is not difficult to develop an algorithm that will
ered in a time series of N points, most of these distances are yield numbers that can be called dimension, but it
1068 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

is far more difficult to be confident that those num- Discretization

bers truly represent the dynamics of the system. Discretized time series are of the form xi = kie, where ki is an
integer and e is the discretization level. Such discretization
Here is an incomplete survey of the kinds of problems that is a natural artifact of digital measuring devices. (In fact,
may arise, usually in the computation of correlation dimen- many algorithms work much faster with integer input than
sion. In some cases, remedies are suggested. Figure 4 real input, so that it may be computationally wise to convert.
shows what the ideal correlation integral curves look like and Usually this involves multiplication by some large factor,
how they are distorted by the various sources of error. followed by rounding to the nearest integer. The multipli-
cative factor obviously does not affect the slope of a log-log
Finite Sampling and StatisticalError plot, but the rounding to an integer is equivalent to a discre-
Ultimately, what limits most estimates of dimension is that tization.) It follows that distances between pairs of discre-
there are a limited number of points sampling the attractor. tized points will themselves be discrete multiples of e. This
Many of the systematic effects discussed below can be elimi- effect is most prominent at small r-indeed, pairs of points
nated in the N -- limit. with r = 0 occur with finite probability, and a log C(r) versus
First, a finite sample of N points limits the range over log r plot must deal with the r = 0 points. A model of points
which interpoint distances can scale. The correlation inte- on an m-dimensional lattice, with lattice points separated by
gral itself varies from 2/N2 to 1, so that any fit to a slope has e, leads to a scaling of C(r) (r + e/2)m to leading order in
only this range. In particular, any fit over a range R of e.10' This suggests that an appropriate plot for a general
distances (R is the ratio of largest to smallest distance scales) discretized time series is log C(r) versus log(r + E/2).
requires that N2 /2 2 RD,so that at least N = points are An alternative approach2 0 is deliberately to add noise of
required. This scaling, however, is an absolute lower bound. amplitude e (this is called dithering) to the original time
The minimum number of points necessary to get a reason- series.
able estimate of dimension for a D-dimensional fractal is
probably given by a formula of the form Edges and Finite Size
The finite size of a compact fractal object limits the range
Nmin = N0 OD, (71) over which C(r) scales as r. For r > 6, where 60 is the
diameter of the attractor, the correlation integral saturates
although it is difficult to provide generically good values for at C(r) = 1. This finite size effect is not necessarily a prob-
No and 0. Experience indicates that 0 should be of the lem in dimension calculations. As long as the effect is con-
order of 10, but experience also indicates the need for more fined to length scales that are larger than some r 6, then
experience. See Refs. 114-116 for other points of view. accurate estimates of dimension can still be obtained from
The statistical error in an estimate of dimension typically the slope of a log C(r) versus log r plot in the r r regime.
scales as (1/,/N), although the coefficient of that scaling The real problem stems from the edges that all finite-sized
can be surprisingly small; in some special cases, 0(1/N) objects in R- have. The neighborhoods around points near
scaling is observed.' 02 Reference 102 also discusses the the edge have different scaling from neighborhoods farther
trade-off between statistical and systematic error and its into the interior.
implications in the context of Eq. (71). Any model of edge effect will depend on the shape of the
fractal, but a tractible and reasonably generic model is that
Noise the fractal is a uniform hypercube of length 1 and dimension
Noise, the ultimate corrupter of measurements, is usually m. The correlation integral can be derived exactly in that
the first concern of the experimentalist. In the case of case: C(N, r) = (2r-r 2)m. The local slope at r of the log-log
dimension estimation, however, the effect of low-amplitude curve is given by
noise is often not so significant as other effects.
Although the particular trajectory of an initial condition is P = dC/dr m(2 - 2r) 1 r) (72)
-~~l 2 - r -2'
extremely sensitive to noise, the global structure of the
strange attractor is robust. Noise will perturb the trajectory
away from the strange attractor, but the dynamics will al- so that the relative error is Iv(r) - ml/m r/2. Since C(N, r)
ways pull it back toward the attractor. Fractal scaling (of varies from 2/N2 to 1, a natural choice of r for the local slope
bulk with size) will break down only at length scales equal to is that value for which C(N, r) = 1/N. This gives r N(-11-)l
the noise amplitude. Thus, at relatively high signal-to- 2, and the relative error p in the dimension estimate becomes
noise ratios, there is still a good range over which fractal p = N(-1/m)/4. Thus, for a given accuracy to be achieved, p
scales may be observed. requires at least N = (4p)-m points. For example, for 5%
Because the noise often possesses a much higher charac- error the minimum number of points needed is
teristic frequency than the deterministic attractor, it is
tempting to low-pass filter the signal to reduce the effects of Nmin = 5. (73)
noise. As a rule this is not recommended. As was pointed
out in Refs. 117 and 118, linear filtering can artifically raise If one were to use a value of r for which C(N, r) were of the
the measured dimension of the time series. On the other order of 1/N2 and if one employed the Takens best estimator
hand, nonlinear filtering techniques have been successfully instead of the local slope, the minimum number of points
employed by Kostolich and Yorke"19 to reduce the noise in needed for 5%error would be Nmin (1.25)m. It i possible
the time-series characterization of a strange attractor. Sim- to reduce this value even further if one's model for the
ilar methods have been proposed in Ref. 40. strange attractor is, instead of an m-dimensional hypercube,
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1069

12 ,C(N,r)

v
3 -

. 2 -

2 1 2 3 4 5 6 7
N2 r embedding dimension
'5
(a)
10°

Z Z 0.1
05 0

0.01 L
0.001 0.01 0.1

(b) (e)
100

Z z
2-
0

(C) (f)
10° 100

10-1
10-2

10-2

z Z 1o-
03 10-3 0

10 5
4 3
105 lo- lo- 10-2 10.1 100

(d) (9

Fig. 4. Sources of error in the correlation integral. (a) Ideally, the correlation integral C(N, r) scales as rm for m < v, and as r, for m > v over a
range from C(N, r) = 2/N2 to saturation at C(N, r) = 1. Here the dimension is somewhere between 2 and 3. This idealization, however, is only
approximated by correlation integrals computed from actual samples of time series data. (b) An actual correlation integral for a two-
dimensional chaotic attractor is shown here, with embedding dimensions m = 1 through m = 6. The finite sample size leads to poor statistics at
small r, and the finite size of the attractor (the edge effect) limits the scaling at large r. Nonetheless, the slopes are more or less constant over a
range of C(N, r) of the order of N2. (c) The effect of noise. With a the amplitude of the noise, one sees that, for r << a, a slope that approaches
the embedding dimension m is observed. For r >> a, the effect of the noise is unimportant. (d) The effect of discretization is to introduce a
stair step into the correlation integral (solid curve). The steps are all of equal width, but the log-log plot magnifies those at small r. The effect
is minimized if one plots log C(N, r) versus log r for r = (k + 1/2e), where k is an integer and e is the level of discretization (dashed-dotted curve).
(e) Lacunarity leads to an intrinsic oscillation in the correlation integral that inhibits accurate determinations of slope. The example here is
the correlation integral of the middle-thirds Cantor set. (f) Autocorrelation in the time-series data can lead to an anomalous shoulder in the
correlation integral. This effect is most highly pronounced for high-dimensional attractors. In this case the input time series was autocorre-
lated Gaussian noise, and the correlation integral was computed for various (large) embedding dimensions of m = 4 to m = 32. Equation (75)
corrects for this effect. (g) If the time-series data arise from a nonchaotic attractor, then the scaling of C(N, r) as rv begins to break down for
C(N, r) < 1/N. The dotted-dashed curve here has a slope of 2, corresponding to the two-dimensional quasi-periodic input data.
1070 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

an m torus. Indeed, for some fractals, such as the middle- tion time r is long compared with the sampling time, then an
thirds Cantor set, there is no edge effect at all. anomalous shoulder can appear in the correlation integral
On the other hand, the edge effect can be even stronger and can lead to inaccurate and possibly spurious estimates of
than this, if the edge is singularly sharp. An example of this dimension. 6 4
effect is the edge of the attractor that arises from the logistic One approach is to increase the sampling time,"30 although
map xn+1 = 4x(1 - x). The square-root singularity of this this may introduce other problems: It can further limit the
edge makes numerical computation of dimension even slow- available data and can also affect the delay-time embedding
er than was predicted by the uniform-measure model.6' strategy. However, there is a more effective solution. Re-
Smith2' has used the uniform-measure model to argue write the definition of the correlation integral:
that an estimate within 5% of the correct value requires a
minimum of Nmin = 42m data points. This value is quite a C(W, N, r) = (N+1-W)(N-W)
bit different from that given in Eq. (73), which points to the
sensitivity of these kinds of estimate to the assumptions in N-1 N-- n
the model. It is not at all clear that the edge effect is a good X N-E O(r - [lXi - X+n) (74)
model for making a prioriestimates of errors that arise from n=W i=0
dimension-estimation methods.
Cawley and Licht'03 have considered an algorithm that Note that W = 1 is just the standard algorithm. This algo-
computes correlation dimension with a truncated data set rithm computes distances between all pairs of points except
that attempts to sidestep the edge effect by deliberately for those that are closer together in time than W sampling
avoiding points that are far from the centroid of the data units. Eliminating this small selection of offending pairs
points. eliminates the anomalous shoulder without sacrificing the
statistics of O(N') distance calculations.
Lacunarity The definition, in words, of correlation integral in Eq. (45)
Dimension is not the only way to gauge the fractal-like ap- is now adjusted to
pearance of a set. Mandelbrot22 has pointed to lacunarity C(W, N r)
as another measure: He comments that for two fractals
having the same dimension, the one having the greater la- # of distances less than r except for those
cunarity is more textured and appears more fractallike. from pairs of points closer together in time than W
Further discussion of lacunarity as a means of characterizing # of distances altogether
fractals can be found in Refs. 123-125. (75)
From the point of view of dimension computation, lacun-
arity has the effect of introducing an intrinsic oscillation into
the correlation integral.26"127 If the lever arm over which
REVIEWS
the slope is estimated is long enough to encompass several The reader is referred to Gleick's best-seller Chaosl3las the
periods of these oscillations, then the effect of the oscillation easiest and undoubtedly the most enjoyable book from
will be minimized; but if attempts to compute dimension are which to start learning about chaos, fractals, and the dynam-
based on the local slope of the correlation integral, lacunar- ic personalities of scientists in the field. Mandelbrot's
ity can prevent convergence of the dimension estimator. book22 thoroughly discusses the topic of fractals qua frac-
For further discussion of lacunarity as an impediment to tals. This "indispensible, encyclopedic, exasperating"
good dimension calculations, see Refs. 105 and 128. (Gleick's words) volume contains many nice pictures and
On the other hand, it has been argued in Refs. 63 and 129 technical and historical information. Other books with
that the amplitudes of these oscillations generically damp good pictures include Refs. 6, 132, and 133; these books also
out in the limit r - 0. provide significant technical discussion of fractals.
A recent tutorial on nonlinear dynamics aimed at the
Nonchaotic Attractors engineer can be found in Ref. 134. This includes a discus-
The correlation algorithm does not efficiently compute di- sion of dimension and Lyapunov exponents and has recently
mension when the attractor is nonchaotic. Such attractors been expanded into a book.'35 Also see Ref. 136 for a simi-
(limit cycles and limit tori) typically have integer dimension larly motivated paper aimed at optical scientists.
and a Fourier spectrum that is not broadband. Reviews of chaos in general abound: A small sample in-
The problem with using the correlation algorithm is that cludes Refs. 21 and 137-140. A recommended text with
the trajectories of nonchaotic dynamical systems are too many references is Ref. 141. Reprint books provide more
regular, and the points that sample the attractor are not advanced references that are taken directly from the re-
even approximately independent. The observed effect is search literature. 6 142 143 Reviews of dimension and its cal-
that the correlation integral C(N, r) scales as rv from C(N, r) culation can be found in Refs. 59, 101, and 144; a particularly
= 0(1/N) to C(N, r) = 0(1) for nonchaotic attractors, where- rich (and not outdated) source is the workshop proceedings
as the scaling is complete [from 0(1/N2) to 0(1)] for chaotic edited by Mayer-Kress.145 See Ref. 146 for an early review
attractors. For further discussion of this effect, see Ref. 101. of experimental observations of chaos.

Autocorrelation
ACKNOWLEDGMENTS
Autocorrelation is inevitable in time-series data. For con-
tinuous signals x(t), there is always some time r over which I am grateful to Bette Korber, Michael Holz, and Courosh
x(t) and x(t + r) are strongly correlated. If this autocorrela- Mehanian for useful comments on this manuscript. I also
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1071

thank Joyce Michaels for help with the bibliography and Al py from time signals of dissipative and conservative dynamical
Gschwendtner for his generosity and encouragement. This systems," Phys. Rev. A 31, 1872 (1985).
27. J. P. Eckmann and I. Procaccia, "Fluctuations of dynamical
study was supported by the U.S. Air Force and the Defense scaling indices in nonlinear systems," Phys. Rev. A 34, 659
Advanced Research Projects Agency. (1986).
28. P. Sz6pfalusy and T. T61, "Dynamical fractal properties of one-
dimensional maps," Phys. Rev. A 35, 477 (1987).
REFERENCES 29. N. H. Packard, J. P. Crutchfield, J. D. Farmer, and R. S. Shaw,
"Geometry from a time series," Phys. Rev. Lett. 45,712 (1980).
1. L. M. Sander, "Fractal growth," Sci. Am. 256, 94 (1987). 30. F. Takens, "Detecting strange attractors in turbulence," in
2. L. M. Sander, "Fractal growth processes," Nature (London) Dynamical Systems and Turbulence, Warwick, 1980, D. A.
322, 789 (1986). Rand and L.-S. Young, eds., Vol. 898 of Springer Lecture Notes
3. J. Nittman, G. Daccord, and H. E. Stanley, "Fractal growth of in Mathematics (Springer-Verlag, Berlin, 1981), p. 366.
viscous fingers: quantative characterization of a fluid insta- 31. R. Man, "On the dimension of the compact invariant sets of
bility phenomenon," Nature (London) 314, 141 (1985). certain non-linear maps," in Dynamical Systems and Turbu-
4. L. Niemeyer, L. Pietronero, and H. J. Wiesmann, "Fractal lence, Warwick, 1980, D. A. Rand and L.-S. Young, eds., Vol.
dimension of dielectric breakdown," Phys. Rev. Lett. 52, 1033 898 of Springer Lecture Notes in Mathematics (Springer-Ver-
(1984). lag, Berlin, 1981), p. 320.
5. P.-Z. Wong, "The statistical physics of sedimentary rock," 32. A.M. Fraser and H. L. Swinney, "Independent coordinates for
Phys. Today 41(12), 24 (1988). strange attractors from mutual information," Phys. Rev. A 33,
6. M. F. Barnsley, Fractals Everywhere (Academic, Boston, 1134 (1986).
Mass., 1988). 33. W. Liebert, K. Pawelzik, and H. G. Schuster, Institut fir
7. D. Stauffer, Introduction to Percolation Theory (Taylor & Theoretische Physik, Universitfit Frankfurt, Frankfurt, Fed-
Francis, London, 1985). eral Republic of Germany, "Optimal embeddings of chaotic
8. T. A. Witten and L. M. Sander, "Diffusion limited aggregation, attractors from topological considerations," preprint (1989).
a kinetic critical phenomenon," Phys. Rev. Lett. 47, 1400 34. D. S. Broomhead and G. P. King, "Extracting qualitative dy-
(1981). namics from experimental data," Physica 20D, 217 (1986).
9. M. F. Barnsley and S. Demko, "Iterated function systems and 35. S. Sato, M. Sano, and Y. Sawada, "Practical methods of mea-
the global construction of fractals," Proc. R. Soc. London Ser. suring the generalized dimension and the largest Lyapunov
A 399, 243 (1985). exponent in high dimensional chaotic systems," Prog. Theor.
10. H. E. Stanley and N. Ostrosky, On Growth and Form: Fractal Phys. 77, 1 (1987).
and Non-FractalPatternsin Physics (Nijhoff, Boston, Mass., 36. A. I. Mees, P. E. Rapp, and L. S. Jennings, "Singular-value
1986). decomposition and embedding dimension," Phys. Rev. A 36,
11. A. J. Hurd, "Resource letter FR-1: fractals," Am. J. Phys. 56, 340 (1987).
969 (1988). 37. A. M. Albano, J. Muench, C. Schwartz, A. I. Mees, and P. E.
12. L. Kadanoff, "Where is the physics of fractals," Phys. Today Rapp, "Singular-value decomposition and the Grassberger-
39(2), 6 (1986). Procaccia algorithm," Phys. Rev. A 38, 3017 (1988).
13. P. H. Carter, R. Cawley, and R. D. Mauldin, "Mathematics of 38. A. M. Fraser, "Reconstructing attractors from scalar time se-
dimension measurement for graphs of functions," in Fractal ries: a comparison of singular system and redundancy criter-
Aspects of Materials,B. B. Mandelbrot and D. E. Passoja, eds. ia," Physica 34D, 391 (1989).
(Materials Research Society, Pittsburgh, Pa., 1985). 39. A.M. Fraser, "Information and entropy in strange attractors,"
14. B. Dubuc, J. F. Quiniou, C. Roques-Carmes, C. Tricot, and S. Ph.D. dissertation (University of Texas at Austin, Austin,
W. Zucker, "Evaluating the fractal dimension of profiles," Tex., 1988).
Phys. Rev. A 39, 1500 (1989). 40. J. D. Farmer and J. J. Sidorowich, "Exploiting chaos to predict
15. H. Haken, Information and Self-Organization: A Macro- the future and reduce noise," in Evolution, Learningand Cog-
scopic Approach to Complex Systems, Vol. 40 of Springer nition, Y. C. Lee, ed. (World Scientific, Singapore, 1988), p.
Series in Synergetics (Springer-Verlag, Berlin, 1988). 227.
16. R. S. MacKay and J. D. Meiss, HamiltonianDynamical Sys- 41. F. Hausdorff, "Dimension und ausseres Mass," Math. Annalen
tems (Hilger, Philadelphia, 1987). 79, 157 (1919).
17. J. Guckenheimer and P. Holmes, Nonlinear Oscillations,Dy- 42. J. D. Farmer, E. Ott, and J. A. Yorke, "The dimension of
namical Systems, and Bifurcations of Vector Fields,Vol.42 of chaotic attractors," Physica 7D, 153 (1983).
Springer Series in Applied Mathematical Sciences (Springer- 43. H. G. E. Hentschel and I. Procaccia, "The infinite number of
Verlag, New York, 1983). generalized dimensions of fractals and strange attractors,"
18. S. M. Hammel, C. K. R. T. Jones, and J. V. Moloney, "Global Physica 8D, 435 (1983).
dynamical behavior of the optical field in a ring cavity," J. Opt. 44. P. Grassberger, "Generalized dimensions of strange attrac-
Soc. Am. B 2, 552 (1985). tors," Phys. Lett. A 97, 227 (1983).
19. M. H6non, "Atwo-dimensional mapping with a strange attrac- 45. G. Paladin and A. Vulpiani, "Anomalous scaling laws in multi-
tor," Commun. Math. Phys. 50, 69 (1976). fractal objects," Phys. Rep. 156, 147 (1987).
20. K. J. Falconer, The Geometry of FractalSets, Vol. 85 of Cam- 46. T. C. Halsey, M. H. Jensen, L. P. Kadanoff, I. Procaccia, and B.
bridge Tracts in Mathematics (Cambridge U. Press, Cam- I. Shraiman, "Fractal measures and their singularities: the
bridge, 1985). characterization of strange sets," Phys. Rev. A 33, 1141 (1986).
21. J.-P. Eckmann and D. Ruelle, "Ergodic theory of chaos and 47. S. K. Sakar, "Multifractal description of singular measures in
strange attractors," Rev. Mod. Phys. 57, 617 (1985). dynamical systems," Phys. Rev. A 36, 4104 (1987).
22. A. Wolf, J. B. Swift, H. L. Swinney, and J. A. Vastano, "Deter- 48. M. H. Jensen, L. P. Kadanoff, A. Libchaber, I. Procaccia, and
mining Lyapunov exponents from a time series," Physica 16D, J. Stavans, "Global universality at the onset of chaos: results
285 (1985). of a forced Rayleigh-Bernard experiment," Phys. Rev. Lett.
23. J.-P. Eckmann, S. 0. Kamphorst, D. Ruelle, and S. Ciliberto, 55, 2798 (1985).
"Liapunov exponents from a time series," Phys. Rev. A 34, 49. E. G. Gwinn and R. M. Westervelt, "Scaling structure of at-
4971 (1986). tractors at the transition from quasiperiodicity to chaos in
24. R. Stoop and P. F. Meier, "Evaluation of Lyapunov exponents electronic transport in Ge," Phys. Rev. Lett. 59, 157 (1987).
and scaling functions from time series," J. Opt. Soc. Am. B 5, 50. J. A. Glazier, G. Gunaratne, and A. Libchaber, "f(a) curves:
1037 (1988). experimental results," Phys. Rev. A 37, 523 (1988).
25. P. Grassberger and I. Procaccia, "Estimation of the Kolmo- 51. M. J. Feigenbaum, M. H. Jensen, and I. Procaccia, "Time
gorov entropy from a chaotic signal," Phys. Rev. A 28, 2591 ordering and the thermodynamics of strange sets: theory and
(1983). experimental tests," Phys. Rev. Lett. 57, 1503 (1986).
26. A. Cohen and I. Procaccia, "Computing the Kolmogorov entro- 52. D. Katzen and I. Procaccia, "Phase transitions in the thermo-
1072 J. Opt. Soc. Am. A/Vol. 7, No. 6/June 1990 James Theiler

dynamic formalism of multifractals," Phys. Rev. Lett. 58,1169 78. J. D. Farmer, "Chaotic attractors of an infinite dimensional
(1987). dynamical system," Physica 4D, 366 (1982).
53. M. H. Jensen, L. P. Kadanoff, and I. Procaccia, "Scaling struc- 79. K. Ikeda and K. Matsumoto, "Study of a high-dimensional
ture and thermodynamics of strange sets," Phys. Rev. A 36, chaotic attractor," J. Stat. Phys. 44, 955 (1986).
1409 (1987). 80. R. Badii and A. Politi, "Renyi dimensions from local expansion
54. T. Bohr and M. H. Jensen, "Order parameter, symmetry rates," Phys. Rev. A 35, 1288 (1987).
breaking, and phase transitions in the description of multifrac- 81. D. Auerbach, P. Cvitanovi, J.-P. Eckmann, G. Gunarante, and
tal sets," Phys. Rev. A 36, 4904 (1987). I. Procaccia, "Exploring chaotic motion through periodic or-
55. A. Renyi, Probability Theory (North-Holland, Amsterdam, bits," Phys. Rev. Lett. 58, 2387 (1987).
1970). 82. G. Gunarante and I. Procaccia, "Organization of chaos," Phys.
56. H. S. Greenside, A. Wolf, J. Swift, and T. Pignataro, "Impracti- Rev. Lett. 59, 1377 (1987).
cality of a box-counting algorithm for calculating the dimen- 83. D. Auerbach, B. O'Shaughnessy, and I. Procaccia, "Scaling
sionality of strange attractors," Phys. Rev. A 25, 3453 (1982). structure of strange attractors," Phys. Rev. A 37, 2234 (1988).
57. P. Grassberger, R. Badii, and A. Politi, "Scaling laws for invari- 84. C. Grebogi, E. Ott, and J. A. Yorke, "Unstable periodic orbits
ant measures on hyperbolic and non-hyperbolic attractors," J. and the dimension of chaotic attractors," Phys. Rev. A 36,3522
Stat. Phys. 51, 135 (1988). (1987).
58. P. Cvitanovi6, G. H. Gunarante, and I. Procaccia, "Topological 85. C. Grebogi, E. Ott, and J. A. Yorke, "Unstable periodic orbits
and metric properties of H6non-type strange attractors," pre- and the dimensions of multifractal chaotic attractors," Phys.
print (University of Chicago, Chicago, Ill., 1988). Rev. A 37,1711 (1988).
59. G. Mayer-Kress, "Application of dimension algorithms to ex- 86. K. Fukunaga and D. R. Olsen, "An algorithm for finding intrin-
perimental chaos," in Directions in Chaos, Hao Bai-lin, ed. sic dimensionality of data," IEEE Trans. Comput. C-20, 176
(World Scientific, Singapore, 1987), p. 122. (1971).
60. P. Grassberger and I. Procaccia, "Characterization of strange 87. H. Froehling, J. P. Crutchfield, D. Farmer, N. H. Packard, and
attractors," Phys. Rev. Lett. 50, 346 (1983). R. Shaw, "On determining the dimension of chaotic flows,"
61. P. Grassberger and I. Procaccia, "Measuring the strangeness of Physica 3D, 605 (1981).
strange attractors," Physica 9D, 189 (1983). 88. D. S. Broomhead, R. Jones, and G. P. King, "Topological di-
62. F. Takens, "Invariants related to dimension and entropy," in mension and local coordinates from time series data," J. Phys.
Atas do 130 (Col6qkio Brasiliero do Matematica, Rio de Janei- A 20, L563 (1987).
ro, 1983). 89. A. Cenys and K. Pyragas, "Estimation of the number of de-
63. P. Grassberger, "Finite sample corrections to entropy and di- grees of freedom from chaotic time series," Phys. Lett. A 129,
mension estimates," Phys. Lett. A 128, 369 (1988). 227 (1988).
64. J. Theiler, "Spurious dimension from correlation algorithms 90. A. Passamante, T. Hediger, and M. Gollub, "Fractal dimension
applied to limited time series data," Phys. Rev. A 34, 2427 and local intrinsic dimension," Phys. Rev. A 39, 3640 (1989).
(1986). 91. W. A. Brock, W. D. Dechert, and J. A. Scheinkman, "A test for
65. K. Pawelzik and H. G. Schuster, "Generalized dimensions and independence based on the correlation dimension," preprint
entropies from a measured time series," Phys. Rev. A 35, 481 SSRI 8702 (University of Wisconsin, Madison, Wisc., 1987).
(1987). 92. A. Namajinas, J. Poziela, and A. Tamagevicius, "An electronic
66. H. Atmanspacher, H. Scheingraber, and W. Voges, "Global technique for measuring phase space dimension from chaotic
scaling properties of a chaotic attractor reconstructed from time series," Phys. Lett. A 131, 85 (1988).
experimental data," Phys. Rev. A 37, 1314 (1988). 93. A. Destexhe, J. A. Sepulchre, and A. Babloyantz, "A compara-
67. Y. Termonia and Z. Alexandrowicz, "Fractal dimension of tive study of the experimental quantification of deterministic
strange attractors from radius versus," Phys. Rev. Lett. 51, chaos," Phys. Lett. A 132,101 (1988).
1265 (1983). 94. J. D. Farmer and J. J. Sidorowich, "Predicting chaotic time
68. J. Guckenheimer and G. Buzyna, "Dimension measurements series," Phys. Rev. Lett. 59, 845 (1987).
for geostrophic turbulence," Phys. Rev. Lett. 51, 1483 (1983). 95. D. A. Russell, J. D. Hanson, and E. Ott, "Dimension of strange
69. R. Badii and A. Politi, "Statistical description of chaotic at- attractors," Phys. Rev. Lett. 45, 1175 (1980).
tractors: the dimension function," J. Stat. Phys. 40, 725 96. P. Grassberger, "On the fractal dimension of the Henon attrac-
(1985). tor," Phys. Lett. A 97, 224 (1983).
70. P. Grassberger, "Generalizations of the Hausdorff dimension 97. M. J. McGuinness, "A computation of the limit capacity of the
of fractal measures," Phys. Lett. A 107, 101 (1985). Lorenz attractor," Physica 16D, 265 (1985).
71. G. Broggi, "Evaluation of dimensions and entropies of chaotic 98. W. E. Caswell and J. A. Yorke, "Invisible errors in dimension
systems," J. Opt. Soc. Am. B 5, 1020 (1988). calculations: geometric and systematic effects," in Dimen-
72. W. van de Water and P. Schram, "Generalized dimensions sions and Entropies in Chaotic Systems-Quantificationof
from near-neighbor information," Phys. Rev. A 37, 3118 Complex Behavior, G. Mayer-Kress, ed., Vol. 32 of Springer
(1988). Series in Synergetics (Springer-Verlag, Berlin, 1986), p. 123.
73. R. Badii and G. Broggi, "Measurement of the dimension spec- 99. A. Chhabra and R. V. Jensen, "Direct determination of the f(a)
trum f(a): fixed-mass approach," Phys. Lett. A 131, 339 singularity spectrum," Phys. Rev. Lett. 69, 1327 (1989).
(1988). 100. F. Takens, "On the numerical determination of the dimension
74. K. W. Pettis, T. A. Bailey, A. K. Jain, and R. C. Dubes, "An of an attractor," in Dynamical Systems and Bifurcations,Gro-
intrinsic dimensionality estimator from near neighbor infor- ningen, 1984, B. L. J. Braaksma, H. W. Broer, and F. Takens,
mation," IEEE Trans. Pattern Anal. Mach. Intell. PAMI-1, 25 eds., Vol. 1125 of Lecture Notes in Mathematics (Springer-
(1979). Verlag, Berlin, 1985).
75. R. L. Somorjai, "Methods for estimating the intrinsic dimen- 101. J. Theiler, "Quantifying chaos: practical estimation of the
sionality of high-dimensional point sets," in Dimensions and correlation dimension," Ph.D. dissertation (California Insti-
Entropies in Chaotic Systems-Quantification of Complex tute of Technology, Pasadena, Calif., 1988).
Behavior, G. Mayer-Kress, ed., Vol. 32 of Springer Series in 102. J. Theiler, "Statistical precision of dimension estimators,"
Synergetics (Springer-Verlag, Berlin, 1986), p. 137. Phys. Rev. A (to be published).
76. J. L. Kaplan and J. A. Yorke, in FunctionalDifferentialEqua- 103. R. Cawley and A. L. Licht, "Maximum likelihood method for
tions and Approximations of Fixed Points, H. 0. Peitgen and evaluating correlation dimension," in The Physics of Phase
H. 0. Walther, eds., Vol. 730 of Springer Lecture Notes in Space, Y. S. Kim and W. W. Zachary, eds., Vol. 278 of Lecture
Mathematics (Springer-Verlag, Berlin, 1979), p. 204. Notes in Physics (Springer-Verlag, Berlin, 1986), p. 90.
77. P. Fredrickson, J. L. Kaplan, E. D. Yorke, and J. A. Yorke, 104. S. Ellner, "Estimating attractor dimensions for limited data:
"The Liapunov dimension of strange attractors," J. Diff, Eq. a new method, with error estimates," Phys. Lett. A 113, 128
49, 185 (1983). (1988).
James Theiler Vol. 7, No. 6/June 1990/J. Opt. Soc. Am. A 1073

105. J. Theiler, "Lacunarity in a best estimator of fractal dimen- 125. Y. Gefen, A. Aharony, and B. B. Mandelbrot, "Phase transi-
sion," Phys. Lett. A 133,195 (1988). tions on fractals: III. Infinitely ramified lattices," J. Phys. A
106. J. Theiler, "Efficient algorithm for estimating the correlation 17, 1277 (1984).
dimension from a set of discrete points," Phys. Rev. A 36, 4456 126. R. Badii and A. Politi, "Intrinsic oscillations in measuring the
(1987). fractal dimensions," Phys. Lett. A 104, 303 (1984).
107. S. Bingham and M. Kot, "Multidimensional trees, range 127. L. A. Smith, J.-D. Fournier, and E. A. Spiegel, "Lacunarity and
searching, and a correlation dimension algorithm of reduced intermittency in fluid turbulence," Phys. Lett. A 114, 465
complexity," Phys. Lett. A 140, 327 (1989). (1986).
108. F. Hunt and F. Sullivan, "Efficient algorithms for computing 128. A.Arneodo, G. Grasseau, and E. J. Kostelich, "Fractal dimen-
fractal dimensions," in Dimensions and Entropies in Chaotic sions and f(a) spectrum of the H6non attractor," Phys. Lett. A
Systems-Quantification of Complex Behavior, G. Mayer- 124, 426 (1987).
Kress, ed., Vol. 32 of Springer Series in Synergetics (Springer- 129. D. Bessis, J.-D. Fournier, G. Servizi, G. Turchetti, and S.
Verlag, Berlin, 1986), p. 74. Vaienti, "Mellin transforms of correlation integrals and gener-
109. M. Franaszek, "Optimized algorithm for the calculation of alized dimension of strange sets," Phys. Rev. A 36, 20 (1987).
correlation integrals," Phys. Rev. A 39, 5540 (1989). 130. P. Atten, J. G. Caputo, B. Malraison, and Y. Gagne, "D6termi-
110. C.-K. and F. C. Moon, "An optical technique for measuring nation de dimension d'attracteurs pour diff6rents 6coule-
fractal dimensions of planar Poincar6 maps," Phys. Lett. A ments," J. Mec. Theor. Appl. 133 (Suppl.) (1984).
114, 222 (1986). 131. J. Gleick, Chaos: Making a New Science (Viking, New York,
111. P. Grassberger, "Do climatic attractors exist?" Nature (Lon- 1987).
don) 323, 609 (1986). 132. H.-O. Peitgen and P. H. Richter, The Beauty of Fractals
112. A. R. Osborne and A. Provenzale, "Finite correlation dimen- (Springer-Verlag, Berlin, 1986).
sion for stochastic systems with power-law spectra," Physica 133. H.-O. Peitgen and D. Saupe, eds., The Science of Fractal
35D, 357 (1989). Images (Springer-Verlag, New York, 1988).
113. A. Brandstater and H. L. Swinney, "Strange attractors in 134. T. S. Parker and L. 0. Chua, "Chaos: a tutorial for engineers,"
weakly turbulent Couette-Taylor flow," Phys. Rev. A 35, 2207. Proc. IEEE 75, 982 (1987).
(1987). 135. T. S. Parker and L. 0. Chua, PracticalNumericalAlgorithms
114. N. B. Abraham, A. M. Albano, B. Das, G. DeGuzman, S. Young, for Chaotic Systems (Springer-Verlag, New York, 1989).
R. S. Gioggia, G. P. Puccioni, and J. R. Tredicce, "Calculating 136. N. B. Abraham, A. M. Albano, B. Das, T. Mello, M. F. H.
the dimension of attractors from small data sets," Phys. Lett. Tarroja, N. Tufillaro, and R. S. Gioggia, "Definitions of chaos
A 114, 217 (1986). and measuring its characteristics," in Optical Chaos,J. Chros-
115. J. W. Havstad and C. L. Ehlers, "Attractor dimension of non- towski and N. B. Abraham, eds. Proc. Soc. Photo-Opt. In-
stationary dynamical systems from small data sets," Phys. strum. Eng. 667, 2 (1986).
Rev. A 39, 845 (1989). 137. R. S. Shaw, "Strange attractors, chaotic behavior, and infor-
116. J. B. Ramsey and H.-J. Yuan, "Bias and error bars in dimen- mation flow," Z. Naturforsch. 36a, 80 (1981).
sion calculations and their evaluation in some simple models," 138. J. P. Crutchfield, J. D. Farmer, N. H. Packard, and R. Shaw,
Phys. Lett. A 134, 287 (1989). "Chaos," Sci. Am. 255, 46 (1986).
117. R. Badii, G. Broggi, B. Derighetti, M. Ravani, S. Ciliberto, A. 139. E. Ott, "Strange attractors and chaotic motions of dynamical
Politi, and M. A. Rubio, "Dimension increase in filtered chaot- systems," Rev. Mod. Phys. 53, 655 (1981).
ic signals," Phys. Rev. Lett. 60, 979 (1988). 140. A. V. Holden, ed., Chaos (Princeton U. Press, Princeton, N.J.,
118. F. Mitschke, M. Moller, and W. Lange, "Measuring filtered 1986).
chaotic signals," Phys. Rev. A 37, 4518 (1988). 141. H. G. Schuster, DeterministicChaos: An Introduction(VCH,
119. E. J. Kostelich and J. A. Yorke, "Noise reduction in dynamical Weinheim, Federal Republic of Germany, 1988).
systems," Phys. Rev. A 38, 1649 (1988). 142. H. Bai-Lin, Chaos (World Scientific, Singapore, 1984).
120. M. M6ller, W. Lange, F. Mitschke, N. B. Abraham, and U. 143. P. Cvitanovi6, Universality in Chaos (Hilger, Bristol, UK,
Hubner, "Errors from digitizing and noise in estimating attrac- 1986).
tor dimensions," Phys. Lett. A 138, 176 (1989). 144. P. Grassberger, "Estimating the fractal dimensions and entro-
121. L. A. Smith, "Intrinsic limits on dimension calculations," pies of strange attractors," in Chaos, A. V. Holden, ed. (Prince-
Phys. Lett. A 133, 283 (1988). ton U. Press, Princeton, N.J. 1986), Chap. 14.
122. B. B. Mandelbrot, The FractalGeometry of Nature (Freeman, 145. G..Mayer-Kress, ed., Dimensions and Entropies in Chaotic
San Francisco, Calif., 1982). Systems-Quantification of Complex Behavior, Vol. 32 of
123. B. B. Mandlebrot, "Corr6lations et texture dans un nouveau Springer Series in Synergetics (Springer-Verlag, Berlin, 1986).
modele d'univers hi6rarchise, bas6 sur les ensembles tremas," 146. N. B. Abraham, J. P. Gollub, and H. L. Swinney, "Testing
C. R. Acad. Sci. A 288, 81 (1979). nonlinear dynamics," Physica liD, 252 (1984).
124. Y. Gefen, Y. Meir, A. Aharony, and B. B. Mandelbrot, "Geo-
metric implementation of hypercubic lattices with noninteger
dimension," Phys. Rev. Lett. 50, 145 (1983).