Received: 3 January 2017 Revised: 3 September 2017 Accepted: 18 October 2017

DOI: 10.1002/rnc.3999

STOCHASTIC PREDICTIVE CONTROL

An efficient method for stochastic optimal control with

joint chance constraints for nonlinear systems

Joel A. Paulson Ali Mesbah

Department of Chemical and

Biomolecular Engineering, University of Summary
California, Berkeley, CA, USA
Stochastic model predictive control hinges on the online solution of a stochas-
Correspondence tic optimal control problem. This paper presents a computationally efficient
Ali Mesbah, Department of Chemical and solution method for stochastic optimal control for nonlinear systems subject to
Biomolecular Engineering, University of
(time-varying) stochastic disturbances and (time-invariant) probabilistic model
California, Berkeley, CA 94720, USA.
Email: [email protected] uncertainty in initial conditions and parameters. To this end, new methods are
presented for joint propagation of time-varying and time-invariant probabilistic
uncertainty and the nonconservative approximation of joint chance constraint
(JCC) on the system state. The proposed uncertainty propagation method relies
on generalized polynomial chaos and conditional probability rules to obtain
tractable expressions for the state mean and covariance matrix. A moment-based
surrogate is presented for JCC approximation to circumvent construction of the
full probability distribution of the state or the use of integer variables as required
when using the sample average approximation. The proposed solution method
for stochastic optimal control is illustrated on a nonlinear semibatch reactor
case study in the presence of probabilistic model uncertainty and stochastic
disturbances. It is shown that the proposed solution method is significantly
superior to a standard random sampling method for stochastic optimal con-
trol in terms of computational requirements. Furthermore, the moment-based
surrogate for the JCC is shown to be substantially less conservative than the
widely used distributionally robust Cantelli-Chebyshev inequality for chance
constraint approximation.

K E Y WO R D S
generalized polynomial chaos, joint chance constraint, joint propagation of probabilistic model
uncertainty and stochastic disturbances, stochastic optimal control

1 I N T RO DU CT ION

Model predictive control (MPC) has shown exceptional success for the optimal control of a wide range of systems, owing
to its ability to handle multivariable system dynamics and state and input constraints.1,2 However, a key challenge in
MPC results from model uncertainty and system disturbances, which can be detrimental to the closed-loop control
performance. Although its receding-horizon implementation provides MPC with some degree of robustness to system
uncertainties, marginal robust control performance may be inadequate in many applications, such as those that are
safety critical.

Int J Robust Nonlinear Control. 2019;29: 5017–5037. wileyonlinelibrary.com/journal/rnc © 2017 John Wiley & Sons, Ltd. 5017
5018 PAULSON AND MESBAH

Extensive work has been done on robust MPC with the aim to develop optimal control methods that can explicitly
account for set-membership-type uncertainties (ie, deterministic uncertainties in a bounded set). Robust MPC methods
range from min-max formulations, which design the control actions based on worst-case evaluations of the cost func-
tion and therefore disregard the “spread” of state trajectories,3,4 to tube-based MPC methods,5-7 which design an ancillary
feedback controller to retain the system states within an invariant “tube” around their nominal trajectories. On the other
hand, uncertainties are generally of a stochastic nature in engineering systems. When the probabilistic descriptions of
system uncertainties are available, they can be explicitly incorporated into an MPC formulation with the aim to achieve
robustness to uncertainties in a probabilistic sense. This notion has led to stochastic MPC (SMPC) that solves a stochastic
optimal control problem (OCP), in which the cost function and constraints are defined in terms of probability distribu-
tion of the system state or its moments (see the work of Mesbah8 for a review of SMPC methods). A key ingredient of
SMPC is chance constraints, which allow for an acceptable (ie, user-defined) level of state constraint violation in prob-
ability to systematically trade off between minimizing the control cost function and satisfying the state constraints (eg,
the works of Schwarm and Nikolaou,9 Li et al,10 and Nemirovski and Shapiro11 ). This is particularly important when the
high-performance operation of an uncertain system is realized in the vicinity of constraints. Furthermore, defining the
control objectives in terms of the probabilistic knowledge of the state becomes essential when the control cost function is
asymmetric, making the tails of its distribution important for effective control.
Output-feedback implementations of SMPC can be separated into 2 distinct problems: (i) stochastic optimal control and
(ii) state estimation. This article focuses on an efficient solution of the stochastic OCP for nonlinear systems subject to
time-invariant, probabilistic model uncertainty and time-varying stochastic disturbances. To this end, the key challenges
addressed in this article are propagation of “joint” model uncertainty and stochastic disturbances through the nonlinear
system dynamics and joint chance constraint (JCC) approximation. Uncertainty propagation has been a long-standing
challenge in the optimal control of nonlinear, uncertain systems. A widely adopted uncertainty propagation method
relies on linearization of the system dynamics.12 Although this method is relatively simple to implement, it requires the
existence and calculation of Jacobians along all possible input trajectories and can be inaccurate for highly nonlinear
systems.13 Another uncertainty propagation method, originally emerged in nonlinear state estimation, is the unscented
transform (UT), which is a heuristic-based method that propagates a relatively small number of deterministically cho-
sen samples (called sigma points) through the system dynamics.14 The UT method can more effectively deal with system
nonlinearity than the linearization method. However, its accuracy is largely dependent on the choice of the sigma points,
which are typically defined in terms of the predicted state covariance and, therefore, must be adapted in every step of the
optimization as a function of the input profile.15 Alternatively, random sampling approaches to uncertainty propagation
such as Monte Carlo–based (MC-based) methods have been applied for solving the stochastic OCP.16,17 A key advantage of
MC-based uncertainty propagation methods is that the approximation error is independent of the number of uncertainty
sources. However, the error convergence rate is slow, often requiring a large number of samples to achieve high accuracy
even for relatively small problems.18 Another random sampling method used in the context of SMPC (eg, the works of
Schildbach et al19 and Calafiore and Fagiano20 ) is the scenario approach, which provides a bound on the number of sam-
ples required to realize a certain probability of state constraint satisfaction irrespective of the uncertainty distribution.21
However, the scenario approach requires the constraints to be convex for every uncertainty realization, which does not
hold for general nonlinear systems. Furthermore, the scenario approach can require a large number of samples and may
be conservative when the uncertainty distribution is known exactly since it is robust to all possible distributions.
The above discussed methods are mainly geared toward propagation of time-varying probabilistic uncertainty (ie,
stochastic disturbances). Generalized polynomial chaos (gPC)22 has recently been adopted for the SMPC of nonlinear
systems subject to time-invariant probabilistic uncertainty in parameters and initial conditions.23,24 In gPC, the state is
approximated by expansions of orthogonal polynomial basis functions, defined based on the known distributions of proba-
bilistic uncertainties. The gPC expansions provide an efficient machinery for uncertainty propagation. The state moments
can be efficiently computed from the expansions' coefficients or, alternatively, the expansions can be used as a surrogate
for the nonlinear system model for MC-based uncertainty propagation. However, gPC cannot handle stochastic distur-
bances efficiently, since the number of terms in the expansion grows factorially with the number of uncertainties, which
typically becomes large when dealing with time-varying disturbances.25 This article presents an efficient method, based
on gPC and conditional probability rules, for joint propagation of time-invariant, probabilistic model uncertainty, and
time-varying stochastic disturbances.
This article also addresses the problem of JCC approximation in the stochastic optimal control. In general, chance
constraints are nonconvex and their numerical solution is often prohibitive due to multidimensional integration.26,27
When the system dynamics are linear and subject to only additive disturbances, chance constraints can be analytically
PAULSON AND MESBAH 5019

reformulated into deterministic expressions. This reformulation can be done exactly when the noise is Gaussian.28
For other types of distributions, chance constraints are commonly replaced with the distributionally robust
Cantelli-Chebyshev inequality,26 or stochastic tubes are utilized to rewrite chance constraints in terms of a backoff param-
eter that can be computed off-line.29,30 However, these approaches do not readily extend to nonlinear systems, nor can
they naturally handle probabilistic parametric uncertainty. Methods for chance constraint approximations in nonlinear
settings have taken 2 somewhat different directions. One approach involves discretization of the probability distribution
of the uncertainties and solving the underlying combinatorial problem.31 Another approach relies on convex approxima-
tions of chance constraints.27 Recently, sample average approximation (SAA) methods have been developed, where the
actual probability distribution in the chance constraint is replaced with an empirical distribution obtained from randomly
drawn samples.32,33 The main shortcoming of SAA methods is that the optimization problem becomes a mixed integer
program since one binary variable is needed for each sample point. This article presents a moment-based method for JCC
approximation to avoid mixed integer programing. The key difference between the proposed JCC approximation method
and the Cantelli-Chebhsev inequality combined with risk allocation,34 which is robust with respect to any distribution, is
how the weight factor multiplying the covariance is chosen to achieve significantly less conservative approximations.
The proposed uncertainty propagation method and the moment-based JCC approximation method are used to derive
an efficient surrogate optimization problem for the stochastic OCP for nonlinear systems with probabilistic model uncer-
tainty and stochastic disturbances. The proposed solution method for stochastic OCP with JCC is demonstrated on a
highly nonlinear semibatch reactor case study with 7 state variables, where its performance is compared with that of
a standard MC-based method for stochastic optimal control. Furthermore, the effectiveness of the moment-based JCC
approximation method is compared with that of the Cantelli-Chebhsev inequality.
Notation. Hereafter, R and N = {1, 2, … } denote the sets of real and natural numbers, respectively; N0 = N ∪ {0}.
For any square matrix S, S > 0 and S ≥ 0 indicate that S is a positive-definite and positive-semidefinite matrix,
respectively. IN denotes the N×N identity matrix. For given random vectors X and Y, E[X] denotes the expected value,
cov(X, Y ) is the covariance between X and Y, and Var(X ) = cov(X, X ) is the covariance of X with itself (also known as
covariance matrix). P(X ∈ A) denotes the probability that the random vector X lies in the set A. For any vector X, [X]i
denotes the ith element of X.

2 PROBLEM STAT EMENT

Consider a continuous-time, nonlinear system of the form

̇ = f(x(t), u(t), w(t), θ),

x(t) (1)

where t ≥ 0 denotes the time, x ∈ Rnx denotes the system states, u ∈ Rnu denotes the control inputs, θ ∈ Rnθ denotes the
unknown system parameters, and w ∈ Rnw denotes the time-varying system disturbances. The nonlinear system function
f ∶ Rnx × Rnu × Rnw × Rnθ → Rnx is assumed to be known. Due to imperfect knowledge of the system, the initial conditions
x0 and parameters θ are modeled as time-invariant, probabilistic uncertainty with a known joint probability distribution
function (pdf). We denote the concatenated uncertainty vector by p = (x0 , θ)⊤ with a known probability measure Pp .

Assumption 1. The stochastic disturbances w(t) evolve according to a continuous, white-noise random process W
with a known probability measure PW = dFW ; and x0 , θ, and w(t) are mutually independent at all times.
System (1) is subject to hard input constraints u(t) ∈  . The system is also subject to state constraints x(t) ∈ .
However, since the state evolves as a stochastic process, the state constraints are enforced as a JCC

P(x(t) ∈ ) ≥ β, (2)

where β ∈ (0, 1) is the required probability that the state constraints must be satisfied. Without loss of generality due to
the nonlinear system dynamics, it is assumed that  is a polytope of the form

 = {x ∈ Rnx ∶ Gx ≤ d}, (3)

where G ∈ Rnc ×nx , d ∈ Rnc , and nc is the number of state constraints.

5020 PAULSON AND MESBAH

Remark 1. When the probabilistic system uncertainties in (1) are bounded, hard state constraints can be enforced by
setting β = 1 in (2). This implies that x(t) ∈  holds for all realizations of the uncertainties. Input rate constraints can
also be handled by adding integrators in the system dynamics.35
The goal of this article is to develop a tractable solution method for the stochastic OCP for the nonlinear system (1).
The stochastic OCP over a finite time tf can be stated as
[ ]
min E JN (u(t); Pp , PW ) (4a)
u(t)

subject to: ̇ = f(x(t), u(t), w(t), θ),

x(t) t ∈ [0, tf ], (4b)

u(t) ∈  , t ∈ [0, tf ], (4c)

P(x(t) ∈ ) ≥ β, t ∈ [0, tf ], (4d)

w(t) ∼ PW , t ∈ [0, tf ], (4e)

[ ]
x
p = θ0 ∼ Pp . (4f)

The control cost function JN is defined by

JN (u(t); Pp , PW ) = x⊤ (tf )Qx(tf ) + q⊤ x(tf ), (5)
where Q is a quadratic weight matrix and q is a linear weight vector. Note that there is no loss of generality in selecting the
quadratic cost function (5) since nonlinear dynamics are considered in (1). Stage cost and nonquadratic objectives can be
incorporated into (5) by adding an additional state to x. That is,
tf
̇ = L(x(t), u(t))
z(t) → z(tf ) = L(x(t), u(t))dt,
∫0
where L ∶ Rnx × Rnu → [0, +∞) is some semidefinite stage cost function. We now define the state mean and covariance by

μ(t) = E [x(t)] , (6)

[ ]
Σ(t) = Var(x(t)) = E (x(t) − μ(t))(x(t) − μ(t))⊤ . (7)

The cost function (4a) can be rewritten in terms of the state mean and covariance at the final time as
E[JN ] = q⊤ μ(tf ) + μ⊤ (tf )Qμ(tf ) + tr(QΣ(tf )). (8)
The formulation of the stochastic OCP (4) is motivated by the SMPC for nonlinear systems. Stochastic MPC involves
an online solution of (4) in a receding-horizon manner, with (x0 , θ)⊤ ∼ Pp replaced with its estimates obtained at every
measurement sampling time. Various methods have been proposed for solving the stochastic OCP for nonlinear systems,
which generally rely on deterministic or random sampling methods for uncertainty propagation. Typically, the uncer-
tainty propagation methods consider only one source of uncertainty, that is, either the time-invariant uncertainties (x0 , θ)⊤
are assumed known, or the time-varying stochastic disturbances w(t) = w ̄ are fixed at their mean values. As discussed in
Section 1, the linearization and the UT methods are the most commonly used deterministic approaches to uncertainty
propagation in stochastic optimal control.12,15 Linearization is usually computationally inexpensive but requires the exis-
tence and calculation of the Jacobians, which can be cumbersome for complex model functions. On the other hand, the
UT method propagates a set of heuristically chosen sigma points through the system dynamics to circumvent the inac-
curacies associated with linearization. However, the UT method can be expensive for online optimization, as the sigma
points must be adapted at each optimization step for any candidate input profile.15 Furthermore, error bounds and con-
vergence results are not readily available for the UT method. Alternatively, random sampling methods look to replace
the probabilistic operators E[·], Var(·), and P(·) with their corresponding sample approximations. The key√ advantages
of the MC-based methods arise from their straightforward implementation and their convergence rate O(1∕ Ns ) as the
number of samples Ns → ∞.36 The fact that the convergence rate of the MC-based methods is independent of the num-
ber of uncertainty sources is particularly important for systems with a high uncertainty dimension since, irrespective of
the uncertainty dimension, the error will decrease in half when the number of samples is quadrupled. However, even
PAULSON AND MESBAH 5021

for a moderate number of uncertainty sources (on the order of 10), the number of samples required to achieve accurate
uncertainty propagation can be very large, which can make the online solution of (4) computationally intractable.18
The first contribution of this article is an efficient uncertainty propagation method that can handle time-invariant
probabilistic uncertainty in the model parameters and initial conditions and time-varying stochastic disturbances. The
proposed uncertainty propagation method yields tractable approximations for the moments μ(t) and Σ(t) by combin-
ing gPC with traditional stochastic noise propagation methods. The fundamental notion of the proposed method is to
decouple the propagation of the probabilistic model uncertainty from the propagation of stochastic disturbances using
conditional probability rules so that different methods that are best-suited for handling each uncertainty source can be
utilized. The proposed uncertainty propagation method is described in Section 3.
The second contribution of this article is the approximation of the JCC (4d). In order to implement the JCC exactly, the
probability distribution of state must be known as a function of the input u(t). However, determining the distribution of
multivariate transformations of random variables is generally a challenging task even in a linear setting (except in special
cases where the state pdf remains invariant such as in linear transformation of Gaussian random variables). As discussed
in Section 1, SAA is a common approach to JCC approximation for nonlinear systems.32,33 Let c(x) = max1≤ j≤nc (gj⊤ x − dj ),
where gj⊤ is the jth row of G, and dj is the jth element of d. Then, the SAA for (4d) takes the form

1∑
N
[ ] s

P(x(t) ∈ ) = P (c(x(t)) ≤ 0) = E 1(0,∞) (c(x(t))) ≈ 1(0,∞) (c(xi (t))),

Ns i=1

where xi (t), i ∈ {1, Ns } denote the samples of the states, and 1(0,∞) is the indicator function. Since both the indicator
function 1(0,∞) and the max operator c are nonsmooth functions, they must be implemented with binary variables, which
can greatly increase the complexity of the optimization problem even for a relatively small number of samples Ns . This
article presents a nonconservative approximation for the JCC (4d) in terms of the moments of the state, as described in
Section 4. The proposed uncertainty propagation method and the moment-based JCC approximation method are used to
derive an efficient surrogate for the stochastic OCP (4), which is presented in Section 5.

3 UNCERTAINTY PRO PAGATION USING GPC

3.1 Generalized polynomial chaos

The term polynomial chaos was introduced by Norbert Wiener in the seminal paper,37 in which a generalized harmonic
analysis was applied to Brownian motion-like processes. The basic notion was to expand finite-variance random variables
by an infinite series of Hermite polynomials that are functions of a normally distributed input random variable. Later,
convergence of the chaos expansions was established by Cameron and Martin.38
Although Wiener-Hermite chaos expansions converge as long as the random variable y ∈ L2 (Ω, σ(ℋ ), P) is measur-
able with respect to the Gaussian Hilbert space ℋ , their convergence rate is often slow in practice for non-Gaussian
input random variables.39 This is because, when expressed as functions of Gaussian input random variables, y will often
be highly nonlinear and can only be approximated adequately with high-order expansions. Alternatively, the expansions
can be constructed in terms of non-Gaussian input random variables whose distribution is closer to that of y, thus allow-
ing better approximations at lower orders. This approach is known as gPC 22 and essentially involves replacing Hermite
polynomials with polynomials that are orthogonal with respect to possibly non-Gaussian input random variables.

3.1.1 Single random variable

First, consider a single random variable ξ with the following:
1. finite moments of all orders such that E[|ξ|k ] < ∞ for all k ∈ N0 ;
2. a continuous distribution function Fξ (x) = P(ξ ≤ x) for any x ∈ R;
3. Fξ uniquely defined in terms of its moments.
For any random variable y ∈ L2 (Ω, σ(ξ), P), which is measurable with respect to ξ, there must exist a measurable func-
tion g ∶ R → R such that y = g(ξ), according to the Doob-Dynkin lemma. The cumulative distribution function (cdf) Fξ
represents a measure on the real line so that ξ ∶ Ω → R induces a probability space (R, 𝔅(R), dFξ ), where 𝔅(R) is the
5022 PAULSON AND MESBAH

Borel σ-algebra on R. When all the moments of the random variable are finite with respect to this measure, a sequence of
orthogonal polynomials {ϕn }n∈N0 associated with this measure can be defined as

⟨ ⟩
⟨ϕn , ϕm ⟩ = ϕn (ξ)ϕm (ξ)fξ (ξ)dξ = ϕ2n δnm , (9)
∫R

where δnm is the Kronecker delta, and fξ = dFξ ∕dξ is the pdf defined for any random variable with a continuous cdf.
The polynomial basis {ϕn }n∈N0 can be constructed to be orthogonal with respect to any arbitrary pdf fξ using the
well-known Gram-Schmidt process, which yields a basis orthogonal to fξ from an existing arbitrary basis. A common
starting basis is the set of monic polynomials {ξj }j∈N0 . The Gram-Schmidt process is defined as

∑
j−1
ϕ0 = 1, ϕj = ξ j − cjk ϕk , j = 1, 2, … , (10)
k=0

⟨ξj ,ϕk ⟩
where the coefficients cjk = ⟨ϕ2k ⟩
must be computed for every k = 0, … , j − 1.

Definition 1. (gPC expansion)

Define the n-term gPC expansion for any random variable y = g(ξ) as

∑
n
yn = 𝑦̂i ϕi (ξ), (11)
i=0

⟨g,ϕi ⟩
with coefficients 𝑦̂i = ⟨ϕ2i ⟩
and polynomials defined by (9).

Theorem 1. (Convergence and boundedness of a gPC expansion)

For any ξ, the set of polynomials {ϕn }n∈N0 represents an orthogonal basis for the Hilbert space L2 (Ω, σ(ξ), P). Therefore,
any random variable y ∈ L2 (Ω, σ(Ξ), P) in this space can be expanded in terms of an abstract Fourier series of form (11)
that converges in the mean-square sense, ie,
[ ]
lim E ( y − yn )2 = 0. (12)
n→∞
[ ]
Furthermore, the mean-square error E ( y − yn )2 ≤ Mn is bounded for some finite constant Mn ∈ [0, ∞), with Mn+1 ≤
Mn for every integer approximation order n ∈ N0 .

Proof. As shown in theorem 3.3 in the work of Ernst et al,39 the set of polynomial basis {ϕn }n∈N0 associated with the
real-valued random variable ξ is dense in L2 (R, 𝔅(R), dFξ ) if and only if ξ has finite moments of all orders and a pdf
that is uniquely defined by its moments. Thus, the polynomial basis constitutes orthogonal basis for L2 (Ω, σ(ξ), P),
indicating that the Fourier series of form (11) exhibits mean-square convergence. This is a generalization of the
Cameron-Martin theorem38 that ensures (12) holds. Using (11) and (12), it can be shown that the expansion error is
bounded for all approximation orders based on the second moment of y, ie,
( [ ])
E[ y2 ] = lim 2E[ yyn ] − E y2n
n→∞
( n )
∑ ∑ ∑
n n
= lim 2 𝑦̂i E[ yϕi (ξ)] − 𝑦̂i 𝑦̂j E[ϕi (ξ)ϕj (ξ)]
n→∞
i=0 i=0 j=0
( )
∑
n
⟨ 2⟩ ∑ 2 ⟨ 2⟩
n
= lim 2 𝑦̂2i ϕi − 𝑦̂i ϕi
n→∞
i=0 i=0
∑
∞
⟨ ⟩
= 𝑦̂2i ϕ2i .
i=0

The first line of the above expression is derived by rearranging (12) and exploiting linearity of the expectation operator.
The second line follows from the substitution of (11) for yn , whereas the third line results from ⟨f, g⟩ = E[ f(ξ)g(ξ)]
and applying the orthogonality condition (9). The fourth line then immediately follows as the infinite sum must be
PAULSON AND MESBAH 5023

TABLE 1 The Wiener-Askey polynomial chaos basis for input random

variables with continuous distribution22
Random Variable Wiener-Askey Polynomial Chaos Support
Gaussian Hermite-chaos (−∞, ∞)
Beta Jacobi-chaos (a, b)
Gamma Laguerre-chaos [0, ∞)
Uniform Legendre-chaos [a, b]

convergent since y has finite variance by the definition of the L2 space. Thus, the gPC approximation error en =
E[( y − yn )2 ] is given by
[ ] [ ] ∑ ∞
⟨ ⟩ ∑ n
∑
∞
en = E y2 − 2E[ yyn ] + E y2n = 𝑦̂2i ϕ2i − 𝑦̂2i ⟨ϕ2i ⟩ = 𝑦̂2i ⟨ϕ2i ⟩.
i=0 i=0 i=n+1

The assertion that the bounded mean-square error monotonically decreases with increasing the approximation order
∑∞
follows directly since 𝑦̂2i ⟨ϕ2i ⟩ ≥ 0 and E[ y2 ] = i=0 𝑦̂2i ⟨ϕ2i ⟩ < ∞.
Table 1 lists the polynomial basis orthogonal with respect to the commonly used continuous random variables. Note
that the convergence rate of the expansion will depend on how closely the chosen fξ matches the distribution of y.

3.1.2 Extension to multiple random variables

( j)
Let ξ1 , ξ2 , … , ξn be a set of independent random variables, collected into a random vector ξ. Let {ϕi }i∈N0 denote the set of
orthogonal polynomial basis with respect to the distribution of each random variable. The multivariate polynomial basis
is then defined by
∏
m
( j)
Φi (ξ) = ϕ (i) (ξj ), αj(i) ∈ N0 , ∀j = {1, m}, ∀i ∈ N0 , (13)
αj
j=1
where αj(i)
is the jth element of a multi-index whose value corresponds to the order of the basis of the jth random variable
in the ith multivariate polynomial. The set of multivariate polynomial basis constitutes the orthogonal basis of the space
L2 (Ω, σ(ξ), P) such that any random variable y in this space can be expanded using a multivariate gPC of the form
∑
∞
⟨g, Φi ⟩
y = g(ξ) = 𝑦̂i Φi (ξ), with coefficients 𝑦̂i = ⟨ 2 ⟩ . (14)
i=0 Φi
The convergence and boundedness results of Theorem 1 naturally extend to the multivariate case.
To implement the gPC expansion (14), the infinite series must be truncated to a finite number of terms. When
the highest order of the polynomial basis retained in the expansion is d, the total number of terms in the truncated
expansion is
(n + d)!
L+1= , (15)
n!d!
∑
which imposes a constraint on the multi-index nj=1 α(i)
j
≤ d for all i = 0, … , L. The mean and variance of y can be
efficiently approximated from the truncated expansion as
E[ y] = 𝑦̂0 , (16)
∑
L
⟨ ⟩
Var( y) = 𝑦̂2i Φ2i . (17)
i=1

When y is a vector, the coefficients (or mode strengths) 𝑦̂i are vectors of equal size, and the covariance between any 2
∑
elements is given by cov( yi , yj ) = Lk=1 [ 𝑦̂k ]i [ 𝑦̂k ]j ⟨Φ2k ⟩.

3.1.3 Extension to correlated random variables

When the random variables ξ1 , ξ2 , … , ξn are correlated, the gPC expansion (14) is not guaranteed to converge. How-
ever, if the multivariate distribution function Fξ has a compact support (or, if ξ is exponentially integrable), a basis set
{Φi }i∈N0 that is orthogonal with respect to fξ can be constructed using a multivariate generalization of the Gram-Schmidt
5024 PAULSON AND MESBAH

process (10).40,41 Alternatively, correlated random variables can be handled by transforming ξ to a random vector whose
elements are independent.42

3.2 Propagation of probabilistic model uncertainty using gPC

We look to quantify the effect of the probabilistic model uncertainty and the stochastic disturbances on the states of system
(1). However, let us first neglect the stochastic disturbances, ie,
̇ = f(x(t), u(t), θ), x(0) = x0 .
x(t) (18)

To propagate the parametric model uncertainty, the gPC expansions for the states and parameters are substituted in
(18), ie,
( L )
∑L
∑ ∑
L
x̂̇ i Φi (ξ) ≈ f x̂ i Φi (ξ), u(t), θ̂ i Φi (ξ) , (19)
i=0 i=0 i=0

where ξ is a multivariate, standard random variable (referred to as the germ) such that θ(ξ), and L is the expansion trun-
cation order. Expression (19) leads to the problem of computing the expansion coefficients {̂xi }Li=0 . Broadly speaking, the
expansion coefficients can be computed using either the intrusive or nonintrusive methods.43
The intrusive method relies on Galerkin projection of the error of (19) onto the space of the multivariate basis Φk , which
leads to a set of ordinary differential equations (ODEs) for the coefficients44
⟨ ( L ) ⟩
d̂xk (t) 1 ∑ ∑L
⟨x0 , Φk ⟩
= f ̂
x̂ i Φi (ξ), u(t), θi Φi (ξ) , Φk , x̂ k (0) = ⟨ 2 ⟩ . (20)
dt ⟨Φk ⟩
2
i=0 i=0 Φk
When the model equations f are a polynomial function of x, u, and θ, the right-hand side of (20) can be computed analyti-
cally (for example, see Remark 2). However, for general nonlinear systems, it is impractical to derive closed-form equations
for the Galerkin projection. The nonintrusive methods, on the other hand, require the system model be evaluated at par-
ticular samples of the germ ξ. Practically speaking, this suggests that the model equations are treated as a “black box”
and thus can be of any general form. Hence, nonintrusive methods can generally be interpreted as a sum over weights
multiplying each state sample
[ ] [ ][ 1 ]
x̂ 0 (t) π0,1 · · · π0,Ns x (t)
⋮ = ⋮ ⋱ ⋮ ⋮ , (21)
x̂ L (t) πL,1 · · · πL,Ns xNs (t)
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟
Π

where Π is the matrix composed of weights πi, j describing the effect of sample states xj (t; ξj ), j = 1, … , Ns on the expansion
coefficients x̂ i , i = 0, … , L, and Ns is the total number of samples. The choice of a particular nonintrusive method mainly
hinges on the selection of the weight matrix Π. To this end, the most popular approaches include quadrature, collocation,
and least-squares regression. Using the least-squares method, for example, Π takes the form
⎡ Φ0 (ξ1 ) Φ1 (ξ1 ) · · · ΦL (ξ1 ) ⎤
( ⊤ )−1 ⊤ ⎢ Φ (ξ2 ) Φ1 (ξ2 ) · · · ΦL (ξ2 ) ⎥
Π= Λ Λ Λ , Λ=⎢ 0 ,
⋮ ⎥
(22)
⋮ ⋮ ···
⎢ N N Ns ⎥
⎣ Φ0 (ξ s ) Φ1 (ξ s ) · · · ΦL (ξ ) ⎦
N
where the sample set {ξj }j=1
s
can be chosen as the roots of the polynomial basis of one degree higher d + 1, so that
(n+d+1)!
Ns = n!(d+1)!
.

Remark 2. When (18) is bilinear in the uncertain parameters, the Galerkin projection in (20) can be performed
analytically. For example, for the scalar system f(x, u, θ) = −θx + u, (20) reduces to

d̂xk (t) ∑ ∑
L L
=− Cijk θ̂ i x̂ j (t) + uk (t), k = 0, 1, … , L,
dt i=0 j=0

where Cijk = ⟨Φi Φj Φk ⟩∕⟨Φ2k ⟩ is the triple inner product normalized by the double inner product, which can be com-
puted off-line (eg, see the work of Paulson et al45 ). Note that higher-order inner products must be computed and stored
as higher-order polynomial basis is considered. This can pose a significant challenge to the Galerkin projection.
PAULSON AND MESBAH 5025

Remark 3. The number of terms in the gPC expansions scales exponentially with respect to the number of uncertainty
sources (see (15)). This can become a computational limitation of gPC. In practice, however, sensitivity analysis can
be performed to identify the uncertain parameters with the largest influence on the state so that only these parameters
are accounted for in uncertainty propagation.

3.3 Joint propagation of probabilistic model uncertainty and stochastic disturbances

In theory, gPC could be used for the propagation of time-varying stochastic disturbances w(t) in (1) through treating w(t)
as individual random variables at different times t and including all the random variables in the germ ξ. However, this
would lead to an excessively large germ ξ and thus large gPC expansions so that gPC would likely become computationally
impractical for online optimization. To address this challenge, we utilize conditional probability rules to enable the use
of traditional methods for the propagation of stochastic disturbances in conjunction with gPC. Denote the conditional
state mean by m(t; ξ) = E[x(t)|ξ], where the germ ξ is defined based on the model uncertainty in parameters and initial
conditions (as in (19)). Applying the law of iterated expectations, the following expression can be derived for the state
mean μ(t):
[ ] [ ]
μ(t) = Eξ Ew [x(t)|ξ] = E m(t; ξ) . (23)

Similarly, denote the conditional state covariance by V(t; ξ) = Var(x(t)|ξ). Applying the law of total variance yields the
following expression for the state covariance Σ(t):
[ ] [ ]
Σ(t) = Eξ Varw (x(t)|ξ) + Varξ (Ew [x(t)|ξ]) = E V(t; ξ) + Var(m(t; ξ)). (24)

Since there exists no closed-form method for the propagation of stochastic disturbances through nonlinear dynamics,
commonly used approximations such as the linearization or the UT methods can be used to evaluate (23) and (24). When
the linearization method is used, the time evolution of the conditional state mean and covariance are approximated by

̇
m(t) ̄ θ(ξ)),
= f(m(t), u(t), w, m(0) = x0 (ξ), (25)

̇
V(t) = A(t)V(t) + V(t)A⊤ (t) + E(t)Σw E⊤ (t), V(0) = 0, (26)

where A and E are Jacobian matrices defined by

𝜕f || 𝜕f ||
A(t; ξ) = | , E(t; ξ) = . (27)
𝜕x |m(t),u(t),w,θ(ξ)
̄ 𝜕w ||m(t),u(t),w,θ(ξ)
̄

The model uncertainty can be propagated by stacking (25) and (26) into a set of equations of form (19), where an intrusive
or nonintrusive method must be used for determining the coefficients of the gPC expansions

∑
L
m(t; ξ) = ̂ i (t)Φi (ξ),
m (28a)
i=0

∑
L
V(t; ξ) = V̂ i (t)Φi (ξ), (28b)
i=0

with m̂ i ∈ Rnx and V̂ i ∈ Rnx ×nx . We suggest using a nonintrusive method since it can be applied to any general form of
(25)-(26) and alleviates the need for closed-form computation of the Galerkin projections. To this end, substituting the
gPC expansion (28a) into (23) gives

∑
Ns
̂ 0 (t) ≈
μ(t) ≈ m π0, j m j (t), (29)
j=1
5026 PAULSON AND MESBAH

where m j is the solution to (25) at sample ξj . Similarly, substituting the gPC expansions (28) into (24) yields the following
approximation for the elements of the covariance matrix:

[Σ(t)]ij ≈ E[V(t; ξ)]ij ] + cov([m(t; ξ))]i , [m(t; ξ)]j ),

∑
L
⟨ ⟩
≈ [V̂ 0 (t)]ij + ̂ k (t)]i [m
[m ̂ k ]j Φ2k ,
k=1
( L ) (30)
∑ Ns
∑ ∑ ∑
Ns Ns
⟨ 2 ⟩ [ l ] [ l′ ]
≈ l
π0,l [V (t)]ij + πl,k πk,l′ Φk m (t) i m (t) j .
l=1 l=1 l′ =1 k=1
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟
precomputable weights

Let 𝒮 = {ξ1 , … , ξNs } be the collection of samples of the germ ξ and Π be defined as in (21), then (29) and (30) explicitly
define the nonlinear operators
( ) ( )
μ(t) = Tμ m j (t), 𝒮 , Π , Σ(t) = TΣ m j (t), V j (t), 𝒮 , Π , (31)

which are known functions of the input u(t) through (25) and (26).
Remark 4. The moment equations (29) and (30) are derived through first conditioning the state mean and covariance
on the model uncertainty ξ and then propagating the stochastic disturbances w(t) (see (23)-(24)). Similar moment
equations can also be derived through reversing the order of these operations, that is, by first conditioning the state
mean and covariance on w(t) and then propagating the model uncertainty using gPC. In this case, the coefficients of
the gPC expansions become functions of the stochastic disturbances, which must be propagated, for example, using
the linearization method.46

3.4 Numerical illustration: propagation error convergence

For illustration, we demonstrate the application of the above gPC-based method for joint propagation of probabilistic
model uncertainty and stochastic disturbances on a simple example, for which the propagation error can be evaluated
exactly. Consider the scalar ODE
dx(t)
= −θx(t) + w(t), x(0) = 1, (32)
dt
where θ ∼  (0, 1) is a normally distributed decay rate coefficient, and w(t) ∼  (0, 1) is a random process. We first derive
expressions for the exact propagation of the mean and variance of x. The conditional state mean and variance are defined
exactly by
dm(t)
= −θm(t), m(0) = 1,
dt
dV(t)
= −2θV(t) + 1, V(0) = 0.
dt
These 2 decoupled ODEs can be solved exactly as a function of t ≥ 0 and the decay rate, yielding
1 − e−2θt
m(t) = e−θt , V(t) = .
2θ
Applying the laws of iterated expectations and total variance to the above expressions leads to
2 ∕2
μ(t) = E [m(t)] = et , (33a)

∞ ( )
1 − e−2θt θ2 2 2
Σ(t) = E [V(t)] + Var(m(t)) = e− 2 dθ + (e2t − et ), (33b)
∫−∞ 2θ
where E[V(t)] must be integrated numerically.
We now apply the proposed gPC-based uncertainty propagation method to (32). That is, we expand m(t; θ) and V(t; θ)
using the gPC expansions (28a) defined in terms of the Hermite basis (since θ is normally distributed) and, subsequently,
PAULSON AND MESBAH 5027

Mean
100 Variance

-2
10

Error
10-4

10-6

10-8
0 1 2 3 4 5 6 7
gPC order

FIGURE 1 Error convergence of the state mean and variance in the stochastic system (32). The error is calculated as the norm of ϵmean (ti )
and ϵvar (ti ) values for ti = 0, 0.1, … , 1

apply the laws of iterated expectations and total variance to compute the state mean and variance. The uncertainty
propagation error with respect to the exact mean and variance obtained from (33) is quantified in terms of
| μ(t) − μexact (t) | | |
ϵmean (t) = || | , ϵvar (t) = | Σ(t) − Σexact (t) | .
| | |
| μexact (t) | | Σexact (t) |
Figure 1 shows the uncertainty propagation error as a function of the order of the gPC expansion. As can be seen, expo-
nential error convergence is achieved for both the mean and variance. Furthermore, it is insightful to contrast this result
with that of uncertainty propagation through linearizing (32) with respect to both uncertainties θ and w(t), ie,
dx(t) ̄ + 𝜕f (x(t) − μlin (t)) + 𝜕f (θ − θ)
̄ θ)
≈ f(μlin (t), w, ̄ + 𝜕f (w(t) − w),
̄
dt ⏟⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏟ ⏟⏟⏟ 𝜕x 𝜕θ 𝜕w
̄ lin (t)+w
⏟⏟⏟ ⏟⏟⏟
−θμ ̄
=−θ̄ =−μlin (t) =1

where μlin (t) denotes the state mean obtained from the linearization of (32). Substituting the statistics θ̄ = 0, Σθ = 1,
̄ = 0, and Σw = 1 into the above expression results in
w
dμlin (t)
= 0, μlin (0) = 1, −→ μlin (t) = 1,
dt
dΣlin (t)
= 1, Σlin (0) = 0, −→ Σlin (t) = t.
dt
Note that the linearization method predicts that the mean is constant and the variance grows linearly over time, whereas
the exact propagation (33) suggests that the mean increases over time and the variance grows exponentially. This example
illustrates the advantage of gPC for handling time-invariant probabilistic uncertainty, as compared with the linearization
method.
Now that the moments {μ(t), Σ(t)}t∈[0,tf ] can be efficiently computed as a function of the input u(t), we look to derive a
surrogate for the JCC (4d) in terms of the moments of the state.

4 MOMENT-BASED APPROX IMATION OF JCC

Due to the intractability of JCCs, they are often replaced with deterministic surrogates derived in terms of either sam-
ples or moments of the probability distribution of the state (eg, the works of Nemirovski and Shapiro27 and Calafiore
and El Ghaoui47 ). A common approach to JCC handling relies on conservatively approximating a JCC with a collection
of individual chance constraints based on a risk allocation derived from Boole's inequality.48 The individual chance con-
straints can then be approximated using the distributionally robust Cantelli-Chebyshev inequality, which replaces each
chance constraint with an expression in terms of the mean and variance of the state. This approach, however, can be even
more conservative than a worst-case constraint handling approach.34 Since a key goal of stochastic control in general is
5028 PAULSON AND MESBAH

FIGURE 2 Illustration of the moment-based method for joint chance constraint approximation. The key notion is to ensure that the
level-set ellipsoid r (specifying the desired confidence level) is contained within the polytopic feasible region. The δxi quantities represent
the “worst-case” vectors along which the ellipsoid intersects the ith half-space

to “soften” the state constraints systematically based on the probabilistic knowledge of the uncertainty, it is imperative to
handle the JCC (4d) in a nonconservative manner.
When x ∼  (μ, Σ) is a multivariate Gaussian random vector, enforcing x ∈  with at least a probability β can be written
exactly as
1 1 ⊤ −1
P(x ∈ ) = √ e− 2 (x−μ) Σ (x−μ) dx ≥ β. (34)
n ∫
(2π) x det(Σ) 
Enforcing the integral constraint (34) is generally challenging. Here, the approach taken in the work of Van Hessem et al49
is adopted to derive a relaxation for (34). Define the ellipsoid r = {x ∶ x⊤ Σ−1 x ≤ r 2 } with some (user-defined) radius r.
Via a simple transformation, the smallest r can be chosen such that P(x ∈ μ ⊕ r ) = β. Thus, P(x ∈ ) is guaranteed to
hold when
μ ⊕ r ⊂  =⇒ P(x ∈ ) > P(x ∈ μ ⊕ r ) = β. (35)
Let Fχ2n denote the cdf of a chi-squared distribution with n degrees of freedom. It is well-known that P(x ∈ μ ⊕ r ) can
be equivalently described by a chi-squared distributed random variable P(x ∈ μ ⊕ r ) = P((x − μ)⊤ Σ−1 (x − μ) ≤ r 2 ) =
Fχ2n (r 2 ).50 This implies that r can be chosen such that Fχ2n (r 2 ) = β. Hence, appropriate selection of the radius r ensures that
c c
the constraint μ ⊕ r ⊂  is enforced. Note that this is equivalent to requiring that the ellipsoid r lies in the intersection
of half-spaces j = {x ∶ gj⊤ x ≤ dj }. An elegant way of guaranteeing this inclusion is to consider the intersection of the
ellipsoid r with each of the half-spaces j . To this end, the following lemma is presented, the proof of which involves
casting the problem as an optimization and solving it with the method of Lagrange multipliers (omitted for brevity).
Lemma 1. For any random vector x with mean μ and covariance Σ, define the Mahalanobis distance as dM (x) =
√ (b−a⊤ μ)
(x − μ)⊤ Σ−1 (x − μ). Then, the Mahalanobis distance to the hyperplane a⊤ x = b is equal to dM (x⋆ ) = √ ⊤ , where
a Σa
(b−a⊤ μ)
x⋆ = μ + √ δx and δx = √
Σa
.
a⊤ Σa a⊤ Σa

As illustrated in Figure 2, the geometric interpretation of Lemma 1 is that x⋆ is the “worst-case” vector at which the
ellipsoid r , with radius dM (x⋆ ), intersects the hyperplane, while δx is the direction along which the worst-case vector lies
away from the mean μ. Applying the result of Lemma 1 to the collection of constraints in  directly leads to the assertion
that μ ⊕ r ⊂  is equivalent to
( )
dj − gj⊤ μ(t)
√ ≥ r, j ∈ {1, nc }. (36)
gj⊤ Σ(t)gj

Constraints (36) ensure that the radius of the ellipsoid r when intersecting the hyperplane j is at least as large as
the desired radius chosen based on β. An attractive feature of constraints (36) is that they are defined in terms of the
Mahalanobis distance, which can handle non-Gaussian uncertainties. In this article, constraints (36) are used to replace
the JCC (4d) with a moment-based surrogate in terms of the mean and covariance of the state. Note that the state mean
and covariance {μ(t), Σ(t)}t∈[0,tf ] can be explicitly expressed in terms of the input u(t) using the proposed uncertainty
propagation method in Section 3.
Remark 5. Approximation (36) does not provide any guarantees with respect to the original JCC (4d). However, the
above discussed choice of r is expected to yield a close approximation for unimodal distributions. In general, r can
be thought of as a tuning parameter that can be selected to systematically trade off control performance and state
constraint satisfaction. Based on the Cantelli-Chebyshev inequality, there must exist some r that is “distributionally
PAULSON AND MESBAH 5029

robust” and ensures that the original chance constraint holds. Therefore, a Pareto front can always be constructed
from these 2 extremes to determine the best value of r for the stochastic OCP at hand. Note that when implement-
ing the stochastic OCP (4) in a receding-horizon fashion, the resultant closed-loop control law does not guarantee
closed-loop satisfaction of the chance constraints.51 This suggests that the proposed JCC approximation method can be
particularly useful in SMPC for nonlinear systems, as it would not be desirable to introduce significant conservatism
in JCC handling without providing guarantees.

5 EFFICIENT SURRO GATE FO R STOCHASTIC OCP

A computationally tractable surrogate can now be formulated for the stochastic OCP (4). To this end, the cost function
(8), the dynamic equations for the mean and covariance of state (31), and the JCC surrogate equations (36) are used to
derive the deterministic optimization problem
( )
min q⊤ μ(tf ) + μ⊤ (tf )Qμ(tf ) + tr QΣ(tf ) (37a)
u(t)

( )
s.t. ṁ j (t) = f mj (t), u(t), w,
̄ θj , t ∈ [0, tf ], j ∈ {1, Ns }, (37b)

V̇ j (t) = Aj (t)V j (t) + V j (t)Aj⊤ (t) + Ej (t)Σw Ej⊤ (t), t ∈ [0, tf ], j ∈ {1, Ns }, (37c)

∑
L
mj (0) = x̂ 0,k Φk (ξj ), j ∈ {1, Ns }, (37d)
k=0

V j (0) = 0, j ∈ {1, Ns }, (37e)

∑
L
θj = θ̂ k Φk (ξj ), j ∈ {1, Ns }, (37f)
k=0
( )
μ(t) = Tμ m j (t), 𝒮 , Π , t ∈ [0, tf ], (37g)

Σ(t) = TΣ (m j (t), V j (t), 𝒮 , Π), t ∈ [0, tf ], (37h)

u(t) ∈ U, t ∈ [0, tf ], (37i)

√
μ(t) + r gk⊤ Σ(t)gk ≤ dk , t ∈ [0, tf ], k ∈ {1, nc }. (37j)
Expressions (37b)-(37c) are the dynamic equations for the conditional mean and covariance of the state for each sample
ξj . Expressions (37d)-(37f ) define the initial conditions for the conditional mean and covariance and model parameters
for each sample ξj . Expressions (37g)-(37h) represent the mean and covariance of the state predicted over the horizon
[0, tf ]. The mean and covariance are defined explicitly in terms of the samples of mj (t) and V j (t) through (29) and (30),
respectively (eg, μ = π0,1 m1 + π0,2 m2 + · · · + π0,Ns mNs ). Expression (37i) enforces the control input constraints over the
prediction horizon, whereas (37j) is the moment-based surrogate for the JCC.
Notice that in the optimization problem (37) the number of constraints scales linearly with respect to the number of
samples Ns . As discussed in Section 3, Ns is user-specified, and typically depends on the order of the gPC expansions and
the germ dimension nξ . Since gPC selects the “optimal” basis set for propagating the model uncertainty, high propagation
accuracy is often achieved with a low expansion order, which enables using a much smaller number of samples than what
is typically required for MC-based uncertainty propagation methods, particularly when nξ is lower than approximately
10. When Σw = 0, the covariance Vj (t) = 0 for all t ∈ [0, tf ]; thus, the covariance dynamics can be excluded from the opti-
mization problem. In addition, when Ns = 1, (37) reduces to certainty equivalence optimal control. Hence, the structure
of (37) provides flexibility in addressing the OCP for uncertain, nonlinear systems.
Since t is a continuous variable in (37), this problem is an infinite-dimensional optimization that (in general) cannot
be solved exactly. A number of methods have been developed for casting dynamic optimization problems as nonlinear
5030 PAULSON AND MESBAH

programs with a finite number of decision variables. These methods include single-shooting optimization strategies, where
only the input is discretized,52 or multiple-shooting and simultaneous optimization strategies, where both the state and
input are discretized.53,54 Alternatively, the system dynamics can be discretized, for example, using a first-order Euler
method to obtain a discrete-time analog of the continuous formulation presented in (37).

Remark 6. The proposed solution method for stochastic optimal control can handle general parametrizations of the
control input u(t). For example, a common control policy is the state-feedback parametrization u(t) = Kx (t) + v(t),
where K is a fixed gain matrix and v(t) comprises the control moves. The decision variables in (37a) would then
comprise of {K, v(t)}.

6 S I M UL ATION CASE ST U DY

The proposed efficient solution method for stochastic optimal control is demonstrated on a nonlinear semibatch reactor
in the presence of probabilistic model uncertainty and stochastic disturbances. An exothermic reaction takes place in
a semibatch reactor equipped with a cooling jacket, where reactants A and B react to produce product C. The reactor
dynamics are described by Thangavel et al,55 ie,
dV
= V̇ in , V(0) = V0 ,
dt
dcA V̇ in
=− cA − kcA cB , cA (0) = cA,0 ,
dt V
dcB V̇ in
= (cB,in − cB ) − kcA cB , cB (0) = cB,0 ,
dt V
dcC V̇
= − in cC + kcA cB , cC (0) = cC,0 ,
dt V
dT V̇ in α A(T − TJ ) kcA cB H
= (Tin − T) − − , T(0) = T0 ,
dt V ρVcp ρcp
dTJ V̇ J,in α A(T − TJ )
= (TJ,in − TJ ) + , TJ (0) = TJ,0 ,
dt VJ ρVJ cp
dTJ,in 1
= (TJ,in,set − TJ,in ), TJ,in (0) = TJ,in,0 ,
dt τc
where V is the reactor volume; cA , cB , and cC are the concentration of species A, B, and C, respectively; T is the reactor
temperature; TJ is the jacket temperature; TJ,in is the jacket inlet temperature; V̇ in is the feed rate of component B with
concentration cB,in ; and TJ,in,set is the setpoint of the jacket inlet temperature, with time constant τc . The heat transfer
area in the reactor is denoted by A = 2V∕r + πr2 , where r is the reactor radius. Density ρ, heat capacity cp , and heat
transfer coefficient α are assumed to be known constants. On the other hand, the reaction rate constant k and enthalpy
H are considered to be uncertain parameters described by the uniform distributions k ∼ U(2.343 × 10−7 , 4.351 × 10−7 )
and H ∼ U(−461.5, −248.5), respectively. The control inputs consist of u = (V̇ in , TJ,in,set )⊤ . A zero-mean Gaussian process
noise is added to all of the state equations, where the noise covariance matrix is Σw = diag(0, 0, 0, 0, 1, 1, 1) × 10−4 . The
model parameters, initial conditions, and constraints are summarized in Table 2. The control objective is to maximize the
expected number of moles of product C at the end of the batch time tf while satisfying a chance constraint on the reactor
temperature T, ie,
[ ]
max E cC (tf )V(tf ) , s.t. P (322 ≤ T(t) ≤ 326) ≥ β, (38)
V̇ in (t),TJ,in,set (t)

where the control inputs are also subject to box constraints given in Table 2. In this case study, the surrogate optimization
problem (37) was solved using a direct single-shooting method, where the control inputs were treated as piecewise con-
stant over 15 intervals and the differential equations for m(t) and V(t) were integrated using a fourth-order Runge-Kutta
method. The resulting nonlinear program problem was solved with CasADi,56 which is a software tool for automatic dif-
ferentiation geared toward large-scale dynamic optimization. The Jacobians and Hessians were automatically created
efficiently in CasADi based on a symbolic implementation of the equations. These derivatives were then passed to the
interior point optimization routine57 with the tolerance set to 10−8 . The results were obtained on a MacBook Pro with 8
GB of RAM and a 2.6-GHz Intel i5 processor.
PAULSON AND MESBAH 5031

TABLE 2 Model parameters, initial conditions, and

constraints on the states and inputs for the semibatch
reactor case study55
Parameters Values Units
H [−461.5, −248.5] kJ mol−1
k [2.343, 4.351] × 10−7 m3 mol−1 s−1
ρ 1000 kg m−3
cp 4.2 kJ kg−1 K−1
r 0.092 m
VJ 2.22 × 10−3 m3
V̇ J 9.167 × 10 −5
m3 s−1
α 0.14844 kJ K−1 s−1 m−2
τc 900 s
cB,in 3000 mol m−3
Tin 300 K
tf 1800 s
V̇ in [0.0, 9.0] × 10−6 m3 s−1
TJ,in,set [280, 350] K
Vmax 7 × 10−3 m3
T [322, 326] K
V0 3.5 × 10−3 m3
cA,0 2000 mol m−3
cB,0 0 mol m−3
cC,0 0 mol m−3
T0 , TJ,0 , TJ,in,0 325 K

0
10
MC mean
MC variance
-1 gPC mean (order 3)
10 gPC variance (order 3)

-2
10
Error

-3
10

-4
10

101 102 103 104 105

Number of samples

FIGURE 3 Average error in predictions of the mean (blue) and variance (red) of the state variables over a simulation time of 1800 seconds
using Monte Carlo sampling. For comparison, the corresponding error in the mean and variance predictions using gPC is shown with dotted
lines [Colour figure can be viewed at wileyonlinelibrary.com]

Since the uncertain parameters k and H have uniform distributions, the Legendre polynomial basis was selected in
accordance with the Weiner-Askey scheme (see Table 1), where the uncertain parameters were expressed in terms of 2
standard uniform random variables ξ = (ξ1 , ξ2 )⊤ through a linear transformation
(Hub + Hlb ) (Hub − Hlb ) (kub + klb ) (kub − klb )
H= + ξ1 , k = + ξ2 .
2 2 2 2
An error convergence analysis, similar to that shown in Section 3.4, was first performed to choose the order of the
polynomial basis used in the gPC expansions. It was observed that gPC expansions with third-order basis result in
5032 PAULSON AND MESBAH

TABLE 3 Computational time for solving the stochastic

optimal control problem (38) using Monte Carlo sampling
Number of Samples Computational Time, s
16 16.1
20 37.2
50 50.1
100 219.7
200 223.7
500 746.1
1000 1284.5

5!
accurate uncertainty propagation for the problem at hand. This implies that the expansions consist of 3!2! = 10
terms. The set of samples used in the nonintrusive method to determine the coefficients of the gPC expansions
were calculated from the roots of the polynomials of one degree higher (ie, fourth-order polynomials in this case),
which corresponds to all combinations of {−0.86, −0.34, 0.34, 0.86} so that Ns = 16. The weight matrix Π was cho-
sen based on least-squares regression as in (22). The accuracy of gPC in predicting the state mean and covariance
was compared with that of MC sampling. Figure 3 shows the average error in predictions of the mean and vari-
ance of the state variables (for a given input profile) over a simulation time of 1800 seconds as a function of the
number of samples used in the MC sampling. As can be seen, the MC sampling requires as much as approxi-
mately 50 000 samples to yield the same accuracy as gPC with the third-order basis, where the expansion coefficients
are determined using only Ns = 16 symmetric sample points. The substantially lower number of samples used
in gPC results in significantly lower computational times for solving the surrogate optimization problem (37), as
compared with solving the stochastic OCP (4) using MC sampling. Table 3 indicates that increasing the number of
samples from 16 to just 1000 samples leads to an almost 80-fold increase in the computational time required for solv-
ing the stochastic OCP (38) for the case study at hand using MC sampling. It is worth emphasizing that even with 1000
samples, the MC sampling suffers from an order-of-magnitude higher error than gPC with the third-order basis solved in
this example with only 16 samples (Figure 3). This comparison suggests that the proposed gPC-based uncertainty prop-
agation method is significantly more efficient than MC sampling for model uncertainty handling. Note that other types
of random sampling methods exist (e.g., Latin-Hypercube sampling), which are known to have certain advantages over
standard MC. Comparing these alternatives to the proposed gPC-based method as well as possibly incorporating them
into the nonintrusive approaches for estimating the gPC coefficients will be the subject of future research.
The semibatch reactor exhibits a trade-off between the optimal value of the cost function nC (tf ) = cC (tf )V(tf ) and the
desired level of state constraint satisfaction β (see (38)). Larger values of β restrict the feasible region of the optimization
problem further, possibly leading to lower values for the cost function (ie, worse control performance). To select the value
of β systematically, a Pareto optimality curve, shown in Figure 4 , was constructed by solving (37) for various values of β.
The computational times for solving (37) ranged from 5 to 20 seconds depending on the value of β. Figure 4 shows a sharp
drop in the control performance when β is in the range of 0.2 to 0.3. As β is increased further, the control performance
drops linearly with a fairly small slope. Note that there is a one-to-one mapping between r (ie, the covariance weight in
(37j) and β, where β = 0 corresponds to r = 0.
We now investigate stochastic optimal control of the nonlinear semibatch reactor for 3 different levels of state constraint
satisfaction, that is, a low, intermediate, and high value of β = 0, β = 0.3, and β = 0.9, respectively. The case of β = 0
is referred to as “nominal” since the state constraint in (38) is enforced only in terms of the state mean. The optimal
control inputs for the 3 cases, obtained by solving (37), are shown in Figure 5, which suggests that the operation changes
drastically to realize the desired level of state constraint satisfaction. A total of 10 000 MC simulation runs were performed
for each case to compare the performance of the system under the optimal control inputs. Figure 6 shows the estimated
pdf of the cost function nC (tf ) (ie, the final amount of product). It is observed that the mean and variance of the pdf of nC (tf )
decrease and the pdf becomes skewed (no longer uniformly distributed) as β increases. Figure 6 clearly illustrates that
relaxing the state constraint allows maximizing the cost function nC (tf ) more effectively, which is achieved to its fullest
extent when β = 0. In addition, this figure highlights the importance of adequate uncertainty propagation in the stochastic
optimal control of a nonlinear system, as nC (tf ) exhibits considerably different distributions under different control inputs.
To illustrate the chance constraint handling, the mean reactor temperature T and its predicted 90% confidence region
during the batch time are shown in Figure 7 for the 3 cases of β. When β = 0.9, the predicted 90% confidence region
PAULSON AND MESBAH 5033

4.5

3.5

2.5

1.5

0.5
0 0.2 0.4 0.6 0.8 1
β

FIGURE 4 Pareto optimality curve showing the optimal value of the cost function nC (tf ) (ie, the final amount of product) as a function of
various levels of temperature constraint satisfaction probability β in the stochastic optimal control problem (38)

10

β
5 β
β

0
0 200 400 600 800 1000 1200 1400 1600 1800

340

320

300

280
0 200 400 600 800 1000 1200 1400 1600 1800

FIGURE 5 Optimal control inputs obtained by solving the surrogate optimization problem (37) for the semibatch reactor case study for 3
different levels of state constraint satisfaction β [Colour figure can be viewed at wileyonlinelibrary.com]

is fully contained within the temperature constraints, as enforced by the chance constraint surrogate (36). On the other
hand, significant portions of the predicted 90% confidence region are outside the temperature constraints when β = 0
and β = 0.3. Approximately 55%, 45%, and 3% state constraint violation is observed for β = 0, β = 0.3, and β = 0.9,
respectively, suggesting nonconservative and reliable handling of the chance constraint in (38).
As discussed earlier, a commonly used approach to provide explicit guarantees for chance constraint satisfaction is the
distributionally robust Cantelli-Chebyshev inequality. Figure 8 shows the empirical cdf of the reactor temperature at the
final batch time obtained for β = 0.9 using the proposed moment-based surrogate for JCC and the Cantelli-Chebyshev
inequality with a uniform risk allocation. While both approaches yield less than 10% actual constraint violation, the
Cantelli-Chebyshev inequality results in 0% violation and the minimum and maximum temperature are about 1◦ C away
from the temperature constraints. This clearly indicates that the Cantelli-Chebyshev inequality is excessively conservative.
In fact, a robust implementation of the state constraint in this case (ie, requiring the full support of the temperature
distribution be within the temperature constraints) would be less conservative and this would lead to a better control
performance in terms of maximizing nC (tf ). Although the Cantelli-Chebyshev inequality provides theoretical guarantees,
its conservatism would be unacceptable in practice, which can be partly addressed using the proposed moment-based
surrogate for JCC.
5034 PAULSON AND MESBAH

8
β
7 β
β

0
0 1 2 3 4 5 6

FIGURE 6 Estimated probability density of the cost function nC (tf ) constructed based on 10 000 Monte Carlo simulation runs using the
optimal control inputs depicted in Figure 5 [Colour figure can be viewed at wileyonlinelibrary.com]

332 β
β
β
330

328

326

324

322

320

318

316
0 200 400 600 800 1000 1200 1400 1600 1800

FIGURE 7 Mean (solid line) and 90% confidence region (shaded) of the reactor temperature calculated based on 10 000 Monte Carlo
simulation runs using the optimal control inputs depicted in Figure 5. The dashed lines represent the temperature constraints [Colour figure
can be viewed at wileyonlinelibrary.com]

7 CO NCLUSIONS A ND FUTURE WO RK

This article presents an efficient solution method for stochastic optimal control for nonlinear systems with 2 dis-
tinct sources of uncertainty: time-invariant probabilistic uncertainty in the model parameters and initial conditions
and time-varying stochastic disturbances. To this end, the first contribution of this article is an uncertainty propaga-
tion method that can handle both probabilistic model uncertainty and stochastic disturbances through exploiting gPC
and conditional probability rules. The uncertainty propagation method is shown to be significantly more efficient than
a standard random sampling method. The second contribution is a moment-based surrogate for JCC approximation,
which avoids the expensive computation of the multivariate probability distribution of the state, or the use of nons-
mooth operators. We demonstrate how a parameter weight multiplying the covariance in the moment-based surrogate
for the JCC can be easily tuned to achieve any desired probability of constraint satisfaction. The effectiveness of the
proposed solution method for stochastic optimal control is illustrated on a nonlinear semibatch reactor case study,
where a significant decrease in the computational cost is achieved compared with a standard random sampling method
for stochastic optimal control. Furthermore, we demonstrate that the moment-based JCC approximation method is
substantially less conservative than the widely used distributionally robust Cantelli-Chebyshev inequality for chance
constraint approximation.
PAULSON AND MESBAH 5035

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
321 322 323 324 325 326 327

FIGURE 8 Empirical cumulative distribution of the reactor temperature at the final batch time when the moment-based surrogate and the
distributionally robust Cantelli-Chebyshev inequality are used for approximating the state chance constraint in the stochastic optimal control
problem (38). The distributions are constructed based on 10 000 Monte Carlo simulation runs for the case of β = 0.9 [Colour figure can be
viewed at wileyonlinelibrary.com]

Since the proposed solution method for stochastic optimal control is computationally efficient, it can be implemented
in a receding-horizon manner, ie, SMPC. This requires updating the initial conditions in the stochastic OCP based on new
system measurements at every sampling time. Since the stochastic disturbances are assumed to be Markov, the augmented
system dynamics composed of system state and parameter evolve as a Markov process. This allows for a theoretical sep-
aration between the stochastic OCP and the Bayesian estimation of the augmented state (see Mesbah58 ). However, the
Bayesian estimation problem cannot be solved exactly for nonlinear systems, suggesting that one must resort to approxi-
mations such as the particle filter59 or other moment-based methods like the extended, unscented, or ensemble Kalman
filters.60 Future work will explore how to best combine the Bayesian estimation methods with the proposed solution
method for stochastic optimal control. Since the proposed surrogate optimization problem relies on the mean and covari-
ance of the “estimated” augmented state, one approach is to fit the gPC expansion coefficients to the moments computed
from the Bayesian estimator at every sampling time. To this end, the main challenges are that higher-order moments may
be required to obtain accurate estimates of the coefficients of the gPC expansions, and that the distribution of the aug-
mented state may change substantially over time (ie, to an extent that the gPC expansions may no longer be accurate). The
latter issue can be mitigated by adapting the basis online using arbitrary polynomial chaos41 as long as the computational
cost of basis adaptation is manageable for real-time implementation.
Another important future research direction is to investigate parallel computing of the proposed surrogate optimiza-
tion problem. As shown in Section 5, the surrogate optimization problem is mostly composed of decoupled dynamic
equality constraints for each realization of the parameters. This structure makes the problem highly amenable to dis-
tributed optimization. The parallel structure of the Bayesian estimation problem can also be exploited so that the overall
receding-horizon algorithm scales favorably with respect to the number of samples and uncertainty sources.

ORCID

Ali Mesbah http://orcid.org/0000-0002-1700-0600

REFERENCES
1. Morari M, Lee JH. Model predictive control: past, present and future. Comput Chem Eng. 1999;23:667-682.
2. Mayne DQ. Model predictive control: recent developments and future promise. Automatica. 2014;50:2967-2986.
3. Zheng A, Morari M. Robust stability of constrained model predictive control. Paper presented at: Proceedings of the American Control
Conference; 1993; San Francisco.
4. Bemporad A, Morari M. Robust model predictive control: a survey. In: Garulli A, Tesi A, eds. Robustness in Identification Control. Vol. 245.
London: Springer; 1999: 207-226.
5. Langson W, Chryssochoos I, Raković SV, Mayne DQ. Robust model predictive control using tubes. Automatica. 2004;40:125-133.
5036 PAULSON AND MESBAH

6. Goulart PJ, Kerrigan EC, Maciejowski JM. Optimization over state feedback policies for robust control with constraint. Automatica.
2006;42:523-533.
7. Raković SV, Kouvaritakis B, Findeisen R, Cannon M. Homothetic tube model predictive control. Automatica. 2012;48:1631-1638.
8. Mesbah A. Stochastic model predictive control: an overview and perspectives for future research. IEEE Control Syst. 2016;36:30-44.
9. Schwarm A, Nikolaou M. Chance-constrained model predictive control. AIChE J. 1999;45:1743-1752.
10. Li P, Wendt M, Wozny G. A probabilistically constrained model predictive controller. Automatica. 2002;38:1171-1176.
11. Nemirovski A, Shapiro A. Convex approximations of chance constrained programs. SIAM J Optim. 2006;17:969-996.
12. Darlington J, Pantelides CC, Rustem B, Tanyi BA. Decreasing the sensitivity of open-loop optimal solutions in decision making under
uncertainty. Eur J Oper Res. 2000;121:343-362.
13. Nagy ZK, Braatz RD. Open-loop and closed-loop robust optimal control of batch processes using distributional and worst-case analysis.
J Process Control. 2004;14:411-422.
14. Julier SJ, Uhlmann JK. Unscented filtering and nonlinear estimation. Proc IEEE. 2004;92(3):401-422.
15. Telen D, Vallerio M, Cabianca L, Houska B, Van Impe J, Logist F. Approximate robust optimization of nonlinear systems under parametric
uncertainty and process noise. J Process Control. 2015;33:140-154.
16. Lecchini A, Glover W, Lygeros J, Maciejowski J. Monte Carlo optimisation for conflict resolution in air traffic control. In: Blom HAP,
Lygeros J, eds. Stochastic Hybrid Systems. Lecture Notes in Control and Information Science, vol 337. Berlin, Heidelberg: Springer;
2006:257-276.
17. Kantas N, Maciejowski J, Lecchini-Visintini A. Sequential Monte Carlo for model predictive control. In: Magni L, Raimondo DM, Allgöwer
F, eds. Nonlinear Model Predictive Control. Vol. 384. Berlin, Germany: Springer; 2009:263-273.
18. Diwekar U. Introduction to Applied Optimization. Springer US: Springer Science & Business Media; 2008.
19. Schildbach G, Fagiano L, Frei C, Morari M. The scenario approach for stochastic model predictive control with bounds on closed-loop
constraint violations. Automatica. 2014;50:3009-3018.
20. Calafiore GC, Fagiano L. Stochastic model predictive control of LPV systems via scenario optimization. Automatica. 2013;49:1861-1866.
21. Calafiore GC, Campi MC. The scenario approach to robust control design. IEEE Trans Autom Control. 2006;51:742-753.
22. Xiu D, Karniadakis GE. The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM J Sci Comput. 2002;24:619-644.
23. Fagiano L, Khammash M. Nonlinear stochastic model predictive control via regularized polynomial chaos expansions. Paper presented
at: Proceedings of the 51st IEEE Conference on Decision and Control; 2012; Maui.
24. Mesbah A, Streif S, Findeisen R, Braatz RD. Stochastic nonlinear model predictive control with probabilistic constraints. Paper presented
at: Proceedings of the American Control Conference; 2014; Portland.
25. Wan X, Karniadakis GE. An adaptive multi-element generalized polynomial chaos method for stochastic differential equations. J Comput
Phys. 2005;209:617-642.
26. Ben-Tal A, Ghaoui LE, Nemirovski A. Robust Optimization. Princeton, NJ: Princeton University Press; 2009.
27. Nemirovski A, Shapiro A. Convex approximations of chance constrained programs. SIAM J Optim. 2006;17:969-996.
28. Oldewurtel F, Parisio A, Jones CN, et al. Use of model predictive control and weather forecasts for energy efficient building climate control.
Energy Build. 2012;45:15-27.
29. Kouvaritakis B, Cannon M. Model Predictive Control: Classical, Robust and Stochastic. Switzerland: Springer; 2015.
30. Lorenzen M, Dabbene F, Tempo R, Allgöwer F. Constraint-tightening and stability in stochastic model predictive control. IEEE Trans
Autom Control. 2017;62:3165-3177.
31. Dentcheva D, Prékopa A, Ruszczynski A. Concavity and efficient points of discrete distributions in probabilistic programming. Math
Program. 2000;89:55-77.
32. Pagnoncelli K, Ahmed BS, Shapiro A. Sample average approximation method for chance constrained programming: theory and applica-
tions. J Optim Theory Appl. 2009;142:399-416.
33. Blackmore L, Ono M, Bektassov A, Williams BC. A probabilistic particle-control approximation of chance-constrained stochastic
predictive control. IEEE Trans Robot. 2010;26:502-517.
34. Paulson JA, Buehler EA, Braatz RD, Mesbah A. Stochastic predictive control with joint chance constraints. Int J Control. 2017:1-14.
35. Rawlings JB, Mayne DQ. Model Predictive Control: Theory and Design. Madison: Nob Hill Publishing; 2009.
36. Caflisch RE. Monte Carlo and quasi-Monte Carlo methods. Acta Numer. 1998;7:1-49.
37. Wiener N. The homogeneous chaos. Am J Math. 1938;60:897-936.
38. Cameron RH, Martin WT. The orthogonal development of non-linear functionals in series of Fourier-Hermite functionals. Ann Math.
1947;48:385-392.
39. Ernst OG, Mugler A, Starkloff H, Ullmann E. On the convergence of generalized polynomial chaos expansions. ESAIM: Math Model Numer
Anal. 2012;46:317-339.
40. Navarro M, Witteveen J, Blom J. Polynomial chaos expansion for general multivariate distributions with correlated variables. arXiv
preprint:1406.5483.
41. Paulson JA, Buehler EA, Mesbah A. Arbitrary polynomial chaos for uncertainty propagation of correlated random variables in dynamic
systems. Paper presented at: Proceedings of the IFAC World Congress Conference; 2017; Toulouse.
42. Rosenblatt M. Remarks on a multivariate transformation. Ann Math Stat. 1952;23:470-472.
PAULSON AND MESBAH 5037

43. Kim K-KK, Shen DE, Nagy ZK, Braatz RD. Wiener's polynomial chaos for the analysis and control of nonlinear dynamical systems with
probabilistic uncertainties. IEEE Control Syst. 2013;33:58-67.
44. Ghanem R, Spanos P. Stochastic Finite Elements - A Spectral Approach. New York: Springer; 1991.
45. Paulson JA, Streif S, Mesbah A. Stability for receding-horizon stochastic model predictive control. Paper presented at: Proceedings of the
American Control Conference; 2015; Chicago.
46. Bavdekar V, Mesbah A. Stochastic nonlinear model predictive control with joint chance constraints. Paper presented at: Proceedings of
the 10th IFAC Symposium on Nonlinear Control Systems; 2016; Monterey, CA.
47. Calafiore GC, El Ghaoui L. On distributionally robust chance-constrained linear programs. J Optim Theory Appl. 2006;130:1-22.
48. Blackmore L, Ono M. Convex chance constrained predictive control without sampling. Paper presented at: Proceedings of the AIAA
Guidance, Navigation and Control Conference; 2009; Chicago.
49. Van Hessem DH, Scherer CW, Bosgra OH. LMI-Based closed-loop economic optimization of stochastic process operation under state and
input constraints. Paper presented at: Proceedings of the 40th IEEE Conference on Decision and Control; 2001; Orlando.
50. Bard Y. Nonlinear Parameter Estimation. London: Academic Press; 1974.
51. Paulson JA, Harinath E, Foguth LC, Braatz RD. Nonlinear model predictive control of systems with probabilistic time-invariant
uncertainties. Paper presented at: Proceedings of the 5th IFAC Conference on Nonlinear Model Predictive Control; 2015; Seville.
52. Cervantes A, Biegler LT. Optimization strategies for dynamic systems. In: Floudas C, Pardalos P, eds. Encyclopedia of Optimization.
Springer; 2008:2847-2858.
53. Leineweber DB, Bauer I, Bock HG, Schlöder JP. An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process
optimization Part 1: theoretical aspects. Comput Chem Eng. 2003;27:157-166.
54. Biegler LT. An overview of simultaneous strategies for dynamic optimization. Chem Eng Process Process Intensif . 2007;46:1043-1053.
55. Thangavel S, Lucia S, Paulen R, Engell S. Towards dual robust nonlinear model predictive control: a multi-stage approach. Paper presented
at: Proceedings of the American Control Conference. 2015; Chicago.
56. Andersson J, Åkesson J, Diehl M. CasADi: a symbolic package for automatic differentiation and optimal control. In: Forth S, Hovland P,
Phipps E, Utke J, Walther A, eds. Recent Advances in Algorithmic Differentiation. Lecture Notes in Computational Science and Engineering,
Vol 87. Berlin, Heidelberg: Springer; 2012:297-307.
57. Biegler LT, Zavala VM. Large-scale nonlinear programming using IPOPT: an integrating framework for enterprise-wide dynamic. Comp
Chem Eng. 2009;33:575-582.
58. Mesbah A. Stochastic Model Predictive Control with Active Uncertainty Learning: A Survey on Dual Control. Annual Reviews in
Control. 2017.
59. Arulampalam MS, Maskell S, Clapp T. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans
Signal Process. 2002;50:174-188.
60. Chen Z. Bayesian filtering: from Kalman filters to particle filters, and beyond. Statistics. 2003;182:1-69.

How to cite this article: Paulson JA, Mesbah A. An efficient method for stochastic optimal control
with joint chance constraints for nonlinear systems. Int J Robust Nonlinear Control. 2019;29:5017–5037.
https://doi.org/10.1002/rnc.3999