IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO.

8, AUGUST 1998 1163

for !  3:4868. We conclude that the minimal volume is vmin  Constrained Linear Quadratic Regulation
8:45 and the relative volume is
Pierre O. M. Scokaert and James B. Rawlings
vmin  11:2667:
v(r0 ) Abstract— This paper is a contribution to the theory of the infinite-
horizon linear quadratic regulator (LQR) problem subject to inequality
constraints on the inputs and states, extending an approach first proposed
VI. CONCLUSION by Sznaier and Damborg [16]. A solution algorithm is presented, which
requires solving a finite number of finite-dimensional positive definite
As an alternative to the classical robustness margin, in this paper
quadratic programs. The constrained LQR outlined does not feature
we introduced the notion of minimum destabilizing volume. We have the undesirable mismatch between open-loop and closed-loop nominal
shown that this volume can be easily computed if the control system is system trajectories, which is present in the other popular forms of model
affected by interval and affine parametric uncertainties. A problem of predictive control (MPC) that can be implemented with a finite quadratic
subsequent interest is the extension of these results to more general programming algorithm. The constrained LQR is shown to be both
optimal and stabilizing. The solution algorithm is guaranteed to terminate
uncertainty structures. in finite time with a computational cost that has a reasonable upper
bound compared to the minimal cost for computing the optimal solution.
Inherent to the approach is the removal of a tuning parameter, the
REFERENCES control horizon, which is present in other MPC approaches and for which
no reliable tuning guidelines are available. Two examples are presented
[1] J. Ackermann, A. Bartlett, D. Kaesbauer, W. Sienel, and R. Steinhauser, that compare constrained LQR and two other popular forms of MPC.
Robust Control, Systems with Uncertain Physical Parameters. London, The examples demonstrate that constrained LQR achieves significantly
U.K.: Springer-Verlag, 1993. better performance than the other forms of MPC on some plants, and
[2] B. R. Barmish, New Tools for Robustness of Linear Systems. New York: the computational cost is not prohibitive for online implementation.
Macmillan, 1994.
[3] B. R. Barmish, P. P. Khargonekar, Z. C. Shi, and R. Tempo, “Robustness Index Terms—Constraints, infinite horizon, linear quadratic regulation,
margin need not be a continuous function of the problem data,” Syst. model predictive control.
Contr. Lett., vol. 15, pp. 91–98, 1990.
[4] B. R. Barmish and B. Polyak, “The volumetric singular value and
robustness of feedback control systems,” in Proc. 32nd Conf. Decision I. INTRODUCTION
and Control, San Antonio, TX, 1993, pp. 521–522.
[5] A. C. Bartlett, C. V. Hollot, and L. Huang, “Root locations of an entire
In 1960, Kalman [4] showed that the Riccati equation associated
polytope of polynomials: It suffices to check the edges,” Math. Contr., with the finite-horizon linear quadratic regulator (LQR) has a well-
Signals, and Syst., vol. 1, pp. 61–71, 1988. defined limit and used that result to solve the infinite-horizon LQR
[6] S. P. Bhattacharyya, H. Chapellat, and L. H. Keel, Robust Control: The problem. To date, this remains one of the most influential discoveries
Parametric Approach. Englewood Cliffs, NJ: Prentice-Hall, 1995. of the modern control era. In the late 1970’s, Richalet et al. [12]
[7] T. E. Djaferis, Robust Control Design: A Polynomial Approach.
Boston, MA: Kluwer, 1995. and Cutler and Ramaker [3] emulated the finite-horizon LQR for
[8] T. E. Djaferis and C. V. Hollot, “The stability of a family of polynomials constrained processes, marking the beginning of the industrial imple-
Ok
can be deduced from a finite number ( 3 ) of frequency checks,” IEEE mentation of what comes to be known as model predictive control
Trans. Automat. Contr., vol. 34, pp. 982–986, 1989. (MPC). In the 1980’s, the theoretical development of MPC with
[9] R. Horst and H. Tui, “Global optimization: A deterministic approach,” in
Control and Information Sciences. Berlin, Germany: Springer-Verlag,
constraints ran into serious difficulties, and it became increasingly
1996. apparent that a return to an infinite-horizon formulation is required
[10] V. L. Kharitonov, “Asymptotic stability of an equilibrium position of to produce stabilizing control laws [1]. Various contributions are
a family of systems of linear differential equations,” Differential’nye made in the early 1990’s, introducing infinite horizons into the MPC
Uraveniya, vol. 14, pp. 1483–1485, 1978. framework for constrained linear processes [11]. However, rather
[11] J. Kogan, “Robust stability and convexity,” in Control and Information
Sciences. Berlin, Germany: Springer-Verlag, 1995. than address the full infinite-horizon constrained LQR problem, all
[12] M. Marden, Geometry of Polynomials, Mathematical Surveys, Provi- emerging MPC variants, except the one by Sznaier and Damborg [16]
dence, RI: Amer. Math. Soc., no. 3. discussed below, rely on a finite and suboptimal parameterization of
the postulated control sequence. In this paper, the use of a finite-input
parameterization is relaxed, leading to the formulation of a control
scheme we call constrained LQR.
In their remarkably succinct paper, Sznaier and Damborg [16]
present the basics of the approach. They treat a more restricted
class of problems than the one considered here. In their problem
statement, the state and input are constrained to lie in bounded convex
polyhedrons. We make no restriction on boundedness of the constraint
region. A major concern in their work is the real-time implementation
limits on calculation time, which can cause early termination of their
algorithm prior to optimal solution. In this work we have found that
Manuscript received December 13, 1995. This work was supported in part
by the National Science Foundation under Grant CTS-9311420 and by the
Texas Advanced Technology Program under Grant 003658-078.
The authors are with the Department of Chemical Engineering, University
of Wisconsin, Madison, Madison, WI 53706 USA.
Publisher Item Identifier S 0018-9286(98)04634-0.

0018–9286/98$10.00  1998 IEEE

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on July 13,2022 at 01:46:06 UTC from IEEE Xplore. Restrictions apply.
1164 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO. 8, AUGUST 1998

for many examples, the cost of our algorithm is not much greater We summarize the three relevant control problems of interest in
and is sometimes less than the cost of the standard suboptimal MPC this paper.
algorithms already in use in industrial process control applications. Problem 0—LQR:
Therefore, many industrial applications exist for which this approach
min (xt ; )
can be implemented as an improvement over current practice. The 
constrained LQR approach also removes what we consider in current subject to:
MPC approaches to be a nuisance tuning parameter, the control xj +1jt = Axj jt + Buj jt ; t  j: (6)
horizon N , i.e., the number of future control moves considered in
the optimization. Problem 0 was formulated and solved by Kalman, and the solution
Sznaier and Damborg do not provide a full discussion of the is the well-known linear feedback control law
ut = 0Kxt
conditions under which their algorithm is stabilizing nor prove that
(7)
it terminates in finite time. The stability of their controller is not
guaranteed if their algorithm terminates before the optimal solution in which the controller gain K can be calculated from the solution
is calculated. In Sznaier’s thesis [15], a contraction constraint is of the discrete algebraic Riccati equation. The linear feedback law of
introduced, which, if feasible, does provide asymptotic stability; (7) is stabilizing under the above assumptions of stabilizability and
however, this added constraint leads to suboptimal control, even when detectability.
ample computational time is available. We provide more complete Problem 1—Constrained LQR:
results on termination and stability and also highlight the benefits of
constrained LQR compared to other MPC variants in current use. min (xt ;  )

The paper is organized as follows. The constrained LQR law is subject to:
defined in Section II, and its stabilizing properties are established.
xj +1jt =Axj jt + Buj jt
Section III outlines the details of a practical implementation of
constrained LQR, which is shown to require the solution of a finite Hxj +1jt h; tj
number of finite-dimensional positive definite quadratic programs. In Duj jt d: (8)
Section IV, we discuss the computational aspects of the constrained
LQR algorithm and show that the computational cost has a reasonable Problem 1 is a natural extension of the infinite-horizon LQR (Problem
upper bound, compared to the minimal cost for computing the optimal 0) that includes constraints. The only difficulty is the infinite number
solution. Finally, examples are presented in Section V, where it of decision variables in the optimization and the infinite number of
is demonstrated that constrained LQR achieves significantly better constraints.
performance than other forms of MPC on some plants, and the Problem 2—An MPC Problem:
computational cost is not prohibitive for online implementation. min (xt ; )
Concluding remarks are made in Section VI. 
subject to:
II. OVERVIEW OF THE PROBLEM STATEMENT xj +1jt = Axj jt + Buj jt ; tj
We consider time-invariant, linear, and discrete-time systems de- Hxj +1jt  h;
scribed by the state-space equation tj t+N 01
xt+1 = Axt + But (1) Duj jt  d;
in which xt 2 n
IR and ut 2 m
IR are the state and input vectors uj jt = 0 Kxj jt ; t+N  j (9)
at discrete time t and A and B are the state transition and input
This form of MPC has a finite number of decision variables, N , and
distribution matrices, respectively. It is assumed throughout the paper
a finite number of constraints, N (nd + nh ); it can therefore be solved
that (A; B ) is stabilizable.
with standard quadratic programming methods. The only interest in
The control objective is to regulate the state of the system optimally
this problem here is as an aid in solving Problem 1. The idea of
appending the u = 0Kx unconstrained linear law to the finite set of
to the origin. Optimality is defined in terms of a quadratic objective
and a set of inequality constraints.
The objective is defined over an infinite horizon and is given by
N decision variables is used in Michalska and Mayne’s dual-mode
1 controller for nonlinear systems [8]; see also [10] for a brief review.
Concentrating on Problem 1, the open-loop optimal control is
(xt ; ) = x0j jt Qxj jt + uj0 jt Ruj jt (2)
j =t obtained by minimization of the objective over all control pro-
in which Q  0 and R > 0 are symmetric weighting matrices, such
files that satisfy the inequality constraints. The feedback law is
that (Q1=2 ; A) is detectable, and
then defined by receding horizon implementation of the optimal
open-loop control. Given the open-loop optimal strategy ? (xt ) =
 = futjt ; ut+1jt ; 1 1 1g (3) fut?jt(xt ); ut?+1jt (xt ); 1 1 1g, we therefore have the control law
xj +1jt = Axj jt + Buj jt ; tj (4) ut = g(xt )  ut?jt (xt ) (10)
with xtjt = xt . The constraints are also defined on an infinite horizon
in which g : IRn ! IRm denotes the nonlinear map between state
and take the form
Hxj +1jt  h; tj
and control.
(5) Remark 1: For notational simplicity, we drop the xt argument in
Duj jt  d uj?jt (xt ) in the sequel, i.e., uj?jt  uj?jt (xt ).
where h 2 IR+ and d 2 IR+ define the constraint levels, with nh
n n
and nd denoting the number of state and input constraints respec- A. Stabilizing Properties
tively, and H and D are the state and input constraint distribution Not surprisingly, the constrained LQR law (10) benefits from
matrices. similar stabilizing properties as recent formulations of MPC.

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on July 13,2022 at 01:46:06 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO. 8, AUGUST 1998 1165

Theorem 1 (Nominal Stability): The origin is an exponentially Problem 1—Constrained LQR: Given x 2 , find a control
stable fixed point of the closed-loop system xt+1 = Axt + Bg (xt ), profile  ? (x) for which
with a region of attraction equal to the domain of g .
? (x) = min (x; ) (12)
Proof: The proof proceeds by showing that, under the assump- 2IP
tion that xt belongs to the domain of g [i.e., there exists a control
is attained.
profile  that satisfies (4) and (5)], the optimal value of the objective,
Remark 4: Existence of  ? (x) for all x 2
? (x), is finite. This implies that Q1=2 xj jt ; uj jt ! 0, as j ! 1. is easy to establish
and follows from the results in the sequel. Uniqueness follows from
Because nominal closed-loop performance is identical to the open-
loop predictions, it follows that Q1=2 xt ; ut ! 0, as t ! 1; as
the uniqueness of the minimum of a positive definite quadratic
(Q
1=2 ; A) is detectable, this implies that xt ! 0. Note that the function on a convex set.
A contribution of this paper is to show that under the usual
assumption of stabilizability ensures that the domain of g includes at
assumption that all elements of h and d are strictly positive (which
least an open neighborhood of the origin.
implies that no constraint passes through the origin and r > 0—see
Moreover, in view of [13], we also have a perturbed stability
Definition 2), the solution to Problem 1 can be calculated in finite
guarantee.
computational time. We therefore show that suboptimality need not be
Theorem 2 (Perturbed Stability): The origin is a locally asymptot-
a property of implementable constrained predictive control schemes.
ically stable fixed point of the perturbed closed-loop system xt+1 =
We start by considering the related MPC problem in which the
Axt + Bg(xt ) + pt , if limt!1 pt = 0.
following finite input parameterization is employed:
Proof: Under the assumption of feasibility, Muske [9] shows
that g (x) is Lipschitz continuous. As g (x) is a nominally expo- uj jt = 0Kxj jt ; j t+N (13)
nentially stabilizing control law, the perturbed stability result of the
theorem follows directly from [13]. where N 2 is the finite control horizon.
Remark 2: The result of Theorem 2 is local; this means that pt In this setting, the open-loop optimal control is obtained by solution
must be small enough, for all t, such that the trajectory of the of the familiar MPC optimization.
perturbed system remains in the domain of g , i.e., the perturbation Problem 2—An MPC Problem: Given N 2 , x 2 N , find
must not cause infeasibility. the control profile N (x) = futNjt ; utN+1jt ; 1 1 1 ; 0KxN t+N jt ;
Remark 3: Note also that, if the perturbation pt converges to 0KxNt+N +1jt ; 1 1 1g for which
zero exponentially, the result of Theorem 2 can be strengthened to
N (x) = min (x; ) (14)
exponential stability. 2IP
This result is significant because it leads directly to a stability proof
for the cascade of a stable state observer and the constrained LQR is attained.
law. This constrained minimization is an Nm-dimensional positive
definite quadratic program with N (nc + nd ) constraints. For any
finite horizon N , efficient solutions for Problem 2 are therefore easily
III. IMPLEMENTATION
formulated.
Remark 5: Existence of N (x) follows, by definition, for all x 2
A. Notation
N . Uniqueness results from positive definiteness of the Hessian,
As in [16], we make the following definition. which is guaranteed because R is positive definite. Note that in the
Definition 1: K  IRn denotes the set of states xt for which the context of Problem 2, we have
unconstrained LQR law, uj jt = 0Kxj jt (t  j ), satisfies (4) and N 01
(5). (x; ) = 0 0 0
(xj jt Qxj jt + uj jt Ruj jt ) + xt+N jt Qx
~ t+N jt (15)
It is immediately apparent that  ? (xt ) = f0Kxtjt ; 0Kxt+1jt ; j =t
1 1 1g, for all xt 2 K . Under the assumption that all elements of
h and d are strictly positive, K contains an open neighborhood of where Q
~ is the solution of the matrix Lyapunov equation

Q~ = Q + K 0 RK + (A 0 BK )0 Q~ (A 0 BK ):
the origin [16].
Definition 2: Br denotes a ball of radius r > 0, centered on the (16)
origin, such that Br  K .
Definition 3: Given N 2 and the current value of the state xt ,
IPN (xt ) denotes the set of control profiles  such that
C. Properties of ? ; N ;  ? , and N
Hxj +1jt  h; tj t+N 01 The interesting relationship between  ? (x) and N (x) is the
Duj jt  d motivation for our interest in Problem 2. We have the following
uj jt = 0 Kxj jt ; t + N  j: (11) results.
Definition 5: We denote by x?jjt the state predictions that corre-
Further, we define IP(xt ) to be the set of control profiles that satisfy spond to the optimal open-loop control profile  ? (xt ) and by xN j jt
(4) and (5). IP(xt ) thus may be thought of as the limit of IPN (xt ) those that correspond to N (xt ).
as N ! 1. Remark 6: Note that x?jjt and xN j jt are functions of xt . That
N  IR denotes the set of states x for which
Definition 4: n
IPN (x) 6= ;. Similarly,  IRn is the set of states for which dependence is, however, left implicit for notational simplicity.
Lemma 1: x?jjt 2 K () u?kjt = 0Kx?kjt ; 8k  j:
IP(x) 6= ;, i.e., is the domain of g . Proof: The control profile that is optimal with respect to the
constrained LQR objective is uj jt = 0Kxj jt , for all xj jt 2 K —see
B. Related Problem Statements Definition 1. In view of Bellman’s principle of optimality, the
Using the notation defined above, the optimal open-loop con- sequence of controls fut?jt ; ut?+1jt ; 1 1 1g is optimal over all time
strained control problems described in Section II may be stated intervals [k; 1), k  t. The forward implication of the lemma
concisely as follows. therefore follows from optimality.

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on July 13,2022 at 01:46:06 UTC from IEEE Xplore. Restrictions apply.
1166 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO. 8, AUGUST 1998

By definition, K is the set of states outside which u = 0Kx can therefore be warm-started from the solution at the previous cycle.
violates the constraints of (4) and (5). As the optimal control profile As the solution of Problem 2 approaches that of Problem 1 when N
fu?t t ;?ut?+1 t; 1 1 1g? satisfies these constraints, it follows that uj? t =
j j j increases, the computational burden of the quadratic program with
0Kxj t only if xj t 2 K , which establishes the reverse implication
j j warm-start should not rise dramatically as Algorithm 1 is cycled and
of the lemma. N is increased.
Theorem 3: For every x 2 , there exists a finite N (x) 2 , 1 Although the set K may be expensive to calculate, the check
such that ? (x) = N (x);  ? (x) = N (x), for all N  N (x): 1 that xNt+N t 2 K , in Step 2), is inexpensive. In fact, the check
j
Proof: As K contains an open neighborhood of the origin and is performed with no need for detailed information about K .
? (x) drives the state predictions x?j t to the origin, there exists
j Predictions of the state and input are simply propagated until either a
a finite integer N (x) such that x?t+N (x) t 2 K . In view of
1 j constraint violation is detected or the predicted state enters Br . Note
Lemma 1, we therefore have for all N  N (x),  ? (x) 2 IPN ,
1 that this only requires prediction on a horizon that is guaranteed to
i.e.,  ? (x) 2 IPN \ IP. As N (x) minimizes (x;  ) in IPN , it be finite. The conclusion that xN t+N t 2 K is made if and only
j
follows that ? (x) = N (x), and consequently  ? (x) = N (x) for if the state prediction enters Br with no prior constraint violation.
all N  N (x).1 The two operations, prediction and constraint violation check, can be
This result is key to our approach to solving Problem 1, as implemented efficiently. A simple method to determine the radius r
explained below. of a ball Br  K is available but omitted for brevity.
The horizon increase made in Step 3) is worthy of a brief
D. The Control Algorithm discussion, as the heuristic used in increasing N is closely linked
In view of the discussion in Section III-C, an algorithm that is to the number of cycles of Algorithm 1 to termination. The simplest
guaranteed to identify the optimal control profile  ? (x) in finite approach is to increment N at each iteration of the algorithm, as
computational time is the following. 1
suggested in [16]. This leads to identification of N (x) but is likely
to result in rather slow implementation of the control scheme. As
? (x) = N (x) for all N  N (x), however, only an upper bound
Algorithm 1—Constrained LQR:
Step 0) Choose a finite horizon N0 , set N = N0 .
1
Step 1) Solve Problem 2.
1
on N (x) is required and values of N may therefore be skipped in

t+N t 2 K , go to Step 4).

an effort to obtain a quicker solution.
Step 2) If xN j An interesting approach is to increase N geometrically, starting
Step 3) Increase N , go to Step 1).
from a nonzero initial horizon. Assume that the current state is x, that
Step 4) Terminate:  ? (x) = N (x):
we start with N = N0  1, and that we multiply the horizon by an
Sznaier and [16] Damborg discuss a similar algorithm, although integer k  2 at each cycle of Algorithm 1. Then termination occurs
they do not show that termination occurs in a finite number of with N = k(x) N0 after (x) + 1 cycles, where  is an integer-
1
iterations. Finiteness of N (x) in Theorem 3, however, ensures that valued function of x. Let c(N ) be the computational price of solving
termination occurs after a finite number of cycles, regardless of the Problem 2 with horizon N , and further assume that c(N )  O(N ),
choice of initial horizon in Step 0) and of the heuristic used to increase so that there exists a finite real  > 0 such that c(N )  N , which is
it in Step 3). a reasonable assumption with infeasible interior point methods [18].
The computational cost of Algorithm 1 is then
IV. COMPUTATIONAL ISSUES
C (x) = c(N ) + c(kN ) + 1 1 1 + c[k(x) N ]
0 0 0
The spirit of optimal control, which lies at the root of MPC and
LQR, leaves no question that Problem 1 is a more natural formulation  N [1 + k + 1 1 1 + k x ]
0
( )

of the constrained predictive control law than other MPC formulations k x 0 1 ( )+1
= N :
k010 (17)
in current use. The solution of Problem 1 can be computationally
If N  N1 (x), (x) = 0, and C (x)  N . If N < N1 (x),
taxing, however. It is therefore necessary to assess the computational
0 0 0
we get termination with N  kN1 (x) 0 k so that k(x) N0 
demands of the method and it is desirable to minimize them. In this
section, we discuss the computational aspects of Algorithm 1. We
kN1 (x) 0 k. It then follows that
also propose a modification of Algorithm 1 that gives guaranteed
k2
C (x)  N (x) 0
k01 1
stability, even when there is not sufficient time for the algorithm to 1
(k2 + N0 ):
1 k01
(18)
terminate. Finally, we discuss upper bounds on N (x) and how they
may be used to obtain one-shot solutions to Problem 1. For any choice of N0 , therefore, we have
k2
C (x)   max N0 ; N (x) 0
k01 1
1
(k + N0 ) :
2
k01
A. Computational Aspects of Algorithm 1 (19)
The choice of the initial horizon, N0 , in Step 0), has an effect on
the efficiency of the algorithm, since an initial horizon that is close to Consequently, we find that C (x)  O[N (x)]. Also, it appears
1
1
or greater than N (x) leads to early termination. However, N (x) 1 wise to set k to its minimal value 2, leading to
varies with x, and the effect of the initial horizon is generally not C (x)   max[N0 ; 4N1 (x) 0 4 0 N0 ]: (20)
critical. An initial choice N0 = 0 can therefore be recommended as
the default, with the desirable side effect that no optimization needs 1
Now, assume N (x) is known; then calculation of  ? (x) requires
to be performed close to steady state when the state belongs to K . only the one-shot solution of Problem 2 with N = N (x), resulting 1
The solution of Problem 2, in Step 1), is the computationally in a minimal computational price c[N (x)]  N (x). When
1 1
expensive part of Algorithm 1. Efficient quadratic program solutions 1
N (x) is large (i.e., when computational time is an issue), a good
should of course be implemented. Infeasible interior point methods approximation is c[N (x)]  N (x). In view of (18), we then find
1 1
appear to be well suited. On one hand, the computational price of that the price of Algorithm 1 is better than approximately k2 =(k 0 1)
that solution is O(N ) [18], c.f. active set methods for which it is times the minimal computational cost required to calculate  ? (x).
O(N 2 ). On the other hand, the starting point for the optimization can Another alternative is to use the time-index of the last constraint
be infeasible and the quadratic program on one cycle of Algorithm 1 violation detected at the previous cycle of the algorithm as the new

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on July 13,2022 at 01:46:06 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO. 8, AUGUST 1998 1167

Fig. 1. Constrained LQR for Van de Vusse reactor.

horizon N . This approach seems to make good control sense, and with x1 and x2 representing the concentration of A and B and u
simulations indicate that it is a reasonable heuristic. the dilution (feed) rate [17]. We assume that k1 = 50, k2 = 100,
k3 = 10. The control objective is to maintain x2 at a set-point of
B. A Modification of Algorithm 1 one. Consequently, the desired steady states for x1 and u are 2.5
and 25, respectively [14], [7]. We linearize (21) around this desired
The stability guarantees of Theorems 1 and 2 apply only to the
steady state and discretize the result with a sampling time of 0.002.
control law given by the exact solution of Problem 1. The stability
This gives
results are therefore invalid if there is not sufficient time to cycle
Algorithm 1 to termination. For cases when time restrictions do not
A= 0:951 23 0
; B= 00:004 877 1 :
allow termination, Sznaier and Damborg suggest using the last control
sequence calculated by the algorithm. They, however, point out that 0:088 33 0:818 73 00:002 042 9
this approach may not lead to stabilizing control; furthermore, it is
likely to cause constraint violations. We implement the constrained LQR law on this linear system, with
A modification of Algorithm 1 can, however, be proposed, that an appropriate shift of the origin to account for the nonzero set-
leads to guaranteed stability, both nominally and under decaying point. The tuning we choose is Q = I , R = I and the constraint
perturbations, regardless of whether or not the algorithm terminates. is specified that x2 should be no greater than 0.12. A simulation
Let MPCs denote a stabilizing (suboptimal) MPC scheme for is performed that is started from x0 = [0:5 0:1]0 (in the shifted
the system under consideration. We may then modify Step 3), in coordinates) and the results are presented in Fig. 1. In this example,
Algorithm 1, to the following. we obtain N1 (x0 ) = 7 and ? (x0 ) = 7 (x0 ) = 143:8.
Modified Step 3)—Algorithm 1: For comparison, we also present simulation results with two other
stabilizing MPC laws. First, we consider an MPC scheme with
Step 3(a): Increase N
Step 3(b): If N  Nmax , go to Step 1).
end–point constraint [5], in which the input parameterization is such
that u = 0 and x = 0 after a finite horizon N . For this control law,
Step 3(c): Terminate and implement MPCs :
if a small horizon N is used, the end–point constraint on the state
Implementation of Algorithm 1, with the above modification of can become incompatible with the constraint that x2  0:12. The
Step 3), is guaranteed to lead to stabilizing control, even if the minimum horizon required for feasibility is, of course, a function of
algorithm does not terminate in Step 4); of course, when there is the state. For instance, with x0 = [1 0:1]0 , N must be no smaller
sufficient computation time, the algorithm does terminate in Step 4), than eight and with x0 = [2 0:1]0 , no smaller than 11.
and this leads to optimal constrained LQR performance. Returning to the initial condition x0 = [0:5 0:1]0 , the minimum
The stabilizing scheme, MPCs , may, for instance, be the subop- horizon required for feasibility is N = 5. The open and closed-
timal, infinite horizon, stabilizing controller of Rawlings and Muske loop costs of the control strategy obtained with different horizons,
[11], or a finite-horizon MPC scheme, similar to that discussed in from this initial condition are plotted in Fig. 2. (The open-loop cost
[5]. These control laws provide feasible points for Problem 1 that are is the cost associated with the control profile postulated at the first
suboptimal for any horizon N ; therefore, a by-product of Modified sample, i.e., 1 j =0 xj j0 Qxj j0 + uj j0 Ruj j0 ; the closed-loop cost is
0 0

Step 3) is also to give an upper bound for the optimal cost, which the cost associated with the actual controls that are implemented in
may be used to derive an upper bound for N . 1 the receding-horizon implementation, i.e., 1 t=0 xt Qxt + ut Rut .)
0 0

Considering these results, we find for the end–point constrained

V. EXAMPLES MPC that we have a difficult problem with the design of N , for
which the minimum value depends on the system state, and we also
A. Van de Vusse Reactor obtain a cost that can be an order of magnitude worse, both in open
We consider the isothermal continuous stirred-tank reactor (CSTR) and closed loop, than with constrained LQR.
using dilution rate as the manipulated variable. The reactor has We now consider an MPC scheme with infinite costing [11], in
constant volume and its dynamics are described by which the input parameterization is simply u = 0 after a finite horizon
N , and there is no end–point state constraint. For this control law,
x_ 1 = 0 k1 x1 0 k3 x21 0 x1 u with a horizon N = 1, the optimal cost (163.92) is approximately
x_ 2 = k1 x1 0 k2 x2 0 x2 u (21) 14% worse than with constrained LQR. With larger N , performance
becomes similar to constrained LQR. However, it must be noted
which models the Van de Vusse series of reactions
that implementation of a stabilizing control law of this type requires
A 0!
k
B 0!
k
C; 2A 0!
k
D enforcement of the state constraint over an infinite horizon, which

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on July 13,2022 at 01:46:06 UTC from IEEE Xplore. Restrictions apply.
1168 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO. 8, AUGUST 1998

Fig. 2. Cost comparison of constrained LQR and MPC with end–point constraint for Van de Vusse reactor.

Fig. 3. Constrained LQR for double integrator, x0 = [0:2 0:2]0 .

Fig. 4. MPC for double integrator, x0 = [0:2 0:2]0 .

leads to solving a finite sequence of quadratic programs just as in the x0 = [0:2 0:2]
0
, we get N ?
(x0 ) = 0 and  (x0 ) = 2:23; with
1

constrained LQR case [6]. x0 = [20 20] ,

0 ?
N1 (x0 ) = 33; and  (x0 ) = 60 056. We compare
We conclude for this example that constrained LQR not only leads with the two stabilizing MPC laws used in Example 1. Because both
to improved performance over alternative MPC formulations, it also modes of the system are unstable, the two MPC formulations are
releases the user from the design of N . Finally, we point out that the equivalent. From initial condition x0 = [0:2 0:2]0 , N = 4 provides
observations made in this example are not specific to the process we a feasible problem. The open and closed-loop MPC costs (70, 118)
use; they would apply to many stable systems. are 30 to 50 times worse than with constrained LQR. Simulation
results are displayed in Fig. 4. From initial condition x0 = [20 20]0 ,
B. Double Integrator the minimum horizon, after which the state can be forced to zero, is
We consider a double integrator system sampled at a frequency of N = 60. We implement the control law with this horizon and get
10 Hertz, for which performance that compares favorably with constrained LQR; open
and closed-loop MPC costs are about 2% larger than the constrained
1 0 0:1
A= 0:1 1
; B= 0:005
: LQR cost. However, the dimension of the optimization problem
required to obtain this control law is almost twice as large as that
The constraint is specified that the input should not exceed amplitude needed in constrained LQR.
ten. The constrained LQR law is implemented with Q = I , R = I . To summarize this example we have the following.
We consider two different initial conditions, x0 = [0:2 0:2]0 and 1) For operation close to the steady state, constrained LQR uses
x0 = [20 20]0 ; a representative simulation is shown in Fig. 3. With control that is not even close to the constraints. With MPC, a

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on July 13,2022 at 01:46:06 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO. 8, AUGUST 1998 1169

N
small can be used for online efficiency, but that choice leads ACKNOWLEDGMENT
to large inputs which are unnecessary and undesirable when
compared to those obtained with constrained LQR. Moreover, The authors would like to thank Dr. S. J. Wright for discussion
the computational cost of implementing MPC is higher than of the computational cost of the algorithm with infeasible interior
constrained LQR. point methods.
2) For operation away from steady state, a larger number of
decision variables (almost twice as many) is needed for MPC
REFERENCES
compared to constrained LQR. Although MPC can be imple-
mented in this case by solving a single quadratic program [1] R. R. Bitmead, M. Gevers, and V. Wertz, Adaptive Optimal Control,
(QP), the dimension of that QP is much larger than any of The Thinking Man’s GPC. Englewood Cliffs, NJ: Prentice-Hall, 1990.
the QP’s solved in the constrained LQR approach. Again, the [2] D. Chmielewski and V. Manousiouthakis, “On constrained infinite-time
computational cost for MPC is higher and the performance is linear quadratic optimal control,” Syst. Contr. Lett., vol. 29, pp. 121–129,
1996.
worse. [3] C. R. Cutler and B. L. Ramaker, “Dynamic matrix control—A computer
control algorithm,” in Proc. Joint Automatic Control Conf., 1980.
VI. CONCLUSIONS [4] R. E. Kalman, “Contributions to the theory of optimal control,” Bull.
Soc. Math. Mex., vol. 5, pp. 102–119, 1960.
In this paper, we further developed the constrained LQR problem [5] S. S. Keerthi and E. G. GIlbert, “Optimal infinite-horizon feedback laws
proposed by Sznaier and Damborg. In addition to being optimal, the for a general class of constrained discrete-time systems: Stability and
constrained LQR removes the mismatch between open and closed- moving-horizon approximations,” J. Optim. Theory Appl., vol. 57, no.
2, pp. 265–293, May 1988.
loop nominal behavior, which is an undesirable side effect of current [6] E. S. Meadows, K. R. Muske, and J. B. Rawlings, “Implementable model
MPC approaches that are based on finite-input parameterizations. predictive control in the state space,” in Proc American Control Conf.,
The main practical advantage of the constrained LQR compared 1995, pp. 3699–3703.
to currently available stabilizing MPC approaches, however, is not [7] E. S. Meadows and J. B. Rawlings, “Model predictive control,” in
optimality but simplicity. The constrained LQR removes the tuning Nonlinear Process Control, M. A. Henson and D. E. Seborg, Eds.
N
parameter , for which no reliable tuning guidelines are available.
Englewood Cliffs, NJ: Prentice Hall, 1997, pp. 233–310.
[8] H. Michalska and D. Q. Mayne, “Robust receding horizon control of
Our recommendation is that current industrial practice in which constrained nonlinear systems,” IEEE Trans. Automat. Contr., vol. 38,
control engineers purposefully use a suboptimal and small value of pp. 1623–1633, Nov. 1993.
N in order to decide the performance robustness tradeoff should be [9] K. R. Muske, “Linear model predictive control of chemical processes,”
Ph.D. dissertation, Univ. Texas, Austin, 1995.
reconsidered. The performance robustness tradeoff can be decided, [10] J. B. Rawlings, E. S. Meadows, and K. R. Muske, “Nonlinear model
as in the classic unconstrained LQR case, by choosing appropriate predictive control: A tutorial and survey,” in ADCHEM’94 Proc., Kyoto,
Q and . R Japan.
We presented the outline of an algorithm that allows practical [11] J. B. Rawlings and K. R. Muske, “Stability of constrained receding
implementation of the constrained LQR; the computational cost of horizon control,” IEEE Trans. Automat. Contr., vol. 38, pp. 1512–1516,
Oct. 1993.
this algorithm is reasonable compared to the minimal cost required [12] J. Richalet, A. Rault, J. L. Testud, and J. Papon, “Model predictive
for calculation of the optimal solution. However, examples exist heuristic control: Applications to industrial processes,” Automatica, vol.
for which this minimal cost is large, which may preclude real-time 14, pp. 413–428, 1978.
implementation of the constrained LQR in some situations. It is not [13] P. O. Scokaert, J. B. Rawlings, and E. S. Meadows, “Discrete-time
stability with perturbations: Application to model predictive control,”
clear if these cases arise often enough in practice to warrant serious Automatica, vol. 33, no. 3, pp. 463–470, 1997.
attention, but the algorithm is easily modified so that termination [14] P. B. Sistu and B. W. Bequette, “Model predictive control of processes
occurs for such cases automatically with a stabilizing MPC solution, with input multiplicities,” Chem. Eng. Sci., vol. 6, no. 6, pp. 921–936,
and without user intervention or online tuning. 1995.
It has been assumed throughout this paper that the state and input [15] M. Snazier, “Suboptimal feedback control of constrained linear sys-
constraints are compatible, i.e., IP 6= ;. Even though the assumption
tems,” Ph.D. dissertation, Univ. Washington, 1989.

h d
that 2 IR+ and 2 IR+ ensures that steady state at the origin
n n [16] M. Sznaier and M. J. Damborg, “Suboptimal control of linear systems
with state and control inequality constraints,” in Proc. 26th Conf.
is feasible, the problem of transient infeasibility remains. Relevant Decision and Control, 1987, pp. 761–762.
contributions have been made in the context of MPC and constraint [17] J. G. Van de Vusse, “Plug-flow type reactor versus tank reactor,” Chem.
Eng. Sci, vol. 19, p. 964, 1964.
relaxation techniques have been proposed that do not endanger [18] S. J. Wright, “Applying new optimization algorithms to model predictive
closed-loop stability. These are readily transportable to constrained control,” in Fifth Int. Conf. Chemical Process Control, Lake Tahoe, CA,
LQR. Jan. 1996.
After this paper was submitted, the authors obtained a recent
report of related work by Chmielewski and Manousiouthakis [2].
The main differences are as follows. As in [16], Chmielewski and
Manousiouthakis assume the state and input constraint sets are
compact, convex polyhedrons. In this paper we do not assume
these constraint sets are bounded. In many applications there are
no constraints specified on some states or inputs and the con-
straint regions are therefore unbounded. Because the constrained
LQR problems differ, the algorithms for computing solutions also
differ. The compact state constraint set enables Chmielewski and
Manousiouthakis to compute an upper bound on N1
and solve a
x
single QP. In this paper, because 0 is not in a compact set, is N
increased in a series of QP’s until termination. For cases with compact
constraint regions, further research is required to make a quantitative
comparison between the efficiencies of these two approaches.

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on July 13,2022 at 01:46:06 UTC from IEEE Xplore. Restrictions apply.