IC-32 Optimal Control Spring 2009

Benoı̂t Chachuat
ME C2 426, Ph: 35148, [email protected]

Problem Set #9 (With Solutions)

The objective of this problem is to solve, via the direct sequential approach, the following scalar
optimal control problem:
Z 2
1 2
minimize: 2 [x1 (t)] dt (1)
0
subject to: ẋ1 (t) = x2 (t) + u(t); x1 (0) = 1 (2)
ẋ2 (t) = −u(t); x2 (0) = 1 (3)
x1 (2) = x2 (2) = 0 (4)
− 10 ≤ u(t) ≤ 10, 0 ≤ t ≤ 2. (5)
For simplicity, a piecewise control parameterization over ns stages is considered for approximating,
u(t) = ω k , tk−1 ≤ t ≤ tk , k = 1, . . . , ns ,
2k
with the time stages being equally spaced, tk = ns .

Questions:
1. First, reformulate the optimal control problem into the Mayer form (denote the additional state variable
by x3 ).

Solutions. Reformulating the problem into the Mayer form proceeds by introducing an extra state
variable, x3 , defined by the differential equation,
x˙3 (t) = 12 [x1 (t)]2 ,
with initial condition z(0) = 0. An equivalent optimal control formulation is then obtained as:
minimize: z(2)
subject to: ẋ1 (t) = x2 (t) + u(t); x1 (0) = 1
ẋ2 (t) = −u(t); x2 (0) = 1
1 2
ẋ3 (t) = 2 [x1 (t)] ; x3 (0) = 0
x1 (2) = x2 (2) = 0

2. In MatLab® , write a m-file calculating the values of the cost (1) and terminal constraints (4), for a
given number ns of stages and given values of the control parameters ω 1 , . . . , ω ns .
◦ Use the function ode15s to integrate the differential equations; set both the absolute and relative
integration tolerances to 10−8 .

Application. Calculate the cost and constraint values for the following 2-stage control:

0 10, 0 ≤ t < 1,
u (t) = (6)
−10, 1 ≤ t ≤ 2.

Solutions. A possible implementation is as follows:

1
 fun.m

1 function [ f ] = fun( x0, ns, ts, ws )

2

3 % Options for ODE solver

4 optODE = odeset( ’RelTol’, 1e-8, ’AbsTol’, 1e-8 );
5

6 % Forward state integration

7 z0 = [ x0 ];
8 for ks = 1:ns
9 [tspan,zs] = ode15s( @(t,x)state(t,x,ws,ks), [ts(ks),ts(ks+1)], z0, optODE );
10 z0 = zs(end,:)’;
11 end
12 % Functions
13 f = zs(end,:)’;
14

15 end

state.m

1 function [ dx ] = state( t, x, w, ks )
2

3 % State system
4 dx=[ x(2) + w(ks);
5 - w(ks);
6 x(1)^2 / 2. ];
7

8 end

3. Solve the sequentially-discretized NLP problem using the fmincon function in MatLab’s Optimization
Toolbox:
◦ You are to code a main program, as well as two separate m-files called by fmincon that calculate
the values of the cost and of the constraints, respectively; these latter m-files should invoke the
m-file developed in Question 2.
◦ For simplicity, let fmincon calculate a finite-difference approximation of the cost and constraint
derivatives; make sure that the minimum change in variable for finite differencing is consistent
with the tolerances set previously for the ODE solver (10−8 ): here, a value of 10−5 appears to be
a reasonable choice.
◦ Make sure to select the medium-scale SQP algorithm with quasi-Newton update and line-search,
and set the solution point tolerance, the function tolerance and the constraint tolerance all to
10−9 ; such tight tolerances are needed because the optimal control problem is singular.
◦ Set all the control coefficients equal to zero as the initial guess.
Application. Solve the optimal control problem for ns = 2, 4, 8, 16 and 32 stages, then plot the results.

Solution. A simple implementation satisfying the aforementioned specifications is given subse-

quently.

2
 main.m

1 clear all;
2 clf;
3 format long;
4

5 % Options for NLP Solvers

6 optNLP = optimset( ’LargeScale’, ’off’, ’GradObj’, ’off’, ’GradConstr’, ’off’,...
7 ’DerivativeCheck’, ’off’, ’Display’, ’iter’, ’TolX’, 1e-9,...
8 ’TolFun’, 1e-9, ’TolCon’, 1e-9, ’MaxFunEval’, 300,...
9 ’DiffMinChange’, 1e-5 );
10

11 % Time Horizon and Initial State

12 t0 = 0; % Initial time
13 tf = 2; % Final time
14 x0 = [ 1; 1; 0 ]; % Initial state
15

16 ns = 1;
17 for is = 1:5
18 ns = 2*ns; % Number of stages: ns = 2, 4, 8, 16, and 32
19 ts = [t0:(tf-t0)/ns:tf]; % Time stages (equipartition)
20

21 % Initial Guess and Bounds for the Parameters

22 w0 = zeros(ns,1);
23 wL = -10*ones(ns,1);
24 wU = 10*ones(ns,1);
25

26 % Sequential Approach of Dynamic Optimization

27 [ wopt ] = fmincon( @(ws)obj(x0,ns,ts,ws), w0, [], [], [], [], wL, wU,...
28 @(ws)ctr(x0,ns,ts,ws), optNLP);
29

30 end

obj.m

1 function [ J ] = obj( x0, ns, ts, ws )

2

3 f = fun( x0, ns, ts, ws );

4 J = f(3);
5

6 end

ctr.m

1 function [ c, ceq ] = ctr( x0, ns, ts, ws )

2

3 f = fun( x0, ns, ts, ws );

4 ceq = f(1:2)’;
5 c = [];
6

7 end

4. To increase the reliability and execution speed of the direct sequential procedure, you are to compute
the derivatives of the cost (1) and terminal constraints (4) with respect to the control parameters
ω 1 , . . . , ω ns via the forward sensitivity method:

3
 (a) Write down the state sensitivity equations, as well as the cost and constraint derivatives per the
forward sensitivity method.
(b) Duplicate the m-file developed in Question 2 above and modify it so that it calculates the cost
and constraint derivatives in addition to their values; run this m-file for the 2-stage control u0 (t)
given in (6).
(c) Modify the two m-files passed to fmincon for cost/constraint function evaluation:
◦ In the main program, tell fmincon to use the user-supplied cost/constraint derivatives (instead
of finite-difference derivatives).
◦ Use the DerivativeCheck feature of fmincon to detect inconsistencies in the computed deriva-
tives!

Application. Recalculate the solution to the optimal control problem for ns = 2, 4, 8, 16 and 32
stages, and make sure that you get the same results as previously.

Solution. A possible implementation that calculates the cost and constraints derivatives via the
forward sensitivity approach is given subsequently. The m-file ‘disopt.m’ plots the optimization
results and then saves these plots.
main.m

1 clear all;
2 clf;
3 format long;
4

5 % Options for ODE & NLP Solvers

6 optODE = odeset( ’RelTol’, 1e-8, ’AbsTol’, 1e-8 );
7 optNLP = optimset( ’LargeScale’, ’off’, ’GradObj’, ’on’, ’GradConstr’, ’on’,...
8 ’DerivativeCheck’, ’off’, ’Display’, ’iter’, ’TolX’, 1e-9,...
9 ’TolFun’, 1e-9, ’TolCon’, 1e-9, ’MaxFunEval’, 300 );
10 ’, 300 );
11

12 % Time Horizon and Initial State

13 t0 = 0; % Initial time
14 tf = 2; % Final time
15 x0 = [ 1; 1; 0 ]; % Initial state
16

17 ns = 1;
18 for is = 1:5
19 ns = 2*ns; % Number of stages: ns = 2, 4, 8, 16, and 32
20 ts = [t0:(tf-t0)/ns:tf]; % Time stages (equipartition)
21

22 % Initial Guess and Bounds for the Parameters

23 w0 = zeros(ns,1);
24 wL = -10*ones(ns,1);
25 wU = 10*ones(ns,1);
26

27 % Sequential Approach of Dynamic Optimization

28 [ wopt ] = fmincon( @(ws)obj(x0,ns,ts,ws,optODE), w0, [], [], [], [],...
29 wL, wU, @(ws)ctr(x0,ns,ts,ws,optODE), optNLP);
30 dispopt( x0, ns, ts, wopt, optODE );
31

32 end

obj.m

1 function [ J, dJ ] = obj( x0, ns, ts, ws, optODE )

2

4
 3 if nargout == 1
4 f = fun( x0, ns, ts, ws, optODE );
5 J = f(3);
6

7 else
8 [f,df] = fun( x0, ns, ts, ws, optODE );
9 J = f(3);
10 dJ = df(3,:)’;
11 end
12

13 end

ctr.m

1 function [ c, ceq, dc, dceq ] = ctr( x0, ns, ts, ws, optODE )
2

3 if nargout == 2
4 f = fun( x0, ns, ts, ws, optODE );
5 ceq = f(1:2)’;
6 c = [];
7

8 else
9 [f,df] = fun( x0, ns, ts, ws, optODE );
10 ceq = f(1:2)’;
11 dceq = df(1:2,:)’;
12 c = [];
13 dc = [];
14 end
15

16 end

fun.m

1 function [ f, df ] = fun( x0, ns, ts, ws, optODE )

2

3 % Calculate function values only

4 if nargout == 1
5 % Forward state integration
6 z0 = [ x0 ];
7 for ks = 1:ns
8 [tspan,zs] = ode15s( @(t,x)state(t,x,ws,ks), [ts(ks),ts(ks+1)], z0, optODE );
9 z0 = zs(end,:)’;
10 end
11 % Functions
12 f = zs(end,:)’;
13

14 % Calculate both function and gradient values

15 else
16 % Forward state & sensitivity integration
17 z0 = [ x0; zeros(3*ns,1) ];
18 for ks = 1:ns
19 [tspan,zs] = ode15s( @(t,x)sens(t,x,ws,ks), [ts(ks),ts(ks+1)], z0, optODE );
20 z0 = zs(end,:)’;
21 end
22 % Functions & Gradients
23 f = zs(end,1:3)’;
24 df = [];

5
25 for is = 1:ns
26 df(1:3,is) = zs(end,is*3+1:is*3+3);
27 end
28 end
29

30 end

state.m

1 function [ dx ] = state( t, x, w, ks )
2

3 % State system
4 dx=[ x(2) + w(ks);
5 - w(ks);
6 x(1)^2 / 2. ];
7

8 end

sens.m

1 function [ dx ] = sens( t, x, w, ks )
2

3 % State system
4 dx=[ x(2) + w(ks);
5 - w(ks);
6 x(1)^2 / 2. ];
7

8 % Append sensitivity system

9 for is = 1:length(w)
10 if is == ks
11 uws = 1.;
12 else
13 uws = 0.;
14 end
15 dx = [ dx;
16 x(3*is+2) + uws;
17 - uws;
18 x(1) * x(3*is+1) ];
19 end
20

21 end

dispopt.m

1 function dispopt( x0, ns, ts, ws, optODE )

2

3 % Forward state integration; store optimal state & control

4 z0 = [ x0 ];
5 topt = [];
6 xopt = [];
7 uopt = [];
8 for ks = 1:ns
9 [tspan,zs] = ode15s( @(t,x)state(t,x,ws,ks), [ts(ks),ts(ks+1)], z0, optODE );
10 z0 = zs(end,:)’;
11 topt = [ topt; tspan ];

6
12 xopt = [ xopt; zs ];
13 uopt = [ uopt; ws(ks)*ones(length(tspan),1) ];
14 end
15

16 % Display Optimal Cost

17 disp(sprintf(’Optimal cost for ns=%d: %d’, ns, zs(end,3)));
18

19 % Plot results
20 figure(1)
21 plot(topt,xopt(:,1),’r’,topt,xopt(:,2),’b’)
22 xlabel(’t’)
23 ylabel(’x1(t), x2(t)’)
24 title([’ns=’,int2str(ns)])
25 saveas(gcf,[’x1(t)_x2(t)_’,int2str(ns),’stg.eps’], ’psc2’)
26

27 figure(2)
28 plot(xopt(:,1),xopt(:,2),’r’)
29 xlabel(’x1(t)’)
30 ylabel(’x2(t)’)
31 title([’ns=’,int2str(ns)])
32 saveas(gcf,[’x2(x1)_’,int2str(ns),’stg.eps’], ’psc2’)
33

34 figure(3)
35 plot(topt,uopt,’r’)
36 xlabel(’t’)
37 ylabel(’u(t)’)
38 title([’ns=’,int2str(ns)])
39 saveas(gcf,[’u(t)_’,int2str(ns),’stg.eps’], ’psc2’)
40

41 end

With the foregoing implementation, one gets the following results:

ns J⋆
2 5.858335
4 4.898494
8 4.608658
16 4.592631
32 4.586278

It should be noted that adding more control stages gives very little improvement in terms of the
optimal cost, for ns greater than 8.
The results obtained with ns = 32 are shown in Fig. 1 below. By inspection an optimal control for
the problem (1–5) appears to be composed of 3 arcs:
(a) u⋆ (t) = 10, 0 ≤ t ≤ t⋆1 ;
(b) u⋆ (t) = −x⋆1 (t) − x⋆2 (t), t⋆1 ≤ t ≤ t⋆2 ;
(c) u⋆ (t) = −10, t⋆2 ≤ t ≤ 2.
These numerical results should be compared with the analytical solution given in the Example 4.34
of the class textbook.

7
 ns = 32 stages ns = 32 stages
10 4

8
3

2
4
x⋆1 (t), x⋆2 (t)

2
1
u⋆ (t)

0
−2

−4 −1

−6
−2
−8

−10 −3
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
t t

Figure 1: Optimal results for CVP with ns = 32. Left plot: optimal control u⋆ vs. t; right plot: optimal
responses x⋆1 (red curve), x⋆2 (blue curve) vs. t.