Chapter 1

Introduction to Optimal Control

 Introduction
• Optimal control – is finding a controller that
drives a system towards a desired operating
condition while achieving a given performance
criteria
• The performance criteria is given as a cost
function J
• Most commonly used J

– (x(to),to), (x(tf),tf), is Condition at the beginning

and end of the control and
– (x(t),t) the cost in the entire interval
 Introduction
• The type of function used for ,  in the
performance function determines the type of
optimal control
• Problem is constrained optimization problem
– Constraints may be given time, state condition,
amount of control effort or energy etc
• Solution techniques depend on the type of
function used for the initial and final conditions
 Problem formulation

• Given a system

• With initial condition

• Find a control signal u that can

– drive the system to a final state
– Fulfill state constraint
– Minimize the function
• Min

• Subject to

• U<Umax
•
Classification of optimal control problems

• Optimal control problems can be classified into

various types based on the performance
function used
• These are
– Minimum time control problem
– Terminal control problem
– Minimum control effort problem
– Optimal servo mechanism
– Optimal regulator problem
 Minimum time control problem
• Objective is to minimize the time required to
drive a system from its initial state to final
state
• J or performance function is
tf


J  dt
to

• Example: optimal control of robotic

manipulator to finish a task
Example for Minimum time control problem

• A rocket burn trajectory is desired to minimize a

travel time between a starting point and a final point,
10 units of distance away.
– The thrust can be between an upper limit of 1.1 and a
lower limit of -1.1.
– The initial and final velocity must be zero and the
maximum velocity can never exceed 1.7.
– It is also desirable to minimize the use of fuel to perform
the maneuver. There is a drag resistance that is
proportional to the square of the velocity and mass is lost
as the fuel is burned during thrust operations.
 Example- minimum time problem
• Since the problem is minimum time problem, the
objective functions is minimize J given by
tf


J  dt
to
• This is similar to
𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒𝑡 𝑓

• Once we determined the objective function, next

we have to determine the constraints. The first
constraint usually is the system dynamic equation
 Example-minimum time problem
• The dynamic equation of the rocket is then
=v

𝑑𝑣 2
=(𝑢− 0.2 𝑣 )/𝑚
𝑑𝑡
𝑑𝑚
=−0.01 𝑢2
𝑑𝑡

• When the system is implemented, it will have

additional constraints. These are the maximum
and minimum control signal limits, the maximum
and minimum velocity which the system can have
• The last constraints are the patch constraints
Example- the minimum time control problem

• The final problem formulation will then be

𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒𝑡 𝑓
• Subject to 𝑠=𝑣
˙
˙ ( 𝑢 − 0.2 𝑣 2 ) / 𝑚
𝑣=
𝑚=−0.01
˙ 𝑢2
u ≤ 1.1
𝑢≥− 1.1
𝑣 ≤1.7
𝑣 ( 0 ) =0 𝑣 ( 𝑡 𝑓 )=0
𝑠 ( 0 ) =0 𝑠 ( 𝑡 𝑓 )=10 𝑈
 Terminal control problem
• Objective: is to minimize the final error value,
i.e. given a desired value, find a control signal
that can drive the system towards a point
where final value has minimum deviation from
desired value
• J has the form of
J  [ x(t f )  d ]T S [ x(t f )  d ]
– d is the desired value of the system at final time
Example- Pick and place robotic manipulator

• Consider a third degree of freedom robotic

manipulator shown below
, and are links of the robot
, and are the joints which are driven by
motors.

The objective is to move the robotic

manipulator end effector from its position to
point A with minimum terminal error by
applying torque to the motors which drive the
joints.

Moreover, each angle should not be moved more than

its limits and each motor has a maximum torque limit .
Formulate the problem as a terminal control problem
 Example- pick and place robot
• First we have to identify the objective function. As a
terminal control problem, the objective is to
minimize the terminal error. The error is the
difference between desired terminal
position/orientation and actual position/orientation
of the end effector.
• Assume the end effector desired position and
orientation at A is given by (3 pos. variable and 3
orientation var.) =Desired position and orientation D
 Example- pick and place robot
• If the actual final position and orientation of
the robot is given by

[]
𝑥1
𝑥2
𝑥3
𝑥 (𝑡 𝑓 )=
𝑥4
𝑥5
𝑥6
• Then the error will also be six dimensional
variable given by -D
• Hence the objective function will then be
S = s11(x1-xa)^2+s22*(x2-yA)^2+…
 Minimum control effort
• Objective: is to minimize the energy used to
drive a system from its initial state to a final
state
• J has the form
tf


J  U (t )T RU (t )dt
to

– R is a weighting matrix with dimension mxm

• Example: minimum fuel problem
 Example: Reformulate the minimum time control
problem of the rocket as a minimum effort problem
• In rocket control, one important objective is
minimizing the control effort of fuel
consumed.
• The minimum time problem when formulated
as energy minimization, it will be given as
• J=int(1/2U^2dt)
Minimum effort rocket launch problem
dt

subject to

𝑠=𝑣
˙
˙ ( 𝑢 − 0.2 𝑣 2 ) / 𝑚
𝑣=
𝑚=−0.01
˙ 𝑢2
u ≤ 1.1
𝑢≥− 1.1
𝑣 ≤1.7
𝑣 ( 0 ) =0 𝑣 ( 𝑡 𝑓 )=0
𝑠 ( 0 ) =0 𝑠 ( 𝑡 𝑓 )=0
 Optimal servomechanism or tracking
problem
• Objective: is to design a control signal which
minimizes the error between desired actual
path of a system
• J has the form
tf
tf
 
T
J  [ x(t )  d (t )] Q[ x(t )  d (t )]dt  e(t )T Qe (t )dt
to
to

• Example: satellite tracking, continuous path

control of a robot etc
• Move the robot from A to B with the given
trajectory x1(t)=
• x2(t)=

• J=int(e1^2+e2^2+…)
 Optimal servomechanism contd
• The above problem can be extended to more
advanced forms
– Servomechanism with minimum control effort
tf


J  ([ x(t )  d (t )]T Q[ x(t )  d (t )]  U (t )T RU (t )) dt
to

– Servomechanism with minimum effort and

terminal control tf

J  [ X (t )  D (t )] S [ X (t )  D (t )]   ([ X (t )  d (t )] Q[ x (t )  d (t )]  U (t )
f f
T
f f
T T
RU (t )) dt
to
 Optimal regulator problem
• This is a sub problem of the optimal
servomechanism where the final point is zero
• Objective: is to drive a system towards a point
where the final state value is zero.
• The performance measure J is given by
tf


J  [ X (t f )]T S [ X (t f )]  ([ X (t )]T Q[ x(t )]  U (t )T RU (t )) dt
to
 Examples
• Consider a body of mass M moving along a
frictionless surface
u
M

xo , vo
x f ,v f

• Find a control signal u, bounded as -amax<u<amax

which can transfer the system from its initial
state to final state with minimum time.
 Examples
• Model the system
d d2
F  ma  m (v (t ))  m 2 ( x(t ))
dt dt
x 1  x2
x 2  ( F / m)  u

• The performance measure J is then

tf


J  dt  t f
0
 Examples
• If friction of the surface us considered in the
motion of the above body, the SS model
dx d d2
F b  ma  m (v(t ))  m 2 ( x(t ))
dt dt dt
x 1  x2
F b b
x 2   x2  u  x2
m m m
• The cost function can be taken to be the same
• Min J=tf time optimal problem formulation for the
frictionless problem
• Subject to
– X1d=x2
– X2d=F/M=U
– U<amax
– U>amin

– Min J=int(0.5u^2)dt min energy problem for the

frictionless surface
 Conclusion
• When considering optimal control problems,
the following four points need to be
considered
– Existence of solution
– Uniqueness
– Main features of the optimal solution
– The form of the solution
• Linear system solution usually exists – if the control
signal constraint is not very strict (U<Umax)

• U(t) – control signal – undesired because it is open

loop – change to closed loop
• X(t) – state trajectory
• U(t) is it robust ( when parameters are changed,
external disturbance occur, will the system perform
the same?) optimal robust controller