APPLICATION OF PONTRYAGIN

APPLICATION OF PONTRYAGIN¹S MAXIMUM PRINCIPLE IN DETERMINING THE OPTIMUM CONTROL OF A VARIABLE-MASS VEHICLE

Dahlard Lukes"¹"

Minneapolis-Honeywell Regulator Co., Minneapolis, Minn.

ABSTRACT

The purpose of this paper is to illustrate mathematical techniques by which Pontryagin1s maximum principle can be applied to determine optimum control of systems with boundary conditions. To demonstrate the procedure, the author con- siders the problem of how to regulate the thrust vector of a variable-mass space vehicle in order to transfer the vehicle from an initial point in space with a prescribed initial velocity and mass to a prescribed final position and velocity, minimizing the amount of propellant consumed. The vehicle is assumed to operate in a three-dimensional central gravitational field and the method of optimization allows the minimization to be performed over the class of bounded, piecewise-continuous thrust.

A solution to the problem is known to exist whenever there exists any solution that satisfies the end conditions.

As compared with methods of steepest descent which attempt to generate a sequence of controls that approaches an optimum, the maximum principle characterizes the optimum control by a system of ordinary differential equations and reserves the iteration for its solution. (Numerical methods are discussed.) The mathematical bases of the two methods are related to each other as well as related to the method of dynamic programming and to other steepest descent methods discussed in a forth- coming paper.

THE MAXIMUM PRINCIPLE

A system which may be described by ordinary differential equations written as a first-order vector differential

Presented at ARS Guidance, Control, and Navigation Confer- ence, Stanford, Calif., Aug. γ- 9 , 1 9 6 1 .

Numbers in parentheses, as found in text, indicate Refer- ences at end of paper.

"^Engineer Scientist, Military Products Group, Research Dept.

equation is considered ^ ^

χ = f(x,u,t) where χ = and u =1 u

Η

are vectors called the state vector and control vector, respectively.

By specifying initial conditions on x(t), a time interval te Τ ,T| and sufficient smoothness of f and u(t) to guarantee existence and uniqueness of solutions of the forementioned system, a correspondence is thereby set up from the choice of u(t) to the resulting value of χ^η( τ ) . Calling the class of control vectors considered admissable controls and symbolizing it by U, one can see that the foregoing correspondence is a functional defined on U. (The word functional is used rather than function to emphasize the fact that although the values taken on or range is real numbers, the domain of definition U is a set of vectors.)

A wide class of control problems can be reduced to the form of minimizing χ ( τ ) over u€ U. (Existence conditions exist (2) For example, assuming a functional of the form

ψ(χ(τ), u (t) ) dx

to be minimized, define a new variable xn + 1( t ) = f V( x( T ) , u (t) ) d T

which when adjoined to the original system reduces the problem to minimizing χ (τ) over u eU. In a similar fashion if the functional to be mini- mized is of the form φ(χ(τ),Τ) and has derivatives, one may

let χ η + 1

("*0

n+1 i=l

which when adjoined to the original system reduces the problem to minimizing χ + 1( τ ) over u e U . More generally, functionals generated out or the two previously given forms can be treated the same way. With these remarks, the author returns to the

canonical formulation of the problem of minimizing the last coordinate of x(t) at final time.

The maximum principle is a theorem which, under the appro- priate hypotheses on f and U, provides a technique for making a choice of u €U (to "be denoted by u* and called an optimum con- trol) at which the functional χ (τ) takes on its minimum. The technique is as follows: Another set of dependent variables are introduced

P(t) = / p²

called the adjoint variables which will be required to satisfy the differential equations

η öf

Κ = - ; ^{1 Ô}T ^Ps i=l,2,...,n[²]

S —_L 1

These equations can be written as one vector differential equation

ρ

=

[3]

where $7 f ) is the Jacobi matrix . One then defines

v χ '

Η = Σ p.f..

i=l ^{1 1}

The maximum principle states that if u* is optimum, then there exists a p*(t) satisfying the foregoing conditions with the additional property that

H(x*(t), p*(t), u*(t), t)>_ H(xU(t), pU(t), u(t), t)

for each u eUat each ΐε[τ ,τ] · (The superscript denotes the correspondence of functions u, ρ and χ satisfying the relations of Eqs. 1 and 2 . )

The proof of the theorem assumes that U consists of piece- wise continuous vectors whose range forms a closed set of points, and that f is continuous in the argument (x,u,t) and has continuous second partial derivatives in (x,u).

The maximum principle is used in the following way: H is maximized with respect to the explicit occurrence of u over U, thereby obtaining u = ^(x,p,t). Then the system

' x = f(x, $(x,p,t), t) x(T^o) = x° ^

/ Ρ = - ( ν^χΓ ) ρ p(T) = p^T u = $(x,p,t)

\

is considçred, where χ is the initial value of the state vec- tor and ρ is the final value of the vector p* mentioned in the statement of the maximum principle. Letting x(t) and p(t) denote the solution of the system in Eq. h and assuming it has a unique solution, the maximum principle requires that $(x,p,t)

= u*.

In summary, finding u* amounts to maximizing H to get § and then solving Eq. k to obtain χ and p.

BOUNDARY CONDITIONS AND THE CHOICE OF THE FUNCTIONAL

Τ So far, no indication has been given of the value of ρ . From the way u = H(x,P,t) was obtained from H = p*f, p(t) can be interpreted as the direction in which the projection of f

must be maximized with respect to u. It can be shown that ρ depends upon the end constraints imposed upon χ(τ) (l). In the case χ(τ) is required to liç in some closed convex space (for example a cylinder) then ρ would be determined from the geometry of the cylinder. However, if there is no such restric- tion on X ( T) , then

Τ (Note: This is minus the p"1" = ( I I gradient of χ . )

Now by a judicious choice of the functional to be minimized, even in a problem where there are end constraints, the problem can be reduced to this simpler case. For example, it is sup- posed tha£ in addition to minimizing ^(τ) one requires X / T) = x. where i= l,2,...,n-l. One may replace the old functional x n(T) by

n- 1 m ο

χ (Τ) + Σ (χ.(τ)-χ.Γ ι = 1

for example. This would automatically tend to drive

2 4

x.(t) i=l,2,...,n-l toward the desired end value. Other measures of distance from desired end conditions could be in- jected into the functional to be minimiζede The precaution should be taken that it be a positive function. Other restraints on x(t) could be handled in a similar manner by

using integral metrics, for example, to hold down peaks in x(t).

Τ It should also be pointed out that the determination of ρ was made on the basis that χ (τ) does not take on its absolute minimum value, zero, (which would be called the degenerate problem), so that it would be improper to try solving Eq. k Çy integration of the system backward in time using χ^(τ) = χ., i=l,2,...,n for the case where the functional was

n- 1 m ο

Σ (χ (Τ) - x ^ )2

i=l

To illustrate these techniques, the following example is considered.

EXAMPLE APPLYING THE MAXIMUM PRINCIPLE TO DETERMINE THE OPTIMAL CONTROL (WITH RESPECT TO MINIMUM PROPELLANT) OF A VARIABLE MASS VEHICLE

The motion of a rocket propelled vehicle is considered which operates in a three-dimensional inverse square gravita- tional field. Assuming that it has constant specific impulse and that the thrust vector magnitude can therefore be controlled by controlling the mass flow rate, the equation of motion can be written as follows (using rectangular coordinates).

The System (f)

χ

V A

i

2 7

" HP

ve^{u 2 u}l f ^{G x}3

X7

x5 ⁼ 6 ^X

veu3Ui + G x5

x7

" R

x7 ⁼ ~^uh

Notation: (x^

(x2 > XV X6 ^

X7

(^, u2, u?)

V

e

NI

Control Constraints ( u )

2 2 2

Ul + U2 + U3

The Adjoined Functional

= position

= velocity

= mass

= unit thrust vector

= mass flow rate

= exhaust velocity (constant)

= gravity constant

= Υχπ + x^ + χ-

1 3 5

= 1 0 <_u^<. Β

The control problem considered is one in which the control vector

u

is chosen subject to the foregoing constraints, and in order to satisfy, as nearly as possible, çrescribed end conditions in position and velocity

(χ^(τ)

i=l 1 ο

where λ.^ 0 are weighting constants. Differentiation gives

6 , Τ

1 = 1 ¹

where the f. are given by the right-hand members of the system (f).

The H Function

One may now form the function Η = Σ f.p. whose terms can be rearranged into the form i=l

26

7

γ L e I I

X2 + p⁵- ^{λ 3}( χ³-^{χ 5}) G Ρ.Χ

ρ

ι "

ι (

ι "

ι )

5 5 5 5 '

where it should be pointed out that the symbols X and Ρ denote X = (χ^,χ^,χ^)

Ρ =|[ρ²-λ²(χ²-χ^)] ,[ρ⁴-λ⁴(χ⁴-χ^)], [ρ⁶-λ⁶(χ⁶-χ^) and it will be useful to define

| | p| | 42- X²( x²- x2 T) ]2 +[P i t- X⁴( x^rx J ) ]^{2 +} [ p6- X⁶( x⁶- x ^ ) ]²

It should also be noted that in forming H, use has been made of the fact that pg(t) = -l which follows from the non-occurrence of xg in fg and P Q ( T ) = - 1 .

Maximizing H with Respect to u

Since the problem is to minimize X Q ( T) , H must be maximized with respect to u. Since v^/x ^0 ana (u-^u^u ) is a unit vector and u ^ 0 , H is maximized by making (u^, u ^ u ) parallel to Ρ and choosing u^ as given in the following. ^ The Optimum Control Vector

ul *

p2 - x2 " ^x2 ^

— —

x6 > ] h - ^λ6 ^

(Β when j|p|| - Λ (^{Ρ γ} + ^ J - 0 0 otherwise

(u^* is denoted by B(t).) Replacing the controls in the system by the optimum controls and calculating the adjoint equations

*>y P± = b \ , i = l, 2 , . . . , 7 ; one obtains

Optimization Equations

v^eB(t) •λ²(χ²- G

χ1

X2

η ' l t d

S =

\

φ

• w

~ ^X7

1 11 IM I

S = x6 v^eB(t)

·^λ6^{( χ}6 " •^χ6 ) ] G

Χ5

x6

' ^X7

1

ΙΙχΙ

^7 -B(t)

| Β whenllpll - _ ϊ( ρ ^ ) = ⁺ where B(t) = < e ' '

(^0 otherwise

3(ρ·χ)χ

[p²-X²(x²-x^2T)J -

i>² = λ²χ² -[ρ¹-λ¹(χ¹-χ^)]

λ^χ,.

h = \ \ - [^ρ3 " ^{λ 3 ( χ}3 -^χ^ ]

V 6 +| Φ ^ ί^Ρ6 '^λ6^{( χ}6 -^χ6 ) ]

λ6^χ6

"fvVV^]

^ 5

p6

3(Ρ'Χ)χ^π

v^eB(t) ||P

^7

It is noted that once u* has been determined, the differential equation ig = fg can be dropped from the system. Now the problem is to solve the foregoing system of differential equa- tions with boundary conditions x.(o) = x. and p.(τ) = 0 specified, i = l, 2 , . . . , 7 .

The Numerical Solution

The previously given system of equations must be solved numerically, using a digital computer. A technique that can be used to solve the equations is one in which the χ equations are integrated forward in time and the ρ equations backward in time each time altering the right-hand sides by the previous χ and ρ computed. The process will converge provided the initial guess on x(t) and p(t) is sufficiently close. The Newton- Raphson method is very often useful to establish the initial guess. This method consists of making a guess at initial values of the ρ variables, integrating the optimization equa- tions as an initial value problem and then altering the guessed initial values on the basis of the resulting values of p.(τ)

and χ

( τ ) .

REFERENCES

1 Rozonoer, W.I., "L.S. Pontryagin's maximum principle in the theory of optimum systems - Part II," automatika I

Telemekhanika 2 0 , no. 1 1 , l 4 4 l - l 4 5 8 (Nov. 1 9 5 9 ) ·

2 Markus, L. and Lee, E.B., "Optimal control for nonlinear processes," Archive, Rational Mech. and Anal. 7 ( i 9 6 0 ) .

3 Bryson, A.E., Carroll, F.J., Mikami, K., and Denham, W.,

"Determination of the lift or drag program that minimizes re- entry heating with acceleration or range constraints using a steepest descent computation procedure," Presented at IAS 2 9 t h Annual Meeting, N.Y., Jan. 2 3 - 2 5 , I 9 6 I .

Editor's note: See the following paper ("Optimalizing Tech- niques for Injection Guidance,M by Wayne Schmaedeke and

George Swanlund) for a brief description of the foundations of the Pontryagin Maximum Principle.