Optimal Control Theory

Author(s): Leonard D. Berkovitz

Source: The American Mathematical Monthly, Vol. 83, No. 4 (Apr., 1976), pp. 225-239
Published by: Mathematical Association of America
Stable URL: http://www.jstor.org/stable/2318209
Accessed: 16/12/2010 05:52
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=maa.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Mathematical Association of America is collaborating with JSTOR to digitize, preserve and extend access to
The American Mathematical Monthly.

http://www.jstor.org

NOTICE
BeginningMay 1st,1976,myEditorialOfficewill be in Zurich.Please addressall editorial
to:
correspondence
Alex Rosenberg
Forschungsinstitut
furMathematik
ETH Zurich
8006Zurich
Switzerland
Authorsmay,in orderto save postage,sendonlyone copyof theirmanuscript.
ALEX ROSENBERG,Editorin-Chief

OPTIMAL CONTROL THEORY

LEONARD

D. BERKOVITZ

1. Introduction.
The development
of themathematical
theoryof optimalcontrolbeganin the
earlyto mid 1950's,partiallyin responseto problemsin variousbranchesof engineering
and
economics.
Despiteitsmodernorigins,
optimalcontrol
froma mathematical
theory,
pointofview,is a
variant
ofone oftheoldestandmostimportant
subfields
inmathematics-the
calculusofvariations.
We shall sketchsome of the important
aspectsof optimalcontroltheory,the storyof their
development,
and theirrelationship
to thecalculusofvariations
andto otherbranches
ofmathematics.
Besidesinforming
thereaderaboutcertainaspectsofoptimalcontroltheory,
we hopeto impress
himwithyetanotherexampleofthefruitful
interaction
in puremathematics
betweendevelopments
and applications
and betweenconcretespecialproblemsand generaltheories.The greathistorical
examplesofsuchinteractions
andtheircontributions
to thevigorous
arewell
ofmathematics
growth
known.The storyofthedevelopment
ofoptimalcontroltheory
inthelasttwenty
yearsis a smalland
yetunfinished
chapterin the historyof such interactions.
It illustrates
how meaningful
applied
problemscan lead to new developments
in mathematics
and to a revitalization
of old areas of
research.It also illustrates
theutility
of existing
whichmayhave been developedin
mathematics,
othercontexts,
forthesolutionof appliedproblems.
In orderto keep thelengthof thispaperwithinreasonwe mustlimitour discussion
to a few
selectedtopics.We trust,
however,
thatthesewilladequatelyillustrate
anddocument
theassertions
we madeabove.
2. Theservoproblem.In theearly1950'selectrical
engineers,
influenced
bya paperofMcDonald
in improving
theperformance
of servomechanisms
withrespectto timeof
[21],becameinterested
response.A typicalproblemis thefollowing.
A control
surfaceon an aircraft
is to be keptat restat a fixedposition.A windgustdisplacesthe
surfacefromthedesiredposition.It is assumedthatifnothing
weredone,thenthecontrolsurface
wouldbehaveas a dampedharmonic
oscillator.
Thusif 8 measuresthedeviationfromthedesired
position,whichwe take to be zero, thenthe freemotionof the surfacesatisfies
the differential
equation

Ol?
aO,, + 20 = 0,
withinitialconditions
6(0) = GOand O(0) = 6O.Here 6Ois thedisplacement
ofthesurfaceresulting
225

226

[April

L. D. BERKOVITZ

fromthewindgustand O is thevelocityimparted
to thesurfaceby thegust.On an aircraft
the
ofa control
dampedoscillation
surfacecannotbe permitted.
a servomechanism
is toapply
Therefore,
a restoring
torqueso as to bringthesystemto restat thedesiredpositionin minimum
time.The
equationof motionof thecontrolsurfacethenbecomes
(2.1)

0"+ aO'+wCt)20= u(t)

(0)= 0O 0'(O)= 0O,

therestoring
whereu(t) represents
torqueattimet.Sincethevoltagesavailablearenotunlimited,
the
magnitude
ofthetorqueis constrained
Theproblem
byan inequality
Iu(t) I c,wherec is a constant.
is to applythetorqueas a function
of timein sucha waythatthesystem
willbe brought
to 0 = 0,
0' = 0 in minimum
time.
It is clearthatifOo> 0 and 0 > 0 thenthetorqueshouldinitially
be directedin thedirection
of
negative0 and shouldhavethelargestpossiblemagnitude.
Thusu(t)= - c, initially.
It is also clear
thatifthetorque- c is appliedfortoo longa periodthenwe shallovershoot
thedesiredterminal
condition
of 0 = 0, 0' = 0. Thusitappearsthatat somepointthereshouldbe a torquereversalto + c
inorderto brakethesystem.
The questionariseswhether
thisis indeedso, andifitis,whereshould
theswitch
occur.Or,is itbettertoremovethetorqueat somepoint,allowa smallovershoot,
andthen
is to allowthesystem
to overshoot
undertheinfluence
of - c,then
apply+ c. Stillanother
possibility
brakewith - c. In thisvein,one could ask whethera sequence - c, + c,
apply + c, and finally
- c,.. + c of n stepsis best,andifso,whatis n,andwheredo theswitches
occur.Relatedto these
questionsis thequestionof whatto do iftheservomotorstartsto act at Oo>0 and O <0?
In theengineering
literature
oftheperioditwasalwaysassumed-on thebasisofintuition-that
theoptimalmodeofoperation
wasone inwhichtherestoring
torqueonlytookon thevalues+ c and
- c,andvariousincomplete
theappropriate
analysesweremadeto determine
sequenceof ? c to be
used.A restoring
valuesu(t) = ? c is calleda
torque,or control,u thatonlytakeson theextreme
to the problemwas made in 1952by Bushawin his
"bang-bang"control.A majorcontribution
Princeton
doctoralthesis,laterpublishedin [8]. Bushawalso assumedthatthebang-bang
modeof
wasindeedoptimalandthenproceededtodetermine
forwhatvaluesof(0,6') oneused + c
operation
andforwhatvaluesof(0, 0') one used - c. His arguments
wereelementary
and non-variational.
He
also determined
theoptimaltrajectories
of thesystemin the(0,0')-plane.
3. Thelineartime-optimal
sectioncanbe rewritten
problem.The servoproblemofthepreceding
inthefollowing
form.Let xl = 0 andletx2= 0'. The differential
to
equation(2.1) is thenequivalent
thesystem

dxt
dt
dx2

dt

_
-

xt(O)=0oo

X
2

2 1+ U,

ax -w x+u

2(O) =

x ()0~,

whereu is a function
classoffunctions
to be chosenfroma specified
and is requiredto satisfy
the
constraintu (t) ?c. The problemis to choose u so as to minimize

ldt,
wheret1is thefirsttimesuchthatxl(t) = 0 and x2(t)= 0, providedsucha firsttimeexists.
The servoproblemin theformulation
justgivenis a specialcase of thefollowing
problem.The
stateof a systemat timet is describedby an n-vector
x(t) = (xl(t), . , xn(t)). The systemevolves
toa system
oflineardifferential
formas follows:
according
equationsthatwewriteinvectormatrix
dxldt= A (t)x + B(t)u(t)

x(to)= xo.

HereA is an n x n matrix
whoseentriesare functions
of t,B is an n x m matrix
whoseentriesare

1976]

OPTIMAL CONTROL

227

THEORY

of t, u = (u , um) is an mr-vector
whosecomponents
are functions
of t thatsatisfy
functions
u'(t)? c', whereci, i - 1,--, m are constants,
tois theinitialtime,and xois theinitialstate.It is
requiredto chooseu so as to bringthesystem
to theorigininminimum
time.In otherwords,we are
to choosea u thatminimizes
t1,wheret1is thefirst
timesuchthatx(t) = 0. The functions
u thatwe
chooseare calledcontrols.A controlu * thatminimizes
t1is calledan optimalcontrol.
thisway,Bellman,Glicksberg,
Withtheproblemformulated
and Gross[4] showed,usingrather
thatforspecialmatrices
functional
A andB an optimalcontrol
elementary
analysis,
doesexistandis
indeedbang-bang.
theassumptions
madeaboutthematrix
A in [4] werenotfulfilled
Unfortunately,
in manyexamplesof interest,
theoriginalservoproblem.Nevertheless,
thiswas thefirst
including
in thispaperwereexpanded
generalformulation
of thelinearproblemand themethodsintroduced
and exploitedbyothers.Later,LaSalle [17]showedthatforlinearsystems
moregeneralthanthose
and Grossit is truethatan optimalcontrolexistsand thatit is
treatedby Bellman,Glicksberg,
LaSalle alsoshowed,underappropriate
thatifthesystem
canbe driventothe
bang-bang.
hypotheses,
controlthatwilldrivethesystem
to theoriginin the
originbya controlu,thenthereis a bang-bang
sametime.Thisis theso-called,"bang-bang
Notethatthisprinciple
holdsforanycontrol,
principle."
optimalor not.
is of importance
The validity
of thebang-bang
in engineering
reason.
forthefollowing
principle
One needonlyuse "contact"or"on-off"
sinceoneneedonlyconsider
that
controls
servomechanisms,
takeon thevaluesu' (t) = + c', i = 1, **,m.Suchservomechanismns
are easierto buildand operate
thanthosein whichall valuesin theintervals
[- c', c'] are allowed.
in varying
canbe established
The bang-bang
principle
degreesofgenerality
byvariousarguments.
As withothermathematical
and
theorems,the proofhas undergonea seriesof refinements
At presentit is seen to followwitha minimum
improvements.
of technicaldetail fromthe
Krein-Milman
theoremand thecharacterization
of theextremepointsof convexsetsin a certain
function
space.See [6].
In theSovietUnion,R. V. Gamkrelidze
also attackedthetimeoptimal
[11]andhiscollaborators
problem.Theirmethodsweredifferent,
and we shalldiscusstheirworklater.
4. The nonlinear
optimalcontrolproblem.The lineartimeoptimalproblemdescribedin the
sectionis a specialcaseofthefollowing
preceding
generalnonlinear
problem.
Thestateofa system
at
timet is describedby a pointor vector
x (t) =(x l(t),

Xn
(t))

in n-dimensional
euclideanspace,n > 1. Initially,
at timeto, thestateof thesystemis
x(to)

= Xo = (XI,

,x).

Moregenerally,
wecanrequirethatattheinitialtimeto,theinitialstatexoissuchthatthepoint(to,xo)
set SOin (t,x)-space.The stateofthesystem
varieswithtimeaccording
belongstosomepre-assigned
to thesystem
of differential
equations
(4.1)

x0=
d(,x,xz)
=f

n,

wherez = (z', ***,z') is a vectorin realeuclideanspace Em and thefunctions

fi are realvalued
continuous
functions
of thevariables(t,x,z).
By the'systemvarying
to (4.1)' we meanthefollowing.
A function
u withvaluesin
according
m-dimensional
euclideanspace is chosenfromsome prescribed
class V of functions,
suchas the
piecewisecontinuous
theboundedmeasurable
functions,
functions,
thefunctions
inL, etc.Whenthe
z = u(t) is madeintheright
substitution
handsideof(4.1)weobtaina system
ofordinary
differential
equations:
(4.2)

dx=

1,

n.

228

[April

L. D. BERKOVITZ

For each u in W itis assumedthatthereexistsa point(to,xo)in S0 anda function = (4l,.,

defined
on aninterval
Thatis,werequirethat
[to,t2]withvaluesinE" suchthat(4.2)is satisfied.
(

)()d+ '=
fi(t +(t),u (t)),

i(to)= x

n)

,**,n.

fdescribestheevolutionofthesystemwithtimeandis calleda trajectory.

Thefunction
Thefunction
u is calleda control.
The controlu is further
requiredto be suchthatat sometimet1,whereto< t1' t2, thepoint
4
to
a
set S- andforto' t < t1thepoints(t,+ (t)) do notbelongto 3-,.
(t1, (ti)) belongs
preassigned
setfortheproblem.In theservoproblemtheset 3- is thepoint
The set Sl is calledtheterminal
by:
(t0 4, x0) and theset 3- is defined
1 = {(tl,Xl,x):X1

= O,X2 = O}.

inthepreceding
is sometimes
summarized
inlessprecisebutsomewhat
Thediscussion
paragraphs
thatthecontrolsu arerequiredto transfer
from
moregraphiclanguagebythestatement
thesystem
E
a
state
at
time
where
E
and
3Y1.
initial
state
at
time
to
terminal
Note
an
xl
t1,
(to,xo) S0
xo
to
(t,,xl)
morethanone trajectory
4. Thisresultsfrom
thatto a givenu in W therewillgenerally
correspond
of solutionsof (4.2) if no
different
choicesof initialpoints(to,xo) in -oor fromnon-uniqueness
are madeto guaranteetheuniquenessof solutionsof (4.2).
assumptions
solutionf mustsatisfy
a
It is oftenfurther
requiredthata controlu in W and a corresponding
constraints
systemof inequality
(4.3)

R'(t, +(t), u(t))-'O

i = 1,2,"* , r,

forall to' t '- t, wherethefunctions

of (t,x,z). In thelineartime
R',* , Rr are givenfunctions
are
optimalproblemtheconstraints
ui(t)+ c ?0
c -ui(t)_O

i=l,

,m.

it is requiredthatu(t) E (t,4(t)), whereQ is a mappingthatassignsto each

More generally,
point(t,x) a subsetQ(t,x) of E m.
Let f?be a realvaluedfunction
of(t,x,z) andletg be a realvaluedfunction
x i.
defined
on Yo>X
as follows:
For each controlu in W and corresponding
trajectory
0 we definea payoff
(4.4)

J(4,u) = g(to, 4(t0),t,,

4(t0))+ f(s, ?(s), u(s))ds.

Itis tacitly
assumedherethatthefunction
r?andtheclassWaresuchthattheintegralin(4.4)exists.
In theservoproblemwe canwriteJ intwoequivalent
ways.We caneithersetfo 1 andg 0 or
g(to,xo,t1,x1)= t1 and fo 0.
We definean admissible
controlu to be a function
u in 16suchthata corresponding
solution0 of
that(4.3) holds,theintegral
in (4.4) is defined,
and
(4.2) existswiththeproperty
(4.5)

(to,0 (to),ti,0 40l) E 30 x 3-1.

We call thesolution4 an admissible

and we call thepair(4, u) an admissible
pair.The
trajectory,
nonlinear
optimalcontrolproblemthatwe consideris thefollowing.
PROBLEM: Findan

admissible
pair (o *, u *) suchthat
J(A*,u*)<-J(, u)

J overtheclass ofadmissible
pairs.
forall admissible
pairs (0, u). Thatis, minimize

1976]

OPTIMAL CONTROL THEORY

229

withtheinterest
ofelectrical
in thetime
5. Moreaboutappliedproblems.Concurrent
engineers
optimalcontrolproblem,therewas interestin otherdisciplines
in optimization
problemswhose
formulation
mathematical
was thatofthegeneralcontrolproblem.Notableamongthesedisciplines
weremechanicalengineering,
chemicalengineering,
and economics.Also,
aerospaceengineering,
problemsin electricalengineering
otherthanthe timeoptimalone were arisingthatcould be
as a generalcontrolproblem.A representative
formulated
let alone a
sampleof theseproblems,
completecatalogue,is notpossiblein thispaper.A representative
sampleis givenin [6]. We shall,
however,
presentone simplified
examplefromeconomicsof a production
planning
problem.
Let x(t) denotetherateofproduction
at timet ofsteel.The amountproducedat timet is to be
allocatedto one oftwouses,theproduction
ofconsumer
or investment.
It is assumedthat
products
is usedto increaseproductive
thesteelallocatedtoinvestment
steel
toproducenew
capacity
byusing
u
steel mills,miningequipment,
etc.
Let
where
u
transport
0_ (t) ?1, denotethe
facilities,
(t),
fraction
ofthesteelproducedat timet thatis allocatedto investment.
Then(1 - u(t)) represents
the
allocatedto consumption.
The assumption
thatthereinvested
fraction
steelis used to increasethe
productive
capacitymeansthat
dx= ku(t)x,
wherek is an appropriate
constant.The problemis to choose u(t) so as to maximizethe total
overa fixedtimeperiodof lengthT >0. Thatis, we are to maximize
consumption
fT

J(1 - u(t))x(t)dt.

The problem
we arefacedwithis thefollowing.
Do we alwaysconsumeeverything
produced?Or,
do we investsomeat presentso as to increasecapacitynowso thatwe shallbe able toproducemore,
andhenceconsumemorelater?Do we followa "bang-bang"
offirst
procedure
investing
everything
and thenconsuming
everything?
Theproduction
inthecontrol
canbe written
ofSection4 as follows.
planning
problem
formulation
MinimizeJ(4,u), where

J(, u)= -

f
rT

(1- u(s))4 (s)ds

overtheset of pairs(4, u) thatsatisfy

thedifferential
equation

dx= ku(t)x

x(0)= c

0<u(t)?1

+(t)-0.

and constraints
Here c > 0 is theinitialcapacity.
The constraint
+ (t) ' 0 is presentsincewe do notdestroy
capacity,
and thusnegativecapacityis meaningless.
Althoughthe problemsconsideredin the variousareas of applicationhad the common
mathematical
oftheproblemin Section4, theseproblems
formulation
werenotso formulated
at the
time.Each was formulated
in thelanguageof itsownfield.For themostpart,investigators
were
unawareoftheworkin otherdisciplines.
Each discipline
seemedto favoritsownsetofmethods
for
theproblem.
attacking
In thenextsectionwe shallsee thatthecontrolproblemis variational
incharacter.
Almostall of
theinvestigators
in theareasofapplication
wereawareofthevariational
character
oftheproblem.
The constraint
directapplication
(4.3), however,prevented
of theknownresultsin thecalculusof
In manyspecialproblems
variations.
thebasictechnique
ofthecalculusofvariations,
namelythatof

230

[April

L. D. BERKOVITZ

was
thesideconditions,
whichstillsatisfies
thereof
witha perturbation
theoptimalsolution
comparing
of the
carriedout fortheexamplein question,(see e.g. [2]). Others,triedvarioustransformations
degreesofsuccess(see
(4.3) withvarying
theconstraint
problemwithout
problemintoa variational
in thetimeoptimal
analysiswas used effectively
e.g. [9], [19],[23]). As alreadynoted,functional
equations(4.2) are linear.In someproblemsthedynamic
problem.Thisis becausethedifferential
formalism
was used,(see e.g. [1]).
programming
is thefollowing.
The "simpleproblem"inthecalculusofvariations
6. Thecalculusofvariations.
f: (t,x,z) -f?(t, x,z) is given,wheret is a scalar,x = (xl, ** , x"') is a vector
A realvaluedfunction
ineuclideanspaceEn andz = (zl, ***,z'n) is a vectorinEn. A point(to,xo)anda point(t,,xl) inE"'
4 suchthat0(to)= xo,0(t1)= xi andsuchthattheintegral
aregiven,as is a class(Doffunctions

f (t,+ (t),0'(t))dt

J()=

4* in(D1thatminimizes
suchthat
J(4); i.e.,finda 4 * in(D1
Theproblem
is tofinda function
isdefined.
4
?
to
we callthereader'sattention
withthesimpleproblem,
J(4 *) J(4) forall in(. In connection
willnotbe precise.
Ourstatements
function
mustsatisfy.
conditions
thata minimizing
twonecessary
the
class (D considered,
hold dependon the function
The precisesense in whichthe conditions
function.
purposeswe nowlet 4 designatetheminimizing
of f,etc.For typographic
properties
is thesystemof Euler equations
The firstnecessary
condition
d fCi(t,
+(t), ?'(t)

(6.1)

n.

condition
The secondis theWeierstrass
n

f?z(t,+(t) 01(t))zi

fo(t,+ (t), z)

i =1

of fo.
forall z in theopen domainof definition
is thatof minimizing
a functional
of theform
The Bolza problemin thecalculusof variations
g(to,4(to),t1,4(ti)) +

fo(t,?(t), +'(t)) dt

in an appropriate
class C thatsatisfy
a systemof differential
equations
overthosefunctions
G'(t, + (t), +'(t)) = O

i = 1,

and end conditions

-

(to, 4 (to), t1,4 (t1)) E 3,

set in E2+2n.If g 0 thentheproblemis calleda Lagrange

wherem < n and is a preassigned
areequivalent
Iffo 0 itis calleda Mayerproblem.
It canbe shownthatall threeproblems
problem.
[7].
A fairly
completetheoryfortheBolza problemhad been developedby 1940.Again,fromthe
forthereader's
conditions
wealthof information
abouttheBolza problemwe selecttwonecessary
F bythe
theLagrangian
function
as follows.Introduce
Thesecan be summarized
attention.
roughly
formula

F(t,x,z,1?, l) = 1?f(t,x,z) +

I G (t,x,z).

19761

OPTIMAL CONTROL

231

THEORY

function
thenthereexistsa scalar 0_i/ 0 and a vectorvaluedfunction
If 4 is the minimizing
= (Ofr1',
*fY") definedon [to,t,] suchthatequations(6.1) and (6.2) holdwithf?replacedby F.
*,
Thus:
(6.3)

dt
+)(t), d>'(t),{4?, +(t)) = F'(t, ?(t), 4)'(t),
d-~jFz(t,

?,Oif(t))

i = 1,**, n

and
(6.4)

F* (t)z > F(t,+ (t), 0 (t),00,+ (t)) - ,F*, (t)(O ) (t),

F(t,+ (t), z, 00,+ (t))-

forall z satisfying
G'(t, +(t), z)

0, i = 1,*, m,where

F*(t=

Fz i(t,+ (t), +'(t), 0?, + (t)).

in conjunction
and q1(t1)mustsatisfy
thatthevalues4i, fr(to)
With(6.3) thereis a setofconditions
conditions."
areknownas the"transversality
withtheendvalues(t0,0(to),ti,0(t1)).Theseconditions
rule."Inequality
equationsorthe"Lagrangemultiplier
Equations(6.3)arecalledtheEuler-Lagrange
condition.
(6.4) is calledtheWeierstrass
easy to
conditionforthe simpleproblemare relatively
The Euler equationsand Weierstrass
thatthesameis trueforthemultiplier
establish.
textsgivetheimpression
manyengineering
Although
rulewas first
statedbyLagrangeforthe
notthecase. The Lagrangemultiplier
rule,thisis definitely
contained
Thefirst
wascorrected
twomajorerrors.
problem
thatbearshisname.His proof,however,
byA. Mayerin 1886and thesecondbyKneserin 1900and also byHilbertin 1905.A satisfactory
wasalsolongincomingforth.
fortheBolza problem
Itwasnot
oftheWeierstrass
condition
treatment
was first
givenbyMcShane[22].Priorto thisworktheWeierstrass
until1939thatsucha treatment
In hisproof,McShane
wasestablished
underconditions
thatwerenotentirely
satisfactory.
condition
convexsetsofvariations
andtheseparation
for
a completely
newidea involving
theorem
introduced
convexsets.We shallreturnto thispointlater.
in nature.
In thepreceding
sectionwe statedthatthecontrolproblemofSection4 is variational
We now show this.If we introducea new independentvariable y = (yl, ***,yi) and let y' = u, then

as follows.
thecontrolproblemof Section4 can be written
Minimize
g(to, 4)(to), t1, 4)(to))+

12

f(s, 4(s),

y'(s))ds

thesystemof differential
overthefunctions
classthatsatisfy
equations
(4,y) in an appropriate
f (4 (t),y'(t)) = 0
( i )(t)- fi

j = 1,

n,

theend conditions
(to,4)(to),t1,4(t1)) e Y,

constraints
and inequality
R s(ts+ (t), y'(t))-'` 0

i = 1,* ,r.

We
is a Bolza problemin thecalculusof variations.
constraint
theinequality
The problemwithout
of
thevariational
character
inappliedproblems
havealreadynotedthattheearlyworkers
recognized
thetheoryoftheBolza problembytheinequality
fromapplying
butwereprevented
theproblems,
constraint.
inthe1950'swasnotconfined
inproblems
ofoptimalcontrol
7. Themaximum
principle.Interest
the
werealsopursuedintheSovietUnion,wheretheyattracted
totheUnitedStates.Theseproblems

232

[April

L. D. BERKOVITZ

and his studentsV. G. Boltyanskii,

attention
of L. S. Pontryagin
R. V. Gamkrelidze
and E. F.
Mishchenko.
Theystatedthegeneralcontrol
intheformgiveninSection4, andin
problemessentially
a seriesof papersin the late 1950'sannouncedwhathas come to be knownas the Pontryagin
is a set ofnecessary
Thisprinciple
maximum
principle.
conditions
thatmustholdalongan optimal
The announcement
ofthemaximum
canproperly
trajectory.
as thebirthofthe
be regarded
principle
mathematical
theoryof optimalcontrol.Workersin different
areas of applicationsaw thattheir
problems
had a commonmathematical
formulation
andwereprovidedwitha toolforattacking
their
problems.
We nowgivean imprecise
ofone formofthemaximum
statement
principle.
Other,moregeneral
formsare known,butwe shallnotconsiderthemhere.For a precisestatement
of theprinciple
we
refer
thereaderto [27],whichis an accountinEnglishoftheworkofPontryagin
andhiscollaborators
inthelate1950's,or to anyotherstandard
reference.
As in [27],we shallsupposethattheconstraints
on thecontrolare of theform
u(t)EQ
we shallassumethattheproblemis in
whereQ is a fixedset in Em. In theinterests
of simplicity
inthisassumption.
thereis no lossofgenerality
Lagrangeform,
i.e.,g 0. As intheBolza problem,
H definedas
The statement
of themaximum
of a function
principle
requirestheintroduction
follows:

H(t,x,z,p0,p) = p?fo(t x, z) +>

p ifi(, X z).

The maximum
statesthatif(0, u) is an optimalpair,thenthereexista constant
principle
0+=_ 0 anda
vectorvalued function1= (q,1,.., ifrn)definedand absolutelycontinuouson [to,t,] such that
0
(fO +f(t)) 0 forall t in [to,t1],suchthatthefollowing
systemof equationsis satisfied
(7.1)
(7.2)

d4'= fi(t,+(t), u(t))

dt

= - E Oi(t)fif(t,
dd~~~~~
+ (t),u(t))
dt
j=O

n,
,,n

and suchthat

(7.3)

z,t/0?
H(t,4O(t),
u(t),(/0?
0/(t))->H(t,4O(t),
4/(t))

valuesS- are C"1)manifolds

andif
forall z infl.IfthesetofinitialvaluesSo andthesetofterminal
conditions
certainotherregularity
hold,thenwe also have thatthevector

(7.4)

(H(to,4(to),u(to),q?0q(to)) -(to))

is orthogonal
to SO at (to,4(to)) and thevector

(7.5)

( - H(ti, 0 (ti),u (ti),0?,0 (ti)),0 (ti))

is orthogonal
are the"transversality
to f- at (t1,4(t1)). The orthogonality
conditions
conditions."
Equations(7.1) arejustthestateequations.Ifwe considertheconstantf0andthepair(4, u) as
in vector
knownin (7.2),then(7.2) is a linearsystemin f. Equations(7.1) and (7.2) whenwritten
notationhave thefollowing
Hamiltonian
form

do>= Hp(t,+(t),u(t),0 0
+(t))
dt =- H (t,
4 (t), u(t),
q/ q(t))
0
dt

19761

233

OPTIMAL CONTROL THEORY

A pair(4, u) thatsatisfies
is calledan extremal
we can
themaximum
principle
pair.Theoretically,
use the maximumprincipleto determine
all extremalpairs(4, u), and fromamongthesethen
determine
theoptimalones by someothercriteria.In problemswherethedimension
of thestate
vectoris largeitis difficult
to determine
extremal
pairs.Thisis primarily
becauseequations(7.1) and
(7.2)andinequality
(7.3)whentakeninconjunction
withthetransversality
conditions
aresuchthatthe
endvaluesof 4 and fare noteitherall terminal
or all initial.Rathersomevaluesare givenat the
initialtimeandothersat theterminal
time.In problems
wherethedimension
ofthestateis small,one
caneitherintegrate
or backward
and keeptheunknown
initialor terminal
(7.1)-(7.3)forward
values
as literalconstants
whichcanbe adjustedtothedataat theotherend.Forsystems
ofhighdimension,
whereone is trying
thisprocedure
is notfeasible.
to determine
extremal
pairsbynumerical
methods,
Neitherare procedures
in whichthemissing
valuesare guessedand latercorrected.
We shallnotgo
intotheseproblems.
has not
We mention
themheremerely
to pointoutthatthemaximum
principle
done awaywithall difficulties.
Despitethelimitations
themaximum
is important
and useful.It does
justmentioned,
principle
giveinformation
of thesolution.It has yieldedthesolutionin certainclassesof
aboutthestructure
of the stateis small,the maximum
problems.In manyspecificexamplesin whichthe dimension
principle
hasgiventhecomplete
To appreciate
thislaststatement
thereadershouldcompare
solution.
of theseproblemsin the
the workedout examplesin [27] withsome of the earlytreatments
engineering
literature,
say as collectedin [261.
Pontryagin
andhisco-workers
ofthecontrolproblemand
wereawareofthevariational
character
oftheBolza problem
also believedthatthepresenceoftheconstraints
(4.3)madetheexisting
theory
inapplicable.
Theytherefore
undertookto developnecessaryconditions
forthe controlproblem
reference
without
to existing
variational
theory.
Theydid,however,
makeessentialuse of theideas
In theproofofthemaximum
introduced
byMcShane[22]in hisproofoftheWeierstrass
condition.
principle
convexsetsofvariations
ofthecontrols
areintroduced
anditis shownthatas a consequence
of optimality,
a certainpair of convexsets can be separated.The analyticconsequencesof this
separationconstitute
themaximum
principle.
Further
detailsare givenin theappendix.
afterthepublication
therelationship
betweenthecalculusof
Shortly
of themaximum
principle
variations
andthemaximum
principle
wasclarified.
At theendofSection6 we showedhowa control
problemwithinequalityconstraints
could be transformed
into a Bolza problemwithinequality
constraints
Ifone introduces
thederivatives.
involving
a newvariablef bymeansofthedifferential
equations
(7.4)

R(t,xx,y)()2=

thentheinequality
aretransformed
constraints
intodifferential
andtheproblem
equationconstraints
is of standardBolza type.Underreasonableassumptions
R' and on theoptimal
on thefunctions
thenecessary
conditions
forthevariational
trajectory,
problemapply,andwhentheseare translated
backintothecontrolformulation
language,one getsthemaximum
principle
(see [5]). It is also quite
forthevariational
easytoobtainthenecessary
conditions
(6.3),(6.4)andthetransversality
conditions
fromthemaximum
See [273Chap.5. Underthesetranslations,
problem
principle.
theEuler-Lagrange
toequations(7.2)ofthemaximum
equations(6.3)correspond
andtheWeierstrass
condition
principle
to the maximum
(6.4) corresponds
condition(7.3). Also, the transversality
conditions
of the two
problemscorrespond.
The use of thevariablef as in (6.3) forLagrangeproblemswithinequality
constraints
on the
derivatives
is due to Valentine[283.It was used byHestenes[151in a minimum
timeproblem.
8. A unified
theory
ofnecessary
conditions.
Primarily
in responseto problems
in economicsand
operationsresearch,the theoryof mathematical
programming
also beganits development
in the
1950's.One formof themathematical
programming
problemis thefollowing.

234

[April

L. D. BERKOVITZ

A set V is givenin euclideanspace E" and mappings,

F?:
F=

--E1
gEv-

* *Fv):

(F',

G= (G',,

__>
E'

G'r):

are given.The problemis to minimize

F(x) overall x in V suchthat
F'(x)

G'(x)-'0

i =l,

,v

i =l,

,r.

If the constraints
G (x) 0 are absent,if the set V is open and thefunctions
F? and F are
is attained,theclassicalLagrange
thenat a pointx * in V at whichthe minimum
differentiable,
ruleholds.Variousextensions
oftheclassicalmultiplier
ruleweredevelopedto handlethe
multiplier
non-differentiable
inequality
constraints,
and further
extensions
weredevelopedto accommodate
functions
F?, F, and G.
In theearlyto mid1960'sit was notedthatthecontrolproblemof Section4 can be castas a
inan appropriate
programming
problem
function
space.One wayofso castingthecontrol
problemis
thefollowing.
Let V denotetheset of function
pairs(4, u) suchthat4)and u belongto function
spacesappropriate
to theproblemand suchthat
i = 1,

(0 i)At)= fi(t,+ (t), u(t))

*,

n.

Let themappingF?: V -- El be definedby

F?0(0,u) =

f(t, (t), u(t)) dt.

We nowsupposethattheinitialset SO is a C(l) manifold,

definedby a systemof equations,
xi (td,xo) = ?

i = 1, ** *,y

and thattheterminal
set is a C'") manifold,
definedby a systemof equations
Xi (ti, Xi) = ?

i = y + 1, * ,v.

We thendefine
F =(Fl,...Fv):

VEv

by
Fi(,O,u)= X'(to,+(to))
F(,u)

X
xI(ti, +(tl))

i =1, * ,y
i =y + 1, * ,v.

Let
G = (G',

G ,rG):
9

E'r

be definedby
Gi(0, u) = inf{R'(t, + (t), u(t)): to_ t - t1}.

If we writex =

(4,u), thenformally
thecontrolproblemof Section4 becomesthefollowing.

MinimizeF?(x) over x in

'

subject to F'(x) = 0, i = 1, ** , v and G'(x) ' 0, i = 1, ** , r.The control

problemreadsexactlyas theprogramming
problem,
exceptthatx is nowan elementof a function
thatofNeustadt[24].Other
space.The formulation
justgivenof thecontrolproblemis essentially
have givenslightly
writers
different
formulations.

1976]

OPTIMAL CONTROL

THEORY

235

problems
ofcontroland programming
formulation
in theabstract
Not onlywerethesimilarities
in
conditions
with
concerned
results
necessary
and
the
techniques
that
also
noticed
it
was
but
noticed,
not
were
the
however,
similarities
in
techniques,'
These
similar.
were
problems
optimization
various
variouswriters,
notablyNeustadt[24],Halkin
prompted
alwayson thesurface.Theseobservations
and
to embarkon the
Milyutin
Dubovitskii
and
[12],
and
Haratisvili
Gamkrelidze
[14],Hestenes[16],
and Milyutin.)
of
Dubovitskii
work
the
of
in
English
an
account
for
[13]
(See
program.
following
as specialcases
include
that
would
problem
programming
or
optimization
a
general
Firstformulate
control
control
problemswith
as
problems,
such
ordinary
interest,
of
problems
all optimization
mathematiparameters,
with
distributed
problems
lags,
control
with
problems
boundedstates,control
etc.
problems,
cal programming
set of necessaryconditionsfor the generalproblemunder
Second, develop a meaningful
mustbe suchthatone obtainsusefulnecessary
conditions
necessary
The
hypotheses.
reasonable
are made.
and identifications
when
specializations
appropriate
forthespecialproblems
conditions
specific
be
must
the
is
that
hypotheses
problem
of
the
general
in
formulation
the
One ofthedifficulties
to
be
enough
should
general
they
yet
structure,
with
some
enoughto yieldnecessaryconditions
interest.
of
includeall thespecialproblems
byNeustadt.An accountofhisworkwillbe
outlinedabovewascarriedthefurthest
The program
book
[25].
foundin hisposthumous
9. Othertopics.Althoughwe have writtenabout the timeoptimalproblemand about the
theoryof optimal
in themathematical
thesewerenottheonlydevelopments
principle,
maximum
lista few
is notpossiblein thispaper.We shall,however,
An accountofotherdevelopments
control.
developments.
newmethodsand ideaswere
questionsforcontrolproblems,
existence
In thecourseofstudying
of classicalresultsin the
and generalizations
simplifications,
whichgave refinements,
introduced
as well as the soughtforresultsin controltheory.The fieldtheoryof the
calculusof variations,
byL. C. Youngin hisbook [31]in orderto cope
generalized
was significantly
calculusofvariations
into the
problemsin the calculusof variations.Generalizedcurves,introduced
withsufficiency
formbyinvestigators
indifferent
byL. C. Youngin 1937[30],wererediscovered
calculusofvariations
controls"or "relaxedcontrols."Here again,new
in controltheoryand werenamed"chattering
overtheearlierworkin thisarea.
resultswereobtainedas wereimprovements
werethedistributed
whicharoseinresponsetoappliedproblems
Twootherareasofinvestigation
parameter
parametercontrolproblemsand the stochasticcontrolproblems.In the distributed
equationsand the
problemsthestateof thesystemis governedby a systemof partialdifferential
variable.In
is takenover a regionin the space of the independent
to be minimized
functional
involverandomness.
functional
controlproblemsthestateequationsand payoff
stochastic
methods.
computational
is thatofdeveloping
problemin termsofapplications
A veryimportant
trajectories
extremal
schemesto determine
As we pointedout earlier,thepursuitofcomputational
More promising
principle)has not been too fruitful.
the maximum
and controls(thosesatisfying
ofinterest
appliedproblems
manyspecific
Although
arethosethatinvolvedirectmethods.
techniques
withsolid
withsomesuccess,thereareno widelyapplicablealgorithms
havebeentreatednumerically
basisextant.
theoretical
we couldproceedwithourlistoftopics,we shallstopat thispointandsuggestthatthe
Although
[6],[10],[16],[18],[20],[25],[27],[29],
readerwhowantstolearnmoreconsultsomeofthereferences
[31].
10. Appendix.A proofof the maximumprinciple.We now sketch'a proofof the maximum
andsomewillbe formal.
willnotbe precise,somewillbe heuristic,
Someofourstatements
principle.
and his
ideas of the proofgivenby Pontryagin
We hope, however,to conveythe underlying
of thisproof.
and refinements
and theideas behindthelatergeneralizations
co-workers

236

L. D. BERKOVITZ

[April

In thissectionwe shalluse vector-matrix

notation.Thus,thestateequationswillbe written
do = f(t,4)(t), U(t)).
The symbolf, willdenotethematrix
ofpartialderivatives
(df'l/dx)i = 1,*, n,j = 1**, n andthe
symbol
f? andfuwilldenote
fuwilldenotethematrix
(dfIlduj)i = 1, , n,j = 1,* , m.Thesymbols
If x and y are vectorsin En, thenthesymbol(x,y) willbe used to
vectorsof partialderivatives.
thenMT willdesignatethetransposeof M.
denotetheirinnerproduct.If M is a matrix,
In orderto maketheexposition
as simpleas possiblewe assumethattheinitialpoint(t., xo)and
theterminal
point(t,,xi) are fixed.As in Section7, we assumethattheproblemis to minimize
J(4, u)=

fr(t,4 (t),u(t))dt

subjecttothestateequation(7.1)andto thecontrolconstraints
u(t) E fl.Let x = (x?l,x
x') and
a newstatecoordinate
letf = (f0,
xobymeansofthedifferential
f,*, f" ). Ifweintroduce
equation

dxo
0o)=o(t,x,u(t))
d=

x?(to) = 0,

thentheproblembecomesthatof minimizing
(10.1)

J(,O,u)=

O)0(t1)

subjectto thestateequations
d'>= f(t,4(t), U(t)),

(10.2)
theend conditions

400(t0)=0 O4(to)=xo

C(t1)=x1,

of4 does notappearin

and controlconstraints
u(t) E f. Notethatin (10.2)thezero-thcomponent
therighthandside.
Let (4),u) be an optimaltrajectory-control
pair.We perturbu by Su and obtaina trajectory
4)+ 84)+ e, where84)represents
the"first
order"termsin thenewtrajectory
and e represents
the
Ifwe substitute
"higher
order"terms.
u + Su and O + 84)+ e into(10.2)andexpandtofirst
order,we
getthat84)satisfies
(10.3)

6(t)'(t)= f. 0, ?>(t),U(O))SO(t) + fu01 ( (t), u (O)SU (t)

so) '(t)= (f*(t,4)(t),u(t)),st)(t)) + (fo?(t,

O)(t),u(t)),Su(t)),

with& (to) = 0. Hence by thevariation

of parameters
formulawe getthat

84(t1)= V(t1,to)f V1(s, to)fu(s)8u(s)ds,

(10.4)

whereft(s) = fu(s,4 (s), u(s)) andwhereV(*,to)is thefundamental

matrix
ofsolutions
ofthesystem
d=

u
f(t, 4)(t), (t))z

satisfying
matrix.
From(10.3)and(10.4)we get8400(t1)
bya
V(to,to)= I. Here I is then x n identity
as follows:
simplequadrature

(10.5)

=
800(t1)

[(f(s,+(s), u(s)),6+(s)) + (f?(s,+)(s),u(s)),8u(s))]ds.

1976]

OPTIMAL CONTROL THEORY

237

wecanspeakofa convexset
space.Therefore
insomefunction
Su arefunctions
Theperturbations
Equations(10.4)and(10.5)definea linearmap
andofa convexconeofperturbations.
ofperturbations
inan
ourattention
toperturbations
intoE"'. Hence,ifwerestrict
L fromthespaceofperturbations
the set of
convexcone 0 withvertexat the originin the space of perturbations,
appropriate
points84(t1)in En+1 willbe a convexconein En+1 withvertexat theorigin.Let X
corresponding
of thiscone by thevector4 (t1). Pointsin X are of theform
denotethetranslate
+(t1)+ &f(ti),

(10.6)

whereSo(t1) is givenby (10.4) and (10.5) forsome Su in 9l.

of
parallelto thexo axiswithinitialpointat 0(t1) andin thedirection
Let 2 denotethehalf-line
rather
trajectories
thenegativexo axis.Ifthepointsof X werethetrueendpointsoftheperturbed
valueof thezero-th
thensince40(t1)is theminimum
thereto,
thanthefirstorderapproximations
itwouldfollowthatno pointofX couldalso be a pointofY. Thuswe
ofall trajectories,
coordinate
wouldhavethatthetwoconvexconesX andY havea commonvertexandhaveno otherpointsin
fromthetrueendpoints
thepoints(10.6)differ
It turns
outthatthisis stilltrue,eventhough
common.
function
by higherorderterms.The proofof thisassertioninvolvestheuse of eithertheimplicit
theoremor of a fixedpointtheoremand willnotbe discussedhere.
is theircommonvertex0 (t1),theycan be
SinceX and Y are convexconeswhoseintersection
Hlthrough4(t1). We now considerthe analyticconsequencesof this
separatedby a hyperplane
separation.
Let toC r < t1andletSt> 0 be suchthatr + St< ti.Definea controlSu as follows:Su(t) = 0 fort
notin [r,r + at] and Su(t) = v - u(t) fort E [r,r + at], wherev is a fixedvectorin fQ.Thus we
in
We supposethat0Uis defined
thetrajectory
u(t) byv overa smallinterval.
byreplacing
perturb
to u + Su. Then
corresponding
sucha waythatSu E VU.Let f denotethetrajectory
+(t)= +(t)+ 6+(t)+ e(t),
"higherorderterms."Since
wheree(t) represents
( T+8t

q(T + at)
O

at)=

(r)+
(r) +

f(s, S(S),v)ds,

rT+8t

f(s, 4 (s), u(s))ds,

it followsthatforSt small,
(10.7)

&+(T + at) = [f(S, t(,),V)-tr,

4(r) uQr))]8t

at r.
and thatu is continuous
thatf is continuous
assuming
thatfor
+
SU (t) = u (t) fort 'r + St,itfollows
side
of
Since
u
(t)
hand
(10.7).
Let ( denotetheright
4
of
the
+
to
the
as
first
order
trajectory
we
can
consider
the
perturbation
approximation
t >- T St
So
if
we
solve
is
that obtained
= f(t,x,u(t))
to $(r + t) + e. It is thena consequenceof
withinitialconditionX(r + )t)=(r + &t)perturbed
ofdifferential
thedependenceofsolutions
equationson initialdatathatSo is a
theorems
describing
solutionof
(10.8)

dZ

(t,+ (t),U(t))Z

Z(T + at)

Note that( and z are vectorsin En+1 and thatf0= 0.

238

[April

L. D. BERKOVITZ

Let nibe a normalto thehyperplane

fl thatseparatesX and 2 andletn'pointintothehalf-space
Y. Then(&1(t1),fi) -0 We nowconstruct
through
thehyperplane
(r, +(r)) withnormal
containing
vectorA(T),whereA is thesolutionof
(10.9)

dw

n(tl)=

fw +(t

and theuse of (10.8) and (10.9) that(8+(t), A(t)) is constant.

bydifferentiation
It is readilyverified
n) _ 0, it followsthat
SinceA(t1)= n and since( n/(t1),
(

+ at), A(T + at)) _ 0.

of A we thengetthat
From(10.7) and fromthecontinuity

(6f(Tr,0 (Tr),V)f

(Tr,0 (Tr),U(T))],

A(Tr)+

0 (1))-

0,

whereo (1) is as St-- 0. Since St>0, it followson divisionby St that

(f(r,4 (r), v)f ((r,4 (r), u(r)), A(r)) _0.
forthe
principle
themaximum
and equation(10.9)withw replacedby A constitute
Thisinequality
problemin theformstudiedhere.
The assertionthatA0is constantfollowsfrom(10.9) and fromthefactthat
dftldx0,**, df /dx?)-0.
fjo = (df?/dx?,

multiples
of the E`
by O(t1) of all nonnegative
Since n = A(t1),sinceY is thetranslate
0
in
it
and
all
follows
that
since
for
y
Y,
(y,n
,0),
(-1,0,
1n s

vector

*, 0),(AO,A (t)

and so AO-=,--.
References
Academic Press,
1. R. Aris, The Optimal Design of Chemical Reactors; A Studyin Dynamic Programming,
New York, 1961.
2. K. J. Arrow,S. Karlin and H. Scarf,Studies in the MathematicalTheory of Inventoryand Production,
StanfordUniversityPress, Stanford,California,1958.
3. N. I. Akhiezer,The Calculus of Variations (trans.by A. H. Frink),Blaisdell, Waltham,Mass., 1962.
4. R. Bellman, I. Glicksberg,and 0. Gross, On the "bang-bang" controlproblem,Quart. Appl. Math., 14
(1956) 11-18.
J. Math. Anal. Appl., 3
5. L. D. Berkovitz,Variational methodsin problemsof controland programming,
(1961) 145-169.
New York, 1974.
, Optimal Control Theory,Springer-Verlag,
6.
7. G. A. Bliss, Lectures on the Calculus of Variations,The Universityof Chicago Press, Chicago, 1946.
8. D. Bushaw, Optimaldiscontinuousforcingterms,Contributionsto theTheoryof NonlinearOscillationsIV,
Annals of Math Study 41, S. Lefschetzed., PrincetonUniversityPress, Princeton,(1958) 29-52.
9. C. A. Desoer, The bang bang servoproblemtreatedby variationaltechniques,Informationand Control,2
(1959) 333-348.
10. W. Flemingand R. W. Rishell,Deterministicand StochasticOptimalControl,SpringerVerlag,New York,
Heidelberg,Berlin, 1975.
processesforlinearsystems,Izv. Akad. Nauk SSSR. Ser. Mat.,
11. R. V. Gamkrelidze,Theoryoftime-optimal
22 (1958) 449-474 (Russian).
12. R. V. Gamkrelidzeand G. L. Haratisvili,Extremalproblemsin lineartopologicalspaces, Izv. Akad. Nauk
SSSR, 33 (1969) 781-839.
13. I. V. Girsanov, Lectures on Mathematical Theory of ExtremumProblems, Springer-Verlag,Berlin,
Heidelberg,New York, 1967.
14. H. Halkin, An abstractframeworkfor the theoryof process optimization,Bull. Amer. Math. Soc., 72
(1966) 677-678.

19761

NOTE ON THE BERNOULLI-L'HOSPITAL

RULE

239

15. M. R. Hestenes, A generalproblemin the calculus of variationswithapplicationsto paths of least time,

RAND CorporationRM-100 (1950).
16.
, Calculus of Variations and Optimal Control Theory,Wiley, New York-London-Sydney,1966.
17. J. P. LaSalle, The time optimal controlproblem,Contributionsto the Theory of NonlinearOscillations
Vol. 5, Annals of Math. Study No. 45, PrincetonUniversityPress, Princeton(1960) 1-24.
18. E. B. Lee and L. Markus,Foundationsof Optimal Control Theory,Wiley,New York-London-Sydney,
1967.
19. G. Leitmann,On a class of variationalproblemsin rocketflight,J. Aero and Space Sciences, 26 (1959)
586-591.
20. J.L. Lions, OptimalControlof SystemsGovernedbyPartialDifferential
Equations, Springer-Verlag,
New
York, Heidelberg, Berlin 1971.
21. D. McDonald, Non lineartechniquesforimprovingservo performance,National ElectronicsConference,
6 (1950) 400-421.
22. E. J. McShane, On multipliersfor Lagrange problems,Amer. J. Math., 61 (1939) 809-819.
23. A. Miele, The calculus of variations in applied aerodynamics and flightmechanics, Optimization
TechniqueswithApplicationsto Aerospace Systems,G. Leitmanned., Academic Press,New York-London,1962.
24. L. W. Neustadt,A general theoryof extremals,J. Comput. SystemSci., 3 (1969) 57-92.
25.
, Optimization:A Theoryof NecessaryConditions,PrincetonUniversityPress,Princeton,N. J.,To
Appear.
26. R. Oldenburger,ed. Optimaland Self-Optimizing
Control,The M.I.T. Press,Cambridge,London, 1966.
27. L. S. Pontryagin,V. G. Boltyanskii,R. V. Gamkrelidze,E. F. Mishchenko,The MathematicalTheoryof
L. W. Neustadt,editor) Wiley,New York, 1962.
Optimal Processes, (Translated by K. N. Trirogoff,
28. F. A. Valentine, The problem of Lagrange with differentialinequalities as added side conditions,
Contributionsto the Calculus of Variations 1933-37, Department of Mathematics,Universityof Chicago,
Universityof Chicago Press, Chicago.
and FunctionalEquations, Academic Press,New York, 1972.
29. J.Warga,OptimalControlof Differential
30. L. C. Young, Generalized curves and the existenceof an attainedabsolute minimumin the calculus of
variations,Compt. Rend. Soc. Sci. et Lettres.Varsovie, Cl III, 30 (1937) 212-234.
, Lectures on the Calculus of Variations and Optimal Control Theory, Saunders, Philadelphia,
31.
London, Toronto, 1969.
DEPARTMENT OF MATHEMATICS, PURDUE UNIVERSITY, WEST LAFAYETTE, IN 47907.

NOTE ON THE BERNOULLI-L'HOSPITAL RULE

A. M. OSTROWSKI

1. The Bernoulli-L'Hospital
Rule on theconvergence
off(x)Ig(x) in thecase thatg(x )*oc can
as follows(See [2], 45-46; [3], 128-129):
be formulated
I. Consider
a fixedone of thefollowing
limiting
processes:
(1)

oo,

x txo,

xJ Xo,

wherexo is finite.
Assumethatf(x) and g(x), forthecorresponding
and have a
x, are continuous
Assumethatg'(x) iseither
derivative.
g(x) m0.Thenwehave
always < 0 oralways > 0 andthatJ
(2)

limf'(x)Ig'(x) -' limf(x)Ig(x) c limf'(x)Ig'(x).

has to be appliedin thecase wheref(x) or g(x) are definedas integrals,

the
If thisformulation
arisesthattherelation
difficulty
(f(x)dx

(x)