2012

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 53, NO. 9, OCTOBER 2008

Safe Adaptive Switching Control: Stability and Convergence

Margareta Stefanovic, Member, IEEE, and Michael G. Safonov, Fellow, IEEE
AbstractA formal theoretical explanation of the model-mismatch instability problem associated with certain adaptive control design schemes is proposed, and a solution is provided. To address the model-mismatch problem, a primary task of adaptive control is formulated as nding an asymptotically optimal, stabilizing controller, given the feasibility of adaptive control problem. A class of data-driven cost functions called cost-detectable is introduced that detect evidence of instability without reference to prior plant models or plant assumptions. The problem of designing adaptive systems that are robustly immune to mismatch instability problems is thus placed in a setting of a standard optimization problem. We call the result safe adaptive control because it robustly achieves adaptive stabilization goals whenever feasible, without prior assumptions on the plant model and, hence, without the risk of model-mismatch instability. The result improves the robustness of previous results in hysteresis switching control, both for discrete and for continuously-parameterized candidate controller sets. Examples are provided. Index TermsAdaptive control, convergence, robustness, stability, switching.

I. INTRODUCTION HE book Adaptive Control [2] begins in the following way: In everyday language, to adapt means to change a behavior to conform to new circumstances. Intuitively, an adaptive controller is thus a controller that can modify its behavior in response to changes in the dynamics of the plant and the character of the disturbances. Whether it is conventional, continuous adaptive tuning or more recent adaptive switching, adaptive control has an inherent property that it orders controllers based on evidence found in data. Any adaptive algorithm can thus be associated with a cost function, dependent on available data, that it minimizes, though this may not be explicitly present. The differences among adaptive algorithms arise in part due to the specic algorithms employed to approximately compute cost-minimizing controllers. And, major differences arise due to the extent to which additional assumptions are tied with this cost function. The cost function needs to be chosen to reect
Manuscript received February 6, 2007; revised January 30, 2008. Current version published October 8, 2008. This work was supported in part by the Air Force Ofce of Scientic Research under Grant F49620-01-1-0302. Recommended by Associate Editor J. Hespanha. M. Stefanovic is with the Department of Electrical and Computer Engineering, University of Wyoming, Laramie, WY 82071 USA (e-mail: mstefano@uwyo.edu). M. G. Safonov is with the Department of Electrical Engineering-Systems, University of Southern California, Los Angeles, CA 90089-2563 USA (e-mail: msafonov@usc.edu). Color versions of one or more of the gures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TAC.2008.929395

control goals. The perspective adopted in this paper hinges on the notion of feasibility of adaptive control. An adaptive control problem is said to be feasible if the plant is stabilizable and at least one (a priori unknown) stabilizing controller exists in the candidate controller set that achieves the specied control goal for the given plant. Given feasibility, our view of a primary goal of adaptive control is to recognize when the accumulated experimental data shows that a controller fails to achieve desired stability and performance objectives. If a destabilizing controller happens to be the currently active one, adaptive control should eventually switch it out of the loop, and replace it with an optimal, stabilizing one. An optimal controller is one that optimizes the controller ordering criterion (cost function) given the currently available evidence. This perspective renders the adaptive control problem in a form of a standard constrained optimization. A concept similar to this feasibility notion can be found in [15]. To address the emerging need for robustness for larger uncertainties or achieve tighter performance specications, several recent important advances have emerged, such as [13] and multi-model controller switching formulations of the adaptive control problem, e.g., supervisory based control design in [8], [10], [18], and [19], or data-driven unfalsied adaptive control methods of [23] (based on criteria of falsiability [20], [31], [32]), which exploit evidence in the plant output data to switch a controller out of the loop when the evidence proves that the controller is failing to achieve the stated goal. In both cases, the outer supervisory loop introduced to the baseline adaptive system allows fast discontinuous adaptation in highly uncertain nonlinear systems, and thus leads to improved performance and overcomes some limitations of classical adaptive control. These formulations have led to improved optimization-based adaptive control theories and, most importantly, signicantly weaker assumptions of prior knowledge. Both indirect [18], [19], [33] and direct [6], [15], [33] switching methods have been proposed for the adaptive supervisory loop. Recently, performance issues in switching adaptive control have been addressed in robust multiple model adaptive control schemes (RMMAC) [3]. The results of this paper build on the result of Morse et al. [17], [18] and Hespanha et al. [9][11], and widen the theoretical ground in our paper [25] by allowing the class of candidate controllers to be innite so as to allow consideration of continuously parameterized adaptive controllers, in addition to nite sets of controllers. It is shown that, under some mild additional assumptions on the cost function (designer-based, not plant-dependent), stability of the closed-loop switched system is assured, as well as the convergence to a stabilizing controller in nitely many steps. We show, via an example, that, when

0018-9286/$25.00 2008 IEEE

STEFANOVIC AND SAFONOV: SAFE ADAPTIVE SWITCHING CONTROL: STABILITY AND CONVERGENCE

2013

^ Fig. 1. Switching adaptive control system 6(K

;P

).

there is a mismatch between the true plant and the assumptions on the plant (priors), a wrong ordering of the controllers can in some cases give preference to destabilizing controllers. This phenomenon is called model mismatch instability. However, prior knowledge and plant models, when they can be found, can be incorporated into the design of the candidate controllers, which could be used together with the safe switching algorithm. In certain ways, the paradigm of adaptive control problem cast as a constrained optimization problem bears similarities with the ideas found in machine learning algorithms [16], [26]. As a direct, data-driven switching adaptive method, it is more similar to the reinforcement learning algorithms than supervised/unsupervised learning. In reinforcement learning, the algorithm learns a policy of how to act given an observation of the world. Every action has some impact in the environment, and the environment provides feedback that guides the learning algorithm. The organization of the paper is as follows. Preliminary facts are given in Section II. Section III contains the main results as well as an example of the cost function satisfying sufcient conditions for stability and niteness of switches. An illustrative example is presented in Section IV. Section V provides some nal conclusions and directions for future work. II. PRELIMINARIES Consider the adaptive system shown in Fig. 1, where and are the plant input and output vector sigis the linear vector space of functions whose nals, and norm, dened as , exists for any nite . For any , a truncation operator is a linear projection operator that truncates the signal at . will be used for the truncated signal [22]. The symbol The adaptive controller switches the currently active controller at times with . For brevity, the controller switched in the loop we also denote . If nite, the total number during the time interval of switches is denoted by , so the nal switching time is and the nal controller is . We dene the set where is an unknown plant. Unknown disturbances and noises may also affect the plant relation . Let represent the output data signals measured in one experiment, dened on the time-interval .

We consider a possibly innite set (e.g., containing a continuum) of the candidate controllers. The nite controller set case is included as a special case. The parametrization of , denoted , will initially be taken to be a subset of ; the treatment of the innite dimensional spaces will be discussed in the Comment 5. Recall some familiar denitions from the stability theory. A belongs in class if is function . The -norm of continuous, strictly increasing and is given as . a truncated signal The Euclidean norm of the parameterization of the controller is denoted . A functional is said when . to be coercive [4] if with input and Denition 1: A system output is said to be stable if for every such input there exist constants such that (1) Otherwise, is said to be unstable. Furthermore, if (1) holds for all , then the system with a single pair is said to be nite-gain stable, in which case the gain of is the least such . Comment 1: In general, can depend on the initial state. Specializing to the system in Fig. 1, and (without loss of gen, stability of the closed-loop system erality) disregarding means that for every , there exist such that . Denition 2: The system is said to be incrementally stable and outputs if, for every pair of inputs , there exist constants such that (2) and the incremental gain of , when it exists, is the least satisfying (2) for some and all . Denition 3: The adaptive control problem is said to be feacontains at least one consible if a candidate controller set troller such that the system is stable. A controller is said to be a feasible controller if the system is stable. Safe adaptive control problem goal is then formulated as nding an asymptotically optimal, stabilizing controller, given the feasibility of the adaptive control problem. Under this condition, safe adaptive control should recognize a destabilizing controller currently present in the loop, and replace it with an as-yet-unfalsied controller. Hence, we have the following (and only) assumption on the plant that we will use in this paper. Assumption 1: The adaptive control problem is feasible. A similar notion of the safe adaptive control appears in [1], where Anderson et al. dene the safe adaptive switching control as one that always yields stable frozen closed-loop; the solution to this problem is achieved using the -gap metric. Limitations of the -gap metric are discussed in [12]. Prevention from inserting a destabilizing controller in the loop is not assured (since the adaptive switching system cannot identify with certainty a destabilizing controller beforehand, based on the past data), but if such a controller is selected, it will quickly be switched out as soon as the unstable modes

2014

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 53, NO. 9, OCTOBER 2008

are excited. Under the feasibility condition, an unfalsied controller will always be found and placed in the loop. Whether the optimal, robustly stabilizing controller will eventually be found and connected, depends on whether the unstable modes are sufciently excited. It follows from the above denition of the adaptive control goal that feasibility is a necessary condition for the existence that robustly solves the safe adaptive of a particular control problem. This paper aims to show that feasibility is also a sufcient condition to design a robustly stable adaptive system , even when it is not known a priori that converges to a which controllers in the set are stabilizing. is said to Denition 4: Stability of the system if there exist such be unfalsied by the data that (1) holds; otherwise, we say that stability of the system is falsied by . Unfalsied stability is determined from (1) based on the data from one experiment for one input, while stability requires additionally that (1) hold for the data from every possible input. Any adaptive control scheme has a cost index inherently tied to it, which orders controllers based on evidence found in data. , deThis index is taken here to be a cost functional ned as a causal-in-time mapping (3) An example of the cost function according to the above denition, which satises the desired properties introduced later in the text, is given in Section III-A. Various examples of the cost are discussed in [24], [29], among others. The switched system comprised of the plant and the cur, where is derently active controller (Fig. 1). For the switched system in noted Fig. 1, the is dened as , where . , a controller Denition 5: Given the pair is said to be falsied at time by the past measurement if . Otherwise it is said to be unfalsied at time by . is one that stabilizes Then, a robust optimal controller (in the sense of the Def. 1) the given plant and minimizes the . true cost (and is not necTherefore, essarily unique). Due to the feasibility assumption, at least one exists, and . such , a ctitious reference Denition 6: [23] For every is dened to be an element of signal

tally stable, is called stably causally left invertible controller (SCLI). Denition 8: Let denote the input and denote the resulting plant data collected with as the current of controller. Consider the adaptive control system is said to Fig. 1 with input and output . The pair be cost-detectable if, without any assumption on the plant and for every with nitely many switching times, the following statements are equivalent: is bounded as increases to innity; 1) is unfalsied by the input2) stability of the system output pair . Comment 2: With cost-detectability satised, we can use the to reliably detect any instability exhibited by cost the adaptive system, even when initially the plant is completely unknown. Comment 3: Cost-detectability is different from the plant detectability. Cost-detectability is determined from the knowledge of the cost function and candidate controllers, without reference to the plant. In [9], a problem similar to ours is approached using the following assumptions: 1) the plant itself is detectable and 2) the candidate plant models are stabilized by the corresponding candidate controllers. The difference between the approach in [9], [7] and this paper lies in the denition of cost-detectability introduced in this paper, which is the property of the cost function/candidate-controller-set pair, but is independent of the plant. In the following, we use the notation for a family of functionals . with the common domain , with Let denote the level set in the controller space corresponding to the cost at the rst switching time instant. With the family of functionals with a common domain , a restriction to the is associated, dened as a family of functionals set with a common domain . is identical to on , and is equal to Thus, outside . Consider now the cost minimization hysteresis switching algorithm reported in [17], together with the cost functional . The algorithm returns, at each , a controller which is the active controller in the loop: -HYSTERESIS SWITCHING ALGORITHM A1 [17]

In other words, is a hypothetical reference signal that would have exactly reproduced the measured data had the been in the loop for the entire time period over controller was collected. which the data Denition 7: [28] When for each and there is a unique , then we say is causally left invertible (CLI) the induced causal map . The and we denote by is called the ctitious reference signal causal left inverse is incremengenerator (FRS) for the controller . When

where is the Kroneckers , and is the limit of from . below as The switch occurs only when the current unfalsied cost related to the currently active controller exceeds the minimum (over ) of the current unfalsied cost by at least (Fig. 2). The hysteresis step serves to limit the number of switches on any nite time interval to a nite number, and so prevents the possibility of the limit cycle type of instability. It also ensures a nonzero dwell time between switches. The hysteresis switching lemma of [17] implies that a that minswitched sequence of controllers at each imize (over ) the current unfalsied cost

STEFANOVIC AND SAFONOV: SAFE ADAPTIVE SWITCHING CONTROL: STABILITY AND CONVERGENCE

2015

Proof: It sufces to consider the nal controller . Denote the nal switching time instant . Then, by the denition , and feasibility of the control problem (Def. 3), it of follows that for all (4) (5) Further, by monotonicity in of , it follows that (5) holds for all . Due to the cost-detectability, stability of with is not falsied by , that is, there exist constants corresponding to the given such that
Fig. 2. Cost versus control gain time snapshots.

(6) switch-time , will also stabilize the plant if the cost related to has the following properties: rst, it is each xed controller a monotone increasing function of time and second, it is uniformly bounded above if and only if is stabilizing. But, these properties were demonstrated for the cost functions in [17] only by introducing prior assumptions on the plant, thereby also introducing the possibility of model-mismatch instability. Denition 9: [30]. Let be a topological space. A family of complex functionals with a common if for domain is said to be equicontinuous at a point there exists an open neighborhood such that every . The family is said to be equicontinuous on if it is equicontinuous at each . is said to be uniformly equicontinuous on if such that , where denotes an open neighborhood of size . In a metric space with a metric , uniform equicontinuity means that . is a compact metric space, then any family Lemma 1: If that is equicontinuous on is uniformly equicontinuous on . Proof: In [21]. III. MAIN RESULT The results on stability and niteness of switches are developed in the sequel. Lemma 2: Consider the feedback adaptive control system in Fig. 1 with input (generality is not lost if is taken instead of the input ) and output , together with the hysteresis switching algorithm A1. Suppose there are nitely many switches. If the adaptive control problem is feasible (Def. 3), candidate controllers are SCLI, and the following properties are satised: is cost-detectable (Def. 8) is monotone increasing in time then the nal switched controller is stabilizing. Moreover, the system response with the nal controller satises the performance inequality According to Lemma 6 in Appendix A, there exist such that . This, along with (6), implies , for some . Lemma 3: Let be a continuous and coercive , the level set function on . Then for any scalar is compact. Proof: Since , we show that is closed and be a convergent sequence, and bounded: Let . Since is continuous, . . Then, Also, , so . Hence, is closed. To is bounded, proceed by contradiction. Asshow that is is not bounded; then there exists a sequence sume that such that . Since is coercive, ; in particular, such that , for any xed . Then, , is closed which contradicts the above assumption. Thus, and bounded in , therefore compact. Lemma 4: Consider the feedback-adaptive control system in Fig. 1, together with the switching algorithm A1. If the adaptive control problem is feasible (Def. 3), and the associated is cost-detectable, cost functional/controller set pair is monotone increasing in time and, in addition: , the cost functional is For all coercive on (i.e., ), and The family of restricted cost functionals with a common domain is equicontinuous on , then the number of switches is uniformly bounded above for all by some . Proof: Our proof is similar to the convergence lemmas of [10], [17]. A graphical representation of the switching process, giving insight to the derivation presented below, is shown in is compact. Then, Fig. 3. Due to Lemma 3, the level set is uniformly the family equicontinuous on (Lemma 1), i.e., for a hysteresis step such that for all (i.e., is common to all and all ). Since is , compact, there exists a nite open cover such that , with

2016

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 53, NO. 9, OCTOBER 2008

Fig. 3. Derivation of the upper bound on switches in the continuum

K case.

Equicontinuity assures that the cost functionals in the said family have associated -balls of nite, non-zero radii, which is holds, used to upper bound the number of switches. If the set is compact; otherwise, an additional requirement that is compact is needed. The nite controller set case the set is obtained as a special case of the Lemma 4, with being the number of candidate controllers instead of the number of -balls in the cover of . The main result follows. Theorem 1: Consider the feedback adaptive control system in Fig. 1, together with the hysteresis switching algorithm A1. Suppose that the adaptive control problem is feasible, the assois monotone in time, the pair ciated cost functional is cost-detectable, candidate controllers are SCLI, and the conditions of Lemma 4 hold. Then, the switched closed-loop system is stable, according to Def. 1. In addition, for each , the system converges after nitely many switches to the controller that satises the performance inequality (7) Proof: Invoking Lemma 4 proves that there are nitely many switches. Then, Lemma 2 shows that the adaptive controller stabilizes, according to Def. 1, and that (7) holds. Comment 4: Note that, due to the coerciveness of is bounded below (by a nonnegative number, if the range of is a subset of ), for all . Comment 5: The parametrization of the candidate controller ; in fact, it can belong set can be more general than to an arbitrary innite dimensional space; however has to be compact in that case, in order to ensure uniform equi-continuity property. Note that the switching ceases after nitely many steps for all . If the system input is sufciently rich so as to increase the cost more than above the level at the time of the latest switch, a switch to a new controller that minimizes the current cost will eventually occur at some later time. The values of these cost minima at any time are monotone increasing and bounded . Thus, sufcient richness of the system above by input (external reference signal, disturbance or noise signals) . will affect the cost to approach Comment 6: The minimization of the cost functional over the innite set is tractable if the compact set can be represented as a nite union of convex sets, i.e., the cost minimization is a convex programming problem. A. Cost Function Example

where depends on the chosen hysteresis step (this is a direct consequence of the denition of a compact set). Let be the controller switched into the loop at the time , and the corresponding minimum cost achieved is . a switch occurs at the Consider that at the time same cost level , i.e., where . Therefore, is falsied, and so are all the controllers . Let be the index set of the as-yet-unfalsied -balls of controllers , for some at the time . Since ( is not necessarily a singleton as may belong to more , but it sufces for the proof that there than one balls is at least one such index ), also falsied are all the controllers , so that . Thus, is updated according to the following algorithm ( is the index of the switching time ). Unfalsied index set algorithm: 1) Initialize: Let 2) . If : Set //

Optimal cost increases Else . 3) go to (2) Therefore, the number of possible switches to a single cost level is upper-bounded by , the number of -balls in the cover of . The next switch (the very rst after the th one), if any, must occur to a cost level higher than , due to the monotonicity of . But then, according , to Algorithm 1, with and . Combining the two bounds, the overall number of switches is upper-bounded by ,where is such that

An example of the cost function and the conditions under which it ensures stability and niteness of switches according to Theorem 1 may be constructed as follows. Consider (a not in necessarily zero-input zero-output) system Fig. 1. Choose as a cost functional (8) are arbitrary positive numbers. The constant is where (unless used to prevent division by zero when has zero-input zero-output property), ensures even , and ensures coerciveness of . when

STEFANOVIC AND SAFONOV: SAFE ADAPTIVE SWITCHING CONTROL: STABILITY AND CONVERGENCE

2017

Fig. 4. MFD of a controller in closed loop with the plant [14]: (a) feedback loop ^ with the current controller K written in MFD form and (b) ctitious feedback loop associated with the candidate controller K written in MFD form (in both cases, (u; y ) are the actually recorded data).

Alternatively, in order to avoid the restriction to the minimum phase (SCLI) controllers (which would assure causality and in), the denominator of cremental stability of the map instead of [14], [5], where is dened (8) can contain via the matrix fraction description (MFD) form of the controller , as and (Appendix B), where the mentioned signals are shown in Fig. 4 (9) Both (8) and (9) satisfy the required properties of Theorem 2, i.e.,: monotonicity in time, coerciveness on , equicontinuity of the restricted cost family , and cost detectability. The rst two properties are evident by inspection of (8) and (9). The justication for the last two properties is as follows. in (8) and (9) is continuous in , then , Since restricted dened as the family of the cost functionals to the level set , is equiconare either equal to , or tinuous (since for any and ). clamped at Lemma 5: Consider the cost functions (8) and (9) with . For to be cost-detectable, it is sufcient are SCLI, or that that the candidate controllers in the set they admit matrix fraction description (MFD) form considered in [14]. with in (8) follows Proof: Cost-detectability of is bounded as from the following: 1) the fact that if and only if stability is unfalsied by the input-output [28]; 2) SCLI property of the controllers; 3) stapair bility of the mapping (Lemma 6 in Appendix A); and (see Appendix C). 4) unfalsied stability by the data These results can be elaborated further using [14] for the class of non-SCLI controllers and the cost function (9), which also ensure internal stability of the adaptive system designed using cost-detectable cost-functions of the forms (8) or (9). IV. SIMULATION EXAMPLE The algorithm A1 in Section II originated as the hysteresis switching algorithm in [17]. We emphasized that the power of the hysteresis switching lemma was clouded in the cited work by

imposing unnecessary assumptions on the plant in the demonstrations of the algorithm functionality. One of the plant properties required in [17] for ensuring tunability was the minimum phase property of the plant. We have shown in theory that the cost detectability is assured by properly choosing a cost function, and is not dependent on the plant or exogenous signals. In the following, we present a simulation example that demonstrates these ndings. Assume that a true, unknown plant . transfer function is given by Given is the set of three candidate controllers: and , each of which stabilizes a different possible plant model. The task of the adaptive control is to select one of these controllers, based on observed data. The problem is complicated by the fact that in is this case, as is often the case in practice, the true plant not in the model set, i.e., there exists a model mismatch. A simple analysis of the nonswitched system (true plant in feedback with each of the controllers separately) shows that is stabilizing (yielding a nonminimum phase but stable closed and are destabilizing. Next, a simulation was loop) while performed of a switched system, where A1 was used to select the optimal controller, and the cost function was chosen to be a combination of the instantaneous error and a weighted accumulated error (10) where is the ctitious error of the th controller, dened as (11) and and , where is a stable, minimum phase reference model. This is the same cost function used in the multiple model switching adaptive control scheme [19], with replaced by , the identication error of the th plant model [for the special case of the candidate conis equivalent trollers designed based on the MRAC method, to the control error and to the ctitious error (11)]. The simulations assume a band-limited white noise at the plant output and the unit-magnitude square reference signal. The has initially been placed in the loop, stabilizing controller and the switching, which would normally occur as soon as the logic condition of Algorithm A1 is met, is suppressed during the initial 5 s of the simulation. That is, the adaptive control loop is s. The reason for waiting some penot closed until time riod of time before engaging the switching algorithm will be explained shortly. The forgetting factor is chosen to be 0.05. Figs. 5(a) and 6(a) show the cost dynamics and the reference and plant outputs, respectively. ), the alSoon after the switching was allowed (after gorithm using cost function (10) discarded the stabilizing controller initially placed into the loop and latched onto a destabilizing one, despite the evidence of instability found in data. Even though the stabilizing controller was initially placed in the loop, and forced to stay there for some time (5 s in this case), as soon as the switching according to (10) was allowed, it promptly switched to a destabilizing one. This model-match instability happens because the cost function (10) is not cost-detectable.

2018

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 53, NO. 9, OCTOBER 2008

Fig. 6. Switching using cost function (a) (10) and (b) (12). Reference and plant outputs. (a) Non-cost-detectable case; (10), (b) Cost-detectable case; (10). Fig. 5. Without cost-detectability, the optimal cost-minimizing controller destabilizes the system. (a) Non-cost-detectable case; cost (10). (b) Cost-detectable case; cost (12).

Note that the initial idle period of 5s is used only to emphasize that, even when the data are accumulated with the stabilizing controller, the switching algorithm based on (10) can disregard this data, and latch onto a destabilizing controller. This idle period is not the dwell time in the same sense used in the dwell-time switching control literature. Next, a simulation was performed of the same system, but using a cost-detectable cost function (i.e., one that satises conditions of Theorem 1) (12) viz., an -gain type cost function (factor added for coerciveness is not necessary, since the set of candidate controllers follows is nite in this example). Cost-detectability of is from Lemma 5. The modied ctitious reference signal used, instead of , due to the presence of the non-minimum

. It is calculated from the on-line data as [according to (9)], where and for ; and for ; and and for . The corresponding simulation results are shown in Figs. 5(b) (a destaand 6(b). The initial controller was chosen to be bilizing one). The constant was chosen to be 0.01. The foregoing example shows that closing an adaptive loop that is designed using a non-cost-detectable cost function like (10) can destabilize, even when the initial controller is stabilizing. In the example, this happens because there is a large mismatch between the true plant and the plant models used to design the candidate controllers. On the other hand, Theorem 1 implies, and the example conrms, that such model-mismatch instability cannot occur when the adaptive control loop is designed using -gain type cost function (12). the cost-detectable V. CONCLUSION The goal of stabilizing an uncertain plant by means of switching through an innite candidate controller set is solved in the paper, provided that feasibility (existence of at least

phase controller

STEFANOVIC AND SAFONOV: SAFE ADAPTIVE SWITCHING CONTROL: STABILITY AND CONVERGENCE

2019

Fig. 7. Generators of the true and ctitious reference signals.

one stabilizing solution in the candidate controller set) holds. Noting that every adaptive scheme is optimal with respect to some data-driven controller-ordering cost function, sufcient conditions are derived on the data-driven cost function to ensure stability and performance. To this end, we have re-examined the Morse-Mayne-Goodwin hysteresis algorithm for adaptive control from the perspective of unfalsied control theory. An upper bound on the number of switches for a general continuum controller set case is calculated. The result is a solution to the problem of model mismatch instability. Our cost- detectability approach to the adaptive controller switching robustly eliminates the possibility of model-mismatch instability that inevitably accompanies the introduction of any prior assumption on the plant. It does so by replacing the plant-dependent/controllerindependent tunability requirements with the plant-independent/ controller-dependent cost-detectability requirements. In the present paper, our focus has been on how a given candidate controller set should be pruned based on data in order to adaptively converge to a controller in the candidate set that achieves and maintains stable behavior and acceptable performance. Though this data driven process requires no prior knowledge of the plant model, this does not mean that our formulation is model-free. Plant models, when available, play an essential role in determining and designing the candidate controller set . The issue of how one might optimally utilize data for continuously and adaptively generating new candidate controllers on the y remains an open question. APPENDIX A RELATION BETWEEN AND Lemma 6: Consider the switching feedback adaptive control system (Fig. 1), where uniformly bounded reference input , as well as the output are given. Suppose there and denote the nal are nitely many switches. Let switching instant and the nal switched controller, respectively. is SCLI (i.e., the ctitious Suppose that the nal controller reference signal is unique and incrementally stable). Then (13) Proof: By the assumption there are nitely many switches. Consider the control conguration in Fig. 7. The top branch gen-

. Its inerates the ctitious reference signal of the controller , and its output is . The puts are the measured data output is generated by the ctitious reference signal generator , denoted . In the middle interconnecfor the controller , generated as the output of the nal controller tion, the signal excited by the actual applied signals and , is simply in. Finally, verted by passing through the causal left inverse the bottom interconnection has the identical structure as the top and ), except that interconnection (series connection of it should generate the actually applied reference signal . To this end, another input to the bottom interconnection is added (denoted ), as shown in Fig. 7. This additional input can be thought of as a compensating (bias) signal, that accounts for the before difference between the subsystems generating and the time of the last switch. In particular, it can be shown (as seen (due to the fact that in Fig. 7) that ). is incrementally stable. Thus, there exist By denition, such that constants

Whence by the triangle inequality for norms, inequality (13) holds with

APPENDIX B is a left maDenition 10: [27]. The ordered pair and trix fraction description (MFD) of a controller if are stable, is invertible, and . Comment 7: To avoid restricting our attention to only those controllers that are stably causally left invertible. i.e., controllers whose FRS generator is stable, we can use the MFD representation of the controllers and write a modied ctitious reference and [14]. signal as would represent the modied applied refSimilarly, erence signal , that is related to the active controller in the loop . Thus, although may not be stable, which is the case with is, by construction. nonminimum phase controllers, APPENDIX C UNFALSIFIED STABILITY VERIFICATION FOR THE COST FUNCTION (8) IN SECTION III-A Recall that the controller switched in the loop at time is are the denoted ; and indices of the switching instants. When , we have . Let the controller switched (so, ). Then, due to the at time be denoted cost minimization property of the switching algorithm, , and . the time of the nal switch, and the corDenote by . Consider the time interval . responding controller

2020

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 53, NO. 9, OCTOBER 2008

During this time period, the active controller in the loop is

where is the nal switching time. Since switching time

was the nal

since . Since r is uniformly bounded, . , the cost of the current controller exceeds the curAt rent minimum by

Thus,

is nite for any nite

(14) and so, according to the hysteresis switching algorithm, a switch . Expresoccurs to the controller sion in (14) is nite since is nite and From here we conclude that stability of the closed-loop switched is unfalsied by . system with the nal controller where Denoting the sum is nite due to the feasibility assumption. by , we have ACKNOWLEDGMENT The authors would like to thank Prof. J. Hespanha for insightful comments and referral to the hysteresis switching lemma of Morse et al. [17] and the related references. They also thank anonymous reviewers for their valuable comments that helped improve the manuscript quality. REFERENCES
[1] B. D. O. Anderson, T. S. Brinsmead, D. Liberzon, and A. S. Morse, Multiple model adaptive control with safe switching, Int. J. Adapt. Control and Signal Process., vol. 15, pp. 445470, 2001. [2] K. J. strm and B. Wittenmark, Adaptive Control. Reading, MA: Addison-Wesley, 1995. [3] M. Athans et al., Issues, progress and new results in robust adaptive control, Int. J. Adapt. Control Signal Process., vol. 20, pp. 519579, Dec. 2006. [4] D. P. Bertsekas, Nonlinear Programming, 2nd ed. Nashua, NH: Athena, 1999. [5] A. Dehghani, B. D. O. Anderson, and A. Lanzon, Unfalsied adaptive control: A new controller implementation and some remarks, in European Control Conf. ECC07, Kos, Greece, Jul. 2007. [6] M. Fu and B. R. Barmish, Adaptive stabilization of linear systems via switching control, IEEE Trans. Automat. Control, vol. AC-31, no. 12, pp. 10971103, Dec. 1986. [7] J. P. Hespanha and A. S. Morse, Certainty equivalence implies detectability, Syst. Contr. Lettr., vol. 1, no. 1, pp. 113, Jan. 1999. [8] J. P. Hespanha, Logic-Based Switching Algorithms in Control, Ph.D. dissertation, Dept. Elect. Eng., Yale University, New Haven, CT, 1998. [9] J. P. Hespanha, D. Liberzon, and A. S. Morse, Supervision of integralinput-to-state stabilizing controllers, Automatica, vol. 38, no. 8, pp. 13271335, Aug. 2002. [10] J. P. Hespanha, D. Liberzon, and A. S. Morse, Hysteresis-based switching algorithms for supervisory control of uncertain systems, Automatica, vol. 39, no. 2, Feb. 2003. [11] J. P. Hespanha, D. Liberzon, and A. S. Morse, Overcoming the limitations of adaptive control by means of logic-based switching, Syst. Control Lett., vol. 49, pp. 4965, 2003. [12] G. C. Hsieh and M. G. Safonov, Conservatism of the gap metric, IEEE Trans. Automat. Control, vol. 38, no. 4, pp. 594598, Apr. 1993. [13] E. Kosmatopoulos and P. A. Ioannou, Robust switching adaptive control of multi-input nonlinear systems, IEEE Trans. Automat. Control, vol. 47, no. 4, pp. 610624, Apr. 2002.

Now consider the next switching period, . We have: troller in the loop is

. The active con-

where the second inequality from the left follows from the monotone increasing property of . Therefore, . Now

Thus, is nite, and so are we conclude that

. By induction,

STEFANOVIC AND SAFONOV: SAFE ADAPTIVE SWITCHING CONTROL: STABILITY AND CONVERGENCE

2021

[14] C. Manuelli, S. G. Cheong, E. Mosca, and M. G. Safonov, Stability of unfalsied adaptive control with non SCLI controllers and related performance under different prior knowledge, in Proc. European Control Conf. (ECC07), Kos, Greece, Jul. 2007. [15] B. Mrtensson, The order of any stabilizing regulator is sufcient information for adaptive stabilization, Syst. Control Lettr., vol. 6, no. 2, pp. 8791, 1985. [16] T. Mitchell, Machine Learning. New York: McGraw-Hill, 1997. [17] A. S. Morse, D. Q. Mayne, and G. C. Goodwin, Applications of hysteresis switching in parameter adaptive control, IEEE Trans. Automat. Control, vol. 37, no. 9, pp. 13431354, Sep. 1992. [18] A. S. Morse, Supervisory control of families of linear set-point controllersPart I: Exact matching, IEEE Trans. Automat. Control, vol. 41, no. 10, pp. 14131431, Oct. 1996. [19] K. S. Narendra and J. Balakrishnan, Adaptive control using multiple models, IEEE Trans. Automat. Control, vol. 42, no. 2, pp. 171187, Feb. 1997. [20] K. R. Popper, Logic of Scientic Discovery 1934. [21] W. Rudin, Principles of Mathematical Analysis, 3rd ed. New York: McGraw-Hill, 1976. [22] M. G. Safonov, Stability and Robustness in Multivariable Feedback Systems. Cambridge, MA: MIT Press, 1980, Signal Processing, Optimization and Control. [23] M. G. Safonov and T. Tsao, The unfalsied control concept and learning, IEEE Trans. Automat. Control, vol. 42, no. 6, pp. 843847, Jun. 1997. [24] M. Stefanovic, Safe Switching Adaptive Control: Stability and Convergence, Ph.D. dissertation, Univ. Southern California, Los Angeles, 2005. [25] M. Stefanovic, R. Wang, and M. G. Safonov, Stability and convergence in adaptive systems, in Proc. American Control Conf., Boston, MA, 2004. [26] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. [27] M. Vidyasagar, Control System Synthesis: A Factorization Approach. Cambridge, MA: The MIT Press, Sep. 1988. [28] R. Wang, A. Paul, M. Stefanovic, and M. G. Safonov, Cost-detectability and stability of adaptive control systems, in Proc. 44th IEEE Conf. Decision and Control, Seville, Spain, Dec. 2005. [29] R. Wang, Cost Detectability and Safe MCAC, Ph.D. dissertation, Univ. Southern California, Los Angeles, 2005. [30] R. L. Wheeden and A. Zygmund, Measure and Integral: An Introduction to Real Analysis. New York: Marcel Dekker, 1977. [31] J. C. Willems, Paradigms and puzzles in the theory of dynamical systems, IEEE Trans. Automat. Control, vol. 36, no. 3, pp. 259294, Mar. 1991. [32] G. Zames, On the input-output stability of nonlinear time-varying feedback systems, part 1: Conditions derived using concepts of loop gain, conicity and positivity, IEEE Trans. Automat. Control, vol. AC-11, pp. 465476, Apr. 1966. [33] P. Zhivoglyadov, R. H. Middleton, and M. Fu, Localization based switching adaptive control for time-varying discrete-time systems, IEEE Trans. Automat. Control, vol. 45, no. 4, pp. 5275, Apr. 2000.

Margareta Stefanovic (S03M05) received the Dipl.-Ing. degree from the University of Nis, Yugoslavia, in 1996, and the M.Sc. and Ph.D. degrees in electrical engineering from the University of Southern California, Los Angeles, in 2002 and 2005, respectively. She joined the University of Wyoming, Laramie, in 2005 as an Assistant Professor in the Electrical and Computer Engineering Department. Her current research interests include switching adaptive control of uncertain systems and coordinated control and decision making in hyper-spectral imaging nano-satellite networks for Space Situational Awareness.

Michael G. Safonov (F89) was born in Pasadena, CA, on November 1, 1948. He received the B.S., M.S., Eng., and Ph.D. degrees in electrical engineering from the Massachusetts Institute of Technology, Cambridge, in 1971, 1971, 1976, and 1977, respectively. From 1972 to 1975, he served with the U.S. Navy as an Electronics Division Ofcer aboard the aircraft carrier U.S.S. Franklin D. Roosevelt (CVA-42). Since 1977, he has been with the University of Southern California, Los Angeles, where he is presently a Professor of Electrical Engineering. He has been a Consultant to The Analytic Sciences Corp., Honeywell Systems and Research Center, Systems Control, Systems Control Technology, Scientic Systems, United Technologies, TRW, Northrop Aircraft, Hughes Aircraft, and others. His consulting and university research activities have involved ight control system design studies in which modern robust multivariable control techniques were applied to a variety of aircraft including the CH-47 Chinook helicopter (Analytic Sciences Corp., 1976), the NASA HiMAT aircraft (Honeywell/USC, 1980) and the F/A-18 Hornet (Northrop, 19871991). During the academic year 19831984, he was a Senior Visiting Fellow with the Department of Engineering, Cambridge University, Cambridge, U.K., and in summer 1987 he held a similar appointment at Imperial College of Science and Technology, London, U.K. During 19901991, he was a Visiting Faculty Member at Caltech, Pasadena, CA. He has authored or co-authored more than 150 journal and conference papers and the book Stability and Robustness of Multivariable Feedback Systems (Cambridge, MA: MIT Press, 1980). Additionally, he is co-author of the MATLAB Robust Control Toolbox (Natick, MA: MathWorks), a software package for use with MATLAB. He is presently an Editor of the International Journal of Robust and Nonlinear Control and Systems and Control Letters. His research interests include robust control, innity-norm optimal control theory, and nonlinear system theory with applications to aerospace control design problems. Dr. Safonov served as an Associate Editor of the IEEE TRANSACTIONS ON AUTOMATIC CONTROL from 1985 to 1987. He was elected IFAC Fellow in 2008. From 1993 to 1995, he was Chair of the AACC Awards Committee of the American Automatic Control Council.