Asian Journal of Control, Vol. 13, No. 1, pp.

3 14, January 2011 Published online 3 August 2010 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/asjc.261 A FRAMEWORK FOR ANALYSIS OF OBSERVER-BASED ILC Johanna Wallen, Mikael Norrl .. of, and Svante Gunnarsson ABSTRACT A framework for iterative learning control (ILC) is proposed for the situation when an ILC algorithm uses an estimate of the controlled variable obtained from an observer-based estimation procedure. Assuming that the ILC input converges to a bounded signal, a general expression for the asymptotic error of the controlled variable is given. The asymptotic error is exemplified by an ILC algorithm applied to a flexible two-mass model of a robot joint. Key Words: Iterative learning control, framework, estimate, asymptotic, controlled variable. I. INTRODUCTION The idea in iterative learning control (ILC) is to compensate for errors when the system performs a repetitive motion. The first publications [13] were published in 1984, and in [4] a summary of the field and publications can be found. Traditionally ILC has been applied to systems where the controlled variable is the measured variable, one recent example is [5]. In industrial robot applications this is typically not the case; in a standard industrial robot the motor angles are measured, but the control objective is to follow a desired tool path. Obviously it is a problem that the controlled variables cannot be measured, which can be confirmed by ILC experiments performed on a robot, see for example [6]. This motivates the need to use an estimate of the controlled variable in the ILC algorithm to be able to improve the performance. The aim is to present a framework for the situation when an ILC algorithm is combined with a procedure for generating an estimate of the controlled variable. To the best of the authors knowledge, estimation Manuscript received October 15, 2009; revised March 6, 2010; accepted April 29, 2010. The authors are with the Division of Automatic Control, Department of Electrical Engineering, Linkoping University, SE-58183 Linkoping, Sweden (email: {johanna, mino, svante}@isy.liu.se). This work was supported by ABB ABRobotics, Vinnovas Industry Excellence Center LINK-SIC at Linkoping University, and the Swedish Research Council (VR), which are gratefully acknowledged. techniques and ILC have only been combined in a few publications. One example is [7], where the ILC algorithm uses an estimate of the arm angle from measurements of motor angle and arm-angular acceleration of a flexible one-link robot arm. The study [8] also discusses these issues, based on simulations of a realistic robot model. In [9] the model error of a statespace model linearised along the desired trajectory is estimated using a Kalman filter in iteration-domain. The control signal at next iteration is given by minimising the deviation of the states from the desired trajectory. Another example is [10], where the estimated states for a class of time-varying nonlinear systems are used

in an ILC algorithm and asymptotic behaviour of the system is discussed. The focus in these papers is on specific estimation and/or ILC algorithm techniques, while the following aspects are of main interest here: A framework for analysis of properties of the ILC algorithm when an estimate of the controlled variable is used in the ILC algorithm. An expression for asymptotic error of controlled variable when an ILC algorithm based on an estimate of the controlled variable has converged. The paper is organised as follows. Section II presents the system description, and Section III contains a description of the procedure for estimating the controlled variable. In Section IV the considered class of ILC algorithms is presented, followed by the key results concerning the properties of the ILC algorithm in Section V. The results are discussed and also illustrated in a numerical example in Section VI. Finally, Section VII presents some conclusions. q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society 4 Asian Journal of Control, Vol. 13, No. 1, pp. 3 14, January 2011 II. SYSTEM DESCRIPTION 2.1 Notation and prerequisites All signals are defined on a finite time interval t =nts , n {0, . . . , N 1} with N samples and sampling interval ts . The time shift operator q is defined as qu(t)=u(t+ts ). The equations are in the time domain and the arguments t and q are omitted for convenience when there is no risk of confusion. 2.2 System description Consider the system shown in Fig. 1. The linear discrete time system T has two inputs; reference r and ILC input uk , with k denoting the iteration number. The outputs are the measured variable yk and the controlled variable zk . The system T can have internal feedback, which means that T contains the system to be controlled as well as the controller. Denoting the true system by the superscript 0, the system description at iteration k is yk (t)= T 0 ry(q)r (t)+T 0 uy(q)uk (t) (1) zk (t)= T 0 rz(q)r (t)+T 0 uz(q)uk(t) (2) The systems T 0 ry, T 0 uy, T 0 rz and T 0 uz are assumed to be stable and causal. System and measurement disturbances are not included here for simplicity. Example 1 (Open loop system). First, an example of the description (1)(2) is given by the stable open loop system shown in Fig. 2. The true system, denoted G0, is given by the discrete time state space description xk (t+ts ) = A0xk (t)+B0uk (t) yk (t) = C0xk (t) zk (t) = M0xk (t) (3) at iteration k with the ILC input uk . The outputs are

the measured variable yk and controlled variable zk . From (3) the transfer operators T 0 uy and T 0 uz are T 0 uy(q) = C0(q I A0) 1B0 T 0 uz(q) = M0(q I A0) 1B0 (4) Example 2 (Closed loop system). The system (1)(2) is exemplified by the closed loop system shown in Fig. 3. Then the system T has internal feedback and contains both the system to be controlled and the controller in operation. In a real application the controller structure can be more complex than the one shown in Fig. 3, and the structure considered here is used for illustration. Fig. 1. System T with reference r (t), ILC input uk (t), measured variable yk (t) and controlled variable zk (t) at iteration k. Fig. 2. Stable open loop system; example of the system (1)(2) with input uk (t) and outputs yk (t), zk (t) at iteration k. The system to be controlled is given by (3) with input uk , which is the control signal from the controller F at iteration k. The transfer operators are similarly to (4) denoted T 0 uy and T 0 uz. Throughout the paper it is assumed that F is given and that the closedloop system is stable. Noting the choice of adding the ILC input uk to the reference r in this example, it gives the closed loop system from r , uk to yk and zk as yk (t) = F(q)T 0 uy(q) 1+F(q)T 0 uy(q) r (t) + F(q)T 0 uy(q) 1+F(q)T 0 uy(q) uk (t) = T 0 ry(q)r (t)+T 0 uy(q)uk (t) zk (t) = F(q)T 0 uz(q) 1+F(q)T 0 uy(q) r (t) + F(q)T 0 uz(q) 1+F(q)T 0

uy(q) uk (t) = T 0 rz(q)r (t)+T 0 uz(q)uk (t) (5) from which the transfer operators T 0 ry, T 0 uy, T 0 rz and T 0 uz are identified from (1)(2). They are stable according to the assumption about the closed loop system. It can be noted that the system description (1)(2) is a natural extension of the description in [11] (system and measurement disturbances omitted here), where yk (t)=Tr (q)r (t)+Tu(q)uk (t) In [11] the properties of this system, when controlled by an ILC algorithm, are analysed with focus on the measured variable yk used in the ILC algorithm. q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society J. Wallen et al.: A Framework for Analysis of Observer based ILC 5 Fig. 3. Closed loop system; example of the system (1)(2) with inputs r (t), uk (t) and outputs yk (t), zk (t) at iteration k. The system G0 is controlled by uk (t) from the controller F. 2.3 System description in matrix form For the analysis in Section V, the matrix description of the system and ILC algorithm is used, which is closely related to the descriptions in [12], [13] and [14] among others. First, define the vector r of the N sample sequence of the reference r as r=(r (0) . . . r ((N 1)ts))T (6) Next, define the vectors uk , yk and zk similarly. The matrix T0 ry is formed by the pulse response coefficients gT 0 ry , t {0, . . . , (N 1)ts } of the transfer operator T 0 ry in (1), denoted gT 0 ry,0, . . . , gT 0 ry,N1. This results in the N N Toeplitz matrix T0 ry = gTry,0 0 . . . 0 gTry,1 gTry,0 0 ... . . . ... gTry,N1 gTry,N2 . . . gTry,0 (7) The system matrices T0 uy, T0

rz and T0 uz are defined similarly. It can be noted that they are lower triangular, since the transfer operators T 0 ry, T 0 uy, T 0 rz and T 0 uz in (1)(2) are causal. Finally, the system description (1)(2) is rewritten in matrix form as yk =T0 ryr+T0 uyuk (8) zk =T0 rzr+T0 uzuk (9) The vectors and matrices describing the system in matrix form are throughout the paper written in bold face, to easily distinguish the matrix description of the system from the description using transfer operators. III. ESTIMATION OF THE CONTROLLED VARIABLE In the description (1)(2) the measured variable yk and controlled variable zk are involved. Obviously, if applying an ILC input uk to the system based directly on yk , it will drive zk towards an incorrect value since the dynamics between yk and zk is neglected. It is therefore natural to use an estimate of zk , denoted zk, in the ILC algorithm.We propose the following representation where the estimate zk is related to r , uk and yk through zk (t)=Fr (q)r (t)+Fu(q)uk (t)+Fy(q)yk(t) (10) and the filters Fr , Fu and Fy are assumed to be stable. See Fig. 4 for an illustration of the system T together with the estimation procedure (10). The relation (10) will now be illustrated by three examples, showing how the estimate zk can be created. The cases are also discussed in Section VI. Example 3 (Case AMeasured variable).A naive way of estimating zk would be to neglect the dynamics between the measured and controlled variable and to let zk (t)=yk (t) In (10) this means choosing Fr (q)=0, Fu(q)=0, Fy(q)=1 Example 4 (Case BEstimate from model of direct relation between measured and controlled variable). There are situations where the relationship between the measured variable yk and controlled variable zk can be explicitly described by the stable discrete time relation zk (t)=T 0 yz(q)yk(t) (11) (See Appendix A for an example where (11) holds.) By using a nominal model of (11), it implies the estimate zk (t)=Tyz(q)yk(t) (12) which can be incorporated in the description (10) by Fr (q)=0, Fu(q)=0, Fy(q)=Tyz(q) In the third example zk is estimated by an observer. The idea with an observer is to estimate the states using the system input and measurement yk . Information from additional sensors can also be fused with the

original measurements available, see for example [15]. q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society 6 Asian Journal of Control, Vol. 13, No. 1, pp. 3 14, January 2011 Fig. 4. Description of the system T with reference r (t), ILC input uk (t), measured variable yk (t) and controlled variable zk (t) at iteration k. The estimation procedure (10) results in an estimate zk (t) of the controlled variable. One example where estimation techniques are combined with ILC is [8]. The arm angle of a realistic robot model is estimated by an extended Kalman filter (EKF) based on measurements of motor angle and tool acceleration. The estimate is thereafter used in an ILC algorithm. Example 5 (Case CEstimate from observer). Based on a nominal model of the state space description (3) with input uk , an observer can be formed as xk (t+ts ) = A xk (t)+B uk (t) +K(yk (t)C xk(t)) zk (t) = M xk (t) (13) where yk can be a vector consisting of several measurements. Using transfer operators the estimate of the controlled variable can be expressed by zk (t)= M(q I (AKC)) 1B uk (t) +M(q I (AKC)) 1Kyk (t) = F u(q) uk (t)+Fy(q)yk(t) (14) From the closed loop system in Fig. 3 the filters in (10) can then be identified as zk (t)= F u(q)F(q)r (t)+F u(q)F(q)uk(t) +(Fy(q)F u(q)F(q))yk(t) = Fr (q)r (t)+Fu(q)uk (t)+Fy(q)yk(t) (15) To summarise, from Examples 35 it can be seen that the description (10) covers different ways of generating the estimate zk ; by neglecting the dynamics, by using a nominal model of the relation between yk and zk , and from observers. It is therefore natural to use (10) in the ILC algorithm, as is seen in Section IV. Now, define the vectors of the N sample sequence of the estimate zk as in (6), and form the matrices Fr , Fu and Fy similarly as in (7) by the pulse response coefficients of the filters Fr , Fu and Fy . It results in the matrix description of the relation (10) as zk =Fr r+Fuuk+Fyyk (16) IV. ILC ALGORITHM In this paper the following structure for an ILC algorithm is considered, where the update equation in transfer operator form is given by uk+1(t)=Q(q)(uk(t)+L(q)k (t)) (17) The linear filters Q and L are possibly non causal. The error used in the ILC algorithm is the difference between the reference r and the estimate zk in (10) of the controlled variable at iteration k, as in k (t)=r (t)zk (t) (18) On the other hand, the error of the controlled variable ek (t)=r (t)zk (t) (19) is used to evaluate the performance of the system. Rewrite in matrix form by first defining the vectors uk+1, ek and ek similarly as in (6). Using the operators

L and Q, it results in the ILC update equation uk+1 =Q(uk+Lek ) (20) ek =rzk (21) while the system performance (19) is given by ek =rzk (22) Note that L and Q are not necessarily lower triangular, due to a possibly non causal ILC algorithm. V. ANALYSIS To support the analysis and make it easier to follow the derivations in this section, the main equations (8)(9), (16), and (20)(22) from previous sections are summarised here. The system given by yk =T0 ryr+T0 uyuk (23) zk =T0 rzr+T0 uzuk (24) is controlled using the ILC input signal uk , updated by uk+1 =Q(uk+Lek ) (25) ek =rzk (26) q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society J. Wallen et al.: A Framework for Analysis of Observer based ILC 7 using the estimate zk =Fr r+Fuuk+Fyyk (27) From the relations (23)(27) the ILC system equation is now derived, which is used in the stability analysis. Lemma 1 (ILC system equation). Consider the system in (23)(24) using the ILC update equation (25) and the estimate of zk from (27). Then the ILC system equation is given by uk+1=Huk+Hr r where H=Q(I L(Fu+FyT0 uy)) Hr =QL(I (Fr +FyT0 ry)) Proof. Using (23) and (27), the error ek in (26) can be written ek =rFr rFuukFy(T0 ryr+T0 uyuk ) =(I (Fr +FyT0 ry))r(Fu+FyT0 uy)uk Inserting the error ek into (25) then gives uk+1 =Q(I L(Fu+FyT0 uy))uk +QL(I (Fr +FyT0 ry))r and the result follows. Stability of the ILC system in Lemma 1 can be expressed based on the results in [11]. Theorem 2 (Stability). Consider the ILC system equation from Lemma 1, uk+1=Huk+Hr r This system is stable if and only if (H)<1, where () is the spectral radius of the matrix. Proof. See [11]; Theorem 1 and proof of Corollary 3.

It is assumed that there exists a unique input trajectory ud on a finite time interval such that z equals r , see [16]. Stability in the sense of Theorem 2 means that it exists a <, see [11], such that sup k=0,1,... uduk< Lemma 3 (Asymptotic behaviour). Under the assumption that the input uk in the ILC system equation from Lemma 1 converges to a bounded signal as k, the limit is given by u =(I Q(I L(Fu+FyT0 uy))) 1 QL(I (Fr +FyT0 ry))r (28) The corresponding asymptotic error rz becomes e =(IT0 rz T0 uz(IQ(IL(Fu+FyT0 uy))) 1 QL(I (Fr +FyT0 ry)))r (29) Proof. Let k and solve the ILC system equation from Lemma 1, u =Q(I L(Fu+FyT0 uy))u +QL(I (Fr +FyT0 ry))r for u. Insert u in the relation for the error of the controlled variable, using (24), e=rz=r(T0 rzr+T0 uzu) to derive the asymptotic error. From a design perspective the following result has an important impact. Theorem 4 (Monotone convergence). Consider the same ILC system as in Theorem 2. If the maximum singular value satisfies (H)<1, then the ILC system is stable, and uukkuu0 with u from (28) in Lemma 3. Proof. See proof of Theorem 9 in [11] and Lemma 3 above for the computation of u. Monotone convergence of uk is a good design criterion, but it is also necessary to check e0 and e so that the error is reduced as desired. From Theorem 4 it can be seen that the smaller the maximum singular value of the matrix H, the faster the convergence. A design criterion therefore should include minimising (H) as well as the asymptotic error e. VI. ILLUSTRATION OF THE RESULTS The relations (23)(27) and the asymptotic error e of the controlled variable from Lemma 3 are discussed in this section. First, in Sections 6.16.4 it is assumed that L can be chosen such that uk converges to q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society

8 Asian Journal of Control, Vol. 13, No. 1, pp. 3 14, January 2011 a bounded signal when Q=I in the ILC algorithm (25). Although this idealised choice of Q is not realistic in practice, it is fruitful in the analysis of e, since then only the effect of the estimation matrices Fr , Fu and Fy is studied and the result is independent of the choice of Q and L. In addition to this idealised discussion, a numerical example with a flexible two mass system is given in Section 6.5 with a general operator Q. The transient and asymptotic behaviour is discussed when an ILC algorithm is applied to the system, using different estimates zk . To support the intuition of the results in Sections 6.16.4, the relations are illustrated for a system where it is possible to write an explicit stable discrete time relationship between the measured variable yk and controlled variable zk as in (11), zk (t)=T 0 yz(q)yk (t) See Appendix A for further details, where such a system is exemplified by a flexible two mass model of a single robot joint. From the pulse response coefficients of the system T 0 yz, the matrix T0 yz is derived as in (7), giving zk =T0 yzyk (30) The ILC algorithm (25) is in this section applied to the system (23)(24) for the following cases: A. The ILC algorithm is based on ek =ryk, that is, the dynamics between yk and zk is neglected. B. The ILC algorithm uses ek =rTyzyk, that is, zk is estimated from a model of the relation (30). C. The ILC signal is updated using ek =rzk , where zk =Fr r+Fuuk+Fyyk . The filters Fr , Fu and Fy are obtained from the observer (13). D. As a comparison, in order to show what ideally can be achieved, it is assumed that the controlled variable zk can be measured and that the ILC algorithm is based on ek =rzk . The choices of filters Fr , Fu and Fy in (10) to implement the cases AC were discussed in Examples 35 in Section III. In Table I the asymptotic error e from Lemma 3 is summarised for the cases above. These relations are discussed in more detail next. 6.1 Case AILC using measured variable The dynamics between the measured variable yk and the controlled variable zk is now neglected, which gives the estimate zk =yk , see Example 3. The actual relation is given by (23)(24). With this estimate, the error ek =ryk is used in the ILC algorithm (25). Assume that L can be chosen such that uk converges to a bounded signal when Q=I . This choice results in e=0 when the algorithm has converged. The asymptotic error e from Lemma 3 is then given by e=(I T0 rz T0 uz(LTuy) 1L(I T0

ry))r (31) Assume that the system can be described with an explicit relation between the measured and controlled variable as in (30). This results in the approximate relations for the finite dimensional matrices T0 rz =T0 yzT0 ry, T0 uz =T0 yzT0 uy (32) For the SISO system (1), (11), the relation (31) can be rewritten in transfer operator form using (32), giving e(t)=(1T 0 yz(q))r (t) Whenever T 0 yz =1, this results in an asymptotic error e=0 of the controlled variable, as expected. 6.2 Case BILC using estimate from model of direct relation between yk and zk Under the assumption that the relation (30) between yk and zk holds, a straightforward way to estimate zk is by using the nominal dynamics, giving zk =Tyzyk . The asymptotic error e from Lemma 3, for Q=I and using the approximation (32), results in e =(I T0 yz(T0 ry +T0 uy(LTyzT0 uy) 1 L(I TyzT0 ry)))r under the assumption that the ILC algorithm converges. For the SISO system (1), (11), the relation above can using transfer operators be rewritten as e(t)=(1T 0 yz(q)T 1 yz (q))r (t) From this expression it can be seen that when the nominal model Tyz is equal to the true system T 0 yz, the error e=0. An obvious question is how close Tyz needs to be to T 0 yz to get better performance using the approach in case B compared to case A. This will depend on the properties of each particular application. 6.3 Case CILC using estimate from observer The estimate zk is now given by (27), zk =Fr r+Fuuk+Fyyk Using Q=I and under the assumption that the ILC algorithm converges, the asymptotic error e is from Lemma 3 given by e =(I T0 rz

T0 uz(L(Fu+FyT0 uy)) 1 L(I (Fr +FyT0 ry)))r q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society J. Wallen et al.: A Framework for Analysis of Observer based ILC 9 Table I. Summary of the description of a system using ILC and performance when t he ILC algorithm uses various errors, cases AD, for the ideal choice Q=1. Case ek e(t)=r (t)z(t), Q(q)=1 A ryk (1T 0 yz(q))r (t) B rTyzyk (1T 0 yz(q)T1 yz (q))r (t) C r(Fr r+Fuuk+Fyyk) (1T 0 yz(q) T 0 ry(q)Fu (q)+(1Fr (q))T 0 uy(q) Fu (q)+Fy (q)T 0 uy(q) )r (t) D rzk 0 Under the assumption that relation (32) holds, the asymptotic error can for the SISO system (1), (11) be rewritten in transfer operator form to e(t)= 1T 0 yz (q) T 0 ry(q)Fu (q)+(1Fr (q))T 0 uy(q) Fu (q)+Fy (q)T 0 uy(q) r (t) (33) Now use the relation (15) from Example 5, zk (t)= F u(q)F(q)r (t)+F u(q)F(q)uk(t) +(Fy(q)F u(q)F(q))yk(t) where F u(q) and Fy(q) are given by (14). The closedloop system is from Example 2 given by (5), yk (t)= F(q)T 0 uy(q) 1+F(q)T 0 uy(q) r (t) + F(q)T 0 uy(q) 1+F(q)T 0 uy(q) uk (t) where the transfer operators in (4) are denoted T 0 uy and

T 0 uz for the system with input uk . The asymptotic error e in (33) then becomes e(t)= 1T 0 yz(q) T 0 uy(q) F u(q)+Fy(q)T 0 uy(q) r (t) (34) Assuming F =0, it can be noted that the expression for e in (34) is independent of the controller F. If the observer is based on the true system, it can be seen that the numerator of (34) is F u(q)+Fy(q)T 0 uy(q)=T 0 yz(q)T 0 uy(q) where the explicit calculations are omitted due to limited space. Using this expression in (34) gives the asymptotic error e=0, which is expected when zk results from an observer based on the true system. 6.4 Case DILC using controlled variable The last case is used to show what ideally can be achieved when the error ek =rzk in (26) is used in the ILC algorithm (25). The asymptotic error e of the controlled variable is from Lemma 3 given by e =(I T0 rz T0 uz(I Q(I LT0 uz)) 1 QL(I T0 rz))r This expression can for a SISO system described in transfer operator form (1)(2) be simplified to e(t)= (1Q(q))(1T 0 rz(q)) 1Q(q)(1L(q)T 0 uz(q)) r (t) Clearly this expression is zero when the ILC algorithm converges for the choice Q=1, achieving perfect tracking of the controlled variable zk . 6.5 Numerical example In this section the approaches AD are illustrated and compared when the ILC algorithm (25)(26) is applied to the two mass system given in Appendix A. A model error is introduced in the system by reducing the spring coefficient by 50% from the nominal value. This means that the true system has a less rigid spring than expected. The resonance frequency of the closed loop system from the arm angle reference ra to arm angle qa changes from approximately 1.2 Hz for the nominal system to around 0.9Hz for the system with model errors. The bandwidth is also reduced from

approximately 1.8 Hz to circa 1.4 Hz. The reference ra is a smooth step, shown in Fig. 6. The two mass system, controller and ILC algorithm is implemented in SIMULINK, and further details can be found in Appendix A. No measurement noise is added in the simulation to facilitate the comparison. q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society 10 Asian Journal of Control, Vol. 13, No. 1, pp. 3 14, January 2011 6.5.1 Tuning of observers The linear observer (13) is used, based on the nominal model of the state space description (3). For case C1 the motor angle qm is measured, while also the arm angular acceleration qa is measured in case C2, y(t)= qm(t), case C1 (qm(t) qa(t))T , case C2 The controlled variable, the arm angle qa in Fig. A1, is only used for evaluation purposes here. The objective is now to find an observer gain K giving as small estimation error qa qa as possible, that is, a robust estimation problem. The observer gain K is here given from a stationary discrete time Kalman filter with the covariance matrices, R1 for the process noise and R2 for the measurement noise, as design variables. The tuning is solved using the following optimisation problem min R1,R2 qaqa2 subject to R1=diag(0 10a 0 10b ) R2= 1, case C1 diag(1 10c ), case C2 where the 2 norm of the estimation error is minimised based on measurements from the true system. Large elements in the observer gain K are possible due to ideal sensors, where no offset or noise are added to the measurements.Moreover, the minimisation of the estimation error is performed over the reference trajectory shown in Fig. 6 for the specific model error. Hence, the robustness of the observer is not evaluated with respect to other model errors. Although the observer is tuned for a special operating point with ideal measurements, it can be used for illustrative purposes to discuss and exemplify the approaches AD. 6.5.2 Tuning of ILC algorithms For each case AD an ILC algorithm is designed, where L(q)=q. The Q filter is a low pass secondorder Butterworth filter with cutoff frequencyn above the bandwidth and resonance frequency of the closedloop system, so that errors associated with the resonance frequency can be corrected for. The Q filter is applied by filtering the signal forwards and backwards to give zero phase characteristics. The corresponding matrices Q and L are found by forming the Toeplitz matrices similarly as in (7) from the pulse response coefficients

of the transfer operators Q and L. The ILC design parametersn (resolution 1 Hz), (integer) and (resolution 0.1) are heuristically tuned for the cases AD, based on the reduction of the 2 norm of the error ek (26) used in the ILC algorithm. The ILC algorithms are tuned such that e2 is approximately 3% of e02, that is, when no ILC algorithm is applied. The tuning is a trade off between transient performance and same level of error reduction for ek , wheren is chosen as large as possible to suppress also higher frequencies of the error. For all cases (H)<1 for the true system, giving monotone convergence with respect to the error ek used in the ILC update according to Theorem 4. The tuning results in: Case A: ILC design parametersn=9Hz, =10, =0.4, giving (H)0.9692. Case B: ILC design parametersn=9Hz, =27, =0.3, giving (H)0.9898. Case C1: Measured motor angle qm used in a discrete time Kalman filter. ILC design parameters n=9Hz, =17, =1.4 gives (H)0.9990. Case C2: Measured motor angle qm and armangular acceleration qa used in a discrete time Kalman filter. ILC design parametersn=7Hz, =27, =0.5 gives (H)0.9816. Case D: ILC design parametersn=7Hz, =28, =0.5 gives (H)0.9823. 6.5.3 Evaluation measure The evaluation of the simulation results is based on the asymptotic error e=raqa, of the controlled variable in Lemma 3. First, the reduction of the 2 norm of the arm angle error is given by ered =100 e2 e02 [%] (35) The reduction of the 2 norm of the error ek used in the ILC algorithm is on the other hand red =100 e2 e02 [%] (36) where e=rqa, is given similarly as e. The quantities (35)(36) measure the relative reduction of the 2 norm of the asymptotic error, expressed in percentage of the 2 norm of the error when no ILC algorithm is applied. Here it is possible to compute these quantities, since the true system is known. 6.5.4 Discussion The results are summarised in Table II. For each case AD the expression for the actual estimate qa,k is shown, which is used in the ILC algorithm (25). q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society J. Wallen et al.: A Framework for Analysis of Observer based ILC 11 Table II. Summary of the results for the two mass system when the ILC algorithm applied uses various estimates of the controlled variable qa; cases AD. Case qa,k red ered A rgqm,k 3.09 109.95

B Tyzqm,k 2.97 54.83 C1 Fr ra+Fuuk+Fyqm,k 2.98 44.14 C2 Fr ra+Fuuk+Fy qm,k qa,k

3.05 3.12 D qa 3.07 3.07 0 50 100 150 200 250 3 4 5 10 20 50 100 Error of controlled variable Iteration || ek ||2 A B C1 C2 D Fig. 5. Reduction of the error (22) for the first 250 iterations for the cases AD, expressed in percent of the error when no ILC algorithm is applied (denoted 0th iteration). The error reduction highly depends on the quality of the estimate. The error reduction is given by the evaluation measures (35)(36). The error reduction of the arm angle qa,k is also illustrated in Fig. 5 for the first 250 iterations. It can be noted from Fig. 5 that we can have non monotone convergence of the arm angle error ek , since the ILC algorithm is tuned with respect to the estimated armangle error ek . In Fig. 6 the resulting arm angle qa is shown after 1000 iterations, and it depends on the relation between the estimate qa,k used in the ILC algorithm and the true arm angle qa,k . It is seen that case A gives a poor error reduction, which is also concluded from Fig. 5 and Table II with ered 110 %, although the ILC algorithm manages to significantly reduce the error ek used in the ILC algorithm, resulting in red 3.1% from Table II. This result is expected for the two mass model, where there is a significant dynamical relationship between the variables qm and qa which is completely neglected in approach A. When comparing case B, see (12), and case C, see (15), it can be seen that case B is a special case 0 1 2 3 4 5 0 0.5 1 1.5 Controlled variable qa Time [s] 1.75 2 2.25 2.5 0.96 0.98 1 Time [s] qa ra qa, k = 0 qa, A

qa, B qa, C1 ra qa, C2 qa, D Fig. 6. Actual arm angle qa(t) after 1000 iterations for the cases AD, compared to when no ILC algorithm is applied (denoted k=0). The performance highly depends on the quality of the estimate. Note the different scales in the figures. of the observer in case C with a particular choice of the design variables R1 and R2 in the Kalman filter described in Section 6.5.1. The details are omitted here for space limitations. The general observer design has more degrees of freedom and this fact can explain why case C1, which only uses the motor angle in the observer, still gives a slightly better error reduction of the actual arm angle compared to case B, with ered 44% compared to ered 55% in Table II. Case B still gives a significant improvement compared to case A (ered 110%) and the simple design compared to approach C can still make approach B a preferred choice in a real application. q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society 12 Asian Journal of Control, Vol. 13, No. 1, pp. 3 14, January 2011 The major advantage of the general observer, especially considering state space formulation and Kalman filter design, is that it is straightforward to include additional measurements in the estimation. Additional measurements will make the tuning somewhat more complicated, but can be done using optimisation techniques where the estimation error is minimised based on real measurements from the true system, for example as described in Section 6.5.1. Using both measured motor angle qm and arm angular acceleration qa in the observer improves the result on the arm side significantly, because of the improved quality of the estimate. The result can be seen for case C2 in Table II with ered 3.1%, from Fig. 5, and from the true arm angle in Fig. 6, which is very close to the reference. VII. CONCLUSIONS A framework for analysis of an ILC algorithm based on an estimate of the controlled variable obtained from an observer based estimation procedure is proposed. The need for such a framework is motivated by the fact that in some applications the controlled variable is not the measured variable. The description is a natural extension of the system description in [11] and is focusing on the performance of the controlled variable. A general expression for the asymptotic error of the controlled variable when the ILC algorithm has converged is derived using the framework. The asymptotic error is discussed for three cases, especially if the error used in the ILC update equation goes to zero, and is exemplified by an ILC algorithm applied to a flexible two mass model of a robot joint. A numerical example of the two mass system when having model errors is thereafter given. The simulation shows better performance of the controlled variable when the estimate

used in the ILC algorithm is based on observer design instead of a nominal model. The benefit of fusing information from additional sensors with the original measurements available is illustrated, with a more accurate estimate improving the ILC system performance. It is also shown how the expression for the asymptotic error can be used in analysis and design. APPENDIX A. TWO MASS MODEL The two mass system in Fig. A1 is a common first approximation of a mechanical system. The input u(t) is the torque applied to the first mass. The angular position Fig. A1. A flexible two mass model of a mechanical system, for example a single robot joint, characterised by spring k, damper d, viscous friction fm, gear ratio rg, moments of inertia Jm, Ja, torque u(t), motor angle qm(t) and arm angle qa(t). qm(t) of the first mass, referred to as motor angle, is the measured variable, and the angular position qa(t) of the second mass, called arm angle, is the controlled variable. The system is described by the relations Jm qm(t)=fm qm(t)rgk(rgqm(t)qa(t)) rgd(rg qm(t) qa(t))+ku u(t) Ja qa(t) = k(rgqm(t)qa(t)) +d(rg qm(t) qa(t)) (A1) with moments of inertia Jm =0.0021 and Ja =0.0991, viscous friction fm =0.0713, spring coefficient k=8, damping coefficient d=0.0924 and gear ratio rg=0.2. A state space description of the two mass system can be derived using, for example, the states x(t)=(qa(t) qa(t) qm(t) qm(t))T Measuring only the motor angle gives the measured variable yk =qm,k and controlled variable zk =qa,k according to the state space description (3). From (A1) it is straightforward to compute the transfer functions qm(t)= T 0 uy(p) u(t) (A2) qa(t)= T 0 yz(p)qm(t) (A3) where p is the derivative operator. In Fig. A2 a system given by (A2)(A3) is illustrated, where the system T 0 uy is stabilised by a controller F and the ILC input uk is added to the reference r at iteration k. It is not possible to find an exact discrete time representation of (A3) since qm(t) is an internal continuous variable in the system described by (A1). When the sampling time is short it is however possible to find a good approximation, using for example the Tustin derivative approximation. Moderate requirements for the feedback can be chosen in Section 6.5, since the desired servo q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society J. Wallen et al.: A Framework for Analysis of Observer based ILC 13 Fig. A2. The two mass system illustrated by T 0 uy and T 0 yz; transfer function from motor torque uk (t) to motor angle qm,k (t) and from motor angle qm,k (t) to arm

angle qa,k (t), respectively, controlled by the feedback controller F. The ILC input uk (t) is added to the reference r (t) at iteration k. performance is achieved by the ILC algorithm. The PD controller including a low pass filter is used, where F(s)=5+ 0.2s 1+0.1s and the controller is implemented in discrete time with sampling time ts =0.1s. The closed loop system and ILC algorithm is simulated using SIMULINK. REFERENCES 1. Arimoto, S., S. Kawamura, and F. Miyazaki, Bettering operation of robots by learning, J. Robot. Syst., Vol. 1, No. 2, pp. 123140 (1984). 2. Casalino, G. and G. Bartolini, A learning procedure for the control of movements of robotic manipulators, Proc. IASTED Sym. Robot. Autom., San Francisco, U.S.A., pp. 108111 (1984). 3. Craig, J. J., Adaptive control of manipulators through repeated trials, Proc. Am. Control Conf., San Diego, CA, U.S.A., pp. 15661572 (1984). 4. Bristow, D. A., M. Tharayil, and A. G. Alleyne, A survey of iterative learning control, IEEE Control Syst. Mag., Vol. 26, No. 3, pp. 96114 (2006). 5.Dabkowski, P., K. Gakowski, B. Datta, and E. Rogers, LMI based stability and stabilization of second order linear repetitive processes, Asian J. Control, Vol. 12, No. 2, pp. 136145 (2010). 6. Wallen, J., M. Norrlof, and S. Gunnarsson, Armside evaluation of ILC applied to a six degreesof freedom industrial robot, Proc. IFAC World Congress, Seoul, Korea, pp. 1345013455 (2008). 7. Gunnarsson, S., M. Norrlof, E. Rahic, and M. O zbek, On the use of accelerometers in iterative learning control of a flexible robot arm, Int. J. Control, Vol. 80, No. 3, pp. 363373 (2007). 8. Wallen, J., S. Gunnarsson, R. Henriksson, S. Moberg, and M. Norrlof, ILC applied to a flexible two link robot model using sensor fusionbased estimates, Proc. IEEE Conf. Decis. Control, Shanghai, China, pp. 458463 (2009). 9. Schollig, A. and R. DAndrea, Optimization based iterative learning control for trajectory tracking, Proc. Eur. Control Conf., Budapest, Hungary, pp. 15051510 (2009). 10. Tayebi, A. and J. X. Xu, Observer based iterative learning control for a class of time varying nonlinear systems, IEEE Trans. Circuits Syst. I Fundam. Theory Appl., Vol. 50, No. 3, pp. 452 455 (2003). 11. Norrlof, M. and S. Gunnarsson, Time and frequency domain convergence properties in iterative learning control, Int. J. Control, Vol. 75, pp. 11141126 (2002). 12. Phan, M. and R. W. Longman, A mathematical theory of learning control of linear discrete multivariable systems, Proc. AIAA/AAS Astrodyn.

Conf., Minneapolis, MN, U.S.A., pp. 740746 (1988). 13. Moore, K. L., Multi loop control approach to designing iterative learning controllers, Proc. IEEE Conf. Decis. Control, Tampa, FL, U.S.A., pp. 151 214 (1998). 14. Tousain, R., E. van der Meche, and O. Bosgra, Design strategy for iterative learning control based on optimal control, Proc. IEEE Conf. Decis. Control, Orlando, FL, U.S.A., pp. 44634468 (2001). 15. Kailath, T., A. H. Sayed, and B. Hassibi, Linear Estimation, Prentice Hall Inc., Upper Saddle River, NJ, U.S.A. (2000). 16. Spong, M. W., F. L. Lewis, and C. T. Abdallah (eds), Robot Control: Dynamics, Motion Planning, and Analysis, IEEE Control Systems Society, IEEE Press, New York, NY, U.S.A. (1992). Johanna Wallen was born in 1979 and grew up in Katrineholm, Sweden. She received her M.Sc. degree in Electrical Engineering from Linkoping University, Sweden, in 2004. Since 2005 she works as a Ph.D. student at the Division of Automatic Control at the Department of Electrical Engineering, Linkoping University, and in 2008 she received her Lic. Eng. degree from the same q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society 14 Asian Journal of Control, Vol. 13, No. 1, pp. 3 14, January 2011 university. Her research interest is iterative learning control, mainly for industrial robots. Mikael Norrlof was born in Karlstad, Sweden in 1971. He received hisM.Sc. in Computer Science and Engineering 1996, Ph.D. degree in Automatic Control in 2000, and became Docent in Automatic Control in 2005, all at Linkoping University, Sweden. He has been post doc at Linkoping University in 20002008. From 2000 until 2006 he has also worked as a consultant in industry, mainly doing research and development in different projects for ABB. From 2007 he is employed at ABB Robotics where he is principal engineer in path planning and motion control. In 2009, Norrlof became adjunct professor in Automatic control at Linkoping University. His current major research interests include industrial robot modeling and control, path and trajectory generation, sensor fusion, and iterative learning control. Svante Gunnarsson was born in Tranas, Sweden in 1959. He received the M.Sc. degree in Electrical Engineering from Linkoping University, Sweden, in 1983. He received his Ph.D. in Automatic Control from Linkoping University, Sweden, in 1988, and is

currently working as Professor in the Division of Automatic Control at the Department of Electrical Engineering, Linkoping University, Sweden. His research interests are in the areas of system identification, iterative learning control, and robot control. q 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society