Sie sind auf Seite 1von 55

CHEE825

System Identification
J. McLellan
Fall 2005
Module 1:
Introduction and Motivation
What do dynamic data look like?

Time Series Plot of Industrial Data


7

0 var1
#1 # 301 # 601 # 901 # 1201 # 1501 # 1801 # 2101 var2
# 151 # 451 # 751 # 1051 # 1351 # 1651 # 1951

chee825 - Fall 2005 J. McLellan 3


Example - Average Weekly Gasoline Prices in the U.S.

Overlay Plot

avg_usa_gas

4/15/92 8/15/97
12/15/86 12/15/02
date

chee825 - Fall 2005 J. McLellan 4


Example - Gasoline Prices

• How can we model this?


• Inputs?
• Inertia?
• Continuous vs. discrete time?
The Time Series model framework
– Structure
– Lagged regression perspective

chee825 - Fall 2005 J. McLellan 5


Outline

• dynamic models
» role
» sources of variability and need for disturbance models
» components in data
• statistical properties of a model - sensor analogy
• making a link to linear regression
» problem formulation and assumptions
» parameter estimates + properties
» role of experimental design
• motivating example - 1st order process
» “lagged regression” problem
» impact of non-ideality in disturbances - serial correlation

chee825 - Fall 2005 J. McLellan 6


Role of Dynamic Models

• process characterization
» analysis of process dynamics
» analysis of disturbance dynamics
• process control
» loop configuration and tuning (e.g., PID control)
» model-based control algorithms
• cf. Internal Model Control (IMC) formalism
• minimum variance control (MVC)
• model predictive control (MPC) - e.g., Dynamic Matrix Control
• process monitoring
» controller performance assessment, process tracking

chee825 - Fall 2005 J. McLellan 7


Control Applications

• models implemented as
» step response models - primarily for MPC
» impulse response models - MPC
» transfer functions - MVC, IMC, occasionally MPC
• range of disturbance estimation approaches
» “step” disturbances - compare predicted vs. current
measured value - e.g., in MPC
» time series approaches
• MPC reference - survey of Badgwell and Qin

chee825 - Fall 2005 J. McLellan 8


Sources of Variability

• process disturbances
– “internal”
• flow fluctuations
• deterioration - e.g., channeling in a reactor
– “external”
• from upstream units
• ambient conditions - e.g., air temperature
• operator interventions

chee825 - Fall 2005 J. McLellan 9


Sources of Variability

• instrumentation and measurements


» electronic noise
» physical location of instrument
• single thermocouple for tank - impact of mixing
• sampling - e.g., for gas chromatograph
• models and process representation
» unmodeled components become lumped as
disturbances
• e.g., ignoring radial profiles in a reactor
» model simplification
» “ignored” dynamic behaviour

chee825 - Fall 2005 J. McLellan 10


Steady-State Data

• no time variation
• does this mean the time trace is a straight line?
– no - we have noise !
– focus on whether variability patterns (distributions) are
changing
– is mean value constant?
– is variance constant?

chee825 - Fall 2005 J. McLellan 11


Dynamic Data

• responses changing in time


• typically have process inertia
– serial or “autocorrelation”
– values now depend on earlier values
– denominator in a transfer function
– variability patterns changing - e.g., variance

chee825 - Fall 2005 J. McLellan 12


Components in Data

• deterministic
– non-random relationships
– physical relationships
» e.g., energy/material balance relationships in column
influence composition in overhead
• stochastic
– random fluctuations - variability pattern
– frequently disturbances
– can evolve in time

chee825 - Fall 2005 J. McLellan 13


Matrix of Model Scenarios

Stochastic Component
Static + Static +
Deterministic Static Dynamic
Component Dynamic + Dynamic +
Static Dynamic

chee825 - Fall 2005 J. McLellan 14


Examples

• step response + white noise


– dynamic deterministic + static stochastic
» white noise is purely uncorrelated noise
• step response + integrating noise
– dynamic deterministic + dynamic stochastic
• inferred property model
– static deterministic + dynamic stochastic

chee825 - Fall 2005 J. McLellan 15


The Task of Dynamic Model Building

partitioning process data into a deterministic component


(the process) and a stochastic component (the
disturbance)

time series
model

process disturbance

transfer
function
model
?
chee825 - Fall 2005 J. McLellan 16
Dynamic vs. Steady State Disturbance Cases

Static (steady state) Disturbance Dynamic


Disturbance

no consistent local trends local trends


chee825 - Fall 2005 J. McLellan 17
Example - Random Shocks into a Tank

• consider tank and vary the time


AR parameter of 0.3 AR parameter of 0.6
constant 4 5
» approach an integrator 2
• transfer function

output
output
0 0

1
-2

-4 -5
−1
1 − δζ
0 100 200 300 0 100 200 300
AR parameter of 0.9 Non-stationary
• referred to as an 10 20

“autoregressive” (AR) time 5 10


series model
output

output
0 0
» “d” is the AR parameter
-5 -10

-10 -20
0 100 200 300 0 100 200 300
Note the vertical scales of the plots time time

chee825 - Fall 2005 J. McLellan 18


Example – Industrial Step Response Data

• Deterministic component
» Step response

• Limited amount of noise


some variability
– Variation about the step
about deterministic
response
trend

chee825 - Fall 2005 J. McLellan 19


Outline

• dynamic models
» role
» sources of variability and need for disturbance models
» components in data
• statistical properties of a model - sensor analogy
• making a link to linear regression
» problem formulation and assumptions
» parameter estimates + properties
» role of experimental design
• motivating example - 1st order process
» “lagged regression” problem
» impact of non-ideality in disturbances - serial correlation

chee825 - Fall 2005 J. McLellan 20


Sensor Analogy

• hard sensor - e.g., thermocouple


– consider accuracy and reproducibility/precision
• accuracy
» is there persistent bias?
» where is the mean of the readings relative to true value?
• reproducibility
» how consistent are the measurements?
» variability of the sensor

chee825 - Fall 2005 J. McLellan 21


Sensor Accuracy

number of
samples

value

sensor avg true value

bias
chee825 - Fall 2005 J. McLellan 22
Sensor Reproducibility

number of better consistency


samples

poor consistency

value

chee825 - Fall 2005 J. McLellan 23


How does accuracy apply to dynamic models?

Bias in model predictions due to


» incorrect model form
• for process component
• for disturbance component
» poor data
• experiment too short
• inadequate dynamic content in MV signal
» violation of model estimation assumptions
• white noise (uncorrelated) vs. correlated noise present

chee825 - Fall 2005 J. McLellan 24


How does reproducibility apply to dynamic models?
• consistency of the predictions
» consistency = variability
• arises from uncertainty associated with the parameter estimates

y$t = a$1 yt − 1 + b$0 ut − 2

= “+”

parameter 1 parameter 2

chee825 - Fall 2005 J. McLellan 25


How does reproducibility apply to dynamic models?


variability associated with parameter estimates
is influenced by data collection, model structure

form of summation dictated by model type

chee825 - Fall 2005 J. McLellan 26


Outline

• dynamic models
» role
» sources of variability and need for disturbance models
» components in data
• statistical properties of a model - sensor analogy
• making a link to linear regression
» problem formulation and assumptions
» parameter estimates + properties
» role of experimental design
• motivating example - 1st order process
» “lagged regression” problem
» impact of non-ideality in disturbances - serial correlation

chee825 - Fall 2005 J. McLellan 27


Linear Regression Model

Y = β 0 + β1x + ε

response
regressor “noise”
“input” (disturbance)
Given -
• N sets of (x,y) data
• estimate parameters

chee825 - Fall 2005 J. McLellan 28


Least Squares Estimation

Minimize sum of squared prediction errors

o
responses
o o o
o

o
x
Ν Ν 2 2
min ∑ ( ψι − ψι = ∑ ει
$)
β0 ,β1 ι=1 ι=1
chee825 - Fall 2005 J. McLellan 29
Linear Regression Assumptions

• independent noise at each measurement point


• normally distributed noise
• constant variability patterns
– mean, variance
– independent, identically distributed (IID) Normal
» noise is white
» constant distribution
» no trends

chee825 - Fall 2005 J. McLellan 30


Matrix Formulation - Linear Case

Model
Y = Xβ + Ε
Least squares estimates

$ Τ −1 Τ
β = (Ξ Ξ ) Ξ Ψ

operating points
for msmts. observations
“design matrix”

chee825 - Fall 2005 J. McLellan 31


Interpretation - Columns of X

– values of a given variable at different operating points -


– entries in XTX
» dot products of vectors of regressor variable values
» related to correlation between regressor variables
– form of XTX is dictated by experimental design
• e.g., 2k design - diagonal form

chee825 - Fall 2005 J. McLellan 32


Parameter Estimation - Graphical View

approximating observation vector residual


vector
observations y

y$

chee825 - Fall 2005 J. McLellan 33


Parameter Estimation - Nonlinear Regression Case

approximating observation vector residual


vector
observations y

y$

model surface
chee825 - Fall 2005 J. McLellan 34
Properties of LS Parameter Estimates

Key Point - parameter estimates are random variables


» because of how stochastic variation in data propagates
through estimation calculations
» parameter estimates have a variability pattern -
probability distribution and density functions
Unbiased
E{β$} = β
» “average” of repeated data collection / estimation
sequences will be true value of parameter vector

chee825 - Fall 2005 J. McLellan 35


Properties of Parameter Estimates

Consistent
» behaviour as number of data points tends to infinity
» with probability 1,
$= b
λιµ b
N →∞
» distribution narrows as N becomes large
Efficient
» variance of least squares estimates is less than that of
other types of parameter estimates

chee825 - Fall 2005 J. McLellan 36


Properties of Parameter Estimates

Covariance Structure
» summarized by variance-covariance matrix

$ Τ −1 2
Cov( β ) = (Ξ Ξ ) σ

structure dictated by variance of


experimental design noise

$  Var (β$0 ) Cov (β$0 , β$1 )


Cov ( β ) =  
 Cov (β$ , $
β ) Var (β$ )
0 1 1 

chee825 - Fall 2005 J. McLellan 37


Prediction Variance

… in matrix form -
T −1 2
$k ) = ξ T
var( y k ( Ξ Ξ ) ξ kσ
where x k is vector of conditions at k-th data point

chee825 - Fall 2005 J. McLellan 38


Joint Confidence Regions

Variability in data can affect parameter estimates jointly


depending on structure of data and model
section of sum of
squares
(or likelihood)
β 2 function

β1
marginal confidence limits
chee825 - Fall 2005 J. McLellan 39
Role of Experimental Design

• role of regressor values in data set is clearly seen from


covariance and prediction variance expressions
• choose experimental factor levels to -
» produce uncorrelated parameter estimates
• XTX will be diagonal
» minimize parameter estimate variances
» minimize prediction variance

chee825 - Fall 2005 J. McLellan 40


Outline

• dynamic models
» role
» sources of variability and need for disturbance models
» components in data
• statistical properties of a model - sensor analogy
• making a link to linear regression
» problem formulation and assumptions
» parameter estimates + properties
» role of experimental design
• motivating example - 1st order process
» “lagged regression” problem
» impact of non-ideality in disturbances - serial correlation

chee825 - Fall 2005 J. McLellan 41


Hot Oil Tank

TI

TIC
Input Sequence
Step

OR
PRBS FI

chee825 - Fall 2005 J. McLellan 42


Process Model #1

In difference equation form:

yt = h1ut −1 + h2ut − 2 + h3ut −3 + h4ut − 4 + et


» impulse response model with additive white noise white
noise
» current output depends on past control moves
» noise structure satisfies assumptions of standard
regression
» for deterministic input, matches standard regression
problem
– deterministic = non-random

In all cases, the et ‘s are independent, identically distributed


Gaussian (Normal) random shocks
chee825 - Fall 2005 J. McLellan 43
Formulation as Linear Regression Problem

 y5   u4 u3 u2 u1   h1   e5 
y   u u4 u3 u2  h2   e6 
6
 = 5   +  
 M  M   h3   M
 y  u u N − 4  h4  e N 
 N   N −1 u N − 2 u N −3

Ψ = ΥΗ + Ε
matrix of uncorrelated
T −1 T
deterministic Η∃ = (Υ Υ ) Υ Ψ (white)
elements noise
impulse weights as
chee825 - Fall 2005 J. McLellan regression parameters 44
Properties of Impulse Estimates - Case #1

In this scenario, the impulse weight estimates are:

» unbiased
» consistent
» best linear unbiased estimator
» essentially, possess standard properties of least squares
linear regression estimates
» if input is white noise, the analysis is more complex
• prove properties by examining dependence of estimates on
stochastic inputs

chee825 - Fall 2005 J. McLellan 45


Process Model #2

In difference equation form:

yt = a1 yt −1 + b1ut −1 + et
» AutoRegressive with eXogenous input (ARX) model
» dependence of current temperature on temperature at
previous sample time
» dependence of current disturbance component on value
at previous time - disturbance is no longer simply white
noise
» inertia in disturbance exactly matches that in process

chee825 - Fall 2005 J. McLellan 46


Another look at the Autoregressive disturbance

• Known as an AR(1) disturbance


AR parameter of 0.3 AR parameter of 0.6
– 1st order 4 5
• transfer function 2

output
output
1 0 0

-2
−1
1 − δζ -4
0 100 200 300
-5
0 100 200 300
• referred to as an AR parameter of 0.9 Non-stationary
“autoregressive” (AR) time 10 20
series model 5 10
» “d” is the AR parameter
output

output
0 0

-5 -10

-10 -20
0 100 200 300 0 100 200 300
Note the vertical scales of the plots time time

chee825 - Fall 2005 J. McLellan 47


Process Model #2

In transfer function form:

(1 − α1θ −1 ) ψτ = β1θ −1υτ + ετ


OR
b1q −1 1
yt = ut + et
(1 − a1q −1 ) (1 − a1q −1 )
{
chee825 - Fall 2005
process

{disturbance
dependence of process and
disturbance on previous
values is the same
J. McLellan 48
Formulation as Linear Regression Problem

 y2   y1 u1   e2 
y   y u  a e 
 3= 2 2  1  3 
  +
 M  M M  b1   M
y  y u  e 
 N   N −1 N −1   N

regressor matrix now


contains random
elements
Ψ = Φθ + Ε noise satisfies

θ∃= ( Φ T Φ) −1 Φ T Ψ
regression
assumptions
chee825 - Fall 2005 J. McLellan 49
Properties of the Estimates - Case #2

In this case, the coefficient estimates are:

» consistent
» asymptotically unbiased
» properties proved by examining dependence of the
estimates on the lagged outputs - “stochastic regressor
elements”
» key point is the manner in which disturbance appears
relative to process dynamics - same autoregressive
dynamics (denominator term)

chee825 - Fall 2005 J. McLellan 50


Process Model #3

nt
In difference equation form:

yt = a1 yt −1 + b1ut −1 + et − a1et −1
» Output Error model-
» dependence of current temperature on temperature at
previous sample time
» form of immediate dependence of current disturbance
component on value at previous time cancels process
inertia term
» scenario - disturbance free process output with added
white noise added - “measurement noise” model

chee825 - Fall 2005 J. McLellan 51


Process Model #3

nt
In transfer function form:

(1 − α1θ −1 ) ψτ = β1θ −1υτ + (1 − α1θ −1 ) ετ


OR
b1q −1
yt = ut + et
(1 − a1q −1 )
{
{
process disturbance

chee825 - Fall 2005 J. McLellan 52


Formulation as Linear Regression Problem

 y2   y1 u1   n2 
y   y u  a n 
 3= 2 2  1  3 
  +
 M  M M  b1   M
y  y u  n 
 N   N −1 N −1   N

regressor matrix now


contains random
elements noise no longer satisfies
Ψ = Φθ + Ε regression assumptions
- dependence on
θ∃= ( Φ T Φ) −1 Φ T Ψprevious shocks

chee825 - Fall 2005 J. McLellan 53


Properties of the Estimates - Case #3

In this case, the coefficient estimates are:

» biased
» not consistent
» likely to have poor precision - variance

chee825 - Fall 2005 J. McLellan 54


Conclusions

• need framework to investigate dynamics of disturbances and


process components, and how random components influence
estimation method and properties of estimates
• standard least squares estimation not always best approach -
need to examine validity of assumptions, particularly nature of
the disturbances

chee825 - Fall 2005 J. McLellan 55