Chapter 4

CURVE

FITTING

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

by Lale Yurttas, Texas

A&M University

Chapter 4

CURVE FITTING
Describes techniques to fit curves (curve fitting) to
discrete data to obtain intermediate estimates.
There are two general approaches for curve fitting:

Data exhibit a significant degree of scatter. The strategy is to

derive a single curve that represents the general trend of the
data.
Data is very precise. The strategy is to pass a curve or a series
of curves through each of the points.

In engineering two types of applications are encountered:

Trend analysis. Predicting values of dependent variable, may
include extrapolation beyond data points or interpolation
between data points.
Hypothesis testing. Comparing existing mathematical model
with measured data.

by Lale Yurttas, Texas

A&M University

Chapter 4

MATHEMATICAL
BACKGROUND

Arithmetic mean. The sum of the individual data

points (yi) divided by the number of points (n).
y

n
i 1, , n

Standard deviation. The most common measure of a

spread for a sample.
St
Sy
n 1
S t ( yi y ) 2
by Lale Yurttas, Texas
A&M University

or

Chapter 4

2
y

y y

2
i

/n

n 1

 Variance. Representation of spread by the square of

the standard deviation.
S y2

2
(
y

y
)
i

n 1

Degrees of freedom

Coefficient of variation. Has the utility to quantify the

spread of data.

c.v.

Sy
y

by Lale Yurttas, Texas

A&M University

100%

Chapter 4

LINEAR REGRESSION

Fitting a straight line to a set of paired

observations: (x1, y1), (x2, y2),,(xn, yn).

y a0 a1 x e
where:

a1

a1- slope
a0- intercept
e- error, or residual, between the
model and the observations

n xi yi xi yi
n x xi
2
i

Mean values

a0 y a1 x
by Lale Yurttas, Texas
A&M University

Chapter 4

Criteria for a Best Fit

Minimize the sum of the residual errors for
all available data:
n

e (y
i 1

i 1

ao a1 xi )

n = total number of points

However, this is an inadequate criterion.
Minimize the sum of the absolute values:
n

e y
i 1

i 1

by Lale Yurttas, Texas

A&M University

a0 a1 xi

Chapter 4

a) Minimizes the sum of

residuals

b) Minimizes the sum of

absolute values of
residuals

c) Minimizes the max

error of any individual
point
by Lale Yurttas, Texas
A&M University

Chapter 4

 Best strategy is to minimize the sum of the

squares of the residuals between the
measured y and the y calculated with the
n
n
n
linear
model:
2
2

S r ei ( yi , measured yi , model) ( yi a0 a1 xi ) 2
i 1

i 1

i 1

Yields a unique line for a given set of data.

Standard error of
S r the estimate, Sy/x:
Sy/x

by Lale Yurttas, Texas

A&M University

n2

Chapter 4

Figure 17.3

by Lale Yurttas, Texas

A&M University

Chapter 4

a) Linear regression with

small residual errors

b) Linear regression with

large residual errors

by Lale Yurttas, Texas

A&M University

Chapter 4

10

Estimation of Errors
St-Sr quantifies the improvement or error reduction due to
describing data in terms of a straight line rather than as an
average value.
r2 -coefficient of determination
r correlation coefficient

St S r
r
St
2

For a perfect fit:

Sr=0 and r=r2=1, signifying that the line explains
100 percent of the variability of the data.
For r=r2=0, Sr=St, the fit represents no
improvement.
by Lale Yurttas, Texas
A&M University

Chapter 4

11

Example 1
Table 1

Fit a straight line to

the x and y values in
table 1. Then
compute the total
standard deviation,
standard error of the
estimate and the
correlation coefficient
for the data

by Lale Yurttas, Texas

A&M University

Chapter 4

xi

yi

0.5

2.5

2.0

4.0

3.5

6.0

5.5
12

POLYNOMIAL
REGRESSION
Some engineering data is poorly
represented by a straight line.
For these cases a curve is better suited to
fit the data.
The least squares method can readily be
extended to fit the data to higher order
polynomials.
y a0 nd
a1 x a2 x 2 e
For example: 2 order polynomial

by Lale Yurttas, Texas

A&M University

Chapter 4

13

 Set of normal equations:

2
y

na

x
a

x
i 0 i 1 i a2

2
3
y
x

x
a

x
a

x
i i i 0 i 1 i a2

2
2
3
4
y
x

x
a

x
a

x
i i i 0 i 1 i a2

In matrix form:

x
x

i
2

by Lale Yurttas, Texas

A&M University

xi
2
x
i
3
x
i

2
x
i
3
x
i
4
x
i

a0
a
1
a2

Chapter 4

y
x y
x y
i

i i
2
i
i

14

 Standard error:
n

Sy/x

Sr

n (m 1)

2 2
(
y

a
x

a
x
i 0 1 i 2 i)
i 1

n (m 1)

Where: m = m-th order polynomial

by Lale Yurttas, Texas

A&M University

Chapter 4

15

Example 2
Table 2

Fit a second-order
polynomial to the x
and y values in table
2. Then compute the
total standard
deviation, standard
error of the estimate
and the correlation
coefficient for the
data.
by Lale Yurttas, Texas
A&M University

Chapter 4

xi

yi

2.1

7.7

13.6

27.2

40.9

61.1

16

MULTIPLE LINEAR
REGRESSION
Useful extension of linear regression in
case where y is a linear function of 2 or
more independent variables.
For example:
y a0 a1 x1 a2 x2 e

by Lale Yurttas, Texas

A&M University

Chapter 4

17

 Set of normal equations:

y na x a x a
y x x a x a x x a
y x x a x x a x a
i

i 1i

1i 1

2i

2
1i 1

1i 0

i 2i

2i 0

1i 2 i 1

1i 2 i

2
2i

In matrix form:

n
x1i
x2 i

by Lale Yurttas, Texas

A&M University

x
x
x x
1i
2
1i

1i 2 i

x
x x
x

a0

1i 2 i a1
2

2 i a2
2i

Chapter 4

y
x y
x y
i

1i

2i

18

 Standard error:
n

Sy/x

Sr

n (m 1)

2
(
y

a
x

a
x
)
i 0 1 1i 2 2i
i 1

n (m 1)

Where: m = dimension

by Lale Yurttas, Texas

A&M University

Chapter 4

19

Example 3
The data from table 3
was calculated from
the equation
y = 5 + 4x1 3x2. Use
multiple linear
regression to fit this
data.

by Lale Yurttas, Texas

A&M University

Table 3
x1

x2

10

2.5

27

Chapter 4

20

GENERAL LINEAR LEAST

SQUARES

General model:

y a0 z0 a1 z1 a2 z 2 am z m e
z0 , z1, , z m are m 1 basis functions

Express in matrix form:

Y Z A E
Z matrix of the calculated values of the basis functions
at the measured values of the independent variable

Y observed valued of the dependent variable

A unknown coefficients
E residuals
by Lale Yurttas, Texas
A&M University

Chapter 4

21

 Outcome / Solution to this model:

Z A Z Y
T

a) Equivalent to the normal equation developed

previously
b)3 solution technique:
(i) Gauss Elimination; (ii) Choleskys ; (iii) matrix
inverse
2

Minimized by taking its partial

derivative with respect to each
of the coefficients and setting
the resulting equation equal to
zero

n
m

Sum
squares
of
the
residuals:

S r yi a j z ji
i 1
j 0

by Lale Yurttas, Texas

A&M University

Chapter 4

22

NONLINEAR
REGRESSION
In engineering many cases where
nonlinear model must be fit to data.
General equation:

yi f ( xi ; a0 , a1 ,..., am ) ei
For example:

f ( x) a0 (1 e

a1 x

)e

Gauss Newton method algorithm for

minimizing the sum of squares of residual
between data and nonlinear equations.
by Lale Yurttas, Texas
A&M University

Chapter 4

23

Procedures
1. Linearized the original model with respect
to the parameter (e.g. : 2 parameter)

yi f ( xi ) j

f ( xi ) j
a0

a0

f ( xi ) j
a1

a1 ei

In matrix form:

D [ Z j ] A E

by Lale Yurttas, Texas

A&M University

Chapter 4

24

Where:
f1 / a0
f / a
2
0

[Z j ]

f1 / a1

f 2 / a1

.
.

by Lale Yurttas, Texas

A&M University

.
.

f n / a0 f n / a1
y1 f ( x1 )
a0
a
y 2 f ( x2 )
1

.
.
; A

.
.

.
.

an
y n f ( xn )
Chapter 4

25

2. Applying linear least square theory; solve

{A} T
T

Z A Z D

3. Improve
a0, j 1 values
a0, j of
a0 parameters:
; a1, j 1 a1, j a1

4. Repeated until the solution converges

falls below an acceptable stopping
ak , j 1 ak , j
criterion.

100%
a k

by Lale Yurttas, Texas

A&M University

ak , j 1

Chapter 4

26

 Sum of squares of the residuals

n

S r yi f ( xi )

i 1

by Lale Yurttas, Texas

A&M University

Chapter 4

27

Example 4
Fit the function

Table 4

f ( x) a0 (1 e a1 x ) e

to data in table 4.
Use initial guesses of
a0 = 1.0 and a1 = 1.0
for the parameters.

by Lale Yurttas, Texas

A&M University

Chapter 4

xi

yi

0.25

0.28

0.75

0.57

1.25

0.68

1.75

0.74

2.25

0.79

28