MATLAB Nonlinear regression A36

Introduction to Nonlinear regression using MATLAB

By Professor Tom Rumsey

Linear regression models are linear in their parameter values while not necessarily in their
independent variables. Nonlinear regression models are nonlinear in their parameters
(Himmelblau, 1970). A typical linear model is

p p 3 3 2 2 1 1 0
x ... x x x ! + ! + ! + ! + ! = "

where
! is the dependent variable
x
i
= independent variable
"
i
= model parameter

Other examples of linear and nonlinear models are given below (Himmelblau, 1970):

a.) Linear in ", nonlinear in x:
( )
1 4 2 1 3 2 2 1 1 0
x ln x x x x ! + ! + ! + ! + ! = "

b.) Linear in x, nonlinear in ":
3 2 2 1 1 2 1 0
x x x ! + ! + ! ! + ! = "

c.) Nonlinear in " and x:
( ) ( )
2 2 1 1
x exp x exp ! + ! = "
2 2 1 1 0
x x ! + ! + ! = "



MATLAB Nonlinear regression A37
The following notes contain examples of using the Matlab function nlinfit to determine
parameters to nonlinear regression models. Matlab can also handle linear regression
problems easily; they are covered in the built in help for Release 12.

1. Example one: using Matlab function nlinfit
Consider the equation

x b
ax
y
+
=

We assume we have experimental data for y versus x and want to determine the
parameters a,b that fit the curve to the data.

To gain confidence in the program, the first example will consider data generated with a=2,
b=5.

x 1 2 3 4 5 6 7 8 9 10
y 0.3333 0.5714 0.7500 0.8889 1.000 1.0909 1.1667 1.2308 1.2857 1.3333

Data were generated in the Matlab script nlinfit_example_one.m. The command window
input and output are shown followed by the plot of the data and the curve fit.

Command window output:

>> nlinfit_example_one
a_mod = 2.0000
b_mod = 5.0000

betaci =
2.0000 2.0000
5.0000 5.0000
>>


MATLAB Nonlinear regression A38


Matlab script: nlinfit_example_one.m

% Example problem using Matlab function nlinfit
% Generate a synthetic data using the equation y=ax/(b+x)
% The parameters a and b are to be determined using nlinfit
% from the data of y versus x.

% x values to be used in equation
x=[1:1:10];

% Set values for a, b to be used in the equation
a=2; b=5;

% Generate the array of "experimental" data
y=a*x./(b+x);

% set initial guesses for a, b
beta=[1 1];

% Use matlab function nlinfit to estimate the parameters
[betahat,f,J]=nlinfit(x,y,'nlin_exam_one_eqn',beta);

% Print out the results found by nlinfit
a_mod=betahat(1)
b_mod=betahat(2)

% Use Matlab function called nlparci to get confidence intervals
% on parameter estimates
betaci=nlparci(betahat,f,J)

% Plot predicted and experimental results
x_mod=linspace(1,10);
y_mod=a_mod*x_mod./(b_mod+x_mod);



MATLAB Nonlinear regression A39
plot(x,y,'ko',x_mod,y_mod,'k-')
xlabel('x')
ylabel('y')
grid
legend('expt','model')
title('Curve Fit for Nlinfit Example 1')
param=['a mod = ',num2str(a_mod),', b mod= ',num2str(b_mod)];
text(2,1.4,param)

Matlab function: nlin_exam_one_eqn.m

function yhat=nlin_exam_one_eqn(beta,z)
% Example for this function follows page 1-81 in
% Matlab Statistics Toolbox Manual

a=beta(1);
b=beta(2);

x=z(:,1);
yhat=a*x./(b+x);



MATLAB Nonlinear regression A40
2. Example using Matlab function nlinfit with simulated noisy data.

Consider the same equation used in part 1:

x b
ax
y
+
=

We assume we have experimental data for y versus x and want to determine the
parameters a,b that fit the curve to the data. This second example will consider data
generated with a=2 and b=5, but this time the values of y were perturbed by using some
random number generators in Matlab. Use of simulated data to evaluate nonlinear
regression methods is a technique used by Himmelblau, 1970. (see example 6.2-1, page
170 of his text).
y
x b
ax
y !
+
=

#y is normally distributed random number with a mean of zero and standard deviation of
sigma (input from program).

Command window output:

>> nlinfit_example_two
a_mod = 2.0054
b_mod = 4.9725
betaci =
1.9010 2.1097
4.3945 5.5505



MATLAB Nonlinear regression A41



Matlab script: nlinfit_example_two.m

% Example problem two using Matlab function nlinfit
% generate a curve y=ax/(b+x)
% a and b are parameters to be determined from
% data of y versus x.

x=[1:1:10];

% give values for a, b and see if nlinfit can retrieve them
% a=2, b=5
a=2; b=5;

% Now add some random noise to y experimental data.
% Matlab function randn generates normally distributed
% random numbers with 0 mean and standard deviation of 1. Multiplying
% it's output by sigma gives a random number with std. dev. of sigma.

sigma=0.02;
for i=1:10
del_y= sigma*randn(1);
yr(i)=a*x(i)./(b+x(i)) + del_y;
end

% plot experimental data points with noise
figure(1)
plot(x,yr,'ko')
grid
xlabel('x')
ylabel('y')
legend('noisy data')
title('Nlinfit Example with Noisy Data')


MATLAB Nonlinear regression A42
% set initial guesses for a, b
beta=[1 1];

% Use matlab function nlinfit to estimate the parameters
[betahat,f,J]=nlinfit(x,yr,'nlin_exam_two_eqn',beta);

a_mod=betahat(1)
b_mod=betahat(2)


% Use Matlab function called nlparci to get confidence intervals
% on parameter estimates

betaci=nlparci(betahat,f,J)

% Plot predicted and experimental results
x_mod=linspace(1,10);
y_mod=a_mod*x_mod./(b_mod+x_mod);

figure(2)
plot(x,yr,'ko',x_mod,y_mod,'k-')
xlabel('x')
ylabel('y')
legend('noisy expt','model')
title('Nlinfit Example with Noisy Data')
param=['a mod = ',num2str(a_mod),', b mod= ',num2str(b_mod)];
text(2,1.4,param)

Matlab function: nlin_exam_two_eqn.m

function yhat=nlin_exam_two_eqn(beta,z)

% Example for this function follows page 1-81 in
% Matlab Statistics Toolbox Manual

a=beta(1);
b=beta(2);

x=z(:,1);

yhat=a*x./(b+x);







MATLAB Nonlinear regression A43
3. Example using Matlab function nlinfit with two independent variables.

Consider the equation:
c b
z ax y =

This is an problem used in the text by Himmelblau (1970) (example 6.2-1, page 179). We
assume we have experimental data for y versus x,z and want to determine the parameters
a,b, and c that fit the curve to the data.

This third example will consider data generated with a=2, b=2.5, c=0.5, and the values of y
were perturbed by using random number generators in Matlab.

y z ax y
c b
! =

where

#y is normally distributed random number with a mean of zero and standard deviation of
sigma (input from program). Wrote Matlab script nlinfit_example_three.m to do the
calculations.

Command window inputs and outputs

>> nlinfit_example_three

sigma = 0.1000
a_mod = 1.9864
b_mod = 2.5151
c_mod = 0.4989

betaci =

1.9576 2.0153
2.4891 2.5411
0.4874 0.5104

>>

MATLAB Nonlinear regression A44





Matlab script: nlinfit_example_three.m

% Example problem three using Matlab function nlinfit

% generate a curve y=a*(x^b)*(z^c)
% a, b, and c are parameters to be determined from
% data of y versus x and z.

clear all

% x=[.5:.2:2];
% z=[.5:.2:2];

% Set values for a, b, and c. Will see if nlinfit can retrieve them.
a=2; b=2.5; c=0.5;

% Generate some "experimental data with random noise.

% Matlab function generates normally distributed
% random numbers with 0 mean and standard deviation of 1.
% Multiplying it's output by sigma gives a random number
% with std. dev. of sigma.

% Set value for sigma and initialize the case counter
sigma=0.2
k=0;
nx=10;
ny=10;

% Loops to generate the synthetic data

MATLAB Nonlinear regression A45
for i=1:nx
for j=1:ny

% update k, the case counter
k=k+1;

% gnerate x and z values from 0.5 to 2
x(i)=.5+(i-1)/(nx-1)*1.5;
z(j)=.5+(j-1)/(ny-1)*1.5;

% generate some random experimental error
del_y= sigma*randn(1);

% calculate experimental data point for kth case
yr(k)=a*(x(i)^b)*(z(j)^c) + del_y;
% store values of independent variables for nlinfit
w(k,1)=x(i);
w(k,2)=z(j);
% store the yr value in a 2D array for plotting
yz(i,j)=yr(k);
end
end

% get the dependent variable into column form
yr=yr';

% set initial guesses for a, b, c
beta=[1 1 1];

% Use Matlab function nlinfit to estimate the parameters
[betahat,f,J]=nlinfit(w,yr,'nlin_exam_three_eqns',beta);

% Print out the results
a_mod=betahat(1)
b_mod=betahat(2)
c_mod=betahat(3)


% Use Matlab function called nlparci to get confidence intervals
% on parameter estimates
betaci=nlparci(betahat,f,J)


% Plot predicted and experimental results

% generate a surface using the parameter estimates
x_mod=linspace(.5,2,40);
z_mod=linspace(.5,2,40);

for i=1:length(x_mod)
for j=1:length(z_mod)
yr_mod(i,j)=a_mod*(x_mod(i)^b_mod)*(z_mod(j)^c_mod);
xmod(i,j)=x_mod(i);
zmod(i,j)=z_mod(j);
end
end


MATLAB Nonlinear regression A46

% Surface plot of curve fit
figure(1)
mesh(zmod,xmod,yr_mod,'edgecolor','black')
xlabel('x')
ylabel('z')
zlabel('y')
title('Curve Fit of Experimental Data')

hold on

% Overlay plot of "experimental" data points
stem3(x,z,yz)

hold off





Matlab function: nlin_exam_three_eqns.m

function yhat=nlin_exam_three_eqns(beta,w)

% Example for this function follows page 1-81 in
% Matlab Statistics Toolbox Manual


a=beta(1);
b=beta(2);
c=beta(3);

x=w(:,1);
z=w(:,2);


yhat=a*(x.^b).*(z.^c);



Reference

Himmelblau, D.M. 1970. Process Analysis by Statistical Methods. Sterling Swift Pub. Co.
Manchaca, Texas.