Curve Fitting & Nonlinear Regression

53 Pages·2012·1.552 MB·English

by  G. David Garson

CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 1
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

@c 2012 by G. David Garson and Statistical Associates Publishing. All rights reserved
worldwide in all media. No permission is granted to any user to copy or post this work in
any format or any media.
The author and publisher of this eBook and accompanying materials make no
representation or warranties with respect to the accuracy, applicability, fitness, or
completeness of the contents of this eBook or accompanying materials. The author and
publisher disclaim any warranties (express or implied), merchantability, or fitness for any
particular purpose. The author and publisher shall in no event be held liable to any party for
any direct, indirect, punitive, special, incidental or other consequential damages arising
directly or indirectly from any use of this material, which is provided “as is”, and without
warranties. Further, the author and publisher do not warrant the performance,
effectiveness or applicability of any sites listed or linked to in this eBook or accompanying
materials. All links are for information purposes only and are not warranted for content,
accuracy or any other implied or explicit purpose. This eBook and accompanying materials is
© copyrighted by G. David Garson and Statistical Associates Publishing. No part of this may
be copied, or changed in any format, sold, or used in any way under any circumstances
other than reading by the downloading individual.
Contact:
G. David Garson, President
Statistical Publishing Associates
274 Glenn Drive
Asheboro, NC 27205 USA

Email: [email protected]
Web: www.statisticalassociates.com

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 2
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

Table of Contents
Overview ......................................................................................................................................... 5
Curve Fitting .................................................................................................................................... 5
Key Concepts and Terms ............................................................................................................ 5
Curve Estimation dialog in SPSS ............................................................................................ 5
Models ................................................................................................................................... 6
Statistical output for the SPSS curve estimation module ........................................................ 19
Comparative fit plots ........................................................................................................... 19
Regression coefficients ........................................................................................................ 20
R-square ............................................................................................................................... 21
Analysis of variance table .................................................................................................... 21
Saved variables .................................................................................................................... 23
Curve Estimation Assumptions ................................................................................................ 23
Data dimensions .................................................................................................................. 23
Data level ............................................................................................................................. 24
Randomly distributed residuals ........................................................................................... 24
Independence ...................................................................................................................... 24
Normality ............................................................................................................................. 24
Curve Fitting: Frequently Asked Questions .............................................................................. 24
Can the SPSS Curve Estimation module tell me what type of model I need (ex., linear,
logarithmic, exponential)? ................................................................................................... 24
I want to use, from the Curve Estimation module, the two best functions of my
independent in a regression equation, but will this introduce multicollinearity? .............. 30
What software other than SPSS is available for curve fitting? ............................................ 30
Nonlinear Regression .................................................................................................................... 32
Overview .................................................................................................................................. 32
Key Concepts and Terms .......................................................................................................... 33
Linearization ........................................................................................................................ 33
Nonlinear regression example ................................................................................................. 36
Entering a model ................................................................................................................. 36
Parameters .......................................................................................................................... 37
Other input options ............................................................................................................. 38
Statistical Output ...................................................................................................................... 41

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 3
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

Parameter Estimates Table.................................................................................................. 42
Correlation of Parameter Estimates Table .......................................................................... 43
ANOVA Table and R2 ............................................................................................................ 44
Modeling multiple individuals .................................................................................................. 44
Overview .............................................................................................................................. 44
Data setup ............................................................................................................................ 44
Segmented models ................................................................................................................... 46
Conditional logic statements ............................................................................................... 46
Alternative models as multiple conditions .......................................................................... 46
Nonlinear regression assumptions ........................................................................................... 47
Data level ............................................................................................................................. 47
Proper specification ............................................................................................................. 47
Nonlinear regression: Frequently asked questions ................................................................. 48
Bibliography .................................................................................................................................. 51

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 4
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

Curve Fitting and Nonlinear Regression
Overview
Both curve fitting and nonlinear regression are methods of finding a best-fit line
to a set of data points even when the best-fit line is nonlinear.
Below, curve-fitting is discussed with respect to the SPSS curve estimation
module, obtained by selecting Analyze > Regression > Curve Estimation. This
module can compare linear, logarithmic, inverse, quadratic, cubic, power,
compound, S-curve, logistic, growth, and exponential models based on their
relative goodness of fit where a single dependent variable is predicted by a single
independent variable or by a time variable. As such it is a useful exploratory tool
preliminary to selecting multivariate models in generalized linear modeling, which
supports nonlinear link functions. (Generalized linear modeling is treated in a
separate Statistical Associates "Blue Book" volume).
The province of nonlinear regression is fitting curves to data which cannot be
fitted using nonlinear transforms of the independent variables or by nonlinear link
functions which transform the dependent variable. This type of data is
"intrinsically nonlinear" and requires approaches treated in a second section of
this e-book, which covers nonlinear regression in SPSS, obtained by selecting
Analyze > Regression > Nonlinear.

Curve Fitting
Key Concepts and Terms
Curve Estimation dialog in SPSS
In SPSS, select Analyze, Regression, Curve Estimation to bring up this dialog, in
which a single dependent and single predictor may be entered, optionally with a
time variable, and any of 11 models requested. If a Case Labels variable is
entered, such as "City" here, then in the Chart Editor (invoked by double-clicking

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 5
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

on a plot in output) one may use the Data Label Mode tool to click on points and
label them, as illustrated in the FAQ section below.

Models
Models are types of linear and nonlinear curves which may be fitted to the data.
SPSS supports these models: linear, logarithmic, inverse, quadratic, cubic, power,
compound, S-curve, logistic, growth, and exponential. Using the SPSS menu
choice Analyze, Legacy Dialog, Scatter/Dot, will allow the researcher to plot
dependent variables, which may aid the researcher in selecting a suitable model
to fit. However, before selecting a more complex model the researcher should
first consider if a transformation of the data might enable a simpler one to be
used, even linear regression.
Residual models. The SPSS Curve Estimation module only supports one dependent
and one independent variable. While this is suitable for bivariate analysis, for
multivariate analysis it is at best a "quick and dirty" tool for assessing if one of
multiple independent variables is related to the dependent in one of the 10

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 6
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

supported nonlinear manners. An alternative strategy is to use OLS, ordinal,
multinomial, or some other form of multivariate regression to regress a given
independent variable on all the other independents, then save the residuals. The
residuals then represent the variance in the given independent once all other
independents are controlled. One may then us these residuals as the independent
variable in the SPSS Curve Fitting module, using it to predict the dependent under
any of the supported linear and nonlinear models.
The choice between a regular (raw data) and a residual model depends on
whether the researcher is interested in uncontrolled or in controlled
relationships. Put another way, the standardized b coefficients in the
uncontrolled, bivariate raw data approach are whole coefficients, equal to the
correlation of the independent with the dependent. The standardized b
coefficients in the controlled, multivariate residual approach are partial
coefficients, partialing out the effect of other independent variables. Generally,
partial coefficients are preferred for most multivariate analysis purposes.
Time series models. In the Curve Estimation dialog, illustrated above, if the "Time"
radio button is turned on, SPSS assumes time series data with a uniform time
interval separating cases in the series. That is, each data row is assumed to
represent observations at sequential times which are uniformly spaced. It is
assumed, of course, that the dependent variable is also a time series variable. A
"Sequence" variable is created automatically and is used as the independent
(other predictor variables cannot be used if the "Time" option is selected). If the
"Time" option is selected, the time variable, t, replaces the independent variable,
x, in the equations given below. and one can specify a forecast period past the
end of the time series.

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 7
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

Linear models. Y = b0 + (b1 * x), where b0 is the constant, b1 the regression
coefficient for x, the independent variable. Note: in this and figures below, the
exact shape of the curve (line) is greatly affected by the parameters. Each figure
only represents one particular set of parameters, of course. In the figure below,
b0 is 4.818 and b1 is .436 in the "Model Summary and Parameter Estimates"
output table.

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 8
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

Logarithmic models. Y = b0 + (b1 * ln(x)) where ln() is the natural log function. In
the figure below, b0 is 5.422 and b1 is 1.113, in the "Model Summary and
Parameter Estimates" output table.

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 9
Single User License. Do not copy or post.
CURVE FITTING & NONLINEAR REGRESSION 2012 Edition

Inverse models. Y = b0 + (b1 / x). In the figure below, b0 is 7.194 and b1 is -1.384,
in the "Model Summary and Parameter Estimates" output table.

Copyright @c 2012 by G. David Garson and Statistical Associates Publishing Page 10
Single User License. Do not copy or post.