Wind Tunnel Database Development using Modern Experiment Design and Multivariate Orthogonal Functions

18 Pages·2003·0.97 MB·English

by  Morelli, Eugene A.

AIAA 2003-0653
Wind Tunnel Database Development
using Modern Experiment Design and
Multivariate Orthogonal Functions
Eugene A. Morelli
NASA Langley Research Center
Hampton, VA
Richard DeLoach
NASA Langley Research Center
Hampton, VA
41stAIAA Aerospace Sciences Meeting and Exhibit
January 6-9, 2003 / Reno, NV
For permission to copy or to republish, contact the American Institute of Aeronautics and Astronautics,
1801 Alexander Bell Drive, Suite 500, Reston, VA, 20191-4344

AIAA-2003-0653
WIND TUNNEL DATABASE DEVELOPMENT USING MODERN EXPERIMENT
DESIGN AND MULTIVARIATE ORTHOGONAL FUNCTIONS
Eugene A. Morelli* and Richard DeLoacht
NASA Langley Resem'ch Center
Hampton, Virginia US,_ 23681- 2199
Abstract Nomenclature
A wind tunnel experiment for characterizing the a parameter vector
aerodynamic and propulsion forces and moments Q,CD, q. lift, drag, and side force coefficients
acting on a research model airplane is described. The
Q rolling moment coefficient
model airplane, called the Free-flying Airplane for
Sub-scale Experimental Research (FASER), is a C,. pitching moment coefficient
modified off-the-shelf radio-controlled model
C. yawing moment coefficient
airplane, with 7 ft wingspan, a tractor propeller
Coy covariance matrix
driven by an electric motor, and aerobatic capability.
FASER was tested in the NASA Langley 12-foot J cost function
Low-Speed Wind Tunnel. using a combination of MDOE Modem Design Of Experiments
traditional sweeps and modem experiment design. number of model terms
n
Power level was included as an independent variable
N total number of data points
in the wind tunnel test, to allow characterization of
OFAT One Factor At a Time
power effects on aerodynamic forces and moments.
PSE predicted squared error
A modeling technique that employs multivariate
orthogonal functions was used to develop accurate pwr power level, percent
analytic models for the aerodynamic and propulsion P modeling function vector
force and moment coefficient dependencies from the T thrust force, Ibf
wind tunnel data. Efficient methods for generating
x independent variable vector
orthogonal modeling functions, expanding the
Y measured output vector
orthogonal modeling functions in terms of ordinary
Oc angle of attack, deg
polynomial functions, and analytical orthogonal
blocking were developed and discussed. The sideslip angle, deg
resulting models comprise a set of smooth, aileron deflection, deg
differentiable functions for the non-dimensional
8e elevator deflection, deg
aerodynamic force and moment coefficients in terms
of ordinary polynomials in the independent variables, _f flap deflection, deg
suitable for nonlinear aircraft simulation.
8r rudder deflection, deg
cr 2 variance
*Research Engineer, Senior Member AIAA ordinary polynomial function vector
_"Senior Research Scientist
Copyright © 2003 by the American Institute of Aeronautics and superscripts
Astronautics, Inc. No copyright is asserted in the United States
T transpose
under Title 17, US. Code The U.S. Government has a royalty-
free license to exercise all rights under the copyright claimed estimate
herein for Governmental purposes. All other rights are reserved by
-1 matrix inverse
the copyright owner.
normalized
1
American Institute of Aerona_tics and Astronautics
subscripts
FASER was designed so that the flight vehicle
max maximum could be installed in the wind tunnel, see Figure 1.
min minimum This avoids any Reynolds number or scaling effects,
o nominal and ensures that there are no physical differences
between the wind tunnel model and the flight vehicle.
In contrast, full scale flight tests and drop model
Introduction tests are expensive and are sometimes separated by
months (and even years) for a particular research
Modem aeronautical research involves expanded
activity. The number of these tests is always tightly
flight envelopes, as a result of the desire for improved
constrained by budget. There is also a substantial
fighter maneuverability for tactical advantages, and the
difference in the cost of overhead if the aircraft is to be
desire to improve flight safety. The expanded flight
kept in flyable condition. Since FASER is inexpensive
envelopes involve nonlinear aerodynamics which must
and unmanned, risks can be taken in research and
be modeled accurately.
development that could never be tolerated in a piloted
Since nonlinear aerodynamics are much more flight test or even in a drop model test. Advances in
complex than linear aerodynamics, more sophisticated instrumentation now make it possible to instrument a
experimentation is required to accurately characterize sub-scale model aircraft with research-quality,
the functional dependencies. Nonlinear aerodynamics miniaturized flight test instrumentation at a reasonable
violate linear modeling assumptions such as cost.
superposition, quasi-steady flow, and no
This paper describes the experiment design, data
interdependence of the effects of states and controls. In
analysis, and mathematical modeling involved in
addition, aircraft designs have evolved with increasing
developing a wind tunnel database for the FASER
numbers of control effectors. Traditional wind tunnel
aircraft. Accuracy of this database is critical for the
testing methods would set each control effector to
development of a high-fidelity nonlinear simulation to
different fixed levels while sweeping through angle of
be used for control law design, flight envelope
attack and sideslip angle, for example. With such an
expansion, flight experiment design, and pilot training.
approach, the number of data points required increases
A preliminary version of the nonlinear simulation for
exponentially with the number of control effectors, if
FASER has already been developed, using U.S. Air
information on control surface interaction effects is
Force DATCOM to generate an aerodynamic model,
desired. These considerations highlight the need to
with experimentally-determined values for the mass
develop more efficient wind tunnel testing and
and inertia characteristics of FASER 1. The work
modeling techniques to accurately characterize
described in this paper will upgrade the aerodynamic
nonlinear aerodynamics, with possible interaction
model with analytic models derived from wind tunnel
effects among a large number of independent variables.
data, add an engine thrust model, and include
At NASA Langley, the Free-flying Airplane for propulsion effects on the aerodynamics. Since FASER
Sub-scale Experimental Research (FASER) is being was intended to be a research vehicle from the outset,
developed to study problems such as that described the approach used for the experiment design and data
above. FASER is a modified off-the-shelf radio control analysis for the wind tunnel testing was not traditional.
model called the Ultra-Stick, manufactured by Hangar This paper explains how the wind tunnel testing was
9, Ltd., see Figure 1. FASER has a conventional high- done, and examines the results. The paper also
wing and tail configuration with 7 ft wingspan, a describes a method for generating orthogonal modeling
foldable tractor propeller driven by an electric motor, functions based on the independent variable data, along
and acrobatic capability. subsequent expansion of the orthogonal modeling
functions in terms of ordinary multivariate
The purpose of FASER is to provide an
polynomials. This method is slightly different from
inexpensive aircraft for developing and demonstrating
that described in Refs. [2] and [3], and represents an
advanced experiment design, data analysis and
evolutionary improvement of the technique. In Ref.
modeling techniques, and control law design methods.
[2], the concept of response surface modeling using
As long as the goal is technology demonstration or
multivariate orthogonal functions was successfully
basic research, a sub-scale model that is not
applied to inference subspaces for limited ranges of
dynamically scaled for a specific full-scale aircraft is a
angle of attack and Mach number with fixed sideslip
completely acceptable test vehicle for these purposes.
angle and control surface deflections. This paper
2
American Institute of Aeronautics and Astronautics
extends the multivariate orthogonal function modeling nt;n-dimensional coefficients, get an idea of the
concept to identify aerodynamic models for a large response levels, collect basic static stability and trim
flight envelope, with more independent variables. infbrmation, and define the boundaries of the
independent variable subspaces to be used for further
Experiment Design experimentation. OFAT sweeps were used because the
data acquisition system in the NASA Langley 12-foot
For this wind tunnel test, the fundamental
Low-Speed Wind Tunnel is set up to collect OFAT data
objective was to find a mathematical description for the
in an automated fashion, making the sweeps very
dependence of non-dimensional aerodynamic force and
efficient in terms of collecting data points in minimum
moment coefficients on independent variables that are
time. However, the initial OFAT sweeps are really
varied during the experiment. Each mathematical
only intended for qualitative use, namely to define the
description or model can be thought of geometrically as
b_,tmdaries for subspaces that will be the focus of
a hyper-surface, also called a response surface. Critical
detailed experimentation and modeling in procedure
issues for successfully identifying an adequate response
four. One advantage of operating in this manner is that
surface model from experimental data include the
any issues related to instrumentation, data collection, or
experiment design (or, how the independent variable
experimental procedures can be worked out during the
values are set when measuring the output responses),
OFAT sweeps without impact on the experiment.
noise level on the measured outputs, identification of a
because the data from the OFAT sweeps is being used
mathematical model structure that can capture the
for qualitative purposes only. This approach also
functional dependence of the output variables on the
provides a good rough overview of the landscape' that
independent variables, accurate estimates of unknown
characterizes the dependence of force and moment
parameters in the identified model structure, and the
coefficients on the independent variables.
ability of the identified model to predict outputs for
data that was not used to identify the response surface The independent variables for the FASER wind
model. ttmnel test were angle of attack a, sideslip angle ft,
p_)wer level pwr, elevator deflection 8e, aileron
The experiment design used for FASER wind
deflection _a, rudder deflection _r, and flap
tunnel testing was a hybrid design broken down into a
series of procedures. The procedures are listed in deflection _y-. The response variables were
Table 1. Randomization 4-6 was used throughout the n_n-dimensional aerodynamic coefficients for lift, drag,
testing, to separate independent variable effects from
a:ld side forces (Cz,CD, andCy), and rolling,
time-dependent systematic errors.
pitching, and yawing moments (CI.C m, and Cn).
The first procedure consisted of randomized
Each data point produced measured values for all
engine power sweeps with the wind tunnel air off, to
independent and response variables.
determine the static thrust from the electric motor and
the propeller. All of these runs were made with the The experiment was designed assuming an), of the
model at zero angle of attack and zero sideslip angle, so independent variables could influence any of the
that the thrust measurement was obtained from the response variables. It was assumed (as an initial guess
longitudinal force measured by the balance mounted in oaly) that the dependencies could be modeled
the model, Figure 2 shows the static thrust plotted as a accurately with polynomial terms in the independent
function of pulses per second from a Hall effect variables of order 3 or less within each independent
transducer on the electric motor, which is proportional variable subspace. In addition, it was assumed that
to the propeller RPM. The model shown in Figure 2is 14mgitudinal controls ( 8e,_f. and pwr) do not interact
the result of a simple least squares fit of thrust to pulse
with lateral/directional controls (t_a and 8r). In
count, using a quadratic model structure. This model
structure was identified automatically from the data, practical terms, this meant that longitudinal and
lateral/directional controls were not varied
using the orthogonal function modeling technique
described later. simultaneously to allow their mutual interaction effects
to be quantified. As a result, the subspaces were called
In the second and third procedures, the approach "longitudinal" if the longitudinal controls were moved,
was to use One Factor At a Time (OFAT) sweeps, and "lateral/directional" if the lateral/directional
wherein one independent variable is changed with all controls were moved.
others held constant, to characterize the general
topology of the response surfaces for the Based on experience with similar airplanes, it was
l,nown that the dependence of non-dimensional
3
American Institute ofAeronautics and Astronautics
aerodynamic force and moment coefficients on control independent variable values are found by mapping the
surface deflections could be modeled with low order independent variable values in engineering units for
polynomials for the entire range of control surface each subspace onto the interval [-1,1]. The
deflections. With that assumption, it was not necessary normalization of each independent variable was
to vary the control surface deflections to search for implemented by
inference subspace boundaries along the dimensions of
the independent variable space corresponding to control
X-- Xmm )
surface deflections. Inference subspace boundaries (1)
"r=-1+ 2(xmax -Xmin )
were therefore sought only for _ fl, and pwr. These
independent variables were varied using OFAT sweeps
to identify the inference subspace boundaries. where _ was the normalized value of the independent
variable, and the independent variable range in
Figure 3 shows an OFAT sweep on angle of
engineering units was [xmi,,Xm_ ]. The inverse
attack. The vertical lines mark the selected subspace
transformation was
boundaries in angle of attack, which are intended to
mark the boundaries of regions where the character of
the response surfaces change. There is a trade-off in (J+l)
(2)
selecting the subspace boundaries, in that more X= Xmin+T( Xmax--Xmi,1)
subspaces mean more individual experimentation
regions in procedure four, while fewer subspaces All modeling for the inference subspaces was
generally require more resources in the data collection done using normalized values of the independent
and modeling for each subspace. Figures 4 and 5show variables. The fmal models used for prediction were
OFAT sweeps on sideslip angle and power level, with written in terms of engineering units.
the selected subspace boundaries marked by vertical
lines. All control surface deflections were zero for the Second-order central composite design 5,6 in four
sweeps shown in Figures 3-5. independent variables (for the power-off subspaces) or
five independent variables (for the power-on
The full independent variable ranges tested are subspaces) was used in each subspace, augmented with
listed in Table 2. Tables 3 and 4, and corresponding a 3ra order D-optimal design 5,6 in the appropriate
Figures 6, 7, and 8, show the definitions of the number of independent variables. A two-dimensional
inference subspaces in terms of boundary values of projection of this constellation of data points in
angle of attack, sideslip angle, and power level. The normalized independent variable space is shown in
symmet D, of the vehicle was used to justify omitting Figure 9. The central composite design points occupy
testing in most of the subspaces with high negative the comers, the centers of each face, and the center of
sideslip angles, see Figures 6, 7, and 8. All the the normalized subspace, while the D-optimal points
subspaces together comprised the full inference space, generally fill in between. Although some of the
defined by the full range of the independent variables in D-optimal points are coincident with the central
Table 2. All control surfaces were tested over their full
composite design points, the number of times that this
physical deflection ranges for each subspace. happens is not represented accurately in Figure 9,
In the fourth and final procedure, Modem Design because of the projection onto two dimensions. This
experiment design enabled identification of up to 3ra
Of Experiment (MDOE) techniques 4-6 were applied to
each defined subspace in order to obtain the most order functional dependencies and interaction effects.
Provisions were made to augment the designed
accurate and complete characterization of the functional
dependencies, and also to make sure all interaction experiment if the data indicated a lack of fit that
required modeling functions with higher than 3_aorder.
effects were adequately modeled. Refs. [2], [4]-[6]
describe and demonstrate in detail the advantages of the
MDOE approach compared to traditional OFAT for Instrumentation and Data Collection
detailed modeling of the functional dependencies, both
FASER was used as the wind tunnel model and
in terms of the modeling accuracy and in the economy
of experimentation resources required to arrive at an tested at a nominal flight speed, thus avoiding scaling,
acceptable result. Reynolds number, or geometric dissimilarity issues for
comparisons with future flight test data. All control
Within each subspace, independent variables were surfaces were instrumented with potentiometers. Air
set according to normalized values. Normalized data vanes and airspeed pinwheels were installed on
4
American Institute of Aeronautics and Astronautics
booms attached at each wing tip and extending 1chord Modeling,
length in front of the wing. The air data sensors, which
Typically, once the experimental data are
will be used for flight testing, were calibrated as part of
collected, polynomials in the independent variables are
the wind tunnel experiment, since aerodynamic
used to model the functional dependence of the output
incidence angles and airspeed were carefully controlled
variables on the independent variables, and the model
and measured in the wind tunnel. The wind tunnel
parameters are estimated from the measured data using
balance was installed near the e.g. of the airplane in the
least squares linear regression 5,6. The question of
space normally occupied by the accelerometer and rate
which polynomial terms should be included in the
gyro package during flight test operations.
model for a given set of data, called model structure
Control surface deflections and power level were determination, gets more difficult with increases in the
automatically set to the values required by the number of independent variables, increased ranges for
experiment design via a serial port interface from a the independent variables, or increased complexity of
laptop computer in the control room to a the underlying functional dependency.
servomechanism controller onboard on the airplane.
Various hypothesis testing techniques 6,7 can be
The same onboard equipment will be used to command
used to identify an adequate model structure, but these
the control surfaces and power level during flight
methods are iterative and require the involvement of an
operations. Angle of attack and sideslip angle were set
experienced analyst. Neural networks using radial
from the control room using servomechanisms driving
basis functions with subspace partitioning, or back
the movable C-strut in the test section. The angle of
propagation with layered and interconnected nonlinear
attack and sideslip angles were set automatically during
activation functions, have also been applied to the
OFAT sweeps, but had to be set manually for the
response surface modeling problem 8. For this type of
MDOE data points, using joystick controllers and a
approach, there is a loss of physical insight and a
measurement feedback to the control room. Dynamic
danger of over-fitting the data, because the model
pressure in the tunnel was regulated to 2 psf by an
structures contain many parameters, typically with no
automatic closed loop control on the wind tunnel fan
mechanism for limiting the size of the model other than
motor speed.
the judgment of the analyst.
The experimental set-up was designed to
In this work, a nonlinear multivariate orthogonal
accommodate the MDOE approach, which typically
modeling technique 2,3 was used to model response
requires changes in more than one independent variable
surfaces from wind tunnel data. The technique
for successive data points. Since the control surface
generates nonlinear orthogonal modeling functions
and power level settings were automated using the
from the independent variable data, and uses those
laptop computer, each data point required only manual
modeling functions with a predicted squared error
setting of angle of attack and sideslip angle using the
metric to determine appropriate model structure. The
joysticks in the control room. Each data point was
orthogonal functions are generated in a manner that
taken as the average of a ten-second dwell using a
allows them to be decomposed without ambiguity into
sampling rate of 100 Hz.
an expansion of ordinary multivariate polynomials.
The data for power-on subspaces was collected by This allows the identified orthogonal function model to
interleaving power-on points with power-off points bc converted to a multivariate ordinary polynomial
from other subspaces, in order to keep the engine expansion in the independent variables, which provides
temperature at low levels for extended testing periods. physical insight into the identified functional
This was necessary to avoid damage to the electric dependencies.
motor. A regulated DC power supply was used to
The next section briefly describes the multivariate
power the electric motor, so that batteries would not
orthogonal function modeling approach. In the Results
have to be swapped in and out. The power-on
section, the multivariate orthogonal function modeling
subspaces were limited to relatively low angle of attack
method is applied to identify response surface models
and sideslip angle (see Figure 7), because of excessive
fi,r non-dimensional aerodynamic force and moment
vibration of the wind tunnel rig and model for powered
coefficients for inference subspaces, based on data
runs at high angles of attack and/or high sideslip
from the FASER wind tunnel test.
angles.
The wind tunnel experiment described above was
conducted over 4 weeks in May 2002, in the NASA
Langley 12-foot Low-Speed Wind Tunnel.
5
American Institute ofAeronautics and Astronautics
Multivariate Orthogonal Functions
Assume an N-dimensional vector of response
variable values, y = [Yl,Y2.....YN ]r, modeled in terms where E is the expectation operator, and the error
of a linear combination of n modeling functions variance o-2 can be estimated from the residuals,
pj, j = 1,2.....n. Each pj is an N-dimensional vector
v = y - Pti (9)
which in general depends on the independent variables.
Then.
Y = al Pl +a2 P2 + .,.+ anPn +£ (3) _2_ 1 Pd)]- vWv
(N-n) [(y_pd:)T (y_ (N-n) (I0)
The as, j = 1,2..... n are constant model parameters to
and n is the number of elements in parameter vector a.
be determined, and _denotes the modeling error vector.
Parameter standard errors are computed as the square
Eq. (3) represents the usual mathematical model used to
root of the diagonal elements of the Coy(d) matrix
fit a response surface to measured data from an
experiment. We put aside for the moment the from Eq. (8), using 6-2 from Eq. (10).
important questions of determining how candidate
modeling functions Ps should be computed from the Estimated model output is
independent variables, as well as which candidate
)=P_i (11)
modeling functions should be included in Eq. (3),
which implicitly determines n. Now define an Nxn
matrix P, For response surface modeling, the modeling
functions (columns of P) are often chosen as
polynomials in the measured independent variables.
P = [t_, P2..... Pn ] (4)
This approach corresponds to using the terms of a
Taylor series expansion to approximate the functional
and let a =[al,a 2..... an]T. Eq. (3) can be written as a dependence of the output response variable on the
independent variables.
standard least squares regression problem,
If the modeling functions are instead multivariate
y =Pa+e (5) orthogonal functions generated from the measured
independent variable data, advantages accrue in the
where y is a vector of measured dependent variable model structure determination for response surface
values, P is amatrix whose columns contain modeling modeling. After the model structure is determined
functions of the measured independent variables, and a using the multivariate orthogonal modeling functions,
is a vector of unknown parameters. The variable each retained modeling function can be decomposed
represents a vector of errors that are to be minimized in into an expansion of ordinary polynomials in the
a least squares sense. The goal is to determine a that independent variables. Combining like terms from this
minimizes the least squares cost function final step puts the final model in the form of a Taylor
series expansion. It is this latter form of the model that
provides the physical insight, particularly in the case of
d = (y -Pa) T(y -Pa)=cTe (6)
modeling non-dimensional aerodynamic force and
moment coefficients. This is the reason that aircraft
The parameter vector estimate that minimizes this cost dynamics and control analyses are nearly always
fimction is computed from 3,5-7 conducted with the assumption of this form for the
dependence of the non-dimensional aerodynamic force
and moment coefficients on independent variables such
it =IP TP1-1 pT y (7)
as angle of attack and sideslip angle.
Ref. [3] describes a procedure for using the
The estimated parameter covariance matrix is
independent variable data to generate multivariate
6
American Institute of Aeronautics and Astronautics
orthogonal modeling functions pj, which have the
following important property: P = __G-1 (17)
p,lpj = 0 i_j, i,j =1, 2..... n (12) The columns of G-1 contain the coefficients for
expansion of each column of P (i.e., each multivariate
It is also possible to generate multivariate orthogonal function) in terms of the ordinar 3'
orthogonal functions by first generating ordinary polynomial functions contained in the columns of --.
multivariate polynomials in the independent variables, Eq. (17) can be used to express each multivariate
then orthogonalizing these functions using orthogonal function in terms of ordinary multivariate
Gram-Schmidt orthogonalization. The process begins polynomials.
by choosing one of the ordinary multivariate The orthogonal functions are generated in a
polynomial functions as the first orthogonal function:
manner that allows them to be decomposed without
_lbiguity into an expansion of ordinary multivariate
p_=_ (13) polynomials. The orthogonalization process can be
repeated using arbitrary ordinal, multivariate
where 41 is the ordinary multivariate polynomial polynomials to generate orthogonal functions of
function chosen to be the first orthogonal function. arbitrary order in the independent variables, subject
Then to make each subsequent ordinary multivariate only to limitations related to the information contained
polynomial function orthogonai to the preceding in the data. For the FASER wind tunnel data response
orthogonal function(s), define the f,h orthogonal surface modeling, the orthogonal modeling functions
were generated in the manner described above.
function pj as:
If an additional independent variable is introduced
j-I
PJ=_J--EYkJPk j=2 ..... n (14) to represent blocking in the experiment, the orthogonal
k=l function modeling can be used to orthogonalize the
block effects with respect to the other independent
where 4i is the jth ordinary multivariate polynomial w_iable effects. A blocking variable is typically used
to indicate some change in the experimental conditions
vector, and the Ykj are scalars determined from
that cannot be controlled by the experimenter. The
blocking variable to the first power can simply be made
p_'¢j k = 1,2..... j -1 one of the ordinary polynomial vectors el. The
(15)
Yk; - pT[Pk j = 2..... n orthogonalization procedure in Eqs. (13)-(15) makes
the blocking variable orthogonal to the other
orthogonal modeling functions, which are generated
where n is the total number of ordinary multivariate
from ordinary multivariate polynomial functions. This
polynomials used as raw material for generating the
approach allows arbitrary blocking of the data points in
multivariate orthogonal functions. Eq. (15) results
the experiment using analytical means alone.
from multiplying both sides of Eq. (13) by p[,
Normally, experiment designers try to arrange the
invoking the mutual orthogonality of Pk, k = 1..... j, normalized settings of the independent variables so that
and solving for Ykj. It can be seen from the blocking variable is orthogonal to the other
independent variables and their polynomial
Eqs. (13)-(15) that each orthogonal function can be
combinations. This separates the block effect from the
expressed in terms of ordinary polynomial functions. If
oiher model terms and allows identification of a block
the pj vectors and the Cj vectors are arranged as
effect independent of the other terms in the model.
columns of matrices P and ---, respectively, and the However, with the analytical orthogonal blocking
y,j are elements in the kth row and f,h column of an described above, the blocking variable is made
orthogonal to the other modeling functions analytically,
upper triangular matrix G with ones on the diagonal,
then so that arranging the experiment so that independent
variable settings and their polynomial combinations are
orthogonal to the blocking variable is not necessary.
--=PG (16)
All the models identified in this work included an
awmlytic blocking variable that accounted for drift in
which leads to
7
American Institute of Aeronaatics and Astronautics
experimental conditions between the time when the
central composite design points were collected and the where o-2,a., is the maximum variance of elements in
time when the D-optimal design points were collected. the error vector tr, assuming the correct model
structure, and n is the number of model terms. The
With the modeling functions orthogonal, using
PSE in Eq. (22) depends on the mean squared fit error
Eqs. (4) and (12) in Eq. (7), the j_ element of the
J/N, and a term proportional to the number of terms
estimated parameter vector _i isgiven by
in the model, n. The latter term prevents over-fitting
+=(,;.)/,i',.;) the data with too many model terms, which is
(18)
detrimental to model prediction accuracy 9. The factor
of 2 in the model over-fit penalty term accounts for the
Combining Eqs. (4), (6), and (11)-(12), fact that the PSE is being used when the model
structure is not correct, i.e., during the model structure
determination stage. Ref. [9] contains further justifying
j= yT),__ .7 (19) statistical arguments and analysis for the form of PSE
)=1 in Eqs. (21)-(22). Note that while the mean squared fit
error J/N must decrease with the addition of each
or, using Eq. (18),
orthogonal modeling function to the model (by Eq. (19)
or (20)), the over-fit penalty term o-2mu,n/N increases
) =/,-,.., pT,>, " pT+p+ (20) with each added model term (n increases). Introducing
j=l the orthogonal modeling functions into the model in
order of most effective to least effective in reducing the
Eq. (20) shows that when the modeling functions
are orthogonal, the reduction in the estimated cost mean squared fit error (quantified by h_(pTs pj) for
resulting from including the term aj pj in the model the fh orthogonal modeling function) means that the
PSE metric will always have a single global minimum
depends only the dependent variable data y and the
value. Figure 10 depicts this graphically, using actual
added orthogonal modeling function pj. This
modeling results from Ref. [2]. Ref. [9] contains
decouples the least squares modeling problem, and details on the statistical properties of the PSE metric,
makes it possible to evaluate each orthogonal modeling including justification for its use in modeling problems.
function in terms of its ability to reduce the least
squares model fit to the data, regardless of which other For wind tunnel testing, repeated runs at the same
orthogonai modeling functions are present in the test conditions are often available. If o-_ is the output
model. When the modeling functions p; are instead variance estimated from measurements of the output for
polynomials in the independent variables (or any other repeated runs at the same test conditions, then o-2,a_
non-orthogonal function set), the least squares problem can be estimated as
is not decoupled, and iterative analysis is required to
find the subset of modeling functions for an adequate "_ 2
model structure. o-,7,,ax'--25 o-o (23)
The orthogonal modeling functions to be included
If the output errors were Gaussian, Eq. (23) would
in the model are chosen to minimize predicted squared
correspond to conservatively placing the maximum
error PSE, defined by9 output variance at 25 times the estimated value
(corresponding to a 5o-0 maximum deviation).
ese=(y-ea) (y- ea)+2O- L (21) However, the estimate of o-o2 may not be very good,
N N
because of relatively few repeated runs available or
or errors in the independent variable settings or drift errors
when duplicating test conditions for the repeat runs.
PSE = + 2o-_,a_-- (22) The 5o-o value was found to give the most accurate
N N
models in model identification algorithm testing done
using simulated data. In Ref. [2], the model structure
8
American Institute of Aeronautics and Astronautics