Linear and Nonlinear Models: Fixed effects, random effects, and total least squares

1027 Pages·2012·9.79 MB·English

by  Erik Grafarend

Springer Geophysics
Forfurthervolumes:
http://www.springer.com/series/10173
•
Erik W. Grafarend Joseph L. Awange
(cid:2)
Applications of Linear and
Nonlinear Models
Fixed Effects, Random Effects, and
Total Least Squares
123
Prof.Dr.-Ing.ErikW.Grafarend Prof.Dr.-Ing.JosephL.Awange
UniversityofStuttgart ProfessorofEnvironmentandEarthSciences
GeodeticInstitute MasenoUniversity
Geschwister-Scholl-Strasse24D Maseno
70174Stuttgart Kenya
Germany
[email protected]
CurtinUniversity
Perth
Australia
KarlsruheInstituteofTechnology
Karlsruhe
Germany
KyotoUniversity
Kyoto
Japan
ISBN978-3-642-22240-5 ISBN978-3-642-22241-2(eBook)
DOI10.1007/978-3-642-22241-2
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2011945776
(cid:2)c Springer-VerlagBerlinHeidelberg2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Foreword
This book is a source of knowledge and inspiration not only for geodesists and
mathematicians,butalso forengineersin general,as wellasnaturalscientists and
economists. Inference on effects which result in observations via linear and non-
linearfunctionsis a generaltask in science.The authorsprovidea comprehensive
in-depth treatise on the analysis and solution of such problems. They start with
thetreatmentofunderdeterminedor/andoverdeterminedlinearsystemsbymethods
of minimum norms, weighted least squares and regularization, respectively. Con-
sideringthe observationsas realizationsof randomvariablesleads to the question
of how to balance bias and variance in linear stochastic regression estimation.
In addition, the causal effects are allowed to be random (general linear Gauss–
Markovmodel of estimation and prediction),even the modelfunction;conditions
areallowed,too(Gauss–Helmertmodel).Nonlinearfunctionsarecomprehendedvia
Taylorexpansionorbytheirspecialstructures,e.g.,withinthecontextofdirectional
observations.
The self-contained monograph is comprehensive in its inclusion of the more
fundamental elements of the topic to the state of the art, complemented by
an extensive list of references. All chapters have an introduction, often with
an indication of fast track reading and with an explanation addressed to non-
mathematicians which are of interest also to me as a mathematician. The book
contains a great variety of case studies and numerical examples ranging from
classical geodesy to dynamic systems theory. The background and much more
is provided by appendices dealing with linear algebra, probability, statistics and
stochastic processes as well as numerical methods. The presentation is lucid and
elegant,withhistoricalremarksandhumorousinterpretations,suchthatItookgreat
pleasureinreadingtheindividualchapters.
I wish all readers of this brilliant encyclopaedic book this pleasure and much
benefit.
Prof.Dr.HarroWalk
v
•
Preface
“Probability does not exist”, such a statement is at the beginning of the book by
B.deFinettidedicatedtothefoundationofprobability,nowadayscalled“probability
theory”. We believe that such a statementis overdone.The targetof the reference
is made very clear, namely the analysis of the random effects or of a random
experiment.TheresultofanexperimentlikethrowingdiceoraLOTTOoutcomeis
notpredictable.Suchanindeterminismischaracteristicanalysisofanexperiment.
In general, we analyze here nonlinear relations between random effects, maybe
stochastic or not. Obviously, the center of interest is on linear models, which are
abletoapproximatenonlinearrelations.Therearetypicalnonlinearmodels,which
cannotbelinearized.Ourexperimentalworldisdividedintolinearorlinearizedand
nonlinearmodels.Sometimes,nonlinearmodelscanbetransformedintopolynomial
equations,alsocalledalgebraic,whichallowa rigoroussolutionset. Aprominent
exampleistheresectionproblemofoverdeterminedtype.Ifweareabletoexpress
the characteristic rotation matrix between reference frames by quaternions, an
algebraic equation of known high order arrives. But, in contrast to linear models,
there exist a whole world of solutions, which have to be discussed. P. Lohse
presentedinhisPh.Dthesiswonderfulexamplesofwhichwerefer.
A.N. Kolmogorov founded the axiomatic (mathematical) theory of probability.
Typical for the axiomatic treatment, there are special rules relating to measure
theory.Here,weonlyrefertoexcellentintroductionsofmeasuretheory,forinstance
H. Bauer(1992),J. Elstrodt (2005)or W. Rudin (1987).Of course, there are very
good textbooks on the theory of probability, for instance H. Bauer (2002), K.L.
Chung(2001),O.Kallenberg(2002),M.Loeve(1977),andA.N.Shiryaev(1996).
It is importantto inform our readers about these many subjects, which are not
treatedhere:robustestimation,Bayesstatistics(onlyexamples),stochasticintegra-
tion, ITOO stochastic processes, stochastic signal processing, martingales, Markov
processes, limit theorems for associated random fields, modeling, measuring, and
managing risks, genetic algorithms, chaos, turbulence, fractals, neural networks,
wavelets,Gibbsfield,MonteCarlo,andmany,indeedveryimportantsubjects.This
statementreferstoanovercriticalreviewerwhocomplainedofthesubject,whichwe
didnottreat.Unfortunatelywehadtostronglyrestrictoursubjects.Ourcontribution
vii
viii Preface
isonlyanatominthe108atoms.
What will be treated?
Itisthetaskoftheobserverorthedesignertodecide:whatisthebestfitbetween
alinearornonlinearmodelorthe“realworld”oftheobserver.Bothtypesofanalysis
havetomakeanaprioristatementaboutalotoffacts.
• “Aretheobservationsrandomornot,astochasticeffectorafixedeffect?”
• “Whataretheparametersinthemodelspaceorwouldyouprefera‘modelfree’
concept?”
• “Whatisthenatureoftheparameters,randomeffectsorfixedeffects?”
• “Couldyoufindtheconditionsbetweentheobservationsorbetweentheparam-
eters?Whatisthenatureoftheconditions,linearornonlinear,randomornot?”
• “Whatisthebiasbetweenthemodel,anditsestimationorprediction?”
There are various criteria, which characterize the observation space or the
parameterspace:
linear or nonlinear, estimation theory, linear and nonlinear distributions,
lr (cid:3)optimality, (LESS: l2), restrictions, mixed models of fixed and random
effects, zero-, first-, and second order design, nonlinear statistics on curved
manifolds,minimalbiasestimation,translationalinvariance,totalleastsquares,
generalized least squares, minimal distance estimation, optimal design, and
optimalprediction.
Of key interest is to find equivalence lemmas, for instance: Assume direct
observations.Thenaleastsquaresfit(LESS)isequivalenttoabestlinearuniformly
unbiasedestimation(BLUUE).
The Chap. 1 deals with the first problem of algebraicregression,the consistent
system of linear observational equations or a system of underdetermined linear
equations. From the first page, examples onwards, we treat MINOS and the
“horizontal rank partitioning”. In detail, we give the eigenvalue decomposition
of weighted minimum norm solution. Examples are FOURIER series, FOURIER-
LEGENDRE series including the NYQUIST frequency for special data. Special
nonlinear models with datum defects in terms of Taylor polynomials and the
generalizedNewtoniterationsareintroduced.
The Chap. 2 is based on the special first problem, the bias problem expressed
bytheEquivalenceTheoremoftheadjustedminimumnormsolution(Gx-MINOS)
andlinearuniformlyminimumbiasedestimator(S-LUMBE).
The second problem of algebraic regression, namely solving an inconsistent
system of linear observational equations as an overdetermined system of linear
equations is extensively treated in Chap. 3. We start with a front example for the
LeastSquaresSolution(LESS).Indeedwediscussalternativesolutionsdepending
on the metric of the observation space like Second Order Design, the Taylor-
Karman structure, an optimal choice of the weight matrix and the Fuzzy sets.
Preface ix
ByaneiqenvaluedecompositionofGy-LESS,weintroduce“canonicalLESS”.Our
case studies range from partial redundancies,latent conditions, the theoryof high
leverage points versus breaking points, direct and inverse Grassman coordinates
andPLU¨CKERcoordinates.WeconcludewithhistoricalnotesonC.F.Gauss,A.M.
Legendreandgeneralizations.
In another extensive review, we concentrate on the second problem of proba-
bilistic regression,namely the specialGauss–Markovmodelwithoutdatumdefect
of Chap.5.Wesetupthebest,linear,uniformlyunbiasedestimator (BLUUE)for
momentsofthefirstorderandthebest,quadraticallyuniformlyunbiasedestimator
for the central moments of the second order (BIQUUE). We depart again from
a detailed example BLUUE and BIQUUE. We set up ˙y-BLUUE and note the
Equivalence Theorem of Gy-LESS and ˙y-BLUUE. For BIQUUE, we study the
block partitioning of the dispersion matrix, the invariant quadratic estimation of
types IQE, IQUUE, HIQUUE versus HIQE according to F.R. Helmert, variance-
componentestimation,simultaneousestimationofthefirstmomentsandthesecond
central moments, the so called E-D correspondence, inhomogeneous multilinear
estimation,andBAYESdesignofmomentsestimation.
ThethirdproblemofalgebraicregressionofChap.5isdefinedastheproblemto
solveinconsistentsystemsoflinearobservationequationswithdatumdefectasan
overdeterminedand indetermined system of linear equations. In the introduction,
webeginwiththefrontpageexample,subjecttotheminimumnorm-leastsquares
solution of the front page example, namely by the technique of additive rank
partitioningas well as multiplicative rank partitioning.In moredetail, we present
MINOSinthesecondsectionincludingtheweightsoftheMINOSaswellasLESS.
Especially, we interpret Gy;Gx-MINOS in terms of the generalized inverse. By
eigenvaluedecomposition,wefindthesolutiontoGy;Gx-MINOS.Aspecialsection
isdevotedtototalleastsquaresintermsof˛-weightedhybridapproximatesolution
(˛-HAPS)withinTykhonov-Phillipsregularization.
ThethirdproblemofprobabilisticregressionasaspecialGauss–Markovmodel
withdatumproblemistreatedinChap.6.Wesetupbestlinearminimumunbiased
estimatorsoftypeBLUMBEandbestlinearestimatorsoftypehomBLE,homS-BLE
andhom-˛-BLEdependingontheweightassumptionforthebias.Forexample,we
introduce continuousnetworks of first and second derivatives. Finally, we discuss
thelimitprocessofdiscretenetworkintocontinuum.
The overdetermined system of nonlinear equations on curved manifolds in
Chap. 7 is presented for inconsistent system of directional observation equations,
namely by minimal geodesic distance (MINGEODISC). A special example is
the directional observation from circular normal distribution of type von MISES-
FISHERwithanoteofangularmetric.
Chapter8isrelatingtothefourthprobabilisticregressionasasetupoftypeBLIP
andVIPforthecentralmomentsoffirstorder.Webeginthedefinitionofarandom
effectmodelaccordingtoourmagictriangle.Threeexamplesrelateto
(i) Nonlinearerrorpropagationwithrandomeffectcomponents
(ii) Nonlinear vector-valued error propagation with random effect in
Geoinformatics