The Design of Optimum Linear Systems

126 Pages·1953·3.693 MB·English

by  Eberhardt Rechtin

External Publication No. 206
"HE DESlCH OF OF'TIW LlNGAB SYSTRS
Eberhardt Rechtin
JEX PBOPULSION UB;)MTOBT
California Institute of Technolorn
Pasadem, miforrda'
April 1953
Sootion Su.bject hg.
Introduction 2
I Critcria for Optimization 4
Elomectary Calculus cf Variattons 9
Cons train+,? 15
IV Curva Fi5ting and I'.!eig:?ting Functions 18
A. Classical Sttitsmcnt of tho Cwvs Fittin2 Problem 18
B. Citissicsl Solution ct thc Curve Fitting F'roblem 20
C. Ektcnsion of the Classical Solution 21
D. Xenn Squcrc Error of thc Zs-timtcd 2eraxters 23
2. Solution by the Calculus of Variations 24
Fourier and 'LF?.?~-?OC Traxsf oms 29
Snectral Eeisttp 44
IX s
57
59
XI1
63
Table of Contents, Cont'd.
XI11 A Physical Interpretatfon of the Wiener Optimum
M V &tensions of Wiener l .s Solution
ZT Abtched Filters
AT1 The Finite-time, Finite-order Filter
XVII Quasi-distortionless Notuozh
mn
The Saturation Constraint
4XJX Transient Error b!inirnization
Zd The Dcsi,p of S e ~ o ~ c h ~ i i s ~
;.;M Re1 ntod Topics
Blbliopaphy
EP 204
-1-
I 7
Criteria for
.I
?htched Filters
n
TrtnsixS i3rrod
: ;. 75-75x etion
+
I
I 1
1 I
I
r--l-
.-_I- -
I
-2-
IN THO-D-U- C-T-I- O N
It is. the purpose of this text t o instruct engineers in the design of
opttmm linear system. dn optimum linear system is defined as an x%5aaeds
linear system which performs a desired linear operation oriththe minimum mbm
square error- The basio cause for the systom error is usually an external
disturbance of a s i q l a statistical nature, System error need not be caused
by a disturbance, .however; the design technique is also applicable to tran-
sient design of syetem with limited power capabilities.
The presentation of the technique is directed toward the design and anal-
ysis of mobile-vehicle control systems, systems in which the control unit and
the element controlled (the Ehicle) are independent except for radio or ap-
tical links, Noise, disturbances of my kind; and deliberate enemy jamming
may enter the system easily, The engineer's job is first,the desi@ of ths
best control system under the circumstances, and second, the analysis of pr-
formance data to determine the degree of success achieved, In a very real
sense, this ?articular problem is the sum of many problems including filtering
signals from noise, dssigninf: stable control systems, preventing internal
saturation of a servomechanism, and analyzing noisy records, Figure 1 in&-
cstes the order and relationship of the topics which descrih the new
toc hique,
The technique and derivations are described in term of Laplace transform.
The author has assumed the reader to have a working knowledge of Laplace trans-
forms roughly equivalent to that attained by reading Gardner and Barnes,
n Transients in Linear Svstem." No knowledge of statistics is assumed, 30
statistics is p-esented in this development which cannot be learned in a
few ninutes.
-3-
Certain portion8 ofthis text must be understood thoroughly in order
that the engineer be able t o do more than plug In and grind out the hmda-
mental equationr
The signifioanoe of the equation will be readily appreciated the develop=
ment proceeds, but it is not always simple to formulate the problem at hand
in such a way as to make the equation upplicable. Regardless of background,
-
the reader nust understand SectEans I X before reading further sections.
Sections xf, XII, and XI11 present the minimum theory necessary to solve
problems, Section 1uv outlines the remainder of the text in order that the
reader may decide whioh further sections are directly related to his hmdi-
ato problem.
The elrlstence of the technique is directly attributable to Norbert
Wiener, Professor of Xathematics at M.I.T. A, 8, Xo'Lmogoroff derived some-
what similar results but in a less useful form. Nith few exceptions, all
extonsions of Wiener's work were done in many places by c a y different people,
all at about the same time. The derivations in this text are original in the
sense of being independently derived a t J.P.L. This text could well have
been writtcn by several other engineer, at J.P.L. I", H. Pickering, F. E- Lehaa,
and R, J, Parks initiated and guided the work. C, W. Berm, R. E, Con, E.
Rcchtin, W, F, Sampson, R. 11 . Stewart, and Do C, Youla continued the develop-
ment, This text is one result,
-4-
Some amazingly violsnt arymente can ensue on the subjeot of optimi-
sation. Because of the great variety of possible systems, all performing
the same task, it ia difficult to get agreement even on the relative valuer
of cost, reliability, simplicity, and accuracy. This disoussion will be
-
limited to coneiderations of accuracy for an extremely inportant reason
when interference is present, it is not possf-ble t o make a system as accurate
as we please, 'Re (possibly) oan increase cast ulthout limit, and can nab
a system as reliable as desired, but we aro definitely limited in the at-
tainable accuracy for the system. The limiting aocuracy of the system
--
depends primarily upon the characteristics of the input functions loosely
--
speaking, upon the characteristics of signal and noise and is realized
only by the optimum system. If this limited accuracy is insufficient for
our purposes, we must either change the input characteristics or abandon
the attempt, No m o u n t of cost, complexity, or ingenuity can yield any
impr ovememt,
The existence of an optimally-accurate system and a knowledge of its
associated "irreducible error" is useful aven though such a system may 00-
casionally be too complicated, costly, and/or unreliable to bLild. Such a
system provides a good standard for evaluation of proposed substitutes. It
i e often possible to design substitutes with accuracies within a few percent
oi' the limiting accuracy.
Let us define an optimum system as optimum in M accuracy sense. Again,
there is much room for differences of opinion. For example, if the problem
is t o transmit information in the prezsnce of noise, systems of different
relative acouraciea may be designed depending on the coding (AI:, FIL, PCEL, ran-
dom) and upon the channel noiso level. If the information nay be quantized,the
number of praotioal oodes may be still further increased. However, for eauh
transmitted code and asaunod intorference, there still exists an optimum re-
. oeiver design. The problem becomes even more intoresting if the interfereuoe
l a assumed intelligent, i.e., if the interferenco is always of the worst
possible type with respect to the seloctcd code, This text will not consider
such problems in the theory of games, but will be restricted t o optimum systems
with respect to spocified inputs.
If all inputs to a system are coipletely specified functions of time ex-
cept for certain parmeters such as absolute mgnitsde or exact tiche of occur-
ronce, elimination of the effects of any of the individual inputs may usually
be accomplished identisally, 60 cycle "hum-bucking" is one example Qf complete
interference elimination, Output hum consists of a pure 60 cycle tone which is
eliminated by addition of an equal and opposite 60 cycle tone to the output, As
another illustration, if the bputs are exactly known except for three constant.
parameters, three operations are usually zufficient to yield error-froe per-
formance, If tho inputs arc statistical in nature, however, there are so mny
unhovrns that exact scparations ai-o impossible, me can only hope for good
perf orxnance averaged over many triese
It is inportant that we lzon bcforchar.d Ln what mi;. the various inputs
differ fro= each othzr, If all ir,puts evcer the system at the same point c1.12
all arc alike, separation is impossible, Conversely, the 'greater tho dif-
ferences, the better tho separation, Inputs my be dcscribed by their exact
time dependence, by their eqectod time dependence, by their complete proba-
bility functions, or by their corrclatton functions (cr spectral densitics).
Fs 204
Statistical descriptions are comparatively weak, but they ray he the only de-
scriptions available. To quote Yiiener, "Statistical prediotion is essentially
a method of' refining a prediction which would be perfect by itself' in an idea-
lized cam but whioh i: aorrupted by statistical errors, either in the observed
quantity itself' or in the observation. Geometrioal facts must be predioted geo-
metrically and analytical facts analytically, 3eeving only statistical facts t o be
predicted statistically.''
No complete the specification of an optimum system by agreeing on a nathe-
matical description of system accuracy. If no restrictions are placed upon
either the probability functions of the inputs, or on the type of operations per-
forned,'it is possible t o specify the accuracy of the system in many different
ways, For example, using ths "maximum likelihood" criterion, one attempts t o
form the observed input by the addition of samples selectsd from horn distri-
butions in such n way that the 3oint probability of occurrence of the smples
is maximized, The selection process may well be non-linear. A linear operation
is describable by an equation of the'first degree i n the dependent variable.
Such equations may bo time-variant, integro-differential or difference, but may
not involve operations on ot'ner than the first. power of the dependent variable,
Accuracy specifications involving probability functions generally lead not to
explicit specification of the operation t o b6 performed, but only to certain
criter5a on such operetio:is, The field of non-linear mathecstics is not yet
in a condition t o 'be ext.ensively exploited by ongineers, Each non-linear
problem is approached independently. No general discipline is available at
this date.
If we limit ourselves t o linear systems,+ not only do we enter a vrell-
developed field, but we can also define system accuracy in a sinrple way. Let
x(t) = desired output at tims t
xo(t) = actual output at time t
-
(t) = x(t) x,(t) = error at timo t
* -
Lot UO repeat.the experiment nnny times and observe the performance at tim t.
The performance will be different for Gach experiment since the input inter-
ferences will never be exactly alike, Not knowing beforehand the exact time
dapendeuce of the interference, we can never guarantee perfect porfomance
(zero error) for any one experiment. After observing many sxperinlents, hmmr,
we certainly nmt to notice good overall performance. F;e desire no error on the
averaga. Letting a bar signify experimental (ensemble) overage, we require. the
condition:
Systems satisfying this condition are called "unbiased." The condition i6 not
LC good measure of- tom partitive performance of various systems, Almost any sys-
c(t)
tern will yield = 0 inasmuch as the average interference is usually zero,
Let us keep this zondition as ti desired feeture and investigate several addi-
tional masuros of sysicn accurecy. Inasmuch as negative errors are ?robably
as serious as positive errors, the measure should so indicate.
*The restriction to linear systems is not as stringent as it might soom. Dis-
turbances projuced by most jnterfercnce phenomena are characterized by Gaussian
distribution funciions. Any linear oparation on such disturbknces w i l i yield
an output which also has a Gaussian distribution function. If we restrict our
desired operations tc linsar operations (i,e., do not look for the square of
the in~ut,tt c.), it has lxcn shown that for Gaussian typo input nmctions the
optimum linsir system is also the opt.imi of all systems, In addition, all
standard descriptions of accuracy reduce t o the one presented here.