Finite Difference and Spectral Methods for Ordinary and Partial Differential Equations

299 Pages·1996·3.809 MB·English

by  Trefethen L.N.

FINITE DIFFERENCE
AND SPECTRAL METHODS
FOR
ORDINARY AND PARTIAL
DIFFERENTIAL EQUATIONS
Lloyd N(cid:2) Trefethen
Cornell University
Copyright (cid:0)c (cid:2)(cid:3)(cid:3)(cid:4) by Lloyd N(cid:5) Trefethen
Note
A trap that academics sometimes fall into is to begin a book and fail to (cid:6)nish it(cid:5) This
has happened to me(cid:5) This book was intended to be published several years ago(cid:5) Alas(cid:7)
two other projects jumped the queue (cid:8)Numerical Linear Algebra(cid:2) with David Bau(cid:7)
to be published in (cid:2)(cid:3)(cid:3)(cid:9) by SIAM(cid:7) and Spectra and Pseudospectra(cid:2) to be completed
thereafter(cid:10)(cid:5) At this point I do not know whether this numerical ODE(cid:11)PDE book will
ever be properly (cid:6)nished(cid:5)
This is a shame(cid:12)as people keep telling me(cid:5) Electronically by email(cid:7) and in person at
conferences(cid:7) I am often asked (cid:13)when that PDE book is going to get done(cid:5)(cid:14) Anyone
who has been in such a situation knows how (cid:15)attering such questions feel(cid:0)(cid:0)(cid:0) at (cid:6)rst(cid:5)
The book is based on graduate courses taught to mathematicians and engineers since
(cid:2)(cid:3)(cid:16)(cid:17) at the Courant Institute(cid:7) MIT(cid:7) and Cornell(cid:5) Successively more complete versions
havebeen usedbymeinthese courseshalfadozentimes(cid:7) andbyahandfulofcolleagues
at these universities and elsewhere(cid:5) Much of the book is quite polished(cid:7) especially the
(cid:6)rst half(cid:7) and there are numerous exercises that have been tested in the classroom(cid:5)
But there are many gaps too(cid:7) and undoubtedly a few errors(cid:5) I have reproduced here
the version of the text I used most recently in class(cid:7) in the spring of (cid:2)(cid:3)(cid:3)(cid:18)(cid:5)
I am proud of Chapter (cid:2)(cid:7) which I consider as clean an introduction as any to the
classicaltheoryofthenumericalsolutionofODE(cid:5) Theothermainpartsofthebookare
Chapter (cid:17)(cid:7) which emphasizes the crucial relationship between smoothness of a function
and decay ofitsFouriertransform(cid:19) Chapters (cid:20) and(cid:18)(cid:7) which present the classicaltheory
ofnumericalstabilityfor(cid:6)nitedi(cid:21)erenceapproximationstolinearPDE(cid:19)andChapters(cid:9)
and (cid:16)(cid:7) which o(cid:21)er ahands(cid:22)on introductiontoFourierandChebyshev spectralmethods(cid:5)
This book is largely linear(cid:7) and for that reason and others(cid:7) there is much more to the
numerical solution of di(cid:21)erential equations than you will (cid:6)nd here(cid:5) On the other hand(cid:7)
Chapters (cid:2)(cid:23)(cid:18) represent material that every numerical analyst should know(cid:5) These
chapters alone can be the basis of an appealing course at the graduate level(cid:5)
Please do not reproduce this text(cid:19) all rights are reserved(cid:5) Copies can be obtained for
(cid:24)(cid:17)(cid:17) or (cid:3)(cid:2)(cid:25) each (cid:8)including shipping(cid:10) by contacting me directly(cid:5) Professors who are
using the book in class may contact me to obtain solutions to most of the problems(cid:5)
(cid:26)This is no longer true (cid:23) the book is freely available online at
http(cid:2)(cid:3)(cid:3)web(cid:4)comlab(cid:4)ox(cid:4)ac(cid:4)uk(cid:3)oucl(cid:3)work(cid:3)nick(cid:4)trefethen(cid:3)pdetext(cid:4)html(cid:5)(cid:27)
Nick Trefethen
July (cid:2)(cid:3)(cid:3)(cid:4)
Department of Computer Science and Center for Applied Mathematics
Upson Hall(cid:2) Cornell University(cid:2) Ithaca(cid:2) NY (cid:3)(cid:4)(cid:5)(cid:6)(cid:7)(cid:2) USA
(cid:8)(cid:9)(cid:10)(cid:11)(cid:12) (cid:13)(cid:6)(cid:6)(cid:14)(cid:4)(cid:13)(cid:13)(cid:13)(cid:2) fax (cid:8)(cid:9)(cid:10)(cid:11)(cid:12) (cid:13)(cid:6)(cid:6)(cid:14)(cid:4)(cid:4)(cid:13)(cid:5)(cid:2) LNT(cid:15)cs(cid:16)cornell(cid:16)edu
CONTENTS TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) v
Contents
Preface viii
Notation x
(cid:2)(cid:5) Big(cid:22)O and related symbols (cid:2)
(cid:3)(cid:5) Rounding errors and discretization errors (cid:3)
(cid:0)(cid:2) Ordinary di(cid:3)erential equations (cid:4)
(cid:2)(cid:5)(cid:2)(cid:5) Initial value problems (cid:5)(cid:6)
(cid:2)(cid:5)(cid:17)(cid:5) Linear multistep formulas (cid:5)(cid:3)
(cid:2)(cid:5)(cid:20)(cid:5) Accuracy and consistency (cid:2)(cid:5)
(cid:2)(cid:5)(cid:18)(cid:5) Derivation of linear multistep formulas (cid:7)(cid:6)
(cid:2)(cid:5)(cid:25)(cid:5) Stability (cid:8)(cid:6)
(cid:2)(cid:5)(cid:4)(cid:5) Convergence and the Dahlquist Equivalence Theorem (cid:3)(cid:7)
(cid:2)(cid:5)(cid:9)(cid:5) Stability regions (cid:3)(cid:4)
(cid:2)(cid:5)(cid:16)(cid:5) Sti(cid:21)ness (cid:9)(cid:3)
(cid:2)(cid:5)(cid:3)(cid:5) Runge(cid:22)Kutta methods (cid:10)(cid:3)
(cid:2)(cid:5)(cid:2)(cid:28)(cid:5) Notes and references (cid:10)(cid:11)
(cid:4)(cid:2) Fourier analysis (cid:4)(cid:6)
(cid:17)(cid:5)(cid:2)(cid:5) The Fourier transform (cid:4)(cid:2)
(cid:17)(cid:5)(cid:17)(cid:5) The semidiscrete Fourier transform (cid:11)(cid:2)
(cid:17)(cid:5)(cid:20)(cid:5) Interpolation and sinc functions (cid:11)(cid:11)
(cid:17)(cid:5)(cid:18)(cid:5) The discrete Fourier transform (cid:5)(cid:6)(cid:5)
(cid:17)(cid:5)(cid:25)(cid:5) Vectors and multiple space dimensions (cid:5)(cid:6)(cid:9)
(cid:17)(cid:5)(cid:4)(cid:5) Notes and references (cid:12)
(cid:5)(cid:2) Finite di(cid:3)erence approximations (cid:5)(cid:6)(cid:4)
(cid:20)(cid:5)(cid:2)(cid:5) Scalar model equations (cid:5)(cid:5)(cid:6)
(cid:20)(cid:5)(cid:17)(cid:5) Finite di(cid:21)erence formulas (cid:5)(cid:5)(cid:3)
(cid:20)(cid:5)(cid:20)(cid:5) Spatial di(cid:21)erence operators and the method of lines (cid:5)(cid:2)(cid:8)
(cid:20)(cid:5)(cid:18)(cid:5) Implicit formulas and linear algebra (cid:5)(cid:7)(cid:8)
(cid:20)(cid:5)(cid:25)(cid:5) Fourier analysis of (cid:6)nite di(cid:21)erence formulas (cid:5)(cid:7)(cid:9)
(cid:20)(cid:5)(cid:4)(cid:5) Fourier analysis of vector and multistep formulas (cid:5)(cid:8)(cid:7)
(cid:20)(cid:5)(cid:9)(cid:5) Notes and references (cid:12)
CONTENTS TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) vi
(cid:6)(cid:2) Accuracy(cid:7) stability(cid:7) and convergence (cid:5)(cid:8)(cid:9)
(cid:18)(cid:5)(cid:2)(cid:5) An example (cid:5)(cid:8)(cid:4)
(cid:18)(cid:5)(cid:17)(cid:5) The Lax Equivalence Theorem (cid:5)(cid:3)(cid:7)
(cid:18)(cid:5)(cid:20)(cid:5) The CFL condition (cid:5)(cid:9)(cid:6)
(cid:18)(cid:5)(cid:18)(cid:5) The von Neumann condition for scalar one(cid:22)step formulas (cid:5)(cid:9)(cid:9)
(cid:18)(cid:5)(cid:25)(cid:5) Resolvents(cid:7) pseudospectra(cid:7) and the Kreiss matrix theorem (cid:5)(cid:10)(cid:8)
(cid:18)(cid:5)(cid:4)(cid:5) The von Neumann condition for vector or multistep formulas (cid:5)(cid:4)(cid:7)
(cid:18)(cid:5)(cid:9)(cid:5) Stability of the method of lines (cid:5)(cid:4)(cid:4)
(cid:18)(cid:5)(cid:16)(cid:5) Notes and references (cid:12)
(cid:8)(cid:2) Dissipation(cid:7) dispersion(cid:7) and group velocity (cid:5)(cid:11)(cid:5)
(cid:25)(cid:5)(cid:2)(cid:5) Dispersion relations (cid:5)(cid:11)(cid:7)
(cid:25)(cid:5)(cid:17)(cid:5) Dissipation (cid:2)(cid:6)(cid:5)
(cid:25)(cid:5)(cid:20)(cid:5) Dispersion and group velocity (cid:2)(cid:6)(cid:7)
(cid:25)(cid:5)(cid:18)(cid:5) Modi(cid:6)ed equations (cid:12)
(cid:25)(cid:5)(cid:25)(cid:5) Stability in (cid:4)p norms (cid:12)
(cid:25)(cid:5)(cid:4)(cid:5) Notes and references (cid:12)
(cid:9)(cid:2) Boundary conditions (cid:2)(cid:5)(cid:11)
(cid:4)(cid:5)(cid:2)(cid:5) Examples (cid:12)
(cid:4)(cid:5)(cid:17)(cid:5) Scalar hyperbolic equations (cid:2)(cid:2)(cid:5)
(cid:4)(cid:5)(cid:20)(cid:5) Systems of hyperbolic equations (cid:12)
(cid:4)(cid:5)(cid:18)(cid:5) Absorbing boundary conditions (cid:2)(cid:2)(cid:3)
(cid:4)(cid:5)(cid:25)(cid:5) Notes and references (cid:12)
(cid:10)(cid:2) Fourier spectral methods (cid:2)(cid:7)(cid:2)
(cid:9)(cid:5)(cid:2)(cid:5) An example (cid:2)(cid:7)(cid:3)
(cid:9)(cid:5)(cid:17)(cid:5) Unbounded grids (cid:2)(cid:7)(cid:4)
(cid:9)(cid:5)(cid:20)(cid:5) Periodic grids (cid:2)(cid:8)(cid:10)
(cid:9)(cid:5)(cid:18)(cid:5) Stability (cid:2)(cid:3)(cid:10)
(cid:9)(cid:5)(cid:25)(cid:5) Notes and references (cid:12)
(cid:11)(cid:2) Chebyshev spectral methods (cid:2)(cid:9)(cid:6)
(cid:16)(cid:5)(cid:2)(cid:5) Polynomial interpolation (cid:2)(cid:9)(cid:2)
(cid:16)(cid:5)(cid:17)(cid:5) Chebyshev di(cid:21)erentiation matrices (cid:2)(cid:9)(cid:11)
(cid:16)(cid:5)(cid:20)(cid:5) Chebyshev di(cid:21)erentiation by the FFT (cid:2)(cid:10)(cid:2)
(cid:16)(cid:5)(cid:18)(cid:5) Boundary conditions (cid:12)
(cid:16)(cid:5)(cid:25)(cid:5) Stability (cid:2)(cid:10)(cid:10)
(cid:16)(cid:5)(cid:4)(cid:5) Some review problems (cid:2)(cid:11)(cid:4)
(cid:16)(cid:5)(cid:9)(cid:5) Two (cid:6)nal problems (cid:7)(cid:6)(cid:6)
(cid:16)(cid:5)(cid:16)(cid:5) Notes and references (cid:12)
CONTENTS TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) vii
(cid:12)(cid:2) Multigrid and other iterations (cid:12)
(cid:3)(cid:5)(cid:2)(cid:5) Jacobi(cid:7) Gauss(cid:22)Seidel(cid:7) and SOR (cid:12)
(cid:3)(cid:5)(cid:17)(cid:5) Conjugate gradients (cid:12)
(cid:3)(cid:5)(cid:20)(cid:5) Nonsymmetric matrix iterations (cid:12)
(cid:3)(cid:5)(cid:18)(cid:5) Preconditioning (cid:12)
(cid:3)(cid:5)(cid:25)(cid:5) The multigrid idea (cid:12)
(cid:3)(cid:5)(cid:4)(cid:5) Multigrid iterations and Fourier analysis (cid:12)
(cid:3)(cid:5)(cid:9)(cid:5) Notes and references (cid:12)
APPENDIX A(cid:5) Some formulas of complex analysis APP(cid:13)(cid:5)
APPENDIX B(cid:5) Vector(cid:7) matrix(cid:7) and operator norms APP(cid:13)(cid:7)
REFERENCES REFS(cid:13)(cid:5)
PREFACE TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) viii
Preface
Many books have been written on numerical methods for partial di(cid:21)er(cid:22)
ential equations(cid:7) ranging from the mathematical classic by Richtmyer and
Morton to recent texts on computational (cid:15)uid dynamics(cid:5) But this is not an
easy (cid:6)eld to set forth in a book(cid:5) It is too active and too large(cid:7) touching a be(cid:22)
wildering variety of issues in mathematics(cid:7) physics(cid:7) engineering(cid:7) and computer
science(cid:5)
My goal has been to write an advanced textbook focused on basic prin(cid:22)
ciples(cid:5) Two remarkably fruitful ideas pervade this (cid:6)eld(cid:29) numerical stability
and Fourier analysis(cid:5) I want the reader to think at length about these and
other fundamentals(cid:7) beginning with simple examples(cid:7) and build in the process
a solid foundation for subsequent work on more realistic problems(cid:5) Numerical
methods for partial di(cid:21)erential equations are a subject of subtlety and beauty(cid:7)
but the appreciation of them may be lost if one views them too narrowly as
tools to be hurried to application(cid:5)
Thisisnotabookofpuremathematics(cid:5) Anumberoftheoremsareproved(cid:7)
but my purpose is to present a set of ideas rather than a sequence of theorems(cid:5)
The book should be accessible to mathematically inclined graduate students
and practitioners in various (cid:6)elds of science and engineering(cid:7) so long as they
arecomfortablewithFourieranalysis(cid:7) basicpartialdi(cid:21)erentialequations(cid:7) some
numerical methods(cid:7) the elements of complex variables(cid:7) and linear algebra in(cid:22)
cluding vector and matrix norms(cid:5) At MIT and Cornell(cid:7) successive drafts of
this text have formed the basis of a large graduate course taught annually
since (cid:2)(cid:3)(cid:16)(cid:25)(cid:5)
One unusual feature of the book is that I have attempted to discuss the
numerical solution of ordinary and partial di(cid:21)erential equations in parallel(cid:5)
There are numerous ideas in common here(cid:7) well known to professionals(cid:7) but
usually not stressed in textbooks(cid:5) In particular I have made extensive use of
the elegant idea of stability regions in the complex plane(cid:5)
Another unusual feature is that in a single volume(cid:7) this book treats spec(cid:22)
tral as well as the more traditional (cid:6)nite di(cid:21)erence methods for discretization
of partial di(cid:21)erential equations(cid:5) The solution of discrete equations by multi(cid:22)
grid iterations(cid:7) the other most conspicuous development in this (cid:6)eld in the
past twenty years(cid:7) is also introduced more brie(cid:15)y(cid:5)
The unifying theme of this book is Fourier analysis(cid:5) To be sure(cid:7) it is well
PREFACE TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) ix
known thatFourieranalysis is connected with numerical methods(cid:7) but here we
shall consider at least four of these connections in detail(cid:7) and point out analo(cid:22)
gies between them(cid:29) stability analysis(cid:7) spectral methods(cid:7) iterative solutions of
elliptic equations(cid:7) and multigrid methods(cid:5) Indeed(cid:7) a reasonably accurate title
would have been (cid:13)Fourier Analysis and Numerical Methods for Partial Dif(cid:22)
ferential Equations(cid:5)(cid:14) Partly because of this emphasis(cid:7) the ideas covered here
are primarily linear(cid:19) this is by no means a textbook on computational (cid:15)uid
dynamics(cid:5) But it should prepare the reader to read such books at high speed(cid:5)
Besides these larger themes(cid:7) a number of smaller topics appear here that
are not so often found outside research papers(cid:7) including(cid:29)
(cid:0)
Order stars for stability and accuracy analysis(cid:7)
(cid:0)
Modi(cid:4)ed equations for estimating discretization errors(cid:7)
(cid:0) p
Stability in (cid:4) norms(cid:2)
(cid:0)
Absorbing boundary conditions for wave calculations(cid:7)
(cid:0)
Stability analysis of non(cid:22)normal discretizations via pseudospectra(cid:2)
(cid:0)
Group velocity e(cid:21)ects and their connection with the stability of boundary
conditions(cid:5)
Like every author(cid:7) I have inevitably given greatest emphasis to those topics I
know through my own work(cid:5)
The reader will come to know my biases soon enough(cid:19) I hope he or she
shares them(cid:5) One is that exercises are essential in any textbook(cid:7) even if the
treatment is advanced(cid:5) (cid:8)The exercises here range from cookbook to philo(cid:22)
sophical(cid:19) those marked with solid triangles require computer programming(cid:5)(cid:10)
Another is that almost every idea can and should be illustrated graphically(cid:5) A
third is that numerical methods should be studied at all levels(cid:7) from the most
formal to the most practical(cid:5) This book emphasizes the formal aspects(cid:7) saying
relatively little about practical details(cid:7) but the reader should forget at his or
her peril that a vast amount of practical experience in numerical computation
has accumulated over the years(cid:5) This experience is magni(cid:6)cently represented
in a wealth of high(cid:22)quality mathematical software available around the world(cid:7)
such asEISPACK(cid:7)LINPACK(cid:7)LAPACK(cid:7)ODEPACK(cid:7)andtheIMSLandNAG
Boeing Fortran libraries(cid:7) not to mention such interactive systems as Matlab
and Mathematica(cid:5)(cid:30) Half of every applied problem can and should be handled
by o(cid:21)(cid:22)the(cid:22)shelf programs nowadays(cid:5) This book will tell you something about
what those programs are doing(cid:7) and help you to think productively about the
other half(cid:5)
(cid:2)Foron(cid:3)lineacquisitionofnumericalsoftware(cid:4)everyreaderofthisbookshouldbesuretobefamiliar
withthe(cid:5)Netlib(cid:6)facilitymaintainedatBellLaboratoriesandOakRidgeNationalLaboratory(cid:7) For
information(cid:4) send thee(cid:3)mail message (cid:5)help(cid:6) to netlib(cid:8)research(cid:7)att(cid:7)com or netlib(cid:8)ornl(cid:7)gov(cid:7)
NOTATION TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) x
Notation
R(cid:5)C (cid:5)Z real numbers(cid:7) complex numbers(cid:7) integers
(cid:6)
marks a number contaminated by rounding errors
O(cid:5)o(cid:5)(cid:31)(cid:5) order relations analogous to (cid:2)(cid:7) (cid:7)(cid:7) (cid:3)(cid:7) !
x(cid:5)y(cid:5)t space and time variables
(cid:8)(cid:5)(cid:9)(cid:5)(cid:10) x wave number(cid:7) y wave number(cid:7) frequency
T initial(cid:22)value problem is posed for t(cid:4)(cid:26)(cid:28)(cid:5)T(cid:27)
N dimension of system of equations
d number of space dimensions
u dependent variable (cid:8)scalar or N(cid:22)vector(cid:10)
h(cid:5)k space and time step sizes h!"x(cid:7) k!"t
i(cid:5)j(cid:5)n space and time indices
(cid:4)(cid:5)r(cid:5)s integers de(cid:6)ning size of (cid:6)nite di(cid:21)erence stencil
n
vij numerical approximation to u(cid:8)ih(cid:5)jh(cid:5)nk(cid:10)
u#(cid:5)v# Fourier transforms
(cid:9)
(cid:11)(cid:5)(cid:12) mesh ratios (cid:11)!k(cid:13)h(cid:7) (cid:12)!k (cid:13)h
n n(cid:10)(cid:11)
K(cid:5)Z space and time shift operators K(cid:29)vj (cid:5)(cid:6)vj(cid:10)(cid:11)(cid:5) Z(cid:29)v (cid:5)(cid:6)v
i(cid:2)h i(cid:3)k
(cid:14)(cid:5)z space and time ampli(cid:6)cation factors (cid:14)!e (cid:5) z!e
(cid:0)(cid:11)
"(cid:5)r forward and backward di(cid:21)erence operators "!K(cid:7)(cid:2)(cid:5) r!(cid:2)(cid:7)K
(cid:15)(cid:8)z(cid:10)(cid:5)(cid:12)(cid:8)z(cid:10) characteristic polynomials for a linear multistep formula
c(cid:5)cg phase and group velocities c!(cid:7)(cid:10)(cid:13)(cid:8)(cid:5)cg!(cid:7)d(cid:10)(cid:13)d(cid:8)
$(cid:8)A(cid:10)(cid:5)$(cid:4)(cid:8)A(cid:10) spectrum and (cid:16)(cid:22)pseudospectrum of A
(cid:2)AND(cid:3) TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) (cid:3)
The followinglittlesections(cid:2)and(cid:3) containessentialbackgroundmaterial
for the remainder of the text(cid:5) They are not chapters(cid:7) just tidbits(cid:5) Think of
them as appetizers(cid:5)
(cid:2)(cid:5) BIG(cid:2)OANDRELATEDSYMBOLS TREFETHEN (cid:2)(cid:3)(cid:3)(cid:4) (cid:0) (cid:13)
(cid:0)(cid:2) Big(cid:3)O and related symbols
How fast is an algorithm% How accurate are the results it produces%
Answeringquestionsliketheserequiresestimatingoperationcountsanderrors(cid:7)
and we need a convenient notation for doing so(cid:5) This book will use the six
symbols o(cid:7) O(cid:7) (cid:31)(cid:7) (cid:7) (cid:8)(cid:7) and (cid:9)(cid:5) Four of these are standard throughout the
mathematical sciences(cid:7) while (cid:31) and have recently become standard at least
in theoretical computer science(cid:5)(cid:30)
Here are the (cid:6)rst four de(cid:6)nitions(cid:5) They are written for functions f(cid:8)x(cid:10) and
g(cid:8)x(cid:10) as x(cid:6)(cid:10)(cid:7) but analogous de(cid:6)nitions apply to other limits such as x(cid:6)(cid:28)(cid:5)
O f(cid:8)x(cid:10)!O(cid:8)g(cid:8)x(cid:10)(cid:10) as x(cid:6)(cid:10) if there exist constants C (cid:17)(cid:28) and x(cid:12)(cid:17)(cid:28) such
that jf(cid:8)x(cid:10)j(cid:2)Cg(cid:8)x(cid:10) for all x(cid:3)x(cid:12)(cid:5) If g(cid:8)x(cid:10)(cid:11)!(cid:28) this means jf(cid:8)x(cid:10)j(cid:13)g(cid:8)x(cid:10) is
bounded from above for all su&ciently large x(cid:5) In words(cid:29) (cid:13)f is O of g(cid:5)(cid:14)
o f(cid:8)x(cid:10)!o(cid:8)g(cid:8)x(cid:10)(cid:10) as x(cid:6)(cid:10) if for any constant C(cid:17)(cid:28) there exists a constant
x(cid:12) (cid:17)(cid:28) such that jf(cid:8)x(cid:10)j(cid:2)Cg(cid:8)x(cid:10) for all x(cid:3)x(cid:12)(cid:5) If g(cid:8)x(cid:10)(cid:11)!(cid:28) this means
jf(cid:8)x(cid:10)j(cid:13)g(cid:8)x(cid:10) approaches (cid:28) as x(cid:6)(cid:10)(cid:5) In words(cid:29) (cid:13)f is little(cid:22)O of g(cid:5)(cid:14)
(cid:31) f(cid:8)x(cid:10)!(cid:31)(cid:8)g(cid:8)x(cid:10)(cid:10) as x(cid:6)(cid:10) if there exist constants C (cid:17)(cid:28) and x(cid:12)(cid:17)(cid:28) such
that f(cid:8)x(cid:10) (cid:3) Cg(cid:8)x(cid:10) for all x (cid:3) x(cid:12)(cid:5) If g(cid:8)x(cid:10)(cid:11)!(cid:28) this means f(cid:8)x(cid:10)(cid:13)g(cid:8)x(cid:10) is
bounded from below for all su&ciently large x(cid:5) In words(cid:29) (cid:13)f is omega
of g(cid:5)(cid:14)
(cid:2)
f(cid:8)x(cid:10)! (cid:8)g(cid:8)x(cid:10)(cid:10) as x(cid:6)(cid:10) if there exist constants C(cid:5)C (cid:17)(cid:28) and x(cid:12) (cid:17)(cid:28)
(cid:2)
such that Cg(cid:8)x(cid:10)(cid:2)f(cid:8)x(cid:10)(cid:2)C g(cid:8)x(cid:10) for all x(cid:3)x(cid:12)(cid:5) If g(cid:8)x(cid:10)(cid:11)!(cid:28) this means
f(cid:8)x(cid:10)(cid:13)g(cid:8)x(cid:10) is bounded from above and below(cid:5) In words(cid:29) (cid:13)f is theta of g(cid:5)(cid:14)
The use of the symbols O(cid:7) o(cid:7) (cid:31)(cid:7) and is a bit illogical(cid:7) for properly
speaking(cid:7) O(cid:8)g(cid:8)x(cid:10)(cid:10) for example is the set of all functions f(cid:8)x(cid:10) with a certain
property(cid:19) but why then don’t we write f(cid:8)x(cid:10)(cid:4)O(cid:8)g(cid:8)x(cid:10)(cid:10)% The answer is sim(cid:22)
ply that the traditional usage(cid:7) illogical or not(cid:7) is well established and very
convenient(cid:5)
The (cid:6)nal two de(cid:6)nitions are not so irregular(cid:29)
(cid:2)Thanks to the delightful note (cid:5)Big omicron and big omega and big theta(cid:6) by D(cid:7) E(cid:7) Knuth(cid:4) ACM
SIGACT News (cid:2)(cid:4) (cid:11)(cid:13)(cid:14)(cid:15)(cid:7)