Undergraduate Texts in Mathematics Editors S. Axler F.W. Gehring K.A. Ribet Paul Cull Mary Flahive Robby Robson Difference Equations From Rabbits to Chaos With 16 Illustrations PaulCull MaryFlahive RobbyRobson Dept.ComputerScience Dept.Mathematics Eduworks DearbornHall KidderHall 3520Northwest OregonStateUniversity OregonStateUniversity HayesAve. Corvallis,OR97331 Corvallis,OR97331 Corvallis,OR97330 USA USA USA [email protected] [email protected] rrobson@eduworks. com EditorialBoard S.Axler F.W.Gehring K.A.Ribet MathematicsDepartment MathematicsDepartment Departmentof SanFranciscoState EastHall Mathematics University UniversityofMichigan UniversityofCalifornia SanFrancisco,CA94132 AnnArbor,MI48109 atBerkeley USA USA Berkeley,CA94720-3840 USA MathematicsSubjectClassification(2000):39-01,39Axx,68Rxx,11B37,11B39 LibraryofCongressCataloging-in-PublicationData Cull,Paul,1943– Differenceequations:fromrabbitstochaos/PaulCull,MaryFlahive, RobbyRobson. p.cm.—(Undergraduatetextsinmathematics) Includesbibliographicalreferencesandindex. ISBN0-387-23234-6(alk.paper) 1. Differenceequations. I. Flahive,MaryE.,1948– II. Title. III. Series. QA431.C85 2004 515′.625—dc22 2004058968 ISBN0-387-23234-6(softcover) Printedonacid-freepaper. ISBN0-387-23233-8(hardcover) ©2005SpringerScience+BusinessMedia,Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc.,233SpringStreet,NewYork,NY10013,USA),exceptforbriefexcerptsinconnec- tionwithreviewsorscholarlyanalysis.Useinconnectionwithanyformofinforma- tionstorageandretrieval,electronicadaptation,computersoftware,orbysimilaror dissimilarmethodologynowknownorhereafterdevelopedisforbidden. The use in this publication of trade names, trademarks, service marks, and similar terms,eveniftheyarenotidentifiedassuch,isnottobetakenasanexpressionof opinionastowhetherornottheyaresubjecttoproprietaryrights. PrintedintheUnitedStatesofAmerica. (EB) 9 8 7 6 5 4 3 2 1 SPIN10950852(softcover) SPIN10967645(hardcover) springeronline.com Preface Some years ago we noticed that various seemingly disparate fields were usingsimilarmodelsandtechniquestosolvesimilarproblems.Frommath- ematicstocomputerscienceto engineeringto biology,variousformsofdif- ference equations were appearing in researchpapers and in textbooks, but with no common background, the same results were being independently derived over and over again. As we were noticing this, some mathematics curricula were being revised with discrete mathematics replacing calculus as the first college mathematics course. New discrete mathematics courses werecreated,andseveralsuperbtextbooks appeared.Insomefields ayear ofdiscrete mathematicsactually replaceda yearofcalculus,while inother fields students took both discrete mathematics and calculus. With these changes,what happened to difference equations?Some texts in discrete mathematics ignored them. Others had a few examples of dif- ference equations as applications of proof by induction. Still others de- voteda chapterto difference equations,but only solveda few specialcases and/or represented generating functions (also called Z-transforms) as the principal or only method for finding solutions. With this lack of common background,textsonalgorithms,signalprocessing,andpopulationbiology werestillforcedtodevotechapterstothedifferenceequationsusedintheir areas.Evenstudentswhotookseveralofthesecourseshaddifficultyseeing thattheywereworkingwiththesamedifferenceequationsindifferentcon- texts.Manyinstructorshadwrittennotestofleshoutthecoveragegivenin texts, but such notes were of necessity usually so terse that students were led to believe that difference equations were very complicated and hard to understand. vi Preface With these problems in mind, we set out to write a book on difference equations that is accessible to undergraduates. As a text, it is meant for undergraduate majors in one of the mathematical sciences, presumably in their junior or senior year. We’ve written it for the student who likes to compute and is comfortable with mathematical proof, but the book can be profitably read by students who approach the subject from either a computational or theoretical point of view. We wanted our text to have an algorithmic spirit. In this book, each chapterleadsto techniques thatcanbe appliedby handto smallexamples and also can be programmed for larger examples. In many cases we give explicit algorithms, which we decided to write in pseudocode rather than in a specific programming language for several reasons. First, it is easy to translate from pseudocode into any reasonable programming language. Second, there are many programming languages available, and translating from one language to another is often more difficult than translating from pseudocode. Third, we are not sure that programming these algorithms is worth the effort, because for almost all of our examples there are high- quality implementations readily available on the Web. It probably makes more sense to use one of these programs rather than to cobble together a programthatwillbeusedonlyafewtimesand/orwillbepronetoproblems when the input is not exactly in the form assumed by the programmer.A number of mathematically oriented computer packages are also available. For example, MATLAB, Maple, and Mathematica all have packages that willsolvedifferenceequationsandrecurrencerelations.Inmanycasesthese packagesgivenumericanswersaswellassymbolicsolutionswhenpossible. Using these packages is much simpler than programming from scratch. In this book we start with the old story of Fibonacci’s rabbits and progressthroughseveralgeneralizations,endingwithsomenonlineardiffer- enceequations.We dealwith familiar mathematical structures suchas the realnumbers, the complex numbers, the integers,and the integers modulo aninteger.We weretemptedto discussmoregeneralstructuresinorderto show,forexample,howtheoriesofcomputationcouldberepresentedasdif- ferenceequations,butwesoondiscoveredthatthiswouldresultineithera verylargebookoraveryformalbook,whichwouldbeatvariancewithour goalofaccessibility.After developingthe theoryandtechniquesforsolving linear difference equations in Chapters 2 to 4, we specialize to equations with nonnegative coefficients in Chapters 5 and 6 and then consider the generalizationto matrix difference equations in Chapter 7. Chapter 8 con- siders equations over other rings, including integers modulo m and finite fields.Chapter9considerssomeissuesincomputationalcomplexity,includ- ingdivide-and-conqueralgorithms.Weendwithsomenonlinearsystemsin Chapter 10. Along the way we use linear algebra, develop formal power series, solve some combinatorial problems, visit Perron–Frobinus theory, use graph theory, discuss pseudorandom number generation and integer factorization, and use the FFT to multiply polynomials quickly. Computation vii Therearefourappendicesservingdifferentpurposes.Thefirstisacollec- tionofworkedexamples,whicharemeanttosupplementtheearlychapters of the book. Because the material in Appendices B and C is essential to an understanding of the book, we suggest working through them before beginning Chapter 2. Although many of the difference equations we con- sider have integer or real coefficients, it is often necessary to consider the coefficients as complex numbers. Appendix B gives the highlights of the complex analysis we use, and no prior experience is necessary to under- standthisappendix.Ontheotherhand,onlythemostexceptionalstudent couldlearnnewmaterialatthe rateatwhichlinearalgebraispresentedin Appendix C. One of the aims of this book is to show students that linear algebra is a powerful and coherent subject whose ideas have diverse appli- cations,andwehope Appendix C is a helpfulreview.Appendix D outlines a method of Morris Marden [105] that can be used to decide when the general solution of a difference equation converges to zero. This appendix is not needed for an understanding of the book. Computation Most of our examples work with “small” difference equations, equations that can be completely solved by hand. In particular, for these equations their characteristic polynomials can be found, the roots of these polyno- mials can be computed exactly, and the associated eigenvector equations can be solved. While the theory we develop applies to both small and largeequations,thesecomputationsmaybedifficultorimpossibleforlarge equations. For example, actually factoring polynomials is not possible in general, and rational computation of characteristic polynomials may re- quire numbers with very many digits. Numerical approximation methods are often used for these computations, and we refer the interested reader to Acton [1], who gives a good introduction to numerical methods. (More serioususersmightrefertothe compendium[131]ortothe classic[170]by Wilkinson.)Ingeneral,wedonotcovernumericalmethods.Theoneexcep- tiontothis ruleisourdiscussionoftheuseofNewton’smethodforfinding the positive root of a nonnegative polynomial. We include this method for several reasons: it rapidly finds this root, the proof of its convergence and its speed of convergenceare relatively easy, and the method is an example of a commonly encountered nonlinear difference equation. viii Preface Notational Preliminaries In this book, we use the following fairly standard notation: Z is the set of integers. Z is the set of integers modulo m. m Zk is the set of all k-tuples with integer coordinates. Zk is the set of all k-tuples of integers modulo m. m N is the set of natural numbers, including 0; N={0,1,2,...}. N+ is the set of positive integers. Q is the set of rational numbers. R is the set of real numbers. F denotes a finite field. C is the set of complex numbers. R[x] is the set of polynomials with real coefficients. C[x] is the set of polynomials with complex coefficients. Z [x] is the set of polynomials whose coefficients are integers modulo m. m F[x] is the set of polynomials with coefficients from the finite fieldF. (cid:1)x(cid:2) is the floor of x∈R, the largest integer n with n≤x. (cid:5)x(cid:6) is the ceiling of x∈R, the smallest integer n with n≥x . k(modm) means the equivalence class {k+jm : j ∈Z}, while k modm means the least nonnegative integer in the class k(modm). Contents Preface v Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Notational Preliminaries . . . . . . . . . . . . . . . . . . . . . . . viii 1 Fibonacci Numbers 1 1.1 The Rabbit Problem . . . . . . . . . . . . . . . . . . . . . . 1 1.2 The Fibonacci Sequence . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Computing Fibonacci numbers . . . . . . . . . . . . 4 1.2.2 A formula for the Fibonacci numbers . . . . . . . . . 5 1.2.3 Further Fibonacci facts . . . . . . . . . . . . . . . . 6 1.3 Notation for Asymptotic Analysis . . . . . . . . . . . . . . . 6 1.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Homogeneous Linear Recurrence Relations 11 2.1 The Solution Space of (HL) . . . . . . . . . . . . . . . . . . 12 2.2 The Matrix Form . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3 A Simpler Basis for the Solution Space . . . . . . . . . . . . 17 2.3.1 Distinct eigenvalues . . . . . . . . . . . . . . . . . . 19 2.3.2 Repeated eigenvalues. . . . . . . . . . . . . . . . . . 21 2.4 The Asymptotic Behavior of Solutions . . . . . . . . . . . . 25 2.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3 Finite Difference Equations 33 3.1 Linear Difference Equations . . . . . . . . . . . . . . . . . . 33 x Contents 3.1.1 First–order equations . . . . . . . . . . . . . . . . . 34 3.2 General and Particular Solutions . . . . . . . . . . . . . . . 36 3.2.1 Finding a particular solution via summation. . . . . 39 3.3 A Special Class of Linear Recurrences . . . . . . . . . . . . 41 3.4 Operator Notation . . . . . . . . . . . . . . . . . . . . . . . 45 3.5 The Shift Operator on the Space of Sequences . . . . . . . . 47 3.6 Formal Power Series . . . . . . . . . . . . . . . . . . . . . . 50 3.6.1 Formal differentiation . . . . . . . . . . . . . . . . . 55 3.6.2 An application of formal power series . . . . . . . . 56 3.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4 Generating Functions 67 4.1 Counting Strings with Some Restrictions . . . . . . . . . . . 67 4.2 An Overview of the Generating Function Technique . . . . 70 4.2.1 Rational representation . . . . . . . . . . . . . . . . 75 4.3 A Review of Partial Fractions . . . . . . . . . . . . . . . . . 76 4.4 Examples of the Generating Function Technique . . . . . . 82 4.4.1 The Catalan numbers . . . . . . . . . . . . . . . . . 83 4.4.2 Stirling numbers of the second kind . . . . . . . . . 85 4.5 Reversion of Generating Functions . . . . . . . . . . . . . . 87 4.5.1 Using the Fourier Transform . . . . . . . . . . . . . 91 4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5 Nonnegative Difference Equations 101 5.1 Nonnegative Polynomials . . . . . . . . . . . . . . . . . . . 102 5.1.1 The dominant root . . . . . . . . . . . . . . . . . . . 102 5.2 When are integer solutions rounded powers of an eigenvalue? 106 5.2.1 Using the Rounding Theorem . . . . . . . . . . . . . 110 5.3 Estimation of the Roots . . . . . . . . . . . . . . . . . . . . 113 5.3.1 Estimation of the dominant root . . . . . . . . . . . 113 5.3.2 Estimation of the second root . . . . . . . . . . . . . 113 5.4 Calculation of the Roots . . . . . . . . . . . . . . . . . . . . 116 5.4.1 The rate of convergence in Newton’s method . . . . 121 5.5 Asymptotic Size of Solutions . . . . . . . . . . . . . . . . . 125 5.5.1 Homogeneous nonnegative recurrences . . . . . . . . 125 5.5.2 Nonhomogeneous nonnegative equations . . . . . . . 127 5.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 6 Leslie’s Population Matrix Model 137 6.1 Leslie’s Model . . . . . . . . . . . . . . . . . . . . . . . . . . 137 6.1.1 How to tell whether a Leslie matrix is primitive . . . 141 6.2 Leslie’s Convergence Theorem . . . . . . . . . . . . . . . . . 142 6.3 Imprimitive Leslie Matrices . . . . . . . . . . . . . . . . . . 144 6.3.1 A simple example . . . . . . . . . . . . . . . . . . . 144 6.3.2 A special case: Only one positive fertility rate . . . . 145 Contents xi 6.3.3 Asymptotically periodic Leslie matrices . . . . . . . 145 6.4 Companion Matrices . . . . . . . . . . . . . . . . . . . . . . 147 6.4.1 Matrices with repeated eigenvalues . . . . . . . . . . 155 6.5 Nonnegative Companion Matrices. . . . . . . . . . . . . . . 157 6.5.1 Periodic nonnegative companion matrices . . . . . . 159 6.6 Back to Leslie Matrices . . . . . . . . . . . . . . . . . . . . 164 6.6.1 Periodic Leslie matrices . . . . . . . . . . . . . . . . 165 6.6.2 Averaging . . . . . . . . . . . . . . . . . . . . . . . . 168 6.7 The Limiting Effect of L on Nonnegative Vectors . . . . . . 169 6.7.1 The period of the total population . . . . . . . . . . 171 6.8 Afterword . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 6.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 7 Matrix Difference Equations 179 7.1 Homogeneous Matrix Equations . . . . . . . . . . . . . . . . 179 7.2 Nonnegative Matrix Equations . . . . . . . . . . . . . . . . 186 7.2.1 Applications to Markov chains . . . . . . . . . . . . 187 7.3 Graphs and Matrices . . . . . . . . . . . . . . . . . . . . . . 189 7.3.1 Next node representation . . . . . . . . . . . . . . . 193 7.3.2 Comments on imprimitivity . . . . . . . . . . . . . . 194 7.4 Algorithms for Primitivity . . . . . . . . . . . . . . . . . . . 198 7.4.1 Algorithm I . . . . . . . . . . . . . . . . . . . . . . . 198 7.4.2 Algorithm II . . . . . . . . . . . . . . . . . . . . . . 202 7.5 Matrix Difference Equations with Input . . . . . . . . . . . 206 7.5.1 Reduction to one dimension . . . . . . . . . . . . . . 207 7.5.2 Reduction to homogeneous form . . . . . . . . . . . 211 7.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 8 Modular Recurrences 217 8.1 Periodicity. . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 8.1.1 Periodicity of linear modular recurrences. . . . . . . 221 8.1.2 Fast modular computations . . . . . . . . . . . . . . 224 8.2 Finite Fields . . . . . . . . . . . . . . . . . . . . . . . . . . 225 8.3 Periods of First–Order Modular Recurrences. . . . . . . . . 227 8.3.1 First–order modular recurrences with maximal period 230 8.4 Periodic Second–Order Modular Recurrences . . . . . . . . 232 8.4.1 Periods of modular Fibonacci sequences . . . . . . . 233 8.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 8.5.1 Application 1: Pseudorandomnumber generation . . 238 8.5.2 Application 2: Integer factorization . . . . . . . . . . 242 8.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 9 Computational Complexity 253 9.1 Analysis of Algorithms . . . . . . . . . . . . . . . . . . . . . 254 9.1.1 Measuring run time . . . . . . . . . . . . . . . . . . 254