Table Of Content

Computational Methods Linearo fAlgebra 3rd Edition 9181_9789814603850_tp.indd 1 12/6/14 10:25 am June5,2014 14:55 BC:9181–ComputationalMethodsofLinearAlgebra 1stReading sewellnew pagevi TThhiiss ppaaggee iinntteennttiioonnaallllyy lleefftt bbllaannkk Computational Methods of Linear Algebra 3rd Edition Granville Sewell University of Texas El Paso, USA World Scientific NEW JERSEY • LONDON • SINGAPORE • BEIJING • SHANGHAI • HONG KONG • TAIPEI • CHENNAI 9181_9789814603850_tp.indd 2 12/6/14 10:25 am Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE Library of Congress Cataloging-in-Publication Data Sewell, Granville, author. Computational methods of linear algebra / by Granville Sewell (University of Texas El Paso, USA). -- 3rd edition. pages cm Includes bibliographical references and index. ISBN 978-9814603850 (hardcover : alk. paper) -- ISBN 978-9814603867 (pbk : alk. paper) 1. Algebras, Linear--Textbooks. I. Title. QA184.2.S44 2014 512'.5--dc23 2014016584 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. Copyright © 2014 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the publisher. For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. Printed in Singapore RokTing - Computational Methods of Linear Algebra.indd 1 17/6/2014 12:14:38 PM June5,2014 14:55 BC:9181–ComputationalMethodsofLinearAlgebra 1stReading sewellnew pagev To my son, Kevin v June5,2014 14:55 BC:9181–ComputationalMethodsofLinearAlgebra 1stReading sewellnew pagevi TThhiiss ppaaggee iinntteennttiioonnaallllyy lleefftt bbllaannkk June5,2014 14:55 BC:9181–ComputationalMethodsofLinearAlgebra 1stReading sewellnew pagevii Contents 0 Reference Material 1 0.1 Miscellaneous Results from Linear Algebra. . . . . . . . . . . . 1 0.2 Special Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 4 0.3 Vector and Matrix Norms . . . . . . . . . . . . . . . . . . . . . 7 1 Systems of Linear Equations 11 1.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2 Gaussian Elimination. . . . . . . . . . . . . . . . . . . . . . . . 12 1.3 Solving Several Systems with the Same Matrix . . . . . . . . . 20 1.4 The LU Decomposition . . . . . . . . . . . . . . . . . . . . . . 24 1.5 Banded Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 27 1.6 Sparse Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 1.7 Application: Cubic Spline Interpolation . . . . . . . . . . . . . 36 1.8 Roundoff Error . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 1.9 Iterative Methods: Jacobi, Gauss-Seidel and SOR . . . . . . . . 48 1.10 The Conjugate Gradient Method . . . . . . . . . . . . . . . . . 59 1.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 2 Linear Least Squares Problems 71 2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 2.2 Orthogonal Reduction . . . . . . . . . . . . . . . . . . . . . . . 74 2.3 Reduction Using Householder Transformations . . . . . . . . . 81 2.4 Least Squares Approximation with Cubic Splines . . . . . . . . 87 2.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 3 The Eigenvalue Problem 94 3.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 3.2 The Jacobi Method for Symmetric Matrices . . . . . . . . . . . 96 3.3 The QR Method for General Real Matrices . . . . . . . . . . . 104 3.4 Alternative Methods for General Matrices . . . . . . . . . . . . 117 3.5 The Power and Inverse Power Methods . . . . . . . . . . . . . . 127 3.6 The Generalized Eigenvalue Problem . . . . . . . . . . . . . . . 138 vii June5,2014 14:55 BC:9181–ComputationalMethodsofLinearAlgebra 1stReading sewellnew pageviii viii CONTENTS 3.7 The Singular Value Decomposition . . . . . . . . . . . . . . . . 146 3.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 4 Linear Programming 156 4.1 Linear Programming Applications . . . . . . . . . . . . . . . . 156 4.1.1 The Resource Allocation Problem . . . . . . . . . . . . 156 4.1.2 The Blending Problem . . . . . . . . . . . . . . . . . . . 157 4.1.3 The Transportation Problem . . . . . . . . . . . . . . . 157 4.1.4 Curve Fitting . . . . . . . . . . . . . . . . . . . . . . . . 158 4.2 The Simplex Method, with Artificial Variables . . . . . . . . . 159 4.3 The Dual Solution . . . . . . . . . . . . . . . . . . . . . . . . . 167 4.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 4.5 A FORTRAN90 Program . . . . . . . . . . . . . . . . . . . . . 175 4.6 The Revised Simplex Method . . . . . . . . . . . . . . . . . . . 182 4.7 The Transportation Problem . . . . . . . . . . . . . . . . . . . 198 4.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 5 The Fast Fourier Transform 210 5.1 The Discrete Fourier Transform . . . . . . . . . . . . . . . . . . 210 5.2 The Fast Fourier Transform . . . . . . . . . . . . . . . . . . . . 211 5.3 FORTRAN90 Programs . . . . . . . . . . . . . . . . . . . . . . 214 5.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 6 Linear Algebra on Supercomputers 225 6.1 Vector Computers . . . . . . . . . . . . . . . . . . . . . . . . . 225 6.2 Parallel Computers . . . . . . . . . . . . . . . . . . . . . . . . . 228 6.3 Computational Linear Algebra in a PDE Solver . . . . . . . . . 247 6.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 Appendix A—MATLAB Programs 255 Appendix B—Answers to Selected Exercises 302 References 313 Index 317 June5,2014 14:55 BC:9181–ComputationalMethodsofLinearAlgebra 1stReading sewellnew pageix Preface This text is appropriate for a course on the numerical solution of linear al- gebraic problems, designed for senior level undergraduate or beginning level graduate students. Although it will most likely be used for a second course innumericalanalysis, the onlyprerequisiteto usingthis textis agoodcourse in linear or matrix algebra;however,such a courseis anextremely important prerequisite. Chapter 0 presents some basic definitions and results from linear algebra which are used in the later chapters. By no means can this short chapter be used to circumvent the linear algebra prerequisite mentioned above; it does not even contain a comprehensive review of the basic ideas of linear algebra. It is intended only to present some miscellaneous ideas, selected for inclusion because they are especially fundamental to later developments or may not be covered in a typical introductory linear algebra course. Chapters 1–4 present and analyze methods for the solution of linear systems of equations (direct and iterative methods), linear least squares problems, linear eigenvalueproblems, and linearprogrammingproblems; in short, we attack everything that begins with the word “linear”. Truly “linear” numerical analysis problems have the common feature that they can be solved exactly, in a finite number of steps, if exact arithmetic is done. This means that all errors are due to roundoff; that is, they are attributable to the use of finite precision by the computer. (Iterative methods for linear systems and all methods for eigenvalue problems—which are not really linear—are exceptions.) Thus stability with respect to roundoff error must be a major consideration in the design of software for linear problems. Chapter5discussesthefastFouriertransform. Thisisnotatopicnormally covered in texts on computational linear algebra. However, the fast Fourier transform is really just an efficient way of multiplying a special matrix times anarbitraryvector,andsoitdoesnotseemtoooutofplaceinacomputational linear algebra text. Chapter 6 contains a practical introduction for the student interested in writing computational linear algebrasoftware that runs efficiently on today’s vector and parallel supercomputers. Double-precision FORTRAN90 subroutines, which solve each of the main ix June5,2014 14:55 BC:9181–ComputationalMethodsofLinearAlgebra 1stReading sewellnew pagex x PREFACE problems covered using algorithms studied in the text, are presented and highlighted. A top priority in designing these subroutines was readability. Each subroutine is written in a well-documented, readable style, so that the studentwill be able tofollowthe programlogicfromstarttofinish. All loops are explicit and indented to make it easier for the student to analyze the computationalcomplexityofthealgorithms. Eventhoughwehavesteadfastly resisted the temptation to make them slightly more efficient at the expense of readability, the subroutines that solve the truly linear problems are nearly state-of-the-art with regard to efficiency. The eigenvalue codes, on the other hand, arenotstate-of-the-art,butneitheraretheygrosslyinferiortothebest programs available. MATLAB(cid:2) versions of the codes in Chapters 1–5 are listed in Appendix A. Machine-readable copies of the FORTRAN90 and MATLAB codes in the book can be downloaded from http://www.math.utep.edu/Faculty/sewell/computational methods There is very little difference between the FORTRAN and MATLAB versions; they are almost line-by-line translations, so the student who is familiar with MATLAB (or any other programming language, for that matter) will have no trouble following the logic of the FORTRAN programs in the text. ButstudentscandothecomputerproblemsusingeitherFORTRANorMAT- LAB, with the exception of the problems in Chapter 6. The problems in this chapter require a FORTRAN90 compiler and an MPI library. Subroutines DEGNON, DPOWER, DFFT and NRFFT contain double- precision complex variables, typed COMPLEX*16, which is a nonstandard, butwidelyrecognized,type. Otherwise,thesubroutinesconformtotheFOR- TRAN90 standardandthus shouldbe highly portable. In fact, the programs in Chapters 1–4 will run on most FORTRAN77 compilers. For extensive surveyson other availablemathematical software, including software for the problems studied in this text, the reader is referred to the books by Heath [2002; summary in each chapter] and Kincaid and Cheney [2004; Appendix B]. The author developed this text for a graduate course at the University of Texas El Paso and has also used it for a Texas A&M distance learning graduate course.