Numerical Linear Approximation in C

950 Pages·2008·4.01 MB·English

Numerical
Linear
Approximation
in C
© 2008 by Taylor & Francis Group, LLC
CHAPMAN & HALL/CRC
Numerical Analysis and Scientific Computing
Aims and scope:
Scientific computing and numerical analysis provide invaluable tools for the sciences and engineering.
This series aims to capture new developments and summarize state-of-the-art methods over the
whole spectrum of these fields. It will include a broad range of textbooks, monographs and
handbooks. Volumes in theory, including discretisation techniques, numerical algorithms, multiscale
techniques, parallel and distributed algorithms, as well as applications of these methods in multi-
disciplinary fields, are welcome. The inclusion of concrete real-world examples is highly encouraged.
This series is meant to appeal to students and researchers in mathematics, engineering and
computational science.
Editors
Choi-Hong Lai Frédéric Magoulès
School of Computing and Applied Mathematics and
Mathematical Sciences Systems Laboratory
University of Greenwich Ecole Centrale Paris
Editorial Advisory Board
Mark Ainsworth Peter Jimack
Mathematics Department School of Computing
Strathclyde University University of Leeds
Todd Arbogast Takashi Kako
Institute for Computational Department of Computer Science
Engineering and Sciences The University of Electro-Communications
The University of Texas at Austin
Peter Monk
Craig C. Douglas Department of Mathematical Sciences
Computer Science Department University of Delaware
University of Kentucky
Francois-Xavier Roux
Ivan Graham ONERA
Department of Mathematical Sciences
University of Bath Arthur E.P. Veldman
Institute of Mathematics and Computing Science
University of Groningen
Proposals for the series should be submitted to one of the series editors above or directly to:
CRC Press, Taylor & Francis Group
24-25 Blades Court
Deodar Road
London SW15 2NU
UK
© 2008 by Taylor & Francis Group, LLC
Numerical
Linear
Approximation
in C
Nabih N. Abdelmalek, Ph.D.
William A. Malek, M.Eng.
© 2008 by Taylor & Francis Group, LLC
Chapman & Hall/CRC
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2008 by Taylor & Francis Group, LLC
Chapman & Hall/CRC is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number-13: 978-1-58488-978-6 (Hardcover)
This book contains information obtained from authentic and highly regarded sources Reason-
able efforts have been made to publish reliable data and information, but the author and publisher
cannot assume responsibility for the validity of all materials or the consequences of their use. The
Authors and Publishers have attempted to trace the copyright holders of all material reproduced
in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so
we may rectify in any future reprint
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.
copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC)
222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that
provides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Abdelmalek, Nabih N.
Numerical linear approximation in C / Nabih Abdelmalek and William A.
Malek.
p. cm. -- (CRC numerical analysis and scientific computing)
Includes bibliographical references and index.
ISBN 978-1-58488-978-6 (alk. paper)
1. Chebyshev approximation. 2. Numerical analysis. 3. Approximation theory.
I. Malek, William A. II. Title. III. Series.
QA297.A23 2008
518--dc22 2008002447
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
© 2008 by Taylor & Francis Group, LLC
v
Contents
List of figures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Warranties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
About the authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv
PART 1
Preliminaries and Tutorials
Chapter 1 Applications of Linear Approximation
1.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Applications to social sciences and economics. . . . . . . . . . . . . . . . 5
1.2.1 Systolic blood pressure and age . . . . . . . . . . . . . . . . . . . 6
1.2.2 Annual teacher salaries. . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.3 Factors affecting survival of island species . . . . . . . . . . 7
1.2.4 Factors affecting fuel consumption. . . . . . . . . . . . . . . . . 8
1.2.5 Examining factors affecting the mortality rate. . . . . . . . 8
1.2.6 Effects of forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.7 Factors affecting gross national products. . . . . . . . . . . . 9
1.3 Applications to industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.1 Windmill generating electricity . . . . . . . . . . . . . . . . . . 10
1.3.2 A chemical process. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Applications to digital images. . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4.1 Smoothing of random noise in digital images . . . . . . . 12
1.4.2 Filtering of impulse noise in digital images . . . . . . . . . 14
1.4.3 Applications to pattern classification . . . . . . . . . . . . . . 15
© 2008 by Taylor & Francis Group, LLC
vi Numerical Linear Approximation in C
1.4.4 Restoring images with missing high-frequency
components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.4.5 De-blurring digital images using the Ridge equation. . 18
1.4.6 De-blurring images using truncated eigensystem. . . . . 19
1.4.7 De-blurring images using quadratic programming. . . . 20
Chapter 2 Preliminaries
2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.2 Discrete linear approximation and solution of overdetermined
linear equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3 Comparison between the L , the L and the L norms by a
1 2 ∞
practical example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.1 Some characteristics of the L and the Chebyshev
1
approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.4 Error tolerances in the calculation. . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5 Representation of vectors and matrices in C. . . . . . . . . . . . . . . . . 33
2.6 Outliers and dealing with them . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.6.1 Data editing and residual analysis. . . . . . . . . . . . . . . . . 36
Chapter 3 Linear Programming and the Simplex Algorithm
3.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1.1 Exceptional linear programming problems. . . . . . . . . . 41
3.2 Notations and definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 The simplex algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3.1 Initial basic feasible solution . . . . . . . . . . . . . . . . . . . . 47
3.3.2 Improving the initial basic feasible solution. . . . . . . . . 48
3.4 The simplex tableau. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.5 The two-phase method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.1 Phase 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.2 Phase 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.5.3 Detection of the exceptional cases . . . . . . . . . . . . . . . . 56
3.6 Duality theory in linear programming. . . . . . . . . . . . . . . . . . . . . . 57
3.6.1 Fundamental properties of the dual problems. . . . . . . . 59
3.6.2 Dual problems with mixed constraints. . . . . . . . . . . . . 60
3.6.3 The dual simplex algorithm . . . . . . . . . . . . . . . . . . . . . 61
3.7 Degeneracy in linear programming and its resolution . . . . . . . . . 62
3.7.1 Degeneracy in the simplex method. . . . . . . . . . . . . . . . 62
3.7.2 Avoiding initial degeneracy in the simplex algorithm . 63
© 2008 by Taylor & Francis Group, LLC
Contents vii
3.7.3 Resolving degeneracy resulting from equal θ . . . . . 63
min
3.7.4 Resolving degeneracy in the dual simplex method. . . . 64
3.8 Linear programming and linear approximation. . . . . . . . . . . . . . . 64
3.8.1 Linear programming and the L approximation. . . . . . 64
1
3.8.2 Linear programming and Chebyshev approximation. . 66
3.9 Stability of the solution in linear programming . . . . . . . . . . . . . . 67
Chapter 4 Efficient Solutions of Linear Equations
4.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Vector and matrix norms and relevant theorems. . . . . . . . . . . . . . 72
4.2.1 Vector norms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.2.2 Matrix norms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.2.3 Hermitian matrices and vectors . . . . . . . . . . . . . . . . . . 73
4.2.4 Other matrix norms. . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.2.5 Euclidean and the spectral matrix norms . . . . . . . . . . . 77
4.2.6 Euclidean norm and the singular values. . . . . . . . . . . . 78
4.2.7 Eigenvalues and the singular values of the sum and
the product of two matrices. . . . . . . . . . . . . . . . . . . . . . 79
4.2.8 Accuracy of the solution of linear equations . . . . . . . . 80
4.3 Elementary matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.4 Gauss LU factorization with complete pivoting. . . . . . . . . . . . . . 84
4.4.1 Importance of pivoting . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4.2 Using complete pivoting. . . . . . . . . . . . . . . . . . . . . . . . 86
4.4.3 Pivoting and the rank of matrix A. . . . . . . . . . . . . . . . . 88
4.5 Orthogonal factorization methods. . . . . . . . . . . . . . . . . . . . . . . . . 90
4.5.1 The elementary orthogonal matrix H . . . . . . . . . . . . . . 90
4.5.2 Householder(cid:146)s QR factorization with pivoting. . . . . . . 90
4.5.3 Pivoting in Householder(cid:146)s method . . . . . . . . . . . . . . . . 92
4.5.4 Calculation of the matrix inverse A(cid:150)1. . . . . . . . . . . . . . 93
4.6 Gauss-Jordan method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.7 Rounding errors in arithmetic operations . . . . . . . . . . . . . . . . . . . 95
4.7.1 Normalized floating-point representation. . . . . . . . . . . 95
4.7.2 Overflow and underflow in arithmetic operations . . . . 96
4.7.3 Arithmetic operations in a d.p. accumulator. . . . . . . . . 97
4.7.4 Computation of the square root of a s.p. number . . . . 100
4.7.5 Arithmetic operations in a s.p. accumulator. . . . . . . . 100
4.7.6 Arithmetic operations with two d.p. numbers. . . . . . . 101
4.7.7 Extended simple s.p. operations in a d.p. accumulator 101
4.7.8 Alternative expressions for summations and inner-
product operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
© 2008 by Taylor & Francis Group, LLC
viii Numerical Linear Approximation in C
4.7.9 More conservative error bounds. . . . . . . . . . . . . . . . . 104
4.7.10 D.p. summations and inner-product operations . . . . . 105
4.7.11 Rounding error in matrix computation. . . . . . . . . . . . 106
4.7.12 Forward and backward round-off error analysis. . . . . 107
4.7.13 Statistical error bounds and concluding remarks . . . . 109
PART 2
The L Approximation
1
Chapter 5 Linear L Approximation
1
5.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.1.1 Characterization of the L solution. . . . . . . . . . . . . . . 116
1
5.2 Linear programming formulation of the problem. . . . . . . . . . . . 116
5.3 Description of the algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.4 The dual simplex method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.5 Modification to the algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.6 Occurrence of degeneracy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.7 A significant property of the L approximation . . . . . . . . . . . . . 129
1
5.8 Triangular decomposition of the basis matrix. . . . . . . . . . . . . . . 130
5.9 Arithmetic operations count . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.10 Numerical results and comments. . . . . . . . . . . . . . . . . . . . . . . . . 134
5.11 DR_L1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.12 LA_L1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5.13 DR_Lone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5.14 LA_Lone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Chapter 6 One-Sided L Approximation
1
6.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.1.1 Applications of the algorithm. . . . . . . . . . . . . . . . . . . 186
6.1.2 Characterization and uniqueness. . . . . . . . . . . . . . . . . 186
6.2 A special problem of a general constrained one . . . . . . . . . . . . . 186
6.3 Linear programming formulation of the problem. . . . . . . . . . . . 187
6.4 Description of the algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
6.4.1 Obtaining an initial basic feasible solution. . . . . . . . . 188
6.4.2 One-sided L solution from above . . . . . . . . . . . . . . . 191
1
© 2008 by Taylor & Francis Group, LLC
Contents ix
6.4.3 The interpolation property . . . . . . . . . . . . . . . . . . . . . 191
6.5 Numerical results and comments. . . . . . . . . . . . . . . . . . . . . . . . . 191
6.6 DR_Loneside. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
6.7 LA_Loneside. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
Chapter 7 L Approximation with Bounded Variables
1
7.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.1.1 Linear L approximation with non-negative
1
parameters (NNL1). . . . . . . . . . . . . . . . . . . . . . . . . . . 216
7.2 A special problem of a general constrained one . . . . . . . . . . . . . 216
7.3 Linear programming formulation of the problem. . . . . . . . . . . . 217
7.3.1 Properties of the matrix of constraints . . . . . . . . . . . . 220
7.4 Description of the algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.5 Numerical results and comments. . . . . . . . . . . . . . . . . . . . . . . . . 222
7.6 DR_Lonebv. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
7.7 LA_Lonebv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Chapter 8 L Polygonal Approximation of Plane Curves
1
8.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
8.1.1 Two basic issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
8.1.2 Approaches for polygonal approximation . . . . . . . . . 246
8.1.3 Other unique approaches. . . . . . . . . . . . . . . . . . . . . . . 248
8.1.4 Criteria by which error norm is chosen. . . . . . . . . . . . 249
8.1.5 Direction of error measure . . . . . . . . . . . . . . . . . . . . . 249
8.1.6 Comparison and stability of polygonal
approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
8.1.7 Applications of the algorithm. . . . . . . . . . . . . . . . . . . 251
8.2 The L approximation problem. . . . . . . . . . . . . . . . . . . . . . . . . . 251
1
8.3 Description of the algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
8.4 Linear programming technique. . . . . . . . . . . . . . . . . . . . . . . . . . 253
8.4.1 The algorithm using linear programming. . . . . . . . . . 255
8.5 Numerical results and comments. . . . . . . . . . . . . . . . . . . . . . . . . 255
8.6 DR_L1pol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
8.7 LA_L1pol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
© 2008 by Taylor & Francis Group, LLC