Numerical algorithms : methods for computer vision, machine learning, and graphics

392 Pages·2015·8.51 MB·English

by  Solomon

COMPUTER SCIENCE
Numerical
Numerical Algorithms
WITH VITALSOURCE®
EBOOK
Methods for Computer Vision, Machine Learning, and Graphics
N
Algorithms
“This book covers an impressive array of topics, many of which are paired with a real-world
u
application. Its conversational style and relatively few theorem-proofs make it well suited for
computer science students as well as professionals looking for a refresher.” m
—Dianne Hansford, FarinHansford.com
e
Numerical Algorithms: Methods for Computer Vision, Machine Learning, and Graphics Methods for Computer V ision,
presents a new approach to numerical analysis for modern computer scientists. Using ex- r
amples from a broad base of computational tasks, including data processing, computation- i
Machine Learning, and Graphics

al photography, and animation, the book introduces numerical modeling and algorithmic c
design from a practical standpoint and provides insight into the theoretical tools needed to
a
support these skills.
l
The book covers a wide range of topics—from numerical linear algebra to optimization and

differential equations—focusing on real-world motivation and unifying themes. It incorpo- A
rates cases from computer science research and practice, accompanied by highlights from
in-depth literature on each subtopic. Comprehensive end-of-chapter exercises encourage
l
critical thinking and build your intuition while introducing extensions of the basic material. g
Features
o
• Introduces themes common to nearly all classes of numerical algorithms
• Covers algorithms for solving linear and nonlinear problems, including popular tech- r
Justin Solomon
niques recently introduced in the research community i
• Includes comprehensive end-of-chapter exercises that push you to derive, extend, and t
analyze numerical algorithms h
m
s
• Access online or download to your smartphone, tablet or PC/Mac
• Search the full text of this and other titles you own
• Make and share notes and highlights
• Copy and paste text and figures for use in your own documents
• Customize your view by changing font size and layout Solomon
K23847
ISBN: 978-1-4822-5188-3
6000 Broken Sound Parkway, NW 90000
Suite 300, Boca Raton, FL 33487
711 Third Avenue
an informa business New York, NY 10017 9 781482 251883
2 Park Square, Milton Park
www.crcpress.com
www.crcpress.com Abingdon, Oxon OX14 4RN, UK AN A K PETERS BOOK
Accessing the E-book edition
Using the VitalSource® ebook DOWNLOAD AND READ OFFLINE
Access to the VitalBookTM ebook accompanying this book is To use your ebook offline, download BookShelf to your PC,
via VitalSource® Bookshelf – an ebook reader which allows Mac, iOS device, Android device or Kindle Fire, and log in to
you to make and share notes and highlights on your ebooks your Bookshelf account to access your ebook:
and search across all of the ebooks that you hold on your
On your PC/Mac
VitalSource Bookshelf. You can access the ebook online or
Go to http://bookshelf.vitalsource.com/ and follow the
offline on your smartphone, tablet or PC/Mac and your notes
instructions to download the free VitalSource Bookshelf
and highlights will automatically stay in sync no matter where
app to your PC or Mac and log into your Bookshelf account.
you make them.
On your iPhone/iPod Touch/iPad
1. Create a VitalSource Bookshelf account at Download the free VitalSource Bookshelf App available
https://online.vitalsource.com/user/new or log into via the iTunes App Store and log into your Bookshelf
your existing account if you already have one. account. You can find more information at https://support.
vitalsource.com/hc/en-us/categories/200134217-
2. Redeem the code provided in the panel below
Bookshelf-for-iOS
to get online access to the ebook. Log in to
Bookshelf and click the Account menu at the top right On your Android™ smartphone or tablet
of the screen. Select Redeem and enter the redemption Download the free VitalSource Bookshelf App available
code shown on the scratch-off panel below in the Code via Google Play and log into your Bookshelf account. You can
To Redeem box. Press Redeem. Once the code has find more information at https://support.vitalsource.com/
been redeemed your ebook will download and appear in hc/en-us/categories/200139976-Bookshelf-for-Android-
your library. and-Kindle-Fire
On your Kindle Fire
Download the free VitalSource Bookshelf App available
from Amazon and log into your Bookshelf account. You can
find more information at https://support.vitalsource.com/
hc/en-us/categories/200139976-Bookshelf-for-Android-
and-Kindle-Fire
N.B. The code in the scratch-off panel can only be used once.
When you have created a Bookshelf account and redeemed
the code you will be able to access the ebook online or offline
on your smartphone, tablet or PC/Mac.
SUPPORT
If you have any questions about downloading Bookshelf,
creating your account, or accessing and using your ebook
edition, please visit http://support.vitalsource.com/
Numerical
Algorithms

Numerical
Algorithms
Methods for Computer Vision,
Machine Learning, and Graphics
Justin Solomon
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Group, an informa business
AN A K PETERS BOOK
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2015 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20150105
International Standard Book Number-13: 978-1-4822-5189-0 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
In memory of Clifford Nass
(1958–2013)

Contents
PREFACE xv
ACKNOWLEDGMENTS xix
Section I Preliminaries
Chapter 1(cid:4) Mathematics Review 3
1.1 PRELIMINARIES:NUMBERSANDSETS 3
1.2 VECTORSPACES 4
1.2.1 Defining Vector Spaces 4
1.2.2 Span, Linear Independence, and Bases 5
1.2.3 Our Focus: Rn 7
1.3 LINEARITY 9
1.3.1 Matrices 10
1.3.2 Scalars, Vectors, and Matrices 12
1.3.3 Matrix Storage and Multiplication Methods 13
1.3.4 Model Problem: A(cid:126)x =(cid:126)b 14
1.4 NON-LINEARITY:DIFFERENTIALCALCULUS 15
1.4.1 Differentiation in One Variable 16
1.4.2 Differentiation in Multiple Variables 17
1.4.3 Optimization 20
1.5 EXERCISES 23
Chapter 2(cid:4) Numerics and Error Analysis 27
2.1 STORINGNUMBERSWITHFRACTIONALPARTS 27
2.1.1 Fixed-Point Representations 28
2.1.2 Floating-Point Representations 29
2.1.3 More Exotic Options 31
2.2 UNDERSTANDINGERROR 32
2.2.1 Classifying Error 33
2.2.2 Conditioning, Stability, and Accuracy 35
2.3 PRACTICALASPECTS 36
2.3.1 Computing Vector Norms 37
vii
viii (cid:4) Contents
2.3.2 Larger-Scale Example: Summation 38
2.4 EXERCISES 39
Section II Linear Algebra
Chapter 3(cid:4) Linear Systems and the LU Decomposition 47
3.1 SOLVABILITYOFLINEARSYSTEMS 47
3.2 AD-HOCSOLUTIONSTRATEGIES 49
3.3 ENCODINGROWOPERATIONS 51
3.3.1 Permutation 51
3.3.2 Row Scaling 52
3.3.3 Elimination 52
3.4 GAUSSIANELIMINATION 54
3.4.1 Forward-Substitution 55
3.4.2 Back-Substitution 56
3.4.3 Analysis of Gaussian Elimination 56
3.5 LUFACTORIZATION 58
3.5.1 Constructing the Factorization 59
3.5.2 Using the Factorization 60
3.5.3 Implementing LU 61
3.6 EXERCISES 61
Chapter 4(cid:4) Designing and Analyzing Linear Systems 65
4.1 SOLUTIONOFSQUARESYSTEMS 65
4.1.1 Regression 66
4.1.2 Least-Squares 68
4.1.3 Tikhonov Regularization 70
4.1.4 Image Alignment 71
4.1.5 Deconvolution 73
4.1.6 Harmonic Parameterization 74
4.2 SPECIALPROPERTIESOFLINEARSYSTEMS 75
4.2.1 Positive Definite Matrices and the Cholesky Factorization 75
4.2.2 Sparsity 79
4.2.3 Additional Special Structures 80
4.3 SENSITIVITYANALYSIS 81
4.3.1 Matrix and Vector Norms 81
4.3.2 Condition Numbers 84
4.4 EXERCISES 86