An Introduction to Optimization

642 Pages·2013·19.81 MB·English

by  Edwin K. P. Chong

AN INTRODUCTION
TO OPTIMIZATION
WILEY SERIES IN
DISCRETE MATHEMATICS AND OPTIMIZATION
A complete list of titles in this series appears at the end of this volume.
AN INTRODUCTION
TO OPTIMIZATION
Fourth Edition
Edwin K. P. Chong
Colorado State University
Stanislaw H. 2ak
Purdue University
®WILEY
A JOHN WILEY & SONS, INC., PUBLICATION
Copyright © 2013 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as
permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior
written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to
the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax
[phone], or on the web at www.copyright.com. Requests to the Publisher for permission should
be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ
07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or
completeness of the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and strategies contained herein may not be
suitable for your situation. You should consult with a professional where appropriate. Neither the
publisher nor author shall be liable for any loss of profit or any other commercial damages, including
but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our
Customer Care Department within the United States at [phone], outside the United States at
[phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic formats. For more information about Wiley products, visit our web site at
www.wiley.com.
Library of Congress Cataloging-in-Publication Data
Chong, Edwin Kah Pin.
An introduction to optimization / Edwin K. P. Chong, Colorado State University, Stanislaw H. Zak,
Purdue University. — Fourth edition,
pages cm
Summary: "The purpose of the book is to give the reader a working knowledge of optimization theory
and methods" — Provided by publisher.
Includes bibliographical references and index.
ISBN 978-1-118-27901-4 (hardback)
1. Mathematical optimization. I. Zak, Stanislaw H. II. Title.
QA402.5.C476 2012
519.6—dc23 2012031772
Printed in the United States of America.
10 9 8 7 6 5 4 3 21
To my wife, Yat-Yee,
and to my parents, Paul
and Julienne Chong.
Edwin K. P. Chong
To JMJ; my wife,
Mary Ann; and my
parents, Janina and
Konstanty Zak.
Stanislaw H. Zak

CONTENTS
Preface xiii
PART I MATHEMATICAL REVIEW
1 Methods of Proof and Some Notation 3
1.1 Methods of Proof 3
1.2 Notation 5
Exercises 6
2 Vector Spaces and Matrices 7
2.1 Vector and Matrix 7
2.2 Rank of a Matrix 13
2.3 Linear Equations 17
2.4 Inner Products and Norms 19
Exercises 22
3 Transformations 25
3.1 Linear Transformations 25
viii CONTENTS
3.2 Eigenvalues and Eigenvectors 26
3.3 Orthogonal Projections 29
3.4 Quadratic Forms 31
3.5 Matrix Norms 35
Exercises 40
Concepts from Geometry 45
4.1 Line Segments 45
4.2 Hyperplanes and Linear Varieties 46
4.3 Convex Sets 48
4.4 Neighborhoods 50
4.5 Poly topes and Polyhedra 52
Exercises 53
Elements of Calculus 55
5.1 Sequences and Limits 55
5.2 Differentiability 62
5.3 The Derivative Matrix 63
5.4 Differentiation Rules 67
5.5 Level Sets and Gradients 68
5.6 Taylor Series 72
Exercises 77
PART II UNCONSTRAINED OPTIMIZATION
Basics of Set-Constrained and Unconstrained Optimization 81
6.1 Introduction 81
6.2 Conditions for Local Minimizers 83
Exercises 93
One-Dimensional Search Methods 103
7.1 Introduction 103
7.2 Golden Section Search 104
7.3 Fibonacci Method 108
7.4 Bisection Method 116
7.5 Newton's Method 116
7.6 Secant Method 120
7.7 Bracketing 123