Maxima and Minima with Applications: Practical Optimization and Duality

289 Pages·1998·4.67 MB·English

by  Wilfred Kaplan(auth.)

Maxima and Minima
with Applications
WILEY-INTERSCIENCE
SERIES IN DISCRETE MATHEMATICS AND OPTIMIZATION
ADVISORY EDITORS
RONALD L. GRAHAM
AT & T Laboratories, Flor ham Park, New Jersey, U.S.A.
JAN KAREL LENSTRA
Department of Mathematics and Computer Science,
Eindhoven University of Technology, Eindhoven, The Netherlands
ROBERT E. TARJAN
Princeton University, New Jersey, and
NEC Research Institute, Princeton, New Jersey, U.S.A.
A complete list of titles in this series appears at the end of this volume.
Maxima and Minima
with Applications
Practical Optimization
and Duality
WILFRED KAPLAN
Emeritus Professor of Mathematics
University of Michigan
A Wiley-Interscience Publication
JOHN WILEY & SONS, INC.
New York · Chichester * Weinheim · Brisbane · Singapore * Toronto
This book is printed on acid-free paper.©
Copyright © 1999 by John Wiley & Sons, Inc. All rights reserved.
Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval
system or transmitted in any form or by any means, electronic,
mechanical, photocopying, recording, scanning or otherwise, except as
permitted under Sections 107 or 108 of the 1976 United States Copyright
Act, without either the prior written permission of the Publisher, or
authorization through payment of the appropriate per-copy fee to the
Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA
01923, [phone], fax [phone]. Requests to the Publisher for
permission should be addressed to the Permissions Department, John
Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012,
[phone], fax [phone], E-Mail: [email protected].
Library of Congress Cataloging-in-Publication Data:
Kaplan, Wilfred, 1915
Maxima and minima with applications: practical optimization and
duality / Wilfred Kaplan.
p. cm. (Wiley-Interscience series in discrete mathematics
and optimization)
"A Wiley-Interscience publication."
Includes bibliographical references and index.
ISBN 0-471-25289-1 (alk. paper)
I. Maxima and minima. 2. Mathematical optimization. I. Title.
II. Series.
QA306.K36 1999 98-7318
511'.66—dc21
Printed in the United States of America.
10 9 8 7 6 5 4 3 21
Contents
Preface ix
1 Maxima and Minima in Analytic Geometry 1
1.1 Maxima and Minima; Case of Functions of One Variable 1
Problems 1.1-1.5 5
1.2 Convexity 6
1.3 Convexity and Maxima and Minima 9
Problems 1.6-1.16 12
1.4 Problems in Two Dimensions 15
Problems 1.17-1.27 23
1.5 Some Geometric Extremum Problems 26
Problems 1.28-1.36 30
1.6 Geometry of n-Dimensional Space 31
1.7 Convex Functions of n Variables 36
1.8 Quadratic Forms 38
Problems 1.37-1.55 41
1.9 Convexity and Extrema, Level Sets and Sublevel Sets 45
Problems 1.56-1.63 49
1.10 Stability 50
1.11 Global Asymptotic Stability, Application to Finding Minimizer 56
Problems 1.64-1.73 58
1.12 Extrema of Functions on Unbounded Closed Sets 60
1.13 Shortest Distance from a Linear Variety 63
Problems 1.74-1.84 66
1.14 Other Inner Products and Norms in ΊΖη 68
1.15 More on Minimum Problems for Quadratic Functions 71
Problems 1.85-1.93 74
1.16 Physical Applications 76
Problems 1.94-1.96 78
1.17 Best Approximation by Polynomials 79
Problems 1.97-1.105 82
References 83
vi
2 Side Conditions 85
2.1 Review of Vector Calculus 85
Problems 2.1-2.13 92
2.2 Local Maxima and Minima, Side Conditions 94
Problems 2.14-2.21 101
2.3 Second-Derivative Test 102
Problems 2.22-2.26 105
2.4 Gradient Method for Finding Critical Points 106
Problems 2.27-2.28 108
2.5 Applications 109
Problems 2.29-2.33 115
2.6 Karush-Kuhn-Tucker Conditions 116
Problems 2.34-2.37 121
2.7 SufScient Conditions for the Mathematical Programming
Problem 122
2.8 Proof of the Karush-Kuhn-Tucker Conditions 127
Problems 2.38-2.49 130
References 134
3 Optimization 135
3.1 Convexity 135
Problems 3.1-3.17 139
3.2 Mathematical Programming, Duality 142
3.3 Unconstrained Quadratic Optimization 144
Problems 3.18-3.28 146
3.4 Constrained Quadratic Optimization in 1Zn 148
3.5 QP with Inequality Constraints, QP Algorithm 153
Problems 3.29-3.38 156
3.6 Linear Programming 15 8
3.7 Simplex Algorithm 166
Problems 3.39-3.55 173
3.8 LP with Bounded Variables 175
Problems 3.56-3.62 181
3.9 Convex Functions and Convex Programming 182
Problems 3.63-3.68 188
3.10 The Fermat-Weber Problem and a Dual Problem 188
Problems 3.69-3.76 193
3.11 A Duality Relation in Higher Dimensions 194
Problems 3.77-3.84 200
References 201
4 Fenchel-Rockafellar Duality Theory 203
4.1 Generalized Directional Derivative 203
Contents Vil
Problems 4.1-4.5 205
4.2 Local Structure of the Boundary of a Convex Set 205
Problems 4.6-4.8 209
4.3 Supporting Hyperplane, Separating Hyperplane 209
Problems 4.9-4.15 216
4.4 New Definition of Convex Function, Epigraph, Hypograph 217
Problems 4.16-4.17 221
4.5 Conjugate of Convex and Concave Functions 221
Problems 4.18-4.24 227
4.6 Fenchel Duality Theorem 228
Problems 4.25-4.32 231
4.7 Rockafellar Duality Theorem 232
4.8 Proof of Lemma C 238
Problems 4.33-4.45 240
4.9 Norms, Dual Norms, Minkowski Norms 242
Problems 4.46-4.61 250
4.10 Generalized Fermat-Weber Problem 252
4.11 Application to Facility Location 254
Problems 4.62-4.74 259
References 261
Appendix: Linear Algebra 263
Answers to Selected Problems 271
Index 279
Preface
This book is intended as a text for an intermediate level course on maxima
and minima. Practical applications are very much in the background and
many illustrations are given. The mathematical background assumed is that of
calculus, preferably advanced. Some knowledge of linear algebra is required;
an Appendix summarizes the essential tools ofthat subject.
A prominent role is played by the concept of convexity. On the one hand,
many applications involve finding the minimum (maximum) of a convex
(concave) function of one or more variables; on the other hand, the convex and
concave functions have many properties that assist greatly in finding their
maxima and minima. In particular, they have properties of continuity and dif-
ferentiability that enhance their value.
The four chapters are of increasing depth and difficulty. In particular, the
first chapter requires modest mathematical knowedge, even though it provides
insight into basic concepts of maxima and minima. The last chapter makes
available profound research of Fenchel and Rockafellar in a presentation of
moderate difficulty.
Chapter 1 stresses the geometric aspect of maximum and minimum
problems. Ideas familiar in elementary calculus are reviewed and extended.
Convexity is introduced and developed to modest extent. There is much
emphasis on quadratic functions and quadratic forms, which both illustrate the
theory and are an important tool for the general problems. Level sets are intro-
duced, and their structure near extreme points is studied. They lead naturally to
ordinary differential equations as a method for computing the extreme points;
here the concepts of stability and asymptotic stability are shown to be useful.
Norms and distance functions are developed and related to minimum
problems.
Chapter 2 considers problems with side conditions: first, those of the form
of equations; then those that may also include inequalities. The Lagrange multi-
plier method is explained, and second-derivative tests are treated fully, along
with the concept of index of a critical point. Differential equations are again
shown to be a useful tool in locating critical points. The Karush-Kuhn-Tucker
necessary conditions for side conditions including inequalities are considered at
some length; some corresponding sufficient conditions are also presented.
ix
X Preface
Chapter 3 is an introduction to optimization, with emphasis on convex
problems. More convexity theory is developed. Mathematical programming
and duality are discussed and then considered in detail for linear programing
and quadratic programming. The Fermat-Weber problem and some generali-
zations are considered.
Chapter 4 has as its principal goal the development of duality theorems of
Fenchel and Rockafellar and illustration of their power. Convexity theory is
extended further: in particular, to the fundamental separation theorem and the
concept of conjugacy. Minkowski norms are defined and shown to have signifi-
cant practical applications. A very general form of the Fermat-Weber problem,
involving such norms, is considered; a dual problem is established and shown to
be applicable to general location problems.
Throughout the four chapters many exercises are provided. They illustrate
the concepts presented and, in some cases, provide significant additional theo-
retical results. Starred problems are more difficult.
The author expresses to his friend and colleague Wei H. Yang his profound
appreciation for introducing the author to the field of Optimization and
providing many ideas which are incorporated in this book. In particular, he
provided much of the exposition in Chapter 3 and the author expresses
gratitude for his permission to include this material. The author thanks his
colleague Katta G. Murty for valuable advice about finding an initial feasible
point for programming problems.
The author also expresses to John Wiley & Sons his appreciation for their
fine cooperation during the production of this book and to Aiji K. Pipho his
thanks for the effort she made to produce the many illustrations.
WILFRED KAPLAN
Ann Arbor, Michigan
February 1998
Maxima and Minima with Applications: Practical Optimization and Duality
by Wilfred Kaplan
Copyright © 1999 John Wiley & Sons, Inc.
1
Maxima and Minima in
Analytic Geometry
1.1 MAXIMA AND MINIMA; CASE OF FUNCTIONS OF ONE
VARIABLE
Throughout this book we consider functions f(x),f(x, y), and, in general, real-
valued functions of one or more real variables. One could also consider real-
valued functions defined on general sets. For example, one may have a function
defined on a set of functions; such a function occurs in elementary calculus,
namely the integral of a function:
/= [ g(x)dx. (1.10)
Ja
The value of / depends on the choice of g so / is a function defined on a set of
functions. One refers to such a function as & functional.
On occasion we consider functions defined on finite sets: for example, the
function
f(n) = -^— for n=l,2,...,10. (1.11)
nL + 1
We can give one definition of maximum and minimum to cover all these cases.
The function / defined on a set E has maximum M (finite) on E if / has the value
M for one member of the set E and / has value at most M for all members o fE;
we call a member of £ at which / has its maximum, M, a maximizer of / When /
has a maximum on E, then / must have at least one maximizer, but there may
well be more than one maximizer. It can also occur that / has no maximum at
all.
The definitions of minimum and minimizer are similar: / has minimum m
on E if / has value m for one member of E (a minimizer) and / has value at least
equal to m for all members of E. We use the word extremum to cover the two
cases of maximum and minimum; thus we say that / has an extremum at a
i