Optimization Techniques: An Introduction

511 Pages·1981·9.57 MB·English

by  L. R. Foulds (auth.)

Undergraduate Texts in Mathematics
Editors
F. W. Gehring
P. R. Halmos
Advisory Board
C. DePrima
I. Herstein
L. R. Foulds
Optimization Techniques
An Introduction
With 72 Illustrations
Springer-Verlag
New York Heidelberg Berlin
L. R. Foulds
Department of Economics
University of Canterbury
Christchurch 1
New Zealand
Editorial Board
P. R. Halmos F. W. Gehring
Department of Mathematics Department of Mathematics
Indiana University University of Michigan
Bloomington, IN 47401 Ann Arbor, MI 48109
U.S.A. U.S.A.
AMS Classification: 49-01, 90-01
Library of Congress Cataloging in Publication Data
Foulds, L. R., 1948-
Optimization techniques.
Bibliography: p.
Includes index.
1. Mathematical optimization. 2. Programming
(Mathematics) I. Title.
QA402.5.F68 519 81-5642
AACR2
© 1981 by Springer-Verlag New York Inc.
Softcover reprint of the hardcover 1st edition 1981
All rights reserved. No part of this book may be translated or reproduced in any form
without written permission from Springer-Verlag, 175 Fifth Avenue, New York,
New York 10010, U.S.A.
9 8 765 432 1
ISBN-13:978-1-4613-9460-0 e-ISBN-13:978-1-4613-9458-7
DOl: 10.1007/978-1-4613-9458-7
This book is dedicated to the memory ofm y father
Richard Seddon Foulds
Contents
Preface
IX
Plan of the Book
XI
Chapter I
Introduction
1.1 Motivation for Studying Optimization, 1.2 The Scope of Opti
mization, 1.3 Optimization as a Branch of Mathematics, 1.4 The
History of Optimization, 1.5 Basic Concepts of Optimization
Chapter 2
Linear Programming 10
2.1 Introduction, 2.2 A Simple L.P. Problem, 2.3 The General L.P.
Problem, 2.4 The Basic Concepts of Linear Programming, 2.5 The
Simplex Algorithm, 2.6 Duality and Postoptimal Analysis, 2.7
Special Linear Program, 2.8 Exercises
Chapter 3
Advanced Linear Programming Topics 106
3.1 Efficient Computational Techniques for Large L.P. Problems,
3.2 The Revised Simplex Method, 3.3 The Dual Simplex Method,
3.4 The Primal-Dual Algorithm, 3.5 Dantzig-Wolfe Decomposi
tion, 3.6 Parametric Programming, 3.7 Exercises
vii
Vlll Contents
Chapter 4
Integer Programming 150
4.1 A Simple Integer Programming Problem, 4.2 Combinatorial
Optimization, 4.3 Enumerative Techniques, 4.4 Cutting Plane
Methods, 4.5 Applications of Integer Programming, 4.6 Exercises
Chapter 5
Network Analysis 187
5.1 The Importance of Network Models, 5.2 An Introduction to
Graph Theory, 5.3 The Shortest Path Problem, 5.4 The Minimal
Spanning Tree Problem, 5.5 Flow Networks, 5.6 Critical Path
Scheduling, 5.7 Exercises
Chapter 6
Dynamic Programming 235
6.1 Introduction, 6.2 A Simple D.P. Problem, 6.3 Basic D.P. Struc
ture, 6.4 Multiplicative and More General Recursive Relationships,
6.5 Continuous State Problems, 6.6 The Direction of Computations,
6.7 Tabular Form, 6.8 Multi-state Variable Problems and the Limi
tations of D.P., 6.9 Exercises
Chapter 7
Classical Optimization 257
7.1 Introduction, 7.2 Optimization of Functions of One Variable,
7.3 Optimization of Unconstrained Functions of Several Variables,
7.4 Optimization of Constrained Functions of Several Variables,
7.5 The Calculus of Variations, 7.6 Exercises
Chapter 8
Nonlinear Programming 310
8.1 Introduction, 8.2 Unconstrained Optimization, 8.3 Constrained
Optimization, 8.4 Exercises
Chapter 9
Appendix 370
9.1 Linear Algebra, 9.2 Basic Calculus, 9.3 Further Reading
References 395
Solutions to Selected Exercises 400
Index 499
Preface
Optimization is the process by which the optimal solution to a problem, or
optimum, is produced. The word optimum has come from the Latin word
optimus, meaning best. And since the beginning of his existence Man has
strived for that which is best. There has been a host of contributions, from
Archimedes to the present day, scattered across many disciplines. Many of
the earlier ideas, although interesting from a theoretical point of view, were
originally of little practical use, as they involved a daunting amount of com
putational effort. Now modern computers perform calculations, whose time
was once estimated in man-years, in the figurative blink of an eye. Thus it has
been worthwhile to resurrect many of these earlier methods. The advent of
the computer has helped bring about the unification of optimization theory
into a rapidly growing branch of applied mathematics. The major objective
of this book is to provide an introduction to the main optimization tech
niques which are at present in use. It has been written for final year undergrad
uates or first year graduates studying mathematics, engineering, business, or
the physical or social sciences. The book does not assume much mathemati
cal knowledge. It has an appendix containing the necessary linear algebra
and basic calculus, making it virtually self-contained.
This text evolved out of the experience of teaching the material to finishing
undergraduates and beginning graduates. A feature of the book is that it
adopts the sound pedagogical principle that an instructor should proceed
from the known to the unknown. Hence many of the ideas in the earlier
chapter& are introduced by means of a concrete numerical example to which
the student can readily relate. This is followed by generalization to the
underlying theory. The courses on which the book is based usually have a
significant number of students of Business and Engineering. The interests
ix
x Preface
of these people have been taken into account in the development of the
courses and hence in the writing of this book. Hence many of its arguments
are intuitive rather than rigorous. Indeed plausibility and clarity have been
given precedence before rigour for the sake of itself.
Chapter I contains a brief historical account and introduces the basic
terminology and concepts common to all the theory of optimization. Chap
ters 2 and 3 are concerned with linear programming and complications of
the basic model. Chapter 2 on the simplex method, duality, and sensitivity
analysis can be covered in an undergraduate course. However some of the
topics in Chapter 3 such as considerations of efficiency and parametric pro
gramming, may be best left to graduate level. Chapter 4 deals with only the
basic strategies of integer linear programming. It is of course dependent on
Chapter 2. It does contain a number of formulations of applications of inte
ger programming. Some of this material has never appeared before in book
form. Chapter 5 is on network analysis and contains a section on using net
works to analyze some practical problems.
Chapter 6 introduces dynamic programming. It is beyond the scope of
this book to provide a detailed account of this vast topic. Hence techniques
suitable for only deterministic, serial systems are presented. The interested
reader is referred to the extensive literature. Chapter 7 serves as an introduc
tion to Chapter 8, which is on nonlinear programming. It presents some of
the classical techniques: Jacobian and Lagrangian methods together with the
Kuhn-Tucker conditions. The ideas in this chapter are used in devising the
more computationally efficient strategies of Chapter 8.
This text contains enough material for one semester at the undergraduate
level and one more at the graduate level. The first course could contain Chap
ters 1, 2, the first half of Chapter 3, and parts of Chapter 4 and Chapter 5.
The remainder can be covered in the second course. A plan outlining this
follows.
The book contains a large number of exercises. Students are strongly en
couraged to attempt them. One cannot come to grips with the concepts by
solely looking at the work of others. Mathematics is not a spectator sport!
The author is grateful for this opportunity to express his thanks for the
support of his employers, the University of Canterbury, which he enjoyed
while finishing this book. He is also thankful for the faith and encouragement
of his wife, Maureen, without which it would never have been written. He is
also grateful to a number of friends including David Robinson, Hans
Daellenbach, Michael Carter, Ian Coope and Susan Byrne, who read parts of
the manuscript and made valuable suggestions. A vote of thanks should also
go to his student, Trevor Kearney, who read the entire manuscript and dis
covered an embarrassing number of errors.
Christchurch, New Zealand L. R. Foulds
November 1980
Plan of the Book
Undergraduate
Course
Graduate
Course
xi
Chapter 1
Introduction
1.1 Motivation for Studying Optimization
There exist an enormous variety of activities in the everyday world which
can usefully be described as systems, from actual physical systems such as
chemical processing plants to theoretical entities such as economic models.
The efficient operation of these systems often requires an attempt at the
optimization of various indices which measure the performance ofthe system.
Sometimes these indices are quantified and represented as algebraic vari
ables. Then values for these variables must be found which maximize the
gain or profit of the system and minimize the waste or loss. The variables
are assumed to be dependent upon a number of factors. Some of these
factors are often under the control, or partial control, of the analyst respon
sible for the performance of the system.
The process of attempting to manage the limited resources of a system
can usually be divided into six phases: (i) mathematical analysis of the
system; (ii) construction of a mathematical model which reflects the impor
tant aspects of the system; (iii) validation of the model; (iv) manipulation
of the model to produce a satisfactory, if not optimal, solution to the model;
(v) implementation of the solution selected; and (vi) the introduction of a
strategy which monitors the performance ofthe system after implementation.
It is with the fourth phase, the manipulation of the model, that the theory
of optimization is concerned. The other phases are very important in the
management of any system and will probably require greater total effort
than the optimization phase. However, in the presentation of optimization
theory here it will be assumed that the other phases have been, or will be,
carried out. Because the theory of optimization provides this link in the
chain of systems management it is an important body of mathematical
knowledge.