Table Of ContentA n I n t r o d u c t i o n t o t h e
Analysis of Algorithms
3rd Edition
10875_9789813235908_TP.indd 1 16/1/18 11:29 AM
b2530 International Strategic Relations and China’s National Security: World at the Crossroads
TTTThhhhiiiissss ppppaaaaggggeeee iiiinnnntttteeeennnnttttiiiioooonnnnaaaallllllllyyyy lllleeeefffftttt bbbbllllaaaannnnkkkk
b2530_FM.indd 6 01-Sep-16 11:03:06 AM
A n I n t r o d u c t i o n t o t h e
Analysis of Algorithms
3rd Edition
Michael Soltys
California State University Channel Islands, USA
World Scientific
NEW JERSEY • LONDON • SINGAPORE • BEIJING • SHANGHAI • HONG KONG • TAIPEI • CHENNAI • TOKYO
10875_9789813235908_TP.indd 2 16/1/18 11:29 AM
Published by
World Scientific Publishing Co. Pte. Ltd.
5 Toh Tuck Link, Singapore 596224
USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601
UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE
British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library.
AN INTRODUCTION TO THE ANALYSIS OF ALGORITHMS
Third Edition
Copyright © 2018 by World Scientific Publishing Co. Pte. Ltd.
All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means,
electronic or mechanical, including photocopying, recording or any information storage and retrieval
system now known or to be invented, without written permission from the publisher.
For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance
Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy
is not required from the publisher.
ISBN 978-981-3235-90-8
For any available supplementary material, please visit
http://www.worldscientific.com/worldscibooks/10.1142/10875#t=suppl
Desk Editor: Amanda Yun
Printed in Singapore
Amanda - 10875 - An Introduction to the Analysis of Algorithms.indd 1 16-01-18 9:53:35 AM
October27,2017 10:59 BK032-AnIntroductiontotheAnalysisofAlgorithms(3rdEdition) soltys˙alg pagev
To my family
v
b2530 International Strategic Relations and China’s National Security: World at the Crossroads
TTTThhhhiiiissss ppppaaaaggggeeee iiiinnnntttteeeennnnttttiiiioooonnnnaaaallllllllyyyy lllleeeefffftttt bbbbllllaaaannnnkkkk
b2530_FM.indd 6 01-Sep-16 11:03:06 AM
October27,2017 10:59 BK032-AnIntroductiontotheAnalysisofAlgorithms(3rdEdition) soltys˙alg pagevii
Preface
If he had only learnt a little
less, how infinitely better he
might havetaught much more!
Charles Dickens[Dickens
(1854)], pg. 7
This book is a short introduction to the analysis of algorithms, from the
point of view of proving algorithm correctness. The quote above refers to
Mr. M‘Choakumchild, a caricature of a teacher in Charles Dickens’ Hard
Times, who chokesthe minds ofhis pupils with too muchinformation. We
will avoid M‘Choakumchild’s mistake, and make a virtue out of brevity.
Our theme is the following: how do we argue mathematically, without
a burden of excessive formalism, that a given algorithm does what it is
supposed to do? And why is this important? In the words of C.A.R.
Hoare:
Asfarasthefundamentalscienceisconcerned,westillcertainly
do not know how to prove programs correct. We need a lot of
steadyprogressinthisarea,whichonecanforesee, andalotof
breakthroughswherepeoplesuddenlyfindthere’sasimpleway
todosomethingthateverybodyhithertohasthoughttobefar
too difficult1.
Software engineers know many examples of things going terribly wrong
becauseofprogramerrors;theirparticularfavoritesarethefollowingtwo2.
The blackout in the American North-East during the summer of 2003 was
due to a software bug in an energy management system; an alarm that
1FromAn Interviewwith C.A.R. Hoare,in[Shustek(2009)].
2These two examples come from [van Vliet (2000)], where many more instances of
spectacular failuresmaybefound.
vii
October27,2017 10:59 BK032-AnIntroductiontotheAnalysisofAlgorithms(3rdEdition) soltys˙alg pageviii
viii Anintroduction to the analysis of algorithms
shouldhavebeen triggeredneverwentoff,leadingto achainofeventsthat
climaxed in a cascading blackout. The Ariane 5, flight 501, the maiden
flight of the rocket in June 4, 1996, ended with an explosion 40 seconds
into the flight; this $500 million loss was caused by an overflow in the
conversion from a 64-bit floating point number to a 16-bit signed integer.
WhenRichardA.Clarke,the formerNationalCoordinatorforSecurity,
asked Ed Amoroso, head of AT&T Network Security, what is to be done
about the vulnerabilities in the USA cyber-infrastructure, Amoroso said:
Software is most of the problem. We have to write software
which has many fewer errors and which is more secure3.
Similarly, Fred D. Taylor, Jr., a Lt. Colonel in the United States Air Force
and a National Security Fellow at the Harvard Kennedy School, wrote:
The extensive reliance on software has created new and ex-
panding opportunities. Along with these opportunities, there
are new vulnerabilities putting the global infrastructure and
our national security at risk. The ubiquitous nature of the In-
ternet and the fact that it is serviced by common protocols
andprocesseshasallowed anyonewiththeknowledgetocreate
software to engage in world-wide activities. However, for most
software developers there is no incentive to produce software
that is more secure4.
Softwaresecurityfallsnaturallyundertheumbrellaofsoftwarecorrectness.
Whilethegoalofprogramcorrectnessiselusive,wecandevelopmethods
and techniques for reducing errors. The aim of this book is modest: we
want to present an introduction to the analysis of algorithms—the “ideas”
behind programs, and show how to prove their correctness.
The algorithm may be correct, but the implementation itself might be
flawed. Some syntactical errors in the program implementation may be
uncoveredbyacompilerortranslator—whichinturncouldalsobebuggy—
buttheremightbeotherhiddenerrors. Thehardwareitselfmightbefaulty;
the libraries on which the program relies at run time might be unreliable,
etc. It is the main task of a programmer to write code that works given
such a delicate, error prone, environment. Finally, the algorithmic content
of a piece ofsoftwaremightbe verysmall; the majorityofthe lines of code
could be the “menial” task of interface programming. Thus, the ability to
argue correctly about the soundness of an algorithm is only one of many
facetsofthetaskathand, yetanimportantone,ifonlyforthepedagogical
reason of learning to argue rigorously about algorithms.
3Seepage272in[ClarkeandKnake(2011)].
4HarvardLawSchool NationalSecurityJournal,[FredD.Taylor(2011)].
October27,2017 10:59 BK032-AnIntroductiontotheAnalysisofAlgorithms(3rdEdition) soltys˙alg pageix
Preface ix
We begin this book with a chapter of preliminaries, containing the key
ideasofinductionandinvariance,andtheframeworkofpre/post-conditions
and loop invariants. We also prove the correctness of some classical algo-
rithms, such as the integer division algorithm, and Euclid’s procedure for
computing the greatest common divisor of two numbers.
We present three standard algorithm design techniques in eponymous
chapters: greedy algorithms, dynamic programming and the divide and
conquerparadigm. Weareconcernedwithcorrectnessofalgorithms,rather
than, say, efficiency or the underlying data structures. For example, in the
chapteronthegreedyparadigmweexploreindepththeideaofapromising
partial solution, a powerful technique for proving the correctness of greedy
algorithms. We include online algorithms and competitive analysis, as well
as randomized algorithms with a section on cryptography.
Algorithms solve problems, and many of the problems in this book fall
under the category of optimization problems, whether cost minimization,
such as Kruskal’salgorithmfor computing minimum cost spanning trees—
section 2.1, or profit maximization, such as selecting the most profitable
subset of activities—section 4.4.
The book is sprinkled with problems. Most problems are theoretical,
but many require the implementation of an algorithm; we suggest the
Python 3 programming language for such problems. The reader is ex-
pected to learn Python on their own; see for example, [Dierbach (2013)]
or [Downey (2015)]5. One of the advantages of Python is that it is easy
to start writing small snippets of code that work—and most of the coding
in this book falls into the “small snippet” category. The solutions to most
problems are included in the “Answers to selected problems” at the end of
each chapter. The solutions to most of the programming exercises will be
available for download from the author’s web page6.
The intended audience of this book are graduate and undergraduate
students in Computer Science and Mathematics. The presentation is
self-contained: the first chapter introduces the aforementioned ideas of
pre/post-conditions, loop invariants and termination. The last chapter,
Chapter 9, Mathematical Foundations, contains the necessary background
in Induction, Invariance Principle, Number Theory, Relations and Logic.
The reader unfamiliar with discrete mathematics is encouraged to start
with Chapter 9 and do all the problems therein.
5ThePDFsofearlierversions,upto2.0.17atthetimeofwriting,areavailableforfree
downloadfromGreenTeaPress,http://greenteapress.com/wp/think-python.
6http://www.msoltys.com.
