Extensions of dynamic programming for combinatorial optimization and data mining

277 Pages·2019·1.994 MB·English

by  AbouEisha H. et al.

Intelligent Systems Reference Library 146
Hassan AbouEisha · Talha Amin 
Igor Chikalov · Shahid Hussain 
Mikhail Moshkov
Extensions of Dynamic
Programming for
Combinatorial
Optimization and Data
Mining
Intelligent Systems Reference Library
Volume 146
Series editors
Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland
e-mail: [email protected]
Lakhmi C. Jain, University of Technology Sydney, NSW, Australia
University of Canberra, Canberra, Australia;
KES International, UK
e-mail: [email protected]; [email protected]
URL: http://www.kesinternational.org/organisation.php
The aim of this series is to publish a Reference Library, including novel advances
and developments in all aspects of Intelligent Systems in an easily accessible and
well structured form. The series includes reference works, handbooks, compendia,
textbooks,well-structuredmonographs,dictionaries,andencyclopedias.Itcontains
well integrated knowledge and current information in the field of Intelligent
Systems. The series covers the theory, applications, and design methods of
IntelligentSystems.Virtuallyalldisciplinessuchasengineering,computerscience,
avionics, business, e-commerce, environment, healthcare, physics and life science
areincluded.Thelistoftopicsspansalltheareasofmodernintelligentsystemssuch
as: Ambient intelligence, Computational intelligence, Social intelligence, Com-
putationalneuroscience,Artificiallife,Virtualsociety,Cognitivesystems,DNAand
immunity-based systems, e-Learning and teaching, Human-centred computing and
Machine ethics, Intelligent control, Intelligent data analysis, Knowledge-based
paradigms,Knowledgemanagement,Intelligentagents,Intelligentdecisionmaking,
Intelligent network security, Interactive entertainment, Learning paradigms,
Recommender systems, Robotics and Mechatronics including human-machine
teaming, Self-organizing and adaptive systems, Soft computing including Neural
systems,Fuzzysystems,EvolutionarycomputingandtheFusionoftheseparadigms,
Perceptionand Vision, Web intelligence and Multimedia.
More information about this series at http://www.springer.com/series/8578
Hassan AbouEisha Talha Amin
(cid:129)
Igor Chikalov Shahid Hussain
(cid:129)
Mikhail Moshkov
Extensions of Dynamic
Programming
for Combinatorial
Optimization and Data
Mining
123
Hassan AbouEisha ShahidHussain
Computer, Electrical andMathematical Computer, Electrical andMathematical
SciencesandEngineering Division SciencesandEngineering Division
KingAbdullah University of Science KingAbdullah University of Science
andTechnology andTechnology
Thuwal Thuwal
SaudiArabia SaudiArabia
TalhaAmin Mikhail Moshkov
Computer, Electrical andMathematical Computer, Electrical andMathematical
SciencesandEngineering Division SciencesandEngineering Division
KingAbdullah University of Science KingAbdullah University of Science
andTechnology andTechnology
Thuwal Thuwal
SaudiArabia SaudiArabia
Igor Chikalov
Computer, Electrical andMathematical
SciencesandEngineering Division
KingAbdullah University of Science
andTechnology
Thuwal
SaudiArabia
ISSN 1868-4394 ISSN 1868-4408 (electronic)
Intelligent Systems Reference Library
ISBN978-3-319-91838-9 ISBN978-3-319-91839-6 (eBook)
https://doi.org/10.1007/978-3-319-91839-6
LibraryofCongressControlNumber:2018941984
©SpringerInternationalPublishingAG,partofSpringerNature2019
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbytheregisteredcompanySpringerInternationalPublishingAG
partofSpringerNature
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
To our families
Preface
Dynamic programming is an efficient technique to solve optimization problems. It
is based on decomposing the initial problem into simpler ones and solving these
subproblemsbeginningfromthesimplestones.Theaimofadynamicprogramming
algorithm is to find an optimal object from a given set of objects.
Wedevelopextensionsofdynamicprogrammingwhichallowustodescribethe
setofobjectsunder consideration,toperformamulti-stageoptimizationofobjects
relativetodifferentcriteria,tocountthenumberofoptimalobjects,tofindthesetof
Pareto optimal points for bi-criteria optimization problem, and to study relation-
ships between two criteria.
Wepresentdifferentapplicationsofthistechniqueintheareasof(i)optimization
of decision trees, (ii) optimization of decision rules and systems of decision rules,
(iii)optimizationofelementpartitiontreeswhichareusedinfiniteelementmethods
for solving PDEs, and (iv) study of combinatorial optimization problems.
Theapplicationsincludeoptimizationofdecisiontreesanddecisionrulesystems
as algorithms for problem solving, as ways for knowledge representation, and as
classifiers. In addition, we study optimal element partition trees for rectangular
meshes, and create the multi-stage optimization approach for such classic combi-
natorial optimization problems as matrix chain multiplication, binary search trees,
global sequence alignment, and shortest paths.
Theresultspresentedinthisbookcanbeusefulforresearchersincombinatorial
optimization, data mining, knowledge discovery, machine learning, and finite ele-
mentmethods,especiallyforthosewhoareworkinginroughsettheory,testtheory,
logicalanalysisofdata,andPDEsolvers.Thebookcanbeusedforthecreationof
courses for graduate students.
Thuwal, Saudi Arabia Hassan AbouEisha
December 2017 Talha Amin
Igor Chikalov
Shahid Hussain
Mikhail Moshkov
vii
Acknowledgements
This book is an outcome of our research and teaching work at King Abdullah
University of Science and Technology. We are thankful to the administration of
KAUST and to our university colleagues for bringing together a great community
of people inspired by science that has become a true home for our research group
for many years.
Wearegratefultoourcoauthorsinpapersdevotedtothecreationofextensions
of dynamic programming: Jewahir AbuBekr, Mohammed Al Farhan, Abdulaziz
Alkhalid, Maram Alnafie, Saad Alrawaf, Fawaz Alsolami, Mohammad Azad,
MontherBusbait, Victor Calo,Pawel Gepner,Damian Goik,Piotr Gurgul,Konrad
Jopek,JacekKitowski,KrzysztofKuznik,AndrewLenharth,BartlomiejMedygral,
Donald Nguyen, Szymon Nosek, Enas Odat, Anna Paszynska, Maciej Paszynski,
Keshav Pingali, Marcin Skotniczny, Maciej Wozniak, and Beata Zielosko.
We are thankful to Prof. Andrzej Skowron for stimulating discussions.
We extend an expression of gratitude to Prof. Janusz Kacprzyk, to Dr. Thomas
Ditzinger, and to the Series Intelligent Systems Reference Library staff at Springer
for their support in making this book possible.
ix
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Multi-stage and Bi-criteria Optimization. . . . . . . . . . . . . . . . . . 2
1.2 Directions of Study and Complexity of Algorithms. . . . . . . . . . 2
1.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Some Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Comparison with Other Investigations . . . . . . . . . . . . . . . . . . . 5
1.6 Contents of Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6.1 Part I. Common Tools: Pareto Optimal Points
and Decision Tables. . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6.2 Part II. Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6.3 Part III. Decision Rules and Systems of Decision
Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6.4 Part IV. Element Partition Trees . . . . . . . . . . . . . . . . . 8
1.6.5 Part V. Combinatorial Optimization Problems . . . . . . . 8
1.6.6 Appendix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.7 Use of Book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Part I Common Tools: Pareto Optimal Points and Decision Tables
2 Tools for Study of Pareto Optimal Points. . . . . . . . . . . . . . . . . . . . 15
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3 Some Tools for Decision Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1 Decision Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Uncertainty Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 Directed Acyclic Graph DU;aðTÞ . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 Restricted Information Systems . . . . . . . . . . . . . . . . . . . . . . . . 28
3.5 Time Complexity of Algorithms on DU;aðTÞ. . . . . . . . . . . . . . . 30
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
xi
xii Contents
Part II Decision Trees
4 Different Kinds of Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1 Decision Trees for T. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.2 ðU;aÞ-Decision Trees for T. . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3 Cardinality of the Set TreeðG;HÞ . . . . . . . . . . . . . . . . . . . . . . 40
4.4 Umax-Decision Trees for T . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.5 Usum-Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.6 Cost Functions for Decision Trees . . . . . . . . . . . . . . . . . . . . . . 43
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5 Multi-stage Optimization of Decision Trees with Some
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.1 Optimization of Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . 49
5.2 Totally Optimal Decision Trees for Boolean Functions . . . . . . . 55
5.2.1 On Optimization of Decision Trees for Boolean
Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.2.2 Experiments with Three Total Boolean Functions. . . . . 58
5.2.3 Experiments with Total Boolean Functions . . . . . . . . . 62
5.2.4 Experiments with Partial Boolean Functions . . . . . . . . 64
5.2.5 Proofs of Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.3 Diagnosis of Constant Faults in Iteration-Free Circuits over
Monotone Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.3.1 Basic Notions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.3.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6 More Applications of Multi-stage Optimization of Decision
Trees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.1 Sorting of Eight Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.1.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.1.3 Computer Experiments . . . . . . . . . . . . . . . . . . . . . . . . 76
6.2 Modified Majority Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.2.2 Main Notions and Results. . . . . . . . . . . . . . . . . . . . . . 78
6.3 Optimization of Reducts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.3.2 Algorithm for Reduct Optimization . . . . . . . . . . . . . . . 80
6.3.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . 82
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83