Decision Trees with Hypotheses

148 Pages·2022·2.024 MB·English

by  Mohammad Azad, Igor Chikalov, Shahid Hussain, Mikhail Moshkov, Beata Zielosko

Synthesis Lectures on Intelligent Technologies
Mohammad Azad · Igor Chikalov ·
Shahid Hussain · Mikhail Moshkov ·
Beata Zielosko
Decision Trees with
Hypotheses
Synthesis Lectures on Intelligent
Technologies
SeriesEditor
JanuszKacprzyk,SystemsResearchInstitute,PolishAcademyofScience,Warsaw,
Poland
Synthesis Lectures on Intelligent Technologies provides highly interdisciplinary research
with the potential to change the fundamental principles of our society. It covers appli-
cations such as Intelligent Transportation, Humanoids, Self-Driving Cars, IoT, Ambient
Intelligence, Smart Cities, Human-computer Interaction, Computational Intelligence,
Industry 4.0, Medical Robotics, or Data Science. Synthesis Lectures on Intelligent Tech-
nologies brings together up-to-date resources from trusted authors working around the
world in all aspects of Intelligent Systems.
Mohammad Azad · Igor Chikalov ·
Shahid Hussain · Mikhail Moshkov ·
Beata Zielosko
Decision Trees
with Hypotheses
MohammadAzad IgorChikalov
DepartmentofComputerScience IntelCorporation
CollegeofComputerandInformationSciences Chandler,AZ,USA
JoufUniversity
Sakakah,SaudiArabia MikhailMoshkov
Computer,ElectricalandMathematical
ShahidHussain SciencesandEngineeringDivision
DepartmentofComputerScience ComputationalBioscienceResearchCenter
SchoolofMathematicsandComputerScience KingAbdullahUniversityofScience
InstituteofBusinessAdministration andTechnology
Karachi,Pakistan Thuwal,SaudiArabia
BeataZielosko
FacultyofScienceandTechnology
InstituteofComputerScience
UniversityofSilesiainKatowice
Sosnowiec,Poland
ISSN2731-6912 ISSN2731-6920 (electronic)
SynthesisLecturesonIntelligentTechnologies
ISBN978-3-031-08584-0 ISBN978-3-031-08585-7 (eBook)
https://doi.org/10.1007/978-3-031-08585-7
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerlandAG
2022
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whetherthewhole
orpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformationstorage
andretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodologynowknownor
hereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublicationdoes
notimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective
lawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbookare
believedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsortheeditorsgive
awarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforanyerrorsoromissionsthat
mayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictionalclaimsinpublishedmapsand
institutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Toourfamilies
Preface
Decision trees are widely used in many areas of computer science and related fields as
classifiers, as a means for knowledge representation, and as algorithms to solve various
problems.Theyarestudied,inparticular,intesttheory,roughsettheory,andexactlearn-
ing. These theories are closely related: attributes from rough set theory and test theory
correspondtomembershipqueriesfromexactlearning.Exactlearningstudiesadditionally
the so-called equivalence queries.
In this book, we added to the model considered in test theory and rough set theory
the notion of a hypothesis that allowed us to use an analog of equivalence queries and
studied decision trees using various combinations of attributes, hypotheses, and proper
hypotheses (an analog of proper equivalence queries).
The two main goals of this book are (i) to create tools for the experimental and the-
oretical study of decision trees with hypotheses and (ii) to compare these decision trees
with conventional decision trees that use only queries, each of which is based on one
attribute.
We got both experimental and theoretical results showing that the decision trees
with hypotheses can have less complexity than the conventional decision trees. These
results open up some prospects for using decision trees with hypotheses as a means for
knowledge representation and as algorithms for computation of the Boolean functions.
Theobtainedtheoreticalresultsandtoolsdesignedforthestudyofdecisiontreeswith
hypothesescanbehelpfulforresearcherswhousedecisiontreesandrulesindataanalysis.
This book can also be used for the creation of courses for graduate students.
Sakaka, Saudi Arabia Mohammad Azad
Chandler, USA Igor Chikalov
Karachi, Pakistan Shahid Hussain
Thuwal, Saudi Arabia Mikhail Moshkov
Sosnowiec, Poland Beata Zielosko
March 2022
vii
viii Preface
Acknowledgements WearegreatlyindebtedtoKingAbdullahUniversityofScienceandTechnol-
ogyforitsimmensesupport.
WearethankfultoProf.AndrzejSkowronandtoAlexanderKnyazevforstimulatingdiscussions.
WeextendanexpressionofgratitudetoProf.JanuszKacprzyk,Dr.ThomasDitzinger,andthe
SeriesSynthesisLecturesonIntelligentTechnologiesstaffatSpringerfortheirsupportinmaking
thisbookpossible.
Contents
1 Introduction ......................................................... 1
1.1 Part I. Decision Tables ............................................ 2
1.2 Part II. Infinite Binary Information Systems and Infinite Families
of Concepts ..................................................... 4
1.3 Prospects of Using Decision Trees with Hypotheses .................. 5
1.4 Use of Book ..................................................... 7
References ........................................................... 8
PartI DecisionTables
2 MainNotions ........................................................ 13
2.1 Decision Tables and Uncertainty Measures .......................... 13
2.2 Decision Trees ................................................... 14
2.3 Decision Rules Derived from Decision Trees ........................ 16
References ........................................................... 17
3 DynamicProgrammingAlgorithmsforMinimizationofDecisionTree
Complexity .......................................................... 19
3.1 Construction of Directed Acyclic Graph Δ(T) ....................... 20
3.2 Minimizing the Depth ............................................ 21
3.3 Minimizing the Number of Realizable Nodes ........................ 25
3.4 Minimizing the Number of Realizable Terminal Nodes ................ 28
3.5 Minimizing the Number of Working Nodes .......................... 30
3.6 On Number of Realizable Terminal Nodes .......................... 31
3.7 Results of Experiments ........................................... 34
3.7.1 Depth .................................................... 35
3.7.2 Number of Realizable Nodes ................................ 36
3.7.3 Number of Realizable Terminal Nodes ....................... 37
3.7.4 Number of Working Nodes ................................. 38
3.8 Conclusions ..................................................... 40
References ........................................................... 40
ix
x Contents
4 ConstructionofOptimalDecisionTreesandDerivingDecisionRules
fromThem .......................................................... 41
4.1 Construction of Decision Trees with Minimum Depth ................. 42
4.2 Construction of Decision Trees with Minimum Number of Working
Nodes .......................................................... 45
4.3 On Construction of Optimal Decision Trees for L and L ............. 48
t
4.4 Results of Experiments ........................................... 49
4.4.1 Decision Trees with Minimum Depth ......................... 49
4.4.2 Decision Trees with Minimum Number of Working Nodes ...... 50
4.4.3 Analysis of Experimental Results ............................ 53
4.5 Conclusions ..................................................... 53
References ........................................................... 53
5 GreedyAlgorithmsforConstructionofDecisionTreeswithHypotheses ... 55
5.1 Greedy Algorithms ............................................... 56
5.2 Results of Experiments on Decision Tables from UCI ML Repository ... 57
5.2.1 Results for Misclassification Error me ........................ 57
5.2.2 Results for Relative Misclassification Errorrme ............... 59
5.2.3 Results for Entropy ent ..................................... 61
5.2.4 Results for Gini Index gini ................................. 63
5.2.5 Results for Uncertainty Measure R ........................... 65
5.3 Results of Experiments on Randomly Generated Boolean Functions .... 68
5.4 Analysis of Experimental Results .................................. 71
5.5 Conclusions ..................................................... 71
References ........................................................... 72
6 DecisionTreeswithHypothesesforRecognitionofMonotoneBoolean
FunctionsandforSorting ............................................. 73
6.1 Problem of Recognition of Monotone Boolean Functions .............. 74
6.1.1 Basic Notions and Notation ................................. 74
6.1.2 Results of Experiments ..................................... 75
6.2 Problem of Sorting ............................................... 77
6.2.1 Basic Notions and Notation ................................. 77
6.2.2 Results of Experiments ..................................... 78
6.3 Conclusions ..................................................... 80
References ........................................................... 80
PartII BinaryInformationSystemsandInfiniteFamiliesofConcepts
7 InfiniteBinaryInformationSystems.DecisionTreesofTypes1,2,and3 ... 83
7.1 Basic Notions .................................................... 84
7.2 Main Results .................................................... 86
7.3 Proofs of Theorems 7.1 and 7.2 .................................... 88