ebook img

Case-Based Approximate Reasoning PDF

380 Pages·2007·5.257 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Case-Based Approximate Reasoning

CASE-BASED APPROXIMATE REASONING by EYKE HÜ LLERMEIER University of Magdeburg, Germany AC.I.P. Catalogue record for this book is available from the Library of Congress. ISBN-10 1-4020-5694-X (HB) ISBN-13 978-1-4020-5694-9 (HB) ISBN-10 1-4020-5695-8 (e-book) ISBN-13 978-1-4020-5695-6 (e-book) Published by Springer, P.O. Box 17, 3300 AADordrecht, The Netherlands. www.springer.com Printed on acid-free paper All Rights Reserved © 2007 Springer No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. To Christiane, Jana, and Jae¨l Table of Contents Dedication ................................................... v Foreword..................................................... xiii Preface....................................................... xv Notation ..................................................... 1 1. Introduction.................................................. 5 1.1 Similarity and case-based reasoning ........................... 5 1.2 Objective of this book ....................................... 6 1.2.1 Making case-based inference more reliable................. 9 1.2.2 The important role of models............................ 10 1.2.3 Formal models of case-based inference .................... 11 1.3 Overview .................................................. 13 2. Similarity and Case-Based Inference .......................... 17 2.1 Model-based and instance-based approaches .................... 17 2.1.1 Model-based approaches ................................ 18 2.1.2 Instance-based approaches .............................. 19 2.1.3 Knowledge representation............................... 19 2.1.4 Performance in generalization ........................... 20 2.1.5 Computational complexity .............................. 21 2.2 Similarity-based methods .................................... 22 2.2.1 Nearest neighbor (NN) estimation........................ 22 2.2.2 Instance-based learning................................. 29 2.2.3 Case-based reasoning................................... 30 vii viii TableofContents 2.3 The concept of similarity..................................... 33 2.3.1 Similarity in case-based reasoning........................ 33 2.3.2 Similarity and fuzzy sets................................ 37 2.3.3 Aggregation of local similarity measures .................. 38 2.4 Case-based inference ........................................ 41 2.4.1 Deterministic inference problems......................... 45 2.4.2 Non-deterministic inference problems..................... 48 2.4.3 Formal models of case-based inference .................... 52 2.5 Summary and remarks....................................... 53 3. Constraint-Based Modeling of Case-Based Inference .......... 59 3.1 Basic concepts.............................................. 59 3.1.1 Similarity profiles and hypotheses ........................ 59 3.1.2 Generalized similarity profiles ........................... 62 3.2 Constraint-based inference ................................... 65 3.2.1 A constraint-based inference scheme...................... 65 3.2.2 Non-deterministic problems ............................. 69 3.3 Case-based approximation.................................... 72 3.3.1 Properties of case-based approximation ................... 73 3.3.2 Local similarity profiles................................. 79 3.4 Learning similarity hypotheses................................ 81 3.4.1 The learning task ...................................... 81 3.4.2 A learning algorithm ................................... 85 3.4.3 Properties of case-based learning......................... 86 3.4.4 Experimental results ................................... 94 3.5 Application to statistical inference ............................ 97 3.5.1 Case-based parameter estimation ........................ 97 3.5.2 Case-based prior elicitation.............................. 98 3.6 Summary and remarks....................................... 99 4. Probabilistic Modeling of Case-Based Inference............... 103 4.1 Basic probabilistic concepts .................................. 106 4.1.1 Probabilistic similarity profiles and hypotheses............. 107 TableofContents ix 4.1.2 Generalized probabilistic profiles......................... 110 4.2 Case-basedinference,probabilisticreasoning,andstatisticalinference112 4.3 Learning probabilistic similarity hypotheses .................... 117 4.3.1 Simple hypotheses and credible case-based inference ........ 117 4.3.2 Extended case-based learning............................ 118 4.4 Experiments with regression and label ranking .................. 121 4.4.1 Regression: artificial data ............................... 122 4.4.2 Regression: real-world data.............................. 124 4.4.3 Label ranking ......................................... 126 4.5 Case-based inference as evidential reasoning .................... 129 4.5.1 Transformation of probabilistic evidence .................. 131 4.5.2 Inference from individual cases .......................... 133 4.5.3 Combining evidence from several cases.................... 134 4.6 Assessment of cases ......................................... 140 4.6.1 Similarity-weighted approximation ....................... 140 4.6.2 More general criteria ................................... 142 4.6.3 Assessment of individual cases........................... 144 4.7 Complex similarity hypotheses................................ 146 4.7.1 Inference schemes of higher order ........................ 147 4.7.2 Partially admissible profiles ............................. 150 4.8 Approximate probabilistic inference ........................... 152 4.8.1 Generalized uncertainty measures and profiles ............. 152 4.8.2 An approximate inference scheme ........................ 155 4.9 Summary and remarks....................................... 159 5. Fuzzy Set-Based Modeling of Case-Based Inference I ......... 165 5.1 Background on possibility theory.............................. 165 5.1.1 Possibility distributions as generalized constraints .......... 166 5.1.2 Possibility as evidential support ......................... 168 5.2 Fuzzy rule-based modeling of the CBI hypothesis................ 169 5.2.1 Possibility rules........................................ 170 5.2.2 Modeling the CBI hypothesis............................ 171 5.3 Generalized possibilistic prediction ............................ 176 x TableofContents 5.3.1 Control of compensation and accumulation of support ...... 177 5.3.2 Possibilistic support and weighted NN estimation .......... 178 5.3.3 Upper and lower possibility bounds ...................... 179 5.3.4 Fuzzy logical evaluation ................................ 180 5.3.5 Comparison of extrapolation principles ................... 181 5.3.6 From predictions to decisions............................ 186 5.3.7 An illustrative example ................................. 188 5.3.8 Complexity issues...................................... 191 5.4 Extensions of the basic model ................................ 191 5.4.1 Dealing with incomplete information ..................... 192 5.4.2 Discounting noisy and atypical instances .................. 194 5.4.3 From instances to rules ................................. 198 5.4.4 Modified possibility rules ............................... 200 5.4.5 Combination of several rules............................. 202 5.4.6 Locally restricted extrapolation.......................... 204 5.4.7 Incorporation of background knowledge................... 205 5.5 Experimental studies ........................................ 206 5.5.1 Preliminaries.......................................... 206 5.5.2 Classification accuracy.................................. 207 5.5.3 Statistical assumptions and robustness.................... 209 5.5.4 Variation of the aggregation operator..................... 210 5.5.5 Representation of uncertainty ........................... 211 5.6 Calibration of CBI models ................................... 212 5.7 Relations to other fields...................................... 219 5.7.1 Fuzzy and possibilistic data analysis...................... 219 5.7.2 Fuzzy set-based approximate reasoning ................... 222 5.8 Summary and remarks....................................... 224 6. Fuzzy Set-Based Modeling of Case-Based Inference II......... 229 6.1 Gradual inference rules ...................................... 230 6.1.1 The basic model....................................... 230 6.1.2 Modification of gradual rules ............................ 232 6.2 Certainty rules ............................................. 235 TableofContents xi 6.3 Cases as information sources ................................. 239 6.3.1 A probabilistic model .................................. 239 6.3.2 Combination of information sources ...................... 241 6.4 Exceptionality and assessment of cases......................... 243 6.5 Local rules................................................. 247 6.6 Summary and remarks....................................... 249 7. Case-Based Decision Making ................................. 253 7.1 Case-based decision theory ................................... 255 7.2 Nearest Neighbor decisions ................................... 258 7.2.1 Nearest Neighbor classification and decision making ........ 258 7.2.2 Nearest Neighbor decision rules.......................... 260 7.2.3 An axiomatic characterization ........................... 261 7.3 Fuzzy modeling of case-based decisions......................... 265 7.3.1 Basic measures for act evaluation ........................ 266 7.3.2 Modification of the basic measures ....................... 267 7.3.3 Interpretation of the decision criteria ..................... 268 7.4 Fuzzy quantification in act evaluation.......................... 270 7.5 A CBI framework of CBDM.................................. 274 7.5.1 Generalized decision-theoretic setups ..................... 274 7.5.2 Decision making using belief functions .................... 277 7.5.3 Possibilistic decision making............................. 279 7.6 CBDM models: A discussion of selected issues .................. 282 7.6.1 The relation between similarity, preference, and belief....... 282 7.6.2 The effect of observed cases ............................. 285 7.6.3 Dynamic aspects of decision making...................... 286 7.7 Experience-based decision making............................. 287 7.7.1 Compiled decision models............................... 290 7.7.2 Satisficing decision trees ................................ 292 7.7.3 Experimental evaluation ................................ 296 7.8 Summary and remarks....................................... 301 xii TableofContents 8. Conclusions and Outlook ..................................... 307 A. Possibilistic Dominance in Qualitative Decisions .............. 309 B. Implication-Based Fuzzy Rules as Randomized Gradual Rules 313 B.1 Implication-based fuzzy rules ................................. 314 B.1.1 Gradual rules ......................................... 315 B.1.2 Other implication-based rules............................ 317 B.2 Randomized gradual rules.................................... 320 B.3 A probabilistic representation of implication-based fuzzy rules..... 322 C. Similarity-Based Reasoning as Logical Inference .............. 333 D. Simulation Results of Section 3.4.4 ........................... 335 E. Experimental Results of Section 5.5.4......................... 339 F. Simulation Results of Section 7.4 ............................. 341 G. Computation of an Extended Splitting Measures.............. 343 H. Experimental Results of Section 7.7.2......................... 345 References ................................................... 346 Index......................................................... 369 Foreword Inthelastthreedecades,importanttrendsofresearchinartificialintelligencehave beendevotedtothedesignandthestudyofinferencesystemsthatexhibithuman- likereasoningcapabilities,withspecialemphasisontolerancetowardincomplete informationanduncertainty.Thisprogram,whichcanbealsorelatedtocognitive psychology concerns, has led to the development of computational models, in particularfordefaultreasoningthataccommodatesexceptionsandinconsistency, for approximate reasoning with interpolative abilities, as well as for case-based reasoning. Case-basedreasoning(CBR)reliesontheconceptofsimilarity,andmorepartic- ularlyontheideathatsituationsrecognizedassimilarinimportantaspectsmay be also similar in other respects. CBR thus appears as a yet simple instance of analogical reasoning, but as powerful as a general problem solving method. This explains the success encountered by case-based reasoning, and beyond that, the interest for similarity-based reasoning that has been increasing in the last ten years.TheveryideaofCBRisthustosolvenewproblemsonthebasisofexpe- rience that is represented by already solved problems of the same type, referred to as cases. Thus, a new problem is solved by adapting the solution of a similar case, hoping that the adaptation can be done with much less effort than solving the problem from scratch. ACBRsystemrequiresefficienttechniquesforseveralimportantsubtasks,such as organizing and maintaining the case base, retrieving cases (which are maxi- mally similar to the problem) from the case base, and adapting stored cases to theproblemathand.ThebasicinferencemechanismunderlyingCBR,wherethe concept of similarity plays a major role, is built upon the principle of instance- based learning and nearest neighbor classification. CBR has always been motivated by real-world problems, and research in this area has largely focused on building efficient computer systems. Less work has been done, however, on the theoretical foundations of case-based and similarity- based inference. Note that this strongly contrasts with the situation in default reasoning,andtoalessextentinapproximatereasoning.Thepresentbookseeks toremedythisflaw.ItsobjectiveistocontributetoasoundfoundationofCBR and related fields, such as instance-based learning and analogical reasoning, by providing formal models of similarity-based inference. To accomplish this objective, Eyke Hu¨llermeier embeds case-based inference into different frameworks of knowledge representation and reasoning, namely xiii

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.