Teaching Case-Based Argumentation Through a Model and Examples Ph.D. Dissertation Vincent Aleven Intelligent Systems Program University of Pittsburgh 1997 TEACHING CASE-BASED ARGUMENTATION THROUGH A MODEL AND EXAMPLES by Vincent A. W. M. M. Aleven M.Sc. Computer Science, Delft University of Technology, 1988 M.Sc. Intelligent Systems, University of Pittsburgh, 1992 Submitted to the Graduate Faculty of the Intelligent Systems Program in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 1997 Copyright by Vincent A. W. M. M. Aleven 1997 TEACHING CASE-BASED ARGUMENTATION THROUGH A MODEL AND EXAMPLES Vincent Aleven, Ph.D. University of Pittsburgh, 1997 CATO is an intelligent learning environment designed to help beginning law students learn basic skills of making arguments with cases. Using CATO, students practice tasks of induction and analogical argumentation. They practice testing theories against a body of cases and making written arguments about a problem, comparing and con- trasting it to past cases. CATO’s model addresses arguments in which two opponents analogize a problem to favorable cases, distinguish unfavorable cases, assess the significance of similarities and differences between cases in light of normative knowledge about the domain, and use that knowledge to organize multi-case arguments. CATO communicates the model to students by presenting dynamically-generated argumentation examples and by rei- fying argument structure based on the model. CATO also provides a case database and tools based on the model that help make students’ tasks more manageable. CATO was evaluated in the context of an actual legal writing course, in a study involving 30 first-year law students. We found that instruction with CATO leads to statistically significant improvement in students’ basic argumentation skills, compara- ble to that achieved by an experienced legal writing instructor teaching groups of 4-10 students. However, on a more advanced legal writing assignment, meant to explore the frontier of the CATO instruction, students taught by the legal writing instructor had higher grades, suggesting a need for more integrated practice with the CATO model. CATO contributes to AI research fields modeling educational techniques as well as case-based and legal reasoning. It is a novel result that students can learn basic argu- mentation skills by studying computer-generated examples. It means that an instruc- tional system does not necessarily need to rely on a very sophisticated understanding of students’ arguments, which would be a significant obstacle to developing such sys- tems. Also, CATO presents novel techniques for using background knowledge to support similarity assessment in case-based reasoning. Drawing on its background knowledge, CATO characterizes and re-characterizes cases in order to argue that two cases are similar or different. This is an important skill in the legal domain not previously mod- eled. CATO’s arguments may help a user in assessing the similarity of cases in a more discriminatory way. Foreword Kevin Ashley has been (and is) a most inspiring mentor and friend. It was he who intro- duced me to the research fields of case-based reasoning and AI and Law. His dedication to these fields and his keen sense of what are important research topics have been a great influence. Kevin’s enthusiasm for the field was perhaps never more evident than when I first showed him a set of (English text) arguments generated by the CATO pro- gram, which interpret and re-interpret legal cases in order to argue that two cases are similar or different. Kevin reacted with emotion, because he felt that these computer- generated arguments captured something significant about the way human experts (at- torneys) make arguments with cases. To him, that is what one strives for in developing AI models of case-based or legal reasoning. He teaches by example to distill the essence of research work and put it in writing in the clearest possible way. Working with Kevin has been a wonderful learning experience. The other members of my Ph.D. committee, each in their own extraordinary way, have encouraged me and have helped me to define and state the contribution of my work. Bruce Buchanan is the kind of professor who makes one enjoy being a graduate stu- dent. After talking to Bruce, one always comes away with a new angle, a new perspec- tive, a different way of looking at one’s work. Very early on he made me aware that I should focus on the “So What?” question and state clearly the contribution of my re- search project. I am glad that I had the opportunity to work for him as a teaching assis- tant during my last semester as a graduate student. Alan Lesgold has given me a broad perspective on the many issues involved in de- veloping and evaluating instructional technology. His advice helped greatly in defining the experiments to evaluate CATO and also in interpreting and presenting the results. He pointed out to me that one ought to be adventurous in stating a thesis. Kurt VanLehn’s approach is a unique blend of “high-level” and “hands-on”. With Kurt, I had many interesting discussions about how to develop instructional systems in ill-structured domains. Kurt also provided much useful advice with respect to the ex- periments to evaluate CATO. I am very grateful to Kevin Deasy of the University of Pittsburgh School of Law for giving us the opportunity to evaluate CATO in the context of his legal writing course at the University of Pittsburgh School of Law. He showed extraordinary flexibility and co- operation. Hopefully, one day Kevin will be rewarded for his foresight and efforts, in the form of contributions made to his courses by CATO’s offspring. The experiment to evaluate CATO was a huge undertaking. Without the expert as- sistance of many, it would have been infinitely more difficult to set up and run the vii “CATO Lab” in the University of Pittsburgh School of Law. The people from the LRDC Computing Services were very helpful. Mark Hahn of LRDC gave us important advice. Dan Jones of LRDC was invaluable, always willing on short notice to let us benefit from his expertise about the intricacies of systems and network administration. Steffi Brüninghaus contributed her experience and was a most charming and dedicated assistant, making my part that much more easy. The Intelligent Systems Program at the University of Pittsburgh has been a very stimulating and cooperative environment. I have had many interesting discussions about case-based reasoning with Bruce McLaren, with whom I shared my office at LRDC. Gerard Aleven provided last-minute long-distance expert editorial assistance. Steffi Brüninghaus, with her energetic optimism and good humor, has been wonder- fully supportive during the last part of my graduate career. She patiently put up with my absence during the many hour of dissertation writing and revising. She helped in every way she could, especially when it was needed the most. Finally, it is hard for me to see how I could have completed this journey without the continuous support of my parents, Gerard and Phini. This dissertation is for them. The author has been supported by a Mellon Fellowship. The research has been sponsored by grants from the National Science Foundation, West Publishing Company, Digital Equipment Corporation, Tektronix, the National Center for Automated Information Research, and the Uni- versity of Pittsburgh ECAC Advanced Instructional Technology Program. viii
Description: