DTIC ADA459671: Automated Tutoring Dialogues for Training in Shipboard Damage Control

5 Pages·0.17 MB·English

by  Defense Technical Information Center

Automated Tutoring Dialogues for Training in Shipboard
Damage Control
John Fry, Matt Ginzton, Stanley Peters, Brady Clark & Heather Pon-Barry
Stanford University
Center for the Study of Language Information
Stanford CA 94305-4115 USA
{fry,mginzton,peters,bzack,ponbarry}@csli.stanford.edu
Abstract 4.0: fire, flooding, missile damage, and wall
or firemain ruptures.
This paper describes an application
of state-of-the-art spoken language 2 Previous Work
technology (OAA/Gemini/Nuance)
Eliciting self-explanation from a student has
to a new problem domain: engaging
been shown to be a highly effective tutoring
students in automated tutorial dia-
method (Chi et al., 1994). For this reason,
logues in order to evaluate and im-
a number of automated tutoring systems cur-
prove their performance in a train-
rently useNLP techniques to engage students
ing simulator.
in reflective dialogues. Three notable exam-
ples are the medical Circsim tutor (Zhou et
1 Introduction
al., 1999); the Basic Electricity and Electron-
Shipboarddamagecontrolreferstothetaskof ics (BE&E) tutor (Ros´e et al., 1999); and
containing the effects of fire, explosions, hull thecomputerliteracyAutoTutor(Wiemer-
breaches, flooding, and other critical events Hastings et al., 1999).
that can occur aboard Naval vessels. The Our system shares several features with
high-stakes,high-stressnatureofthistask,to- these three tutoring systems:
gether with limited opportunities for real-life
A knowledge base Our system encodes
training, makedamage controlanideal target
all domain knowledge relevant to supporting
for AI-enabled educational technologies like
intelligent tutoring feedback into a structure
training simulators and tutoring systems.
called an Expert Session Summary (Section
This paper describes the spoken dialogue
4). These expert summaries encode causal
systemwedevelopedforautomatedcritiquing
relationships between events on the ship as
of student performance on a damage control
well as the proper and improper responses to
training simulator. The simulator is DC-
shipboard crises.
Train (Bulitko and Wilkins, 1999), an im-
mersive, multimedia training environment for
Tutoring strategies In our system, as in
damage control. DC-Train’s training sce-
those above, the flow of dialogue is controlled
nariossimulateamixtureofphysicalphenom-
by (essentially) a finite-state transition net-
ena (e.g., fire, flooding) and personnel issues
work (Fig. 1).
(e.g., casualties, communications, standard-
ized procedures). Our current tutoring sys- An interpretation component In our
tem is restricted fire damage scenarios only, system,thestudent’sspeechisrecognizedand
and in particular to the twelve fire scenar- parsed into logical forms (Section 3). A dia-
ios available in DC-Train version 2.5, but logue manager inspects the current dialogue
in future versions we plan to support post- information state to determine how best to
session critiques for all of the damage phe- incorporate each new utterance into the dia-
nomena that will be modeled by DC-Train logue (Lemon et al., 2001).
Report Documentation Page Form Approved
OMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it

does not display a currently valid OMB control number.
1. REPORT DATE 3. DATES COVERED
2001 2. REPORT TYPE 00-00-2001 to 00-00-2001
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Automated Tutoring Dialogues for Training in Shipboard Damage
5b. GRANT NUMBER
Control
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
Center for the Study of Language ad Information (CSLI),Stanford REPORT NUMBER
University,Stanford,CA,94305
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT
NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
The original document contains color images.
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON

a. REPORT b. ABSTRACT c. THIS PAGE 4
unclassified unclassified unclassified
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std Z39-18
Prompt
Summary student review "OK, let’s
of damage of actions continue"
Brief event 1 Summary
summary of Correct of damage Review
session Correct student’s event N... main points
student’s reflections
Evaluate report
START Prompt for
student’s
"You handled reflection on END
performance
this one well" errors
Figure 1: Post-session dialogue move graph (simplified)
However, an important difference is that database queries). Our system uses OAA to
the three systems above are entirely text- coordinate the following five agents:
based, whereas ours is a spoken dialogue sys-
tem. Ourspeech interface offersgreater natu- 1. The Gemini NLP system (Dowding et
ralnessthankeyboard-basedinput. Inthisre- al., 1993). Gemini uses a single unifi-
spect,oursystemissimilartocove(Roberts, cation grammar both for parsing strings
2000), a training simulator for conning Navy of words into logical forms (LFs) and for
ships that uses speech to interact with the generating sentences from LF inputs.
student. Butwhereascoveusesshortconver-
2. A Nuance speech recognition server,
sational exchanges to coach the student dur-
which converts spoken utterances to
ing the simulation, our system engages in ex-
strings of words. The Nuance server re-
tended tutorial dialogues after the simulation
lies on a language model, which is com-
has ended. Besides being more natural, spo-
piled directly from the Gemini grammar,
kenlanguage systemsarealsobettersuitedto
ensuring that every recognized utterance
multimodal interactions (viz., one can point
is assigned an LF.
and click while talking but not while typing).
An additional significant difference between
3. The Festival text-to-speech system,
our system and a number of other automated
which ‘speaks’ word strings generated by
tutoring systems is our use of ‘deep’ process-
Gemini.
ing techniques. While other systems utilize
‘shallow’statisticalapproacheslikeLatentSe- 4. A Dialogue Manager which coordi-
manticAnalysis(e.g. AutoTutor),oursystem nates inputsfromtheuser,interprets the
utilizes Gemini, a symbolic grammar. This user’s dialogue moves, updates the dia-
approach enables us to provide precise and logue context, and delivers speech and
reliable meaning representations. graphical outputs to the user.
3 Implementation 5. A Critique Planner, described below
in Section 4.
To facilitate the implementation of multi-
modal, mixed-initiative tutoring interactions, Agents 1-3 are reusable, ‘off-the-shelf’ dia-
we decided to implement our system within logue system components (apart from the
the OpenAgent Architecture (OAA) (Martin Gemini grammar, which mustbemodifiedfor
et al., 1999). OAA is a framework for coor- each application). We implemented agents 4
dinating multiple asynchronous communicat- and 5 in Java specifically for this application.
ing processes. The core of OAA is a ‘facilita- Variants of this OAA/Gemini/Nuance ar-
tor’ which manages message passing between chitecture have been deployed successfully in
a number of software agents that specialize other dialogue systems, notably SRI’s Com-
in certain tasks (e.g., speech recognition or mandTalk (Stent et al., 1999) and an un-
Figure 2: Screen shot of post-session tutorial dialogue system
manned helicopter interface developed in our Each action node in the ESS therefore falls
laboratory (Lemon et al., 2001). into one of three classes: (i) correct actions;
(ii) errors of commission (e.g., the student
4 Planning the dialogue sets fire containment boundaries incorrectly);
and (iii) errors of omission (e.g., the student
Each student session with DC-Train pro- failstosecurepermissionfromthecaptainbe-
ducesasessiontranscript, i.e.atime-stamped fore flooding certain compartments).
record of every event (both computer- and Our current tutoring system covers scenar-
student-initiated) that occurred during the ios generated by DC-Train 2.5, which covers
simulation. These transcripts serve as the fire scenarios only. Future versions will use
input to our post-session Critique Planner scenarios generated by DC-Train 4.0, which
(CP). coversdamagecontrolscenariosinvolvingfire,
The CP plans a post-session tutorial di- smoke, flooding, pipe and hull ruptures, and
alogue in two steps. In the first step, an equipment deactivation. Our current tutor-
Expert Session Summary (ESS) is cre- ing system is based on an ESS graph that is
ated from the session transcript. The ESS generated by an expert model that consists
is a tree whose parent nodes represent dam- of an ad-hoc set of firefighting rules. Future
age events and whose leaves represent actions versions will be based on an ESS graph that
taken in response to those damage events. is generated by an successor to the Minerva-
Each student-initiated action in the ESS is DCA expert model (Bulitko and Wilkins,
evaluatedastoitstimelinessandconformance 1999), an extended Petri Net envisionment-
to damage control doctrine. Actions that the based reasoning system. The new expert
studentshouldhavetakenbutdidnotarealso model is designed to produce an ESS graph
inserted into the ESS and flagged as such. duringthecourseofproblemsolvingthatcon-
tains nodes for all successful and unsuccessful N000140010660, a multidisciplinary univer-
plan and goal achievement events, along with sityresearchinitiativeonnaturallanguagein-
an explanation structureforeach graphnode. teraction with intelligent tutoring systems.
The second step in planning the post-
session tutorial dialogue is to produce a di-
References
alogue move graph (Fig. 1). This is a di-
rectedgraphthatencodesallpossibleconfigu- VV.BulitkoandDC.Wilkins. 1999. Automated
instructorassistantforshipdamagecontrol. In
rations of dialogue structureandcontent that
Proceedings of AAAI-99, Orlando, FL, July.
can be handled by the system.
Generating an appropriate dialogue move M. T. H. Chi, N. de Leeuw, M. Chiu, and C.
LaVancher. 1994. Eliciting self-explanations
graph from an ESS requires pedagogical
improves understanding. Cognitive Science,
knowledge, and in particular a tutoring strat-
18(3):439–477.
egy. The tutoring strategy we adopted is
J. Dowding, J. Gawron, D. Appelt, J. Bear, L.
based on our analysis of videotapes of fifteen
Cherny, R. C. Moore, and D. Moran. 1993.
actual DC-Train post-session critiques con-
Gemini: Anaturallanguagesystemforspoken-
ducted by instructors at the Navy’s Surface language understanding. In Proceedings of the
Warfare Officer’s School in Newport, RI. The ARPAWorkshoponHumanLanguageTechnol-
ogy.
strategy we observed in these critiques, and
implemented in our system, can be outlined O. Lemon, A. Bracy, A. Gruenstein, and S. Pe-
as follows: ters. 2001. A multi-modal dialogue system for
human-robot conversation. In Proceedings of
1. Summarize the results of the simulation NAACL 2001.
(e.g., the final condition of the ship).
D. Martin, A. Cheyer, and D. Moran. 1999.
TheOpenAgentArchitecture: aframeworkfor
2. Foreach majordamageeventintheESS:
building distributed software systems. Applied
(a) Ask the student to review his ac- Artificial Intelligence, 13(1-2).
tions, correcting his recollections as B. Roberts. 2000. Coaching driving skills in
necessary. a shiphandling trainer. In Proceedings of the
AAAI Fall Symposium on Building Dialogue
(b) Evaluatethecorrectnessofeach stu-
Systems for Tutorial Applications.
dent action.
C.P.Ros´e,B.DiEugenio,andJ.D.Moore. 1999.
(c) If the student committed errors,
Adialoguebasedtutoringsystemforbasicelec-
ask him how these could have been
tricity and electronics. In S. P. Lajoie and
avoided, and evaluate the correct- M. Vivet, editors, Artificial Intelligence in Ed-
ness of his responses. ucation (Proceedings of AIED’99), pages 759–
761. IOS Press, Amsterdam.
3. Finally, review each type of error that
A.Stent,J.Dowding,J.Gawron,E.O.Bratt,and
arose in step (2c).
R.C.Moore. 1999. TheCommandTalkspoken
A screen shot of the tutoring system in dialogue system. In Proceedings of ACL ’99,
pages 183–190,College Park, MD.
action is shown in Fig. 2. As soon as a
DC-Train simulation ends, thedialogue sys- P. Wiemer-Hastings, K. Wiemer-Hastings, and
A. Graesser. 1999. Improving an intelli-
tem starts up and the dialogue manager be-
genttutor’scomprehensionofstudentswithla-
gins traversing the dialogue move graph. As
tent semantic analysis. In S. P. Lajoie and
the dialogue unfolds, a graphical representa- M. Vivet, editors, Artificial Intelligence in Ed-
tion of the ESS is revealed to the student in ucation (Proceedings of AIED’99), pages 535–
piecemeal fashion as depicted in the top right 542. IOS Press, Amsterdam.
frame of Fig. 2. Y. Zhou, R. Freedman, M. Glass, J. A. Michael,
A.A.Rovick,andM.W.Evens. 1999. Deliver-
Acknowledgments inghintsinadialogue-basedintelligenttutoring
system. In Proceedings of AAAI-99, Orlando,
This work is supported by the Depart-
FL, July.
ment of the Navy under research grant