Table Of ContentAFRL-HE-AZ-TR-2007-0001
Exploring the relationship between
distributed training, integrated learning
environments, and immersive training
environments
Adrienne Y. Lee
New Mexico State University
Department of Psychology
New Mexico State University
PO Box 30001, MS 3452
Las Cruces, NM 88003
January 2007
Final Report for June 2004 to February 2006
A i r F o r c e R e s e a r c h Laboratory
Approved for Public Release.
Human Effectiveness Directorate
Distribution is unlimited.
Warfighter Readiness Research Division
NOTICES
This report is published in the interest of scientific and technical information exchange and its
publication does not constitute the Government’s approval or disapproval of its idea or findings.
This technical report is published as received and has not been edited by the Air Force
Research Laboratory, Human Effectiveness Directorate. It is published in the interest of
STINFO exchange.
Using Government drawings, specifications, or other data included in this document for any
purpose other than Government procurement does not in any way obligate the U.S.
Government. The fact that the Government formulated or supplied the drawings, specifications,
or other data does not license the holder or any other person or corporation; or convey any
rights or permission to manufacture, use, or sell any patented invention that may relate to them.
Qualified requestors may obtain copies of this report from the Defense Technical Information
Center (DTIC) at http://www.dtic.mil.
AFRL-HE-AZ-TR-2007-01 HAS BEEN REVIEWED AND IS APPROVED FOR PUBLICATION
IN ACCORDANCE WITH ASSIGNED DISTRIBUTION STATEMENT.
//Signed// //Signed//
DULCINEA E. YABUT, 2D LT, USAF HERBERT H. BELL
Lab Contract Monitor Technical Advisor
Air Force Research Laboratory
//Signed//
DANIEL R. WALKER, Colonel, USAF
Chief, Warfighter Readiness Research Division
Air Force Research Laboratory
Form Approved
REPORT DOCUMENTATION PAGE
OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the
data needed, and completing and reviewing this collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing
this burden to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-
4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently
valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.
1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To)
18 Feb 06 Final report 30 Jun 04 – 18 Feb 06
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Exploring the relationship between distributed training, integrated learning environments, and FA8650-04-1-6529
immersive training environments 5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
Adrienne Y. Lee 1123
5e. TASK NUMBER
AS
5f. WORK UNIT NUMBER
1123AS07
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT
NUMBER
Adrienne Y. Lee
Department of Psychology
New Mexico State University
PO Box 30001, MS 3452
Las Cruces, NM 88003
9. SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
Air Force Research Laboratory AFRL; AFRL/HEA
Human Effectiveness Directorate
Warfighter Readiness Research Division 11. SPONSOR/MONITOR’S REPORT
6030 South Kent Street NUMBER(S)
Mesa AZ 85212-6061
AFRL-HE-AZ-TR-2007 - 01
12. DISTRIBUTION / AVAILABILITY STATEMENT
Approved for public release. Distribution is unlimited.
13. SUPPLEMENTARY NOTES
Published as it was submitted from the author.
14. ABSTRACT
The original goal of this paper was to review the literature on the new areas of integrated learning environments and immersive training
environments, as well as distance and team training and intelligent tutoring systems research, and evaluate the feasibility of integrating
these ideas.
15. SUBJECT TERMS Current literature; distance and team training; distributed training; distributed training literature; immersive distributed
training; immersive training; immersive training environments; integrated learning environments; proficiencies and deficiencies; tutoring
systems; HBCU
16. SECURITY CLASSIFICATION OF: 17. LIMITATION 18. NUMBER 19a. NAME OF RESPONSIBLE PERSON
OF ABSTRACT OF PAGES Dr. Adrienne Y. Lee
a. REPORT b. ABSTRACT c. THIS PAGE 19b. TELEPHONE NUMBER (include area
UNCLASSIFIED UNCLASSIFIED UNCLASSIFIED UNLIMITED 33 code)
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std. Z39.18
This page intentionally left blank.
Table of Contents
Introduction.................................................................................................................................6
Background.............................................................................................................................6
1. Intelligent tutoring systems, Distributed/Distance Training, Team Training.........6
2. New ideas in training: Integrated learning environments and Immersive training
environments...........................................................................................................................8
3. Overall summary of the technical issues........................................................................9
Method.......................................................................................................................................10
Findings .....................................................................................................................................10
General...................................................................................................................................10
What do we know?...............................................................................................................11
Simulations..........................................................................................................................11
Teams...................................................................................................................................12
Distance...............................................................................................................................13
Virtual reality/Virtual environments.................................................................................14
1. Virtual humans:.......................................................................................................14
2. Fine motor skills:......................................................................................................14
3. Building a database of results:...............................................................................15
4. Evaluation instruments:..........................................................................................15
5. Problems with technology:.....................................................................................15
6. Transfer:....................................................................................................................16
7. Therapy:....................................................................................................................16
8. General:.....................................................................................................................17
9. Leadership:...............................................................................................................17
10. Negative side effects:............................................................................................17
Medicine and Education......................................................................................................19
Recommendations for promising areas of research with a discussion of the problems
and scalability of these approaches for current Air Force training practices..................21
Potential advantages............................................................................................................21
Potential problems................................................................................................................22
Other issues...........................................................................................................................23
Future Directions..................................................................................................................23
References..................................................................................................................................25
Exploring the relationship between distributed training, integrated learning
environments, and immersive training environments
Introduction
The original goal of this paper was to review the literature on the new areas of
integrated learning environments and immersive training environments, as well as
distance and team training and intelligent tutoring systems research, and evaluate the
feasibility of integrating these ideas.
Background
With the military’s increased reliance on advanced technology, the push to
acquire and maintain proficiency on the use of that technology has increased as well
(Barry & Runyan, 1995). In order to maintain its preeminence, the Air Force needs to
maintain an ability to use advanced training technology and deliver instruction on
demand (Sikes, Cherry, Durall, Hargrove, & Tingman, 1996). New technologies are
creating novel instructional methods and techniques as part of the design, such as e-
learning, immersive training environments, and interactive learning environments.
However, different views of how effective integrated learning environments and
immersive training environments are, especially in comparison to traditional
technologically-based training methods, is not well known.
While research is necessary to ultimately determine effectiveness, integration,
and scalability of the technologies, thoroughly studying what exists in the literature
and relating those findings to previous, earlier attempts to apply training technology
can allow for more effective focusing of research efforts and avoid duplication of
results. The following short sections briefly review the major areas and provide a brief
background for the summaries of new research directions discussed in this paper.
1. Intelligent tutoring systems, Distributed/Distance Training, Team Training
Intelligent tutoring systems. Instruction can take many forms such as classroom
lecture, group activities, and discovery learning; however, the access to technology
expands the possibilities and provides a potential for individualized instruction. The
basic outline for this type of individualized instruction with computers, intelligent
tutoring systems, has been around since 1973 (Hartley & Sleeman, 1973); however, a
real break through came when validated psychological theories about learning were
applied to the creation of these systems (for summaries see Anderson, Corbett,
Koedinger, & Pelletier, 1995; Corbett, Koedinger & Anderson, 1997). Although a
variety of tutoring systems were proposed and some developed (Sleeman & Brown,
1982; Wenger, 1987), those systems developed by Anderson and colleagues have been
Lee page 6
used in public schools and have also continued to inform learning theory (Anderson &
Gluck, 2001; Corbett et al., 1997; Koedinger, Anderson, Hadley & Mark, 1997). These
tutoring systems are thus unique because they were able to scale up to a real world
situation where many such systems did not.
A majority of individual tutoring systems are not scalable due to development
costs, even though, unlike off-the-shelf educational software (edutainment), they
provide empirical validation of learning. However, these tutoring systems have
applicability in distributed/distance training situations, where trainees are expected to
do some or all of their learning on their own and away from the class.
Distributed/Distance training. In recent years, the advancement of military
technology has resulted in highly complex skills that need to be acquired and
maintained to use that technology (Barry & Ryan, 1995). Consequently, the demand for
ongoing innovations including distance training has increased. While sending
personnel to training sites can be costly, recent surveys have found that considerable
cost savings with comparable training outcomes can be acquired through the use of
distance-learning. For example, the Asynchronous Computer Conferencing (ACC)
used by the Army resulted in comparable performance between resident and ACC
students but cost less (Hahn, 1990). However, even though distance learning in some
form has been available for many years (e.g., correspondence courses), studies focusing
on the internet/web-based training have only recently been performed and relatively
few have examined the effectiveness of the technology for knowledge acquisition
(Boling, & Robinson, 1999; Kerka, 1996). With increasing use of these technologies,
research focused on the factors that could improve learning from this type of training
would be beneficial.
While research in intelligent tutoring systems focused on the individual, issues
in distance training apply equally to individuals and teams. For example, under ideal
training situations, multiple individuals can be trained together to form effective
teams. However, the constraints of distance and time often play a role in limiting team
training. Furthermore, limitations in available personnel and the need to spread these
personnel over wide geographic areas also impedes the ability to train effective teams.
One solution to these problems is distance team learning.
Team/Group training. A parallel development to distance training has occurred in
the area of team training. Teams play an increasingly large role in many aspects of
military work. In particular, the growing complexity of tasks frequently surpasses the
cognitive capabilities of individuals and thus, necessitates a team approach (Cooke,
Salas, Cannon-Bowers, & Stout, 2000; Salas, Cannon-Bowers, Church-Payne, & Smith-
Jentsch, 1998). However, most individuals have little formal training in how to work
within a group, much less how to learn within one. Moreover, the move toward
distributed mission training for many military training programs creates new
challenges for collaborative learning by teams. Trainers are faced with the problem of
both training the skills directly needed for the job and the interpersonal skills needed
Lee page 7
for successful team performance. In addition, trainers and trainees must adapt to
distributed training programs. At the heart of both individual and team training,
however, is the ability to accurately and reliably assess learning and measure the
knowledge that results from training. Although progress has been made on
knowledge measurement at the individual level, the measurement of team knowledge,
and team cognition in general, is still in its infancy (Cooke et al., 2000). In addition,
extensive research has been performed on individual’s cognitive skill acquisition (see
above), however, team learning and computer supported cooperative learning have
only recently become the focus of attention, especially for groups larger than two
individuals. A greater understanding of what an individual learns within
collaborative learning situations and, as opposed to or in conjunction with, what the
team learns is needed in order to provide information for trainers.
Distance learning may have a different meaning in the context of group
(collaborative) or team training. Few studies have examined collaborative (group)
distance learning and even fewer studies examine team distance training (Frost &
Fukami, 1997; Kerka, 1996). Computer internet/web environments have tended to be
built for the individual (Calvani, Sorzio, & Varisco, 1997); however, groupware and
CMC has allowed for the possibility of the delivery of team distance learning. Results
from studies comparing CMC to face-to-face have found mixed results. For example
studies have suggested that CMC can reduce social norms (Sproull & Kiesler, 1986),
can be a hindrance to the creation of meaning (Mantovani, 1996) and can lead to
lengthy decision making (Hedlund, Ilgen, & Hollenbeck, 1998) in comparison to face-
to-face communication. These studies are limited because they tend to focus on
organizational and social factors rather than team process, cognition and performance.
Also, military teams tend to be populated by hierarchical teams of individuals each
with his or her own role, as opposed to homogeneous groups of individuals making
the same judgments or decisions. Therefore, in order to study team distance learning,
new techniques need to be developed.
2. New ideas in training: Integrated learning environments and Immersive training
environments
With the advent of more powerful computing, training systems can provide
more personalized and interactive feedback to a trainee. Integrated learning
environments and immersive training environments are two of the most recent
approaches that leverage off of computing power for improving training.
Immersive training environments have been characterized as virtual reality
technology but also include a simulation component (Furness, Winn, & Yu, 1997).
Simulations and virtual reality have been used extensively for training in the military
(see Tobias & Fletcher, 2000). The difference between non-immersive versions of these
technologies and immersive versions can be defined in terms of the degrees of sensory
impact on the trainee. Because 3D non-immersive environments can provide very
realistic situations for the trainee, determining when to use the more expensive
Lee page 8
immersive technologies becomes a question. Therefore, reviewing the effectiveness of
non-immersive and immersive technologies can be the first step before determining the
relationship between immersive training environments and integrated learning
environments.
Integrated learning environments are characterized by a systems approach and
provide delivery of training on demand (often from the internet), providing alternative
learning methods, and easily allow instructor management (e.g., changes of materials,
monitoring students, providing correction) (Honebein, 1996; Nishinosono, 1992). This
idea is a natural extension of e-learning or web-based instruction. With advancement
in technology and technological proficiency of instructors, the suggestion of integrating
various learning environments not only makes sense but also becomes feasible. Thus,
integrated learning environments would allow an instructor to provide a variety of
digital instructional media (e.g., journal articles, chat rooms, tutoring systems), while
also allowing instructors to monitor and maintain contact with students. However,
many tools for the internet are not integrated. Tools have been independently
developed and instructors who would like to conduct their classes fully or partially via
e-learning may find difficulty in reaching the goal of integration. Therefore, the time is
right for additional research into deciding which tools are critical for successful
integrated learning environments, the development of tools that could be integrated,
usability issues involving these tools, and whether learning is improved when
instructors adapt these environments. In addition, because the shortcomings in
evaluations of large-scale military training simulations have been identified, it would
be useful to find ways to tie integrated learning environments with current large-scale
military training simulation (Simpson, 1999a, b).
Immersive training environments utilize new technology to provide a more
sensory approach to learning, whereas integrated learning environments utilize
combinations of tools and new takes on the learning experience. Relating results from
the previously well tested area of intelligent tutoring systems and the literatures of
distance and team training will provide additional insights as to how these two areas
can practically and conceptually be combined and evaluated.
3. Overall summary of the technical issues
In summary, traditional research in training technology has focused on the
individual. These systems have had successes and provide information about
scalability and empirical evaluations of effectiveness (training validity). In addition,
this research may inform distance training, but will need degrees of modification to be
generalized to team training. Immersive training environments, on the other hand,
extend simulation technology and provide the trainee with additional contextually-
based learning experiences. All of these techniques, tutoring systems, distance and
team training issues, and immersive training environments, can apply ideas derived
from integrated learning environments and may do so to various extents. Reviewing
these literatures to discover proficiencies and deficiencies in application will provide
Lee page 9
information about how integrated learning environments can be achieved, as well as
how immersive training environments should be designed and evaluated.
Method
We focused exclusively on published empirical studies. We also included book
chapters which are unrefereed. Due to the fact that there were so few published
papers, we expanded slightly to include some papers from very recent conferences.
The vast majority of the articles in the newest topics of distance training, team training,
and virtual learning environments are opinion pieces. They also tend to focus only on
a single one of these topics and therefore do not necessarily speak to the combination.
The other source for information is the internet. On the internet, you can often
find information about projects on the topics covered in this paper that are being
performed in academics, industry, and the military. Some of these authors have
published papers that are accessible; many of these papers are conference proceedings.
However, the vast majority of the sites are just descriptions of research being
performed. Some information from these sources may be included where conference
papers are available or citations are at least made. Although APA provides for citing
from the web, web sources are often unreliable or unverifiable. So, although we used
the internet to try to locate additional sources, websites are not generally included
below.
Findings
General
Within the literature, virtual learning environment or virtual environment for learning
can be synonymous with immersive training environment. In order to be complete,
though, the terms virtual reality, simulation/immersive simulation, and training
(learning, tutoring systems or training systems) should also be included. By virtual
environment, though, computer scientists are generally referring to something more
than a simulation. They are referring to an underlying simulation with an interactive
environment and the sense for the user that he/she is immersed and physically
interacting. The tactile (as well as audio) feedback seems to be a critical factor.1
1 In the field of education, virtual reality environments can be broken down into 3
categories: text-based virtual environment, desktop virtual reality, and immersive
virtual environments. Text-based virtual environments are like MUDs (Multi-User
Dungeon or Domain or Dimension) and MOOs (MUD object oriented); and, desktop
virtual reality is like simulation or synthetic training environments. Both of these lack
the sensory experience of an immersive environment.
Lee page 10
