DTIC ADA501744: High-Fidelity e-Learning: SEI's Virtual Training Environment (VTE)

2.6 MB·English

by  Defense Technical Information Center

HHiigghh--FFiiddeelliittyy ee--LLeeaammiinngg:: SSEEII''ss VViirrttuuaall
TTrraaiinniinngg EEnnvviirroonnmmeenntt ((VVTTEE))
JJiimmWWrruubb""
DD""""iiddWWrriittee
JJuulliiaaAAlllleenn
JJ11IIlluuaarryy22000000
TTEECCHHNNIICCAALL RREEPPOORRTT
eeMMllll''SSEEII--22000099--TTRR--000055
EESSCC--TTRR--llOOOO!!ll--000055
CCEERRTT PPrrooggrraamm
11mm........
uu"" ,,"",,""ii,,,,""ttoooo..""""jjooeett!!ootthh..OOOOII''YY'';;iiJJ""
cc .......... __
Thisreportwaspreparedforthe
SEIAdministrativeAgent
ESC/XPK
5EglinStTeet
HanscomAFB, MA01731-2100
TheideasandfindingsinthisreportshouldnotbeconstruedasanofficialDoDposition.Itispublishedinthe
interestofscientificandteclmicalinformationexchange.
Thisworkissponsoredbytheu.s. DepartmentofDefense.The SoftwareEngineeringInstituteisa federally
fundedresearchanddevelopmentcentersponsoredbytheu.s. DepartmentofDefense.
Copyright2009 CarnegieMellonUniversity.
NOWARRANTY
THISCARNEGIEMELLONUNIVERSITYAND SOFTWAREENGINEERINGINSTITUTEMATERIAL IS
FURNISHEDONAN"AS-IS"BASIS. CARNEGIEMELLONUNIVERSITY MAKESNOWARRANTIESOF
ANYKIND,EITHEREXPRESSEDORIMPLIED,ASTOANYMATTERINCLUDING,BUTNOT LIMITED
TO,WARRANTYOFFITNESS FORPURPOSEORMERCHANTABILITY,EXCLUSIVITY, ORRESULTS
OBTAINEDFROMUSEOFTHE MATERIAL. CARNEGIE MELLONUNIVERSITYDOESNOT MAKE
ANYWARRANTYOFANYKINDWITHRESPECTTO FREEDOMFROM PATENT,TRADEMARK,OR
COPYRIGHTINFRINGEMENT.
Useofanytrademarksinthisreportisnotintendedinanywaytoinfringeontherightsofthetrademarkholder.
Internaluse. PeTillissiontoreproducethisdocumentandtoprepare derivativeworksfromthisdocumentfor
internaluseisgranted,providedthe copyTightand"NoWammty"statementsareincludedwithallreproductions
andderivativeworks.
Externaluse. Thisdoclillientmaybereproducedinitsentirety,v{ithoutmodification, andfi·eelydistributedin
writtenorelectronicfOlTIlwithoutrequestingfOlTIlal permission. Permissionisrequiredforanyotherexternal
and/orcommercialuse. RequestsforpeTillissionshouldbedirectedtotheSoftwareEngineeringInstituteat
[email protected].
ThisworkwascreatedintheperformanceofFederalGovernmentContractNumberFA8721-05-C-0003'-'lith
CarnegieMellonUniversityforthe operationofthe SoftwaTeEngineeringInstitute,afederallyfundedresearch
anddevelopmentcenter.TheGovernmentoftheUnitedStateshasaroyalty-free government-purposelicenseto
use, duplicate,ordisclosethework,inwhole orinpartandinanymarmer, andtohaveorpermitotherstodoso,
forgovernmentpurposespursuanttothecopyrightlicenseundertheclause at252.227-7013.
ForinformationaboutSEIreports,pleasevisitthepublicationssectionofourwebsite
(http://www.sei.cmu.edu/publications).
Table of Contents
Abstract v
Introduction 1
1.1 TargetAudience 1
1.2 Desired Outcomes 1
1.3 e-Learning, Classroom Learning, and Blended Learning 1
1.4 High-FidelityE-Learning 3
2 Principles ofHigh Fidelitye-Learning 4
2.1 Personalization 4
2.2 Multimodality 4
2.3 Lean-ForVvard Learning 5
2.3.1 Practices, Exercises, and Labs 6
2.3.2 Formative Assessment 6
2.4 Collaboration 7
2.4.1 Peer-StudentCollaboration 7
2.4.2 InstructorSupport 8
2.5 Accessibility g
2.6 Modularity 10
2.6.1 Design for Re-Use 10
2.6.2 Design for Reference 10
2.7 ProgressTracking 11
2.8 Balancing Control 12
2.9 Usingthe Principles 13
3 Virtual Training Environment- a High-Fidelitye-Learning System 14
3.1 Background and History 14
3.2 Implementingthe Principles- VTEFeaturesand Benefits 15
3.2.1 MimicClassroom Education (Multimodality, Personalization) 15
3.2.2 Practice'<'ith Hands-on Labs (Lean-Forward Learning) 15
3.2.3 Encourage Annotationsand Takeaways (Personalization,
Lean-Forv-vard Learning) 16
3.2.4 Read It, HearIt, See It, Do It, Master It(Multimodality) 16
3.2.5 Composing Recorded Lectures(Modularity) 17
3.2.6 Accessing ContentviaTraining and Library Modes (Modularity) 17
3.2.7 Using VTE Offline (Accessibility, Balancing Control) 17
3.2.8 Providing a Full Transcript (Multimodality, Accessibility) 17
3.2.9 ContentSequencing and Tracking {ProgressTracking} 18
3.2.10 Virtual Office Hours, Forums, and BreakoutRooms (Collaboration) 18
3.3 VTEArchitecture 18
3.4 VTEUse 19
3.4.1 Comparisonsvvith DirectInstruction 21
3.4.2 Scale 21
3.5 Producing VTEContent 22
3.5.1 Publishing aRecorded Class 22
3.5.2 Creating a Hands-On Lab 23
4 Conclusion - The Future ofVTE 25
4.1 Whomto Contact; Howto Proceed 25
i ICMU/SEI-TR-2009-005
AppendixA DefenseInformation SystemsAgency(DISA)Case Study 26
Appendix B U. S. SecretService Experienceswith VTE 32
Bibliography 34
ii ICMU/SEI-TR-2009-005
List of Figures
Figure 1: Spectrum ofLearningfTraining Models 2
Figure 2: Training ModelsShovvn in Two Dimensions 2
Figure 3: The VTE LectureInterface 15
Figure 4: The VTE Learning "PowerCurve" 16
Figure 5: VTEConceptual Architecture 19
Figure 6: VTETraining Hours Delivered per Quarter 20
Figure 7: VTE registrations perquarterfor pasttvvo years 20
Figure 8: Unique Visitsto VTE byMonth forthe PastTwoYears 21
iii ICMU/SEI-TR-2009-005
List of Tables
Table 1: Computer-aided Communication and Collaboration Technologies 8
Table 2: ThreeTypesofStudentControl [Chen & Liu 2008] 12
Table 3: DataforComparison from ASTD Report 28
Table 4: VTE Datafor Comparison 28
iv ICMU/SEI-TR-2009-005
Abstract
High-fidelity e-learning preservesthe quality ofthe educationalexperiencewhileminimizing or
eliminating theneedto be inaspecific place ataspecific timewith alive instructor. The prin
ciples ofthis type oflearning include personalization, multimodality, lean-forward learning, col
laboration, accessibility, modularity, progresstracking, andbalancing control. The SEI's Virtual
Training Environment(VTE) is an example ofahigh-fidelity e-learning success. The VTEwas
developedto drawupontheprinciples ofhigh-fidelity e-learning to provide high-quality learning
with infinitescalability. Thepurpose ofthis document is to describethetenets ofhigh-fidelity e
learning, to describehow VTEreflects these, andto summarizehow organizations have used and
areusing VIE.
v ICMU/SEI-TR-2009-005
Acknowledgments
Theauthors express our deep appreciationto theUnited States Defense Information Systems
Agency (DISA), theUnited StatesMarine CorpsPacific Command(MARFORPAC), andthe
United States SecretService Criminal InvestigativeDivisionfor theirguidance, funding, support,
use of, and improvementsuggestions for the SEI's Virtual Training Environment.
The authors acknowledgethe contributions ofall SEIstaffand instructors who contribute course
contentto the VTE library. Wethanktheteams ofChris May andRichardNolanfor contributing
high-quality training content WethankAnne Connell andDavidZubrowfor theirvaluablere
view comments andBarbaraLaswell, Technical Director, CERTEnterprise andWorkforce De
velopment, for her leadership, guidance, and supportthroughoutthe developmentand deployment
ofVTE.
vi ICMU/SEI-TR-2009-005
Executive Summary
Thistechnicalreportexplores high-fidelity e-learning, whichwe define as asetoftoolsandme
thods thatdelivers education andtrainingtostudents anywhere, anytime, withoutsacrificingeffi
cacy. High-fidelity e-learning preserves the quality ofthe educational experiencewhileminimiz
ing or eliminating the needto be in aspecific placeata specific timewith a live instructor.
The SEI's Virtual Training Environmentdescribed inthis reporthasbeen developedto reflectthe
principlesofhigh-fidelity e-learningfor workforce development Itoffers the user aweb-based,
multimedia, interactive training experience that canbe accessed anytime. This environmentuses
state-of-the-artsoftwareandhardware to trainusers withtechnical andmanagerialroles on awide
range ofinformation assurance topics.
The intended audience for thisreportis responsiblefor building orbuying e-learning programs for
employees, customers, businesspartners, orsuppliers. This reportaids in evaluating alternatives
and selecting solutions thatneedto scaleand changeto meet organizationalawareness, training,
and educationalobjectives. Thereportfirst describes the learning landscape, whichranges from
instructor-led classroomtraining in a single physical locationto "pure" e-Iearning where content
is available anytime, anywhere, using the internet.
High-fidelity e-Iearning environments incorporate richmediatechnologies, immersive simula
tions, scenario-basedinstruction, andhands-on exercises thatcanbe accessedbystudents ontheir
terms and as theirtime permits. Drawingfrom extensive research, thereport summarizestheprin
ciplesofhigh-fidelity e-learning which include thefollowing:
• Personalization: using acasual, conversationalstylefor spoken andwritten content
• Multimodality: delivering training contentusing multiple sensory channels(read it, hear it, see
it)
• Lean-ForwardLearning: using demonstrations, hands-on exercises, and selfassessmentsto
increase retention andrecall (do it, master it)
• Collaboration: providing technologies that allow studentsto engagewith one anotherandwith
instructors
• Accessibility: making training contentbroadly availableto people with arange ofabilities
• Modularity: publishing contentinmanageable, self-containedchunks so thatit canbe easily
reusedby contentowners andreferencedbystudents
• Progress Tracking: supporting courseproviders, instructors, and students withthe abilityto
track studentprogress
• BalancingControl: balancing controlbetweenthe e-Iearning systemandthe studentto achieve
optimal learning
vii ICMU/SEI-TR-2009-005
Inlargepart,thesehigh-fidelitye-learningprinciplesarere~ectedintheSEI'sVirtual Training
Environment VTE(http:lMe.cm.org)waslaunchedin 20[14to irrvrovethescalabilityoftech
nical training. VITholds over 1,800uniquemodules, representing740 hours oftraining,and cur
renUydelivers over 10,000hours oftraininga monthin ,,-,pportofU.S DoD andcivilian gov
ernmentagenciesandthegeneralpublic
Coursematerial onVTEis >WJcturedto allowstudentsto progressattheir Omlpace,u,;nga
Read It,HearIt, See It,Do It,M>sterItapproach
fJ READ Topicsbeginwithbackgrounddocumentation,relatedpolicy,and oth-
IT er foundatioml material
I~ HCAR A video cameraplacedina >tudent seatrecordstheentire class,v.fuch
isthenpublishedtothe\>?ebin I5-minuteblocks "-litable for on-
IT
demandviewing
!l:!. Narrated 'sa-eenca>t' walkthroughs ofspecificapplicationsandbest
"'CIT
practicesreinforcethelecturematerial
•
Hands-onlabsprovideanactivelearning experience Students canre-
motelystandupany ofVTE's70 corrvuternetworkconfigurationsin
DO IT
underthreeminutes. Thisallowsthemto learnby doingandto practice
on softwarethatcouldnotbe "'-felyloadedontheirOmlmachines
,.
All material isavailable ondemand, so technical trainingisn'tsome-
MASTER thing >tudentsattend; itis somethingtheyalwayshave. Students can
IT repeatanyportion0fthe COlrsematerial days,weeks,months, oryears
aftertakingthe class
Rea-eatingthe clas,.-oomin>WJctionexperience,providinga verbatimtexttransaipt,presenting
demonstrationsand opporturitiesto engagehands-onlabsatcritical learningpoints,providingfor
progresstrackingandself-assesmlent,and offeringvirtual officehoursarea fewexamples ofhow
VTEhasincorporatedhigh fidelity e-leamingprinciples
Thereportcloseswithexperiencereports from twoU.S. ~venunent organizationsthatusedVIT
tohelpmeettheirtrainingrequire,,-.,nts. AppendixA desaibesthe experiencesofDISA inad
dressingU. S. Department ofDefenseDirective 8570.I requirements forinfonnationassurance
workforceimprove,,-.,nt Perionmnceandco>t datafromtheDISA contractare comparedto an
nual "-llVeydaB fromtheA,,-.,ricanSociety ofTrainingandDevelopment's"2007 State ofthe
IndustryReport." VITwasusedto deliver38,157 hours oftrainingforDISA duringtheperiod
from III(2007 through 10/3112007. Based ontheASID comparison,DlSA savedmJrethan
$I 2Mwitha retum-on~nves~ntof14I% v.ilen comparingVTE co>tsto v.tJatitcouldhave
expectedto payatprevailingindusllyaverage co>ts
." ICo.oJISEI_TR_2OO>_005