DTIC ADA509039: The Importance of Crew Resource Management Behaviors in Mission Performance: Implications for Training Evaluation PDF
Preview DTIC ADA509039: The Importance of Crew Resource Management Behaviors in Mission Performance: Implications for Training Evaluation
MILITARYPSYCHOLOGY,2003,15(1),77-96 The Importance of Crew Resource Management Behaviors in Mission Performance: Implications for Training Evaluation RobertT. Nullmeyer AircrewTrainingResearchDivision AirForceResearchLaboratory Mesa, Arizona V. Alan Spiker AnacapaSciences SantaBarbara, California Cockpit/crewresource management(CRM) trainingwithinthe military has grown rapidlydespitethepaucityofempiricaldatalinkingCRMtomissionperformance. CRMtrainingobjectives(andcoursecontent)areoftentoovaguetoallowmeaningful trainingevaluationwithinthecontextoftraditionaltransfer-of-trainingparadigms.A multimeasuremethodologythatexploitsallsourcesofarchivalandobservationaldata withinatrainingorganizationhasthepotentialtoadvancetrainingevaluation,particu larlyforcrew-basedskillssuchasCRM.ThisarticlediscussesavarietyofCRMdata sourcesandpresentsfindingsusing2ofthesesources:instructorcommentsinstudent trainingfoldersandover-the-shoulderobservationsofcrewsintacticalsimulators.In structorcommentsrevealedthatCRMproblemsearlyintrainingmostfrequentlyin volvedecisionmakingandcommunicationamongcrewmembers.Over-the-shoulder observationsofexperiencedcrewsshowedhighcorrelationsbetweenindependentrat ingsofCRMproficiencyandmissionperformance.Themosteffectivecrewsexhibited suchcharacteristicCRMbehaviorsasthepresenceofasingleleaderandwillingnessto changeplansbasedonchangingmissionsituations.Thearticleclosesbydescribing how these studydatacanbe usedto restructureCRMtrainingintoa setofbehav ior-basedobjectivesthatwillenablemeaningfulevaluationofitseffectivenessinim provingtheperformancelevelsofallstudentcrews. RequestsforreprintsshouldbesenttoRobertT.Nullmeyer,AirForceResearchLaboratory,6030 SouthKentStreet,Mesa,AZ 85212-6061.E-mail:[email protected] Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 2. REPORT TYPE 3. DATES COVERED 01 DEC 2003 Journal Article 01-01-2001 to 30-11-2003 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER The Importance of Crew Resource Management Behaviors in 5b. GRANT NUMBER Mission Performance: Implications for Training Evaluation 5c. PROGRAM ELEMENT NUMBER 62205F 6. AUTHOR(S) 5d. PROJECT NUMBER Robert Nullmeyer; V. Spiker 1123 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER Air Force Research Laboratory,Aircrew Training Research AFRL/HEA Division,6030 South Kent Street,Mesa,AZ,85212-6061 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) Air Force Research Laboratory, Warfighter Training Research AFRL/RH; AFRL/RHA Division, 6030 South Kent Street, Mesa, AZ, 85212-6061 11. SPONSOR/MONITOR’S REPORT NUMBER(S) AFRL-RH-AZ-JA-2003-0002 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES Published in Military Psychology, 2003, 15(1), 77-96 14. ABSTRACT Cockpit/crew resource management (CRM) training within the military has grown rapidly despite the paucity of empirical data linking CRM to mission performance. CRM training objectives (and course content) are often too vague to allow meaningful training evaluation within the context of traditional transfer-of-training paradigms. A multimeasure methodology that exploits all sources of archival and observational data within a training organization has the potential to advance training evaluation, particularly for crew-based skills such as CRM. This article discusses a variety of CRM data sources and presents findings using two of these sources: instructor comments in student training folders and over-the-shoulder observations of crews in tactical simulators. Instructor comments revealed that CRM problems early in training most frequently involve decision making and communication among crew members. Over-the-shoulder observations of experienced crews showed high correlations between independent ratings of CRM proficiency and mission performance. The most effective crews exhibited such characteristic CRM behaviors as the presence of a single leader and willingness to change plans based on changing mission situations. The article closes by describing how these study data can be used to restructure CRM training into a set of behavior-based objectives that will enable meaningful evaluation of its effectiveness in improving the performance levels of all student crews. 15. SUBJECT TERMS Crew resource management; Mission performance; Training evaluation; Cockpit resource management; Transfer of training; CRM; Decision making; Crewmember communication; Crew-based skills; Flight simulators 16. SECURITY CLASSIFICATION OF: 17. LIMITATION 18. NUMBER 19a. NAME OF RESPONSIBLE OF ABSTRACT OF PAGES PERSON a. REPORT b. ABSTRACT c. THIS PAGE Public 20 unclassified unclassified unclassified Release 78 NULLMEYERANDSPIKER Human erroris frequently linked with aviation accidents and incidents (Kayten, 1993). Infact, HelmreichandFoushee (1993) reportedthataircrewactions were causalfactors inmorethan70%ofhull-lossaccidentsintheworldwidecommer cialjetfleetfrom 1959through 1989.InRuffellSmith's(1979)landmarksimula torstudyoncockpitworkload,thefactorsthatmostdifferentiatedeffectivecrews from weaker ones were leadership, decision making,·and resource management ratherthanmoretechnicallyorientedskills. Inresponsetofindings suchasthese, the National Aeronautics and Space Administration (NASA) sponsored a work shop, Resource Management on the Flightdeck, in 1979 (Cooper, White, & Lauber, 1980),whichiscommonlyviewedastheoriginofformalcockpit/crewre sourcemanagement(CRM)training.NASAandtheMilitaryAirliftCommandco sponsoredafollow-up conferenceonCRMtrainingseveralyearslater(Orlady& Foushee, 1987). This workshop marked the expansion ofCRM training into the militaryservices.EachofthemilitaryservicesquicklyaddedCRMtrainingtose lectedprograms,andwithinafew years,CRMinstructionbecamemandatoryfor allmilitaryaviators. This rapid growth ofCRM training throughoutaviationoccurreddespite sur prisinglylittleempiricalevidencelinkingthistrainingtoimprovedmissionperfor mance.TheCRMvalidationdatathatdoexistaredominatedbytraineejudgments about its value (e.g., lIgen, 1999; Salas, Fowlkes, Stout, Milanovich, & Prince, 1999).Despiteits shorthistory, five distinctgenerationsofCRMtrainingcanal readybedocumentedinthecommercialairlines(Helmreich,Merritt,& Wilhelm, 1999),witheachgenerationrepresentingasubstantialshiftintrainingphilosophy andcontent. The military's approach to measuring the effectiveness oftraining interven tions-be they a revised program ofinstruction, improved courseware, or new training device-has traditionally followed Kirkpatrick's (1996) four-stage model,depictedinthemiddleofFigure 1(Bell&Waag, 1998;SalasetaI., 1999). Althougheffectivenessis ultimatelyequatedwiththe"contributionoftraining to therequiredavailabilityofcombatpower"(Stage4; Bell& Waag, 1998,p. 234), the vastmajorityofeffortisfocused ontheearlierstages. Theseare the trainees' perceived value of the training, the degree to which the to-be-trained knowl edge-skills-attitudes(KSAs)areactuallylearned,andtoamuchlesserextent,the availabilityoftargetedKSAsforuseonthejob.Fromaresearchstandpoint,Stage 3isthedefiningfeatureofeffectiveness,asdemonstratedbyapositivetransferto thejob(combat)environment. Correspondingtoeachstageisatraditional meth odologyforevaluatingtraining,asshownintheleftpartofFigure1.Theseinclude a survey oftrainees' attitudes toward their training, a pre- and postexperimental comparisonoftrainees'performanceontheKSAs,afulltransfer-of-trainingstudy conductedin thejobenvironment, and a cost-effectiveness analysis that demon stratesthedesiredorganizationalproductivityorperformanceimpact(e.g.,combat readiness)atanacceptablecost. BEHAVIORSANDTRAININGEVALUATION 79 Performance Validity TrainingRecords instructorcomments Illstrllt:torratings GainingUnitSurveys I I Accidentreports "Naturalexperiments" Other Traditional Kirkpatrick's4-stagemodel Measurement Measurement "Opportunities" Methods Wasthetraining Wasthetraining contentappropriate effectiveinaiding forthejobandwas jobperformance? itdeliveredeffectively? FIGURE1 Frameworkformeasuringtrainingeffectiveness. Thoughinfluential,themodel'slimitationsasacomprehensivetheoryofeffec tiveness measurementhavebeennotedbyanumberofresearchers. Forexample, Bell and Waag (1998) acknowledged the "brute force" aspectofthe model's se quential stages and cited the need to distinguish between training processes and performance,particularlyinStage2.Salasetal.(1999)usedthemodel'shierarchy primarilyas a theoretical "driver" for collectingdatawithin amultimeasurement frameworkontrainees'reactions,attitudes,knowledge, andbehavior,withnose quentialprioritiesimposedonthemeasures. Bell andWaag (1998) and Thurman and Dunlap (1999) reported apaucity of transferstudiescorrespondingtoStage3ofKirkpatrick's(1996)model.Thisdearth is tiedtocost, methodologicalandoperationalconstraintsassociatedwithtesting newlytrained(andnowvaluable)personnelinactualornear-realisticmissionsce narios,solicitingandpreservingasuitablecontrolgroup,andobtainingtheneces- 80 NULLMEYERANDSPIKER sarypersonnelstabilitywithinaturbulentenvironment(Thurman&Dunlap,1999). Barrierstoeffectivetransferaremany, withperhapsthemostsignificantbeingno cleardefinitionofwhatisbeingtrained.TheKirkpatrickmodelwaspromulgatedin an era when procedurally oriented KSAs from prescribed task to training lists (TTLs)formedthecontentfortraining(Bills&Wood,1999).Withtheproliferation ofsimulationdevicesgearedtowardfacilitatingcognitive-basedprocessessuchas decisionmakingandsituationawareness,thetask-specificbasisoftrainingcontent hasbeenaltered.ThishasbeenparticularlytrueinCRM,where---dependingonthe researcher or the tactical domain-it has been referred to as a training program (lIgen,1999),instructionalstrategy(SalasetaI., 1999),tacitknowledge,metaskill, orenterprise(Bills& Wood, 1999). Indiscussing trainingprogramevaluation, Goldstein(1987)distinguishedbe tween gauging the "training validity" ofthe intervention, that is, determining if trainingcontentisappropriateandeffectivelydelivered, and gaugingits "perfor mancevalidity,"thatis,theextenttowhichtheinterventionfacilitatessubsequent jobperformance. Asdenotedbythe arrows intheright-handportionofFigure 1, thesetwoaspectsoftrainingeffectivenesscanbeassessedinparallel,andtheyex tendacrossallstagesofKirkpatrick's (1996) model. Giventhelimitationsofthestagemodeldiscussedpreviouslyandtheattendant difficultiesinconductingtransfer-of-trainingexperiments,webelievethatamore viableframeworkformeasuringtrainingeffectivenesswillemphasizecapitalizing onmultiplemeasurementopportunities,muchasisdoneintheprogramevaluation area(Cook&Campbell, 1979).Consideringthetypicalmilitarytrainingenviron ment,thereareanumberofalternativedatasourcesthatbeardirectlyontheeffec tiveness of a unit's training. Five of these opportunities are indicated in the right-hand portionofFigure 1.Although theseare orderedaccordingtotheirap proximateplacementwithinKirkpatrick's(1996)stages,eachmethodcanapplyto eithertrainingvalidityorprogramvalidity;consequently,wehavenotattemptedto specifyformal links withanyofthestages. Naturalistic observations ofmilitary aviators during simulator missions have revealedstrongempiricallinksbetweenthequalityofcrewinteractions(process) andmissionperformance(outcome).Povenmire,Rockway,Buenecke,andPatton (1989) observed seven B-52 crews execute a tactically realistic scenario in a high-fidelity simulatorand reported a statistically significantrank ordercorrela = tionbetweenmissionperformanceandCRM(r .83).Practicinginquiryandadvo cacy,avoidingdistractions,distributing workload,andresolvingconflictsemerged asfactors thatweresignificantlycorrelatedwithoverallcrewcoordination. Thornton,Kaempf, Zeller,andMcAnulty(1992)observed 19pairsofaviators astheyflew acombat-orientedmissioninanadvancedUH-60BlackHawksimu lator. Aircrewcoordinationwasdefinedintermsoftherate,pattern, content, and qualityofinteractionsalong 13functionalcategories(inquiry,command,declara tive, etc.). Mission effectiveness was defined in terms of navigation accuracy, BEHAVIORSANDTRAININGEVALUATION 81 threatavoidance,andperformanceofanonprecisionapproach. Patternsandtypes ofcommunicationwererelatedtooutcome-basedindexesofmissionperformance, butrateofcommunicationwasnot. Brannick,Prince,Prince,andSalas(1995)alsoreportedastrongempiricalrela tion between team coordination and performance in a Navy study in which 52 two-personcrewsflew nontacticalscenariosinalow-fidelity,tabletopT-44flight trainer.Sixteamcoordinationdimensionswererated: assertiveness,decisionmak ing, adaptability, situationalawareness, leadership, andcommunication. Arating scaleranging from 1(unacceptable) to 5 (excellent) was developedfor each di mension,andbehaviorsrepresentingtheperformanceexpectedwereprovidedfor the scale points. Forexample, assertive behaviors associated with higherratings includedquestioningsomedirectionsfromairtrafficcontrolandadmittingconfu sionaboutanassignedaltitude. Allsixprocessdimensionswerepositivelycorre latedwithperformance, withcorrelationsrangingfrom .43 to.69. TheremainingthreemethodsinFigure 1illustratehoweffectivelytraininghas preparedstudentsforperformingtheirdesignatedrolesin theiroperationalunits. Thesemeasuresincludereviewingthesurveyscompletedbytrainingunitsupervi sorsconcerninghowwelltraineescouldsupporttheunitwithoutadditionaltraining, trackingandtrendingaccidentreportstoidentifylinkswithcommonsetsofprior trainingexperiences,andexaminingtheconductof"naturalexperiments"thatarise due to operationalcircumstances. An exampleofnatural experimentsinvolveda comparisonofmissionreadinessduringtheGulfWarforgroupsofaviatorswhoei therdidordidnotreceivefull-missionsimulatortraining(thesimulatorwasunavail ablefortechnicalreasonsforacohortofaviators)priortotheirarrivalintheMiddle East(Rakip, Kelly, Appler, &Riley, 1993).Thatsimulator-trainedMH-53JPave Lowoperatorsweremoreabletoconductmissionsimmediatelyonarrivalinacoun trycomparedwiththeirnonsimulator-trainedcounterpartswasconsideredtobeevi dencesupportingtheeffectivenessoftheMH-53Jflightsimulator. Ultimately, CRMtrainingeffectivenessevaluationmustaddress the degree to which actual training needs are satisfied. Competing definitions and poorly de finedtrainingobjectiveshaveundoubtedlycontributedtothecurrentdearthofem pirical CRM training studies. The studies reported here were designed to define CRMinconcrete,observabletermsforMC-l30Pcrewtrainingatthe58thSpecial OperationsWing,KirtlandAirForceBase,NewMexico.Ourobjectiveswereboth toestablishthecontentofCRMtrainingandtodevelopmeasuresfor subsequent trainingevaluation. The top two boxes on the rightportion ofFigure 1highlight the methods de scribedin this article. Althougheachisdescribed inmoredetail inthefollowing sections, analysisoftraining records andnaturalistic observations were methods ofchoicebecausetheyshedlightontherelevanceofexistingCRMcoursecontent forMC-l30PcrewsandtherelationofputativeCRMprocessestomissionperfor mance. By contentanalyzing the instructors' comments in student grade folders 82 NULLMEYERANDSPIKER forsimulatorandflight-linemissions,wemayascertaintheCRMproblemsexpe riencedbystudentsintraining.Byaugmentingtheseanalyseswithin-depth,natu ralistic observations ofcrews during simulator and flight-line training, we may constructabetterpicture ofwhichCRM behaviors contributeto successfulmis sionperformanceand, ofthese, whicharepresentlytaughtduringCRMtraining. STUDY 1: USING STUDENTGRADE FOLDERS TO ASSESS CRM PROFICIENCY Overview Analysis ofarchival training records, such as student grade folders, provides at least three advantages as a source of training effectiveness data. First, training records are routinely maintainedby training wings and do notrequire additional effort to collect. Second, they are populated by input from instructors who are experts in that student's crew position and who have acquired considerable fa miliarity witheachstudent'scapabilities. Third, they areoftenquantitativeinna ture and hence amenable to statistical analysis. MC-130P StudentGrade Folders Grade folders are maintained for eachMC-l30P student as he or she completes missionqualification(MQ)training.Foreachacademic,simulator,andflight-line = = trainingsession,theinstructorassignsalettergrade(P proficiencyadvance, E exceptional, S = satisfactory, T= needs training, U = unsatisfactory, I = incom plete)inthestudent'sForm15,AircrewTrainingRecord.Tobeausefulsourceof proficiencydata,ameasuremustvaryacrossstudents;otherwise,onecannotinfer theimpactofprogramvariablesortraininginterventions.However,inouranalysis ofMC-l30Pstudentrecords,morethan98%ofstudentgradesintheForm15were assignedan"S." MQtraining progresses through seven simulatortrainingevents: aconversion mission,twoday-tacticalmissions,andfournight-tacticalmissions.Eachblockof simulatortrainingisfollowedbyflight-lineinstructionofcorrespondingcomplex ity. To keep track ofthis progression, MC-130P instructors fill out a Form 14, Aircrew Training Progress Record, aftereach simulator and flight-line mission. This preprintedform provides asetofrequiredproficiency levels (RPLs) for the trainingeventsassociatedwiththatmissionprofile.Eventsaretaskbased,suchas airdropchecklist, simulatedenginefailure, nightvisiondeviceoperations,andso forth. Performanceand knowledgearegradedon a4-pointscale, ranging from 1 (extremely limited) to 4 (highly proficient). As the student progresses through training, theRPLs fortheeventsineachtrainingprofilebecomemorestringent. BEHAVIORSANDTRAININGEVALUATION 83 Instructors cross offRPLs onthepreprintedform aseachtrainingeventis ac complished.IfastudentexceedsorfailstomeetanRPL,theinstructormustwrite in the actuallevelofperformance orknowledge demonstrated; however, instruc torsrarelynotedeviationsfromtheRPLbecausetheyarebusyanddonotwishto havestudentdeficienciesnotedinthepermanentrecord.Consequently,analysisof theRPLdata,suchasaggregatingthenumberofsub-RPLevents,willnotproduce usefulmeasuresofproficiencybecausemostentriesaretheunannotatedRPLs. Inadditiontogrades,instructorswritecommentsfollowingeachsimulatorand flight session on Form 13, Training Comments Record. The comments are un structuredandarenotnecessarilytiedtotherequireditemscoveredintheForm14. These comments provide information that is potentially a rich source ofprofi ciencydataasinstructorsarefreetoexpresstheirreservationsregardingastudent, knowing that theirremarks are not reflected in the student's recorded grade. In structors may also laud exemplary performance and can go over the comments withthestudentafteratrainingmission, usingitas ateachingordebriefingaid. In analyses of MC-130P crew training records for each position, we deter mined that instructors make extensive comments that, when aggregated across missions, can be reliably classified into positive and negative cases. Moreover, the comments can be sorted into functional categories characteristic of each crew position (e.g., crew coordination, equipmentknowledge). This analysis re vealed that instructor comments are quite specific (e.g., "missed several radio calls," "must keep checklist flowing to ensure proper crew responses," "need more positive continuous guidance to pilot") and yield valuable insights con cerning areas where student proficiency is strong or weak. To the extent that thesecommentsarerecordedroutinelyandcomprehensively,theycanbecontent analyzed, aggregated, and quantified to yield data-based assessments ofstudent proficiency and, ultimately, training effectiveness. Method Basedonthepreliminaryanalyses notedpreviously, we usedthe instructorcom mentsfrom studentgradefolders toassess theeffectivenessofthe wing'scurrent CRM training. We reviewed arepresentative sample of20records from the five crewspecialtiesintheMC-130P:pilots,navigators,flightengineer(FE),commu nicationsystemsoperator(CSO),andloadmaster(LM).Wethenenlistedthehelp oftwosubjectmatterexperts(SMEs)toreviewtherecordsaccordingtothemeth odology outlinedin thefollowing paragraph. Both wereexperiencedinstructors, oneinairbornecommandandcontrolaircraftandtheotherinspecialoperations, fixed-wing aircraft. Working independently, the SMEs reviewed each training record and high lightedallinstructorcommentsrelevanttoCRM.Thesecommentswerethenpara phrasedandtranscribedontoafour-page, structuredTrainingRecordEvaluation 84 NULLMEYERANDSPIKER Worksheet.TheworksheetwasorganizedaroundthesixCRMareascurrentlycov eredinAirForceInstruction11-290(Cockpit/CrewResource, 1998).TheseCRM areas are mission planning and debrief, task management, situation awareness, crew coordination, communication, and risk management decision making. We addedaseventh area, tacticsemployment,toaddressthecombat-intensiveopera tions requiredofthis aircraft. Eachparaphrasedcommentwasplacedin therele vantCRMcategoryandthenratedona5-pointscale,rangingfrom 1(significantly belowexpectations)to5(exceptional),withamidpointof3(levelexpectedforthis leveloftraining). Oncecomments were transcribed andrated, a summaryrating forthatCRMcategory was assigned. Finally, an overallproficiencyrating, again ona 1to5scale,wasassignedforeachstudent. Results Interraterreliability. Thefirstgoaloftheanalysiswastodetermineifthetwo SMEswereconsistentintheirratingassignments.Thecorrelationbetweenthetwo setsof20overallproficiencyratings was .81, within therecommendedrange for acceptable interrater reliability (Cronbach, 1990). Looking at the ratings them selves, wefound that the two raters producedidenticalratings for 16ofthe stu dentsanddifferedbyonlyonepointfortheotherfourstudents.Wethusconclude thatSMEscanreliablyassignproficiencyratings basedonthecommentsthatin structorsplaceinstudentgradefolders. Inferringstudentproficiency. Havingestablishedthereliabilityoftherating process,wehadoneoftheSMEsextendhisreviewtoincludeallMC-130Pstudent recordsfrom1998,atotalof87records.Wethenexaminedtheabsolutevaluesofthe ratingstodetermineiftherewassufficientvariationacrossstudentstoassessCRM training.Ofthe87records,morethanonethirdreceivedaratingotherthan3,amuch higherpercentagethanwasevidentintheinstructorgradesontheForm 14s. Even greater sensitivity is seen when the comment-based proficiency ratings arebrokendownbyCRMcategory,asshowninFigure2.Computingtheaverage ratingvariabilitywithineachcategory,wedeterminedthatascaledifferenceof.14 canbeconsideredstatisticallymeaningful(Hays, 1973).Onthatbasis,weseethat mission preparation and crew coordination received substantially above-average ratings (3.2-3.3), with decision making and communications (2.8-2.9) signifi cantlybelowaverage. Qualitativeanalysis. Having identified the CRM categories that stand out statistically,wecanexaminetheassociatedinstructorcommentstopinpointareas wherepresentCRMbehaviorsarestrongandweak. Inperformingthisanalysis,it should be noted that such comments have two aspects: evaluative and directive. Theevaluativecomponentisusedtogaugestudentproficiencyinthecommented areaandistypicallyrepresentedbyanadjective,suchas"good"missionplanning, BEHAVIORSANDTRAININGEVALUATION 85 5 ~ 4 - 3 - - avg=3.1 Avg ~ Profi- 2 l IF ciency 1 I-- Jli Rating 1'\. ii ~ MP CC SA TM Com DM TE CRM FIGURE2 AveragestudentproficiencybyCRMcategory.CRM=cockpit/crewresource = = = management; MP mission planningand debrief; CC crewcoordination; SA situation = = = awareness;TM taskmanagement;COM communication;DM riskmanagementdecision making;TE=tacticsemployment. "excellent"missionbriefing,or"weak"situationawareness.Thedirectiveaspects ofeachcommentletus extractthe specific crew behaviors that wereeitherdefi cientorlaudable.Bydirective,wemeansuchcommentsas"slowtopreparebrief," "needstothinkfurtheraheadoftheaircraft,"and"missedradiocalls."Thesecom ments areusuallygiven tothe studentas verbalfeedback duringthetrainingses siontopromoteimmediateimprovementortoreinforcesomeessentialskill.Over the long term, the content of these comments can be collected, analyzed, and foldedbackintoanimprovedtrainingcurriculumas asetoftargetbehaviors. Toillustrate, wereviewedthecompiledsetofinstructorcommentsin mission preparationand crew coordinationto ascertainwhy these categories stoodoutas positiveinstancesofCRM.Forthemostpart,wesawthatinstructorswerecompli mentingstudentsonsuchaspectsas"thorough"planning,"concise"briefings,and "goodbackingupcrewmembers."Theseareasareemphasizedinthepresenttrain ingandseemtohavebeeninternalizedbythestudents. Yettherewerealsonega tivecommentsthatindicatedareasinneedofimprovement,suchasnavigationand leadership.Exampleshereincludetheneedforprovidinggreaterannotationofsig nificantterrainfeatures onmaps,discussingmoreobstaclesinthelow-levelbrief, andtakingfirmercontrolofthecrew. Turning to the weaker CRM areas, decision making exhibited a wide assort mentofdeficiencies thatprimarily involved pilots and navigators. A majordefi ciencyentailedslowreactions toconditionsrequiring morerapidjudgment,such as initiatingemergencyprocedures,respondingto lossofengine,turning tofinal approach, joining up during aerial refueling, correcting the flight profile, and breakingoffformation duringthe onsetofinstrumentmeteorologicalconditions. Ahostofcommunicationproblemareaswerealsoexposedforallcrewmembers.
The list of books you might like
