Human-in-the-Loop Assessment of Alternative Clearances in Interval Management Arrival Operations

302 Pages·2016·6.03 MB·English

https://ntrs.nasa.gov/search.jsp?R=20170000688 2018-11-17T12:40:29+00:00Z
NASA/TP–2016-219362

Human-in-the-Loop Assessment of Alternative
Clearances in Interval Management Arrival
Operations

Brian T. Baxley, Sara R. Wilson, Kurt A. Swieringa, William C. Johnson, and Roy D. Roper
Langley Research Center, Hampton, Virginia
Clay E. Hubbs, Paul A. Goess, and Richard F. Shay
National Institute of Aerospace, Hampton, Virginia

December 2016
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the • CONFERENCE PUBLICATION.
advancement of aeronautics and space science. The Collected papers from scientific and technical
NASA scientific and technical information (STI) conferences, symposia, seminars, or other
program plays a key part in helping NASA maintain meetings sponsored or co-sponsored by NASA.
this important role.
• SPECIAL PUBLICATION. Scientific,
The NASA STI program operates under the auspices technical, or historical information from NASA
of the Agency Chief Information Officer. It collects, programs, projects, and missions, often
organizes, provides for archiving, and disseminates concerned with subjects having substantial
NASA’s STI. The NASA STI program provides access public interest.
to the NTRS Registered and its public interface, the
NASA Technical Reports Server, thus providing one • TECHNICAL TRANSLATION.
of the largest collections of aeronautical and space English-language translations of foreign
science STI in the world. Results are published in both scientific and technical material pertinent to
non-NASA channels and by NASA in the NASA STI NASA’s mission.
Report Series, which includes the following report
types: Specialized services also include organizing
and publishing research results, distributing
• TECHNICAL PUBLICATION. Reports of specialized research announcements and feeds,
completed research or a major significant phase of providing information desk and personal search
research that present the results of NASA support, and enabling data exchange services.
Programs and include extensive data or theoretical
analysis. Includes compilations of significant For more information about the NASA STI program,
scientific and technical data and information see the following:
deemed to be of continuing reference value.
NASA counter-part of peer-reviewed formal • Access the NASA STI program home page at
professional papers but has less stringent http://www.sti.nasa.gov
limitations on manuscript length and extent of
graphic presentations. • E-mail your question to [email protected]

• TECHNICAL MEMORANDUM. • Phone the NASA STI Information Desk at
Scientific and technical findings that are 757-864-9658
preliminary or of specialized interest,
e.g., quick release reports, working • Write to:
papers, and bibliographies that contain minimal NASA STI Information Desk
annotation. Does not contain extensive analysis. Mail Stop 148
NASA Langley Research Center
• CONTRACTOR REPORT. Scientific and Hampton, VA 23681-2199
technical findings by NASA-sponsored
contractors and grantees.
NASA/TP–2016-219362

Human-in-the-Loop Assessment of Alternative
Clearances in Interval Management Arrival
Operations

Brian T. Baxley, Sara R. Wilson, Kurt A. Swieringa, William C. Johnson, and Roy D. Roper
Langley Research Center, Hampton, Virginia
Clay E. Hubbs, Paul A. Goess, and Richard F. Shay
National Institute of Aerospace, Hampton, Virginia

National Aeronautics and
Space Administration

Langley Research Center
Hampton, Virginia 23681-2199

December 2016
Foreword
This NASA Technical Publication is a comprehensive report about the Interval Management Alternative Clearances
(IMAC) human-in-the-loop experiment conducted at the NASA Langley Research Center in July and August of
2015. The authors gratefully acknowledge the many individuals who made important contributions to the research,
and without whom this very challenging and complex experiment would not have occurred. They include: Terry
Abbott, Brenda Andrews, Janice Bayer, Tom Britton, Joel Brockman, John Bunnell, Tony Busquets, Mike Clark,
Jim Davis, Mike Day, Dylan Drake, Peter Franklin, Dennis Frasca, Michael Harper, Stella Harrison, Fred Hibbard,
Missy Hill, Brian Hutchinson, Richard Jessop, Regina Johns, Lon Kelly, Joe King, Troy Landers, Kara Latorella,
Ron Maddox, Carolyn Malloy, Doug Mielke, Brendan Moeller, Sharon Otero, Mike Palmer, Benjamin Remy,
Darrell Sacra, Ed Scearce, Omar Scott, Jim Smail, Jim Sturdy, Erin Thomas, Jon Welters, Dave West, and Chris
Wyatt.

Many key contributions were also provided by NASA Ames Research Center, to include the TBFM and CMS
software. The authors are particularly indebted to Todd Callantine who provided his expertise in air traffic control to
help design and create the IMAC experiment. The authors are also very appreciative of the tremendous support from
Matt Gregory, Joey Mercer, and Thomas Prevot.

The authors benefited greatly from the Project and Program leadership team, and thank the following individuals for
their support: Sherilyn Brown, John Cavolowsky, Melinda Gratteau, Mike Madson, Leighton Quon, John Robinson,
and Jane Thipphavong.

MITRE and the FAA Surveillance and Broadcast Services Program Office also provided input to the experiment,
and the authors thank Randy Bone, Mike Castle, Ian Levitt, and Wes Stoops for their contributions.

And finally, organizing and running the experiment would not have been possible without the assistance of our
interns, and the authors gratefully thank Maryam Amer, Andrew Arkebauer, Stuart Cooke, Rebecca Lidvall, Sarah
Linthicum, Joshua Milot, and Ryan Petit for their great work.

The use of trademarks or names of manufacturers in this report is for accurate reporting and does not constitute an
official endorsement, either expressed or implied, of such products or manufacturers by the National Aeronautics and
Space Administration.

Available from:

NASA STI Program / Mail Stop 148
NASA Langley Research Center
Hampton, VA 23681-2199
Fax: [phone]
Table of Contents
Executive Summary ..........................................................................................................................1
1 Introduction ................................................................................................................................3
1.1 Background ..........................................................................................................................3
1.2 Current Operational Need ....................................................................................................4
1.3 ATD-1 and IM Goals ...........................................................................................................4
1.4 Previous ATD-1 Research ...................................................................................................4
1.5 Terminology .........................................................................................................................5
2 ConOps, Technologies, Clearances, and Procedures .................................................................7
2.1 ATD-1 Concept of Operation ..............................................................................................7
2.2 The ATD-1 Technologies ....................................................................................................8
2.2.1 Traffic Management Advisor with Terminal Metering (TMA-TM) ...................... 8
2.2.2 Controller Managed Spacing (CMS) .................................................................... 10
2.2.3 Interval Management (IM) .................................................................................... 13
2.3 IM Operation Types ...........................................................................................................16
2.3.1 CROSS .................................................................................................................. 17
2.3.2 CAPTURE ............................................................................................................ 17
2.3.3 MAINTAIN .......................................................................................................... 18
2.4 Controller and Pilot Procedures .........................................................................................18
2.4.1 Controller Procedures ........................................................................................... 19
2.4.2 Flight Crew Procedures......................................................................................... 20
3 Experiment Design...................................................................................................................22
3.1 Objectives and Scope .........................................................................................................22
3.2 Assumptions .......................................................................................................................22
3.3 Performance Goals and Hypotheses ..................................................................................22
3.4 Test Matrix .........................................................................................................................24
3.5 Independent Variable .........................................................................................................24
3.6 Dependent Measures ..........................................................................................................25
3.6.1 IM Algorithm Performance................................................................................... 25
3.6.2 Air Traffic System Performance ........................................................................... 25
3.6.3 Controller Objective Performance ........................................................................ 26
3.6.4 Flight Crew Objective Performance ..................................................................... 27
i
3.6.5 Controller Subjective Assessments ....................................................................... 27
3.6.6 Flight Crew Subjective Assessments .................................................................... 28
3.7 Case Studies .......................................................................................................................28
3.8 Environment and scenario description ...............................................................................29
3.8.1 KDEN Instrument Arrival and Approach Procedures .......................................... 29
3.8.2 KDEN Airport and Landing Runways .................................................................. 32
3.8.3 Truth and Forecast Wind ...................................................................................... 33
3.8.4 Training Scenarios ................................................................................................ 34
3.8.5 Data Collection Scenarios ..................................................................................... 35
3.8.6 Example of Aircraft Location at Scenario Start .................................................... 36
3.9 Controller and Pilot Participants ........................................................................................37
3.9.1 Qualification and Currency Requirements ............................................................ 37
3.9.2 Protocol for Training and Data Collection ............................................................ 38
4 Facilities and Software .............................................................................................................40
4.1 Air Traffic Operations Laboratory (ATOL) ......................................................................40
4.1.1 MACS Controller Stations .................................................................................... 41
4.1.2 Time Based Flow Management (TBFM) .............................................................. 43
4.1.3 Aircraft Simulation for Traffic Operations Research (ASTOR) ........................... 44
4.1.4 MACS Pseudo-Pilot Station ................................................................................. 45
4.2 Development and Test Simulator (DTS) ...........................................................................46
4.3 Integration Flight Deck (IFD) ............................................................................................47
5 IMAC Experiment Results .......................................................................................................49
5.1 Caveat to Experiment Results ............................................................................................49
5.2 Preconditioning of Data .....................................................................................................50
5.3 IM Algorithm Performance Results ...................................................................................50
5.3.1 IM Spacing Error .................................................................................................. 50
5.3.2 Frequency of IM Speed Changes during IM Operations ...................................... 51
5.3.3 Number of IM Speed Reversals per IM Operation ............................................... 54
5.4 Air Traffic System Performance Results ...........................................................................55
5.4.1 Inter-Arrival Spacing Error at the Final Approach Fix ......................................... 55
5.4.2 PBN Success Rate ................................................................................................. 57
5.4.3 Slot Marker Deviation in the TRACON ............................................................... 58
5.5 Controller Objective Performance Results ........................................................................59
5.5.1 Percentage of Controller-Interrupted IM Operations ............................................ 59
ii
5.5.2 Schedule Deviation ............................................................................................... 59
5.6 Controller Subjective Assessment Responses ...................................................................62
5.6.1 Controller Acceptability of ATD-1 Operations .................................................... 62
5.6.2 Controller Assessment of Workload of ATD-1 Operations ................................. 63
5.6.3 Controller Comments about ATD-1 ConOps and IM Operations ........................ 66
5.7 Flight Crew Objective Performance Results ......................................................................67
5.7.1 Percentage of Flight Crew Interrupted IM Operations ......................................... 67
5.7.2 Flight Crew Reaction Time to IM Speed Command Change ............................... 67
5.7.3 Flight Crew Conformance to the IM Speed .......................................................... 68
5.7.4 Missed Altitude Constraints .................................................................................. 69
5.7.5 Unstable Approaches ............................................................................................ 72
5.8 Flight Crew Subjective Assessment Responses .................................................................73
5.8.1 Flight Crew Acceptability ..................................................................................... 73
5.8.2 Flight Crew Workload of IM Operations .............................................................. 80
5.8.3 Flight Crew Ratings and Comments about Situation Awareness ......................... 81
5.8.4 Flight Crew Comments about ATD-1 ConOps and IM Operations ..................... 82
5.8.5 Flight Crew Comments about IM Interface and Displays .................................... 87
5.9 Case Study Results .............................................................................................................89
5.9.1 Loss of Separation between Aircraft ..................................................................... 89
5.9.2 Undesirable Propagation of Speed Changes ......................................................... 92
5.9.3 Desirable IM Spacing Algorithm Performance .................................................... 95
5.9.4 Variations to Normal IM Clearances and Operations ........................................... 97
5.9.5 Impact to ASTAR When Target Aircraft not at Published Airspeed ................... 98
6 Conclusions and Lessons Learned .........................................................................................105
6.1 General .............................................................................................................................105
6.2 Meeting Performance Goals and Hypotheses ..................................................................105
6.3 ATD-1 ConOps, Procedures, and Phraseology................................................................106
6.4 IM Cockpit Displays ........................................................................................................107
6.5 IM Spacing Software .......................................................................................................109
6.6 Lessons Learned ...............................................................................................................110
7 Recommendations for Future Research .................................................................................113
References . ..............................................................................................................................115
Appendix A: IM Software ...........................................................................................................117
A.1 IM Software Components ................................................................................................117
iii
A.2 ASTAR Algorithm ...........................................................................................................117
A.3 Speed Control Laws .........................................................................................................120
A.4 ASTAR Logic Wrapper ...................................................................................................123
A.4.1 IM States ............................................................................................................. 123
A.4.2 Transition between IM States ............................................................................. 124
A.4.3 Alert Levels ......................................................................................................... 129
A.4.4 IM Alerting Messages ......................................................................................... 130
Appendix B: IM Displays and Data Entry ...................................................................................132
B.1 EFB Overview .................................................................................................................132
B.2 CGD Overview ................................................................................................................135
B.3 IM Displays by State ........................................................................................................136
B.4 IM Display Sub-Elements ................................................................................................144
B.5 Flight Crew Data Entry into IM Interface ........................................................................148
Appendix C: Procedures and Phraseology for IM Operations .....................................................160
C.1 Pilot Procedures for IM Operations .................................................................................160
C.2 Controller – Pilot Phraseology for IM Operations ...........................................................161
Appendix D: Airlines Operating into Denver International Airport ............................................164
Appendix E: Questionnaires ........................................................................................................165
E.1 Controller Background Questionnaire .............................................................................165
E.2 Pilot Background Questionnaire ......................................................................................166
E.3 Controller Post-Run Questionnaire ..................................................................................167
E.4 Pilot Post-Run Questionnaire ...........................................................................................172
E.5 Controller Post-Experiment Questionnaire ......................................................................179
E.6 Pilot Post-Experiment Questionnaire ...............................................................................189
Appendix F: Questionnaire Results .............................................................................................199
F.1 Controller Background Questionnaire Results ................................................................199
F.2 Pilot Background Questionnaire Results .........................................................................200
F.3 Controller Post-Run Questionnaire ..................................................................................201
F.4 Pilot Post-Run Questionnaire Results ..............................................................................213
F.5 Controller Post-Experiment Questionnaire Results .........................................................238
F.6 Pilot Post-Experiment Questionnaire Results ..................................................................252
Appendix G: Data Removed from Analysis ................................................................................279
Appendix H: Effectiveness of Training .......................................................................................281
Appendix I: Oculometers .............................................................................................................283
iv
List of Figures
Figure 1. Integrated NASA technologies used in the ATD-1 ConOps. .......................................... 8
Figure 2. IM information displayed on the controller’s meter list. ................................................. 9
Figure 3. CMS tools and displays. ................................................................................................ 11
Figure 4. IM status indications on the ARTCC controller workstations. ..................................... 12
Figure 5. IM status indicators on the TRACON controller workstations. .................................... 13
Figure 6. IM Cockpit Interfaces: EFB (left) and CGD (right). ..................................................... 15
Figure 7. Arrivals into KDEN for north-flow landing. ................................................................. 30
Figure 8. Arrivals into KDEN for south-flow landing. ................................................................. 31
Figure 9. KDEN runways used for landing during IMAC. ........................................................... 32
Figure 10. Example of aircraft location at scenario start. ............................................................. 36
Figure 11. Controller and pilot room configuration in ATOL for the IMAC experiment. ........... 40
Figure 12. MACS workstations for controllers. ............................................................................ 41
Figure 13. MACS display for ARTCC controller workstation. .................................................... 42
Figure 14. MACS display for TRACON controller workstation. ................................................. 42
Figure 15. TBFM screen and display. ........................................................................................... 43
Figure 16. Three-monitor ASTOR station for a two-person flight crew. ..................................... 44
Figure 17. MACS workstation for a single pseudo-pilot. ............................................................. 45
Figure 18. The Development and Test Simulator (DTS). ............................................................. 46
Figure 19. The Integration Flight Deck (IFD). ............................................................................. 47
Figure 20. IFD on the cockpit motion base................................................................................... 48
Figure 21. Graphs of IM spacing error at the ABP and PTP. ....................................................... 50
Figure 22. Graph of IM of speed change frequency by scenario type. ......................................... 52
Figure 23. Scatterplot of time between all IM speed changes by distance-to-go. ........................ 53
Figure 24. Histogram of IM speed changes by IM operation type by distance-to-go. ................. 53
Figure 25. Number of IM speed reversals per flight by scenario type. ........................................ 54
Figure 26. Graph of inter-arrival spacing error at Final Approach Fix. ....................................... 55
Figure 27. The latitude and longitude of all aircraft for all four landing runways. ...................... 57
Figure 28. The RMS of the slot marker deviation in the TRACON. ............................................ 58
Figure 29. The schedule deviation at Final Approach Fix. ........................................................... 60
v
Figure 30. The schedule deviation at the TRACON Meter Fix. ................................................... 61
Figure 31. Controller acceptability of ATD-1 operations. ............................................................ 62
Figure 32. Scatter plot of pilot reaction time in seconds by altitude by simulator type. .............. 68
Figure 33. The magnitude of missed altitude constraints by simulator type. ............................... 70
Figure 34. Flight crew acceptability of operations by IM operation type. ................................... 74
Figure 35. Flight crew acceptability of IM procedure by IM operation type. .............................. 76
Figure 36. Flight crew completeness of IM procedure by IM operation type. ............................. 77
Figure 37. Flight crew operational acceptability of IM speeds by IM operation type. ................. 78
Figure 38. Flight crew acceptability of IM speed change frequency by IM operation type. ........ 79
Figure 39. Flight crew acceptability of head down time by IM operation type. ........................... 80
Figure 40. Flight crew perception of workload by IM operation type. ......................................... 81
Figure 41. Flight crew acceptability of use of voice communications by IM operation type. ..... 84
Figure 42. Loss of separation case #2 between IM and Target aircraft. ....................................... 90
Figure 43. Loss of separation case #3 between IM and Target aircraft. ....................................... 91
Figure 44. Non-ideal IM speed reversals of CTD control law during CAPTURE operation. ...... 94
Figure 45. Desired spacing algorithm performance by IM operation type. .................................. 96
Figure 46: Case study of spacing error when Target has a large ground speed deviation. ......... 103
Figure 47: Target aircraft ground speed compensation term. ..................................................... 103
Figure 48. Relationship of IM software, ASTAR algorithm, and speed control laws. ............... 117
Figure 49. Diagram of the TBO speed control law. .................................................................... 121
Figure 50. Diagram of the CTD speed control law. .................................................................... 123
Figure 51. IM state logic flow diagram. ..................................................................................... 124
Figure 52. Legend for the IM Home page on an EFB. ............................................................... 133
Figure 53. Legend for the IM displays on a CGD. ..................................................................... 135
Figure 54. EFB and CGD in the OFF state with no Ownship information. ............................... 136
Figure 55. EFB and CGD in the OFF state with Ownship information. .................................... 137
Figure 56. EFB and CGD in the ARMED state. ......................................................................... 138
Figure 57. EFB and CGD in the AVAILABLE state. ................................................................ 139
Figure 58. EFB and CGD in the PAIRED state. ......................................................................... 140
Figure 59. EFB and CGD in the SUSPENDED-AVAILABLE state. ........................................ 141
Figure 60. EFB and CGD in the SUSPENDED-ARMED state. ................................................ 142
Figure 61. EFB and CGD in the UNABLE state. ....................................................................... 143
Figure 62. EFB and CGD with IM speed in reverse video. ........................................................ 144
vi