NASA Technical Reports Server (NTRS) 20060051774: Application of the H-Mode, a Design and Interaction Concept for Highly Automated Vehicles, to Aircraft

0.46 MB·English

by  NASA Technical Reports Server (NTRS)

APPLICATION OF THE H-MODE, A DESIGN AND INTERACTION CONCEPT
FOR HIGHLY AUTOMATED VEHICLES, TO AIRCRAFT

Kenneth H. Goodrich, NASA Langley Research Center, Hampton VA
Frank O. Flemisch, DLR-Institute of Transportation Systems, Braunschweig Germany
Paul C. Schutte, NASA Langley Research Center, Hampton VA
Ralph A. Williams, Analytical Mechanics Associates, Hampton VA

the beginning of the 21st century, the automation of
Abstract state of the art aircraft can control all its actions
from just after take-off to touchdown and rollout
Driven by increased safety, efficiency, and
during routine operations.
airspace capacity, automation is playing an
increasing role in aircraft operations. As aircraft A large body of experience with highly
become increasingly able to autonomously respond automated airplanes highlights several fundamental
to a range of situations with performance surpassing challenges that limit current system effectiveness
human operators, we are compelled to look for new [1-4]. Briefly summarized, these challenges include
methods that help us understand their use and guide heavy reliance on memorized action sequences,
their design using new forms of automation and workload spikes caused by reformulation of
interaction. We propose a novel design metaphor to operational tasks into subtasks understood by the
aid the conceptualization, design, and operation of automation, stupefying workload lulls,
highly-automated aircraft. Design metaphors complacency, skill attrition, inflexible automation
transfer meaning from common experiences to less support, and sudden, unexpected changes in
familiar applications or functions. A notable automation behavior. Furthermore, unless the basic
example is the “Desktop metaphor” for form of human-machine interaction is improved,
manipulating files on a computer. the adverse effects of these challenges are likely to
be more pronounced as potentially less trained and
This paper describes a metaphor for highly
supervised operators (e.g., single-pilots or remote
automated vehicles known as the H-metaphor and a
operators versus a multi-pilot, commercial flight
specific embodiment of the metaphor known as the
crew) use highly automated vehicles.
H-mode as applied to aircraft. The fundamentals of
the H-metaphor are reviewed followed by an The current state of the art is typified by the
overview of an exploratory usability study flight management and control systems on
investigating human-automation interaction issues contemporary transport aircraft (Figure 1). The
for a simple H-mode implementation. The crew can control the vehicle using three primary
envisioned application of the H-mode concept to
aircraft is then described as are two planned
evaluations.
Introduction
Highly automated vehicles, motivated by
improved performance, efficiency, and safety, are
becoming commonplace in many forms of
transportation. As used here, the term “highly
automated” refers to hardware/software systems
that can perform a majority of the control functions
traditionally performed by a human operator. At
Figure 1. State of the art flight deck
control automation levels corresponding, in general, degree. An example of such an occurrence is the
to manual control, parameter control via an loss of the Learjet carrying Payne Stewart [6].
autopilot/auto-throttle system, or fully coupled
In addition to these conventional, crew
operation through a flight management system
controlled functions, safety and security research
(FMS). The interfaces for the three levels of
activities are currently developing concepts in
control automation are designed as independent
which the automation may autonomously deviate
systems, resulting in three sets of interfaces (i.e.,
from the crew’s instructions in the presence of a
manual control inceptors, mode-control panel, and
significant conflict or hazard [7]. If such
the FMS-Control Display Unit) and system
automation initiated actions occur only during rare,
characteristics that the operators must be proficient
uncertain, time-critical, and potentially hazardous
with, choose, and manage in any given situation.
circumstances, crews are likely to have limited
Control of the lateral path, vertical path and speed
proficiency in predicting and verifying the
may be simultaneously spread across all three
correctness of its operation. This lack of familiarity
interfaces. This situation is complicated by the
increases the likelihood that a crew inappropriately
many sub-modes within the primary automation
disables or overrides the automation or conversely,
levels. These sub-modes are often associated with
delays a needed intervention.
subtle but important changes in system behavior
(e.g., whether or not stall protection is provided) Furthermore, traditional automation interfaces
increasing the likelihood of hazardous operator and associated input parameters (e.g., discrete
confusion. At the highest automation level (i.e., parameter entry via knobs or buttons and alpha-
fully coupled operation via the FMS), commands numeric identifiers via a keyboard or comparable
are typically entered in the form of abstract, alpha- input devices such as voice recognition) are ill-
numeric identifiers rather than a spatially encoded suited to the management of spatially referenced,
form (e.g., physically designating features on a map autonomous behaviors in dynamic situations. The
or in the external scene). This translation from one inability to efficiently communicate simple but
form to another increases the likelihood of situationally referenced actions may force reversion
accidents such as the crash of American Airlines to unaided manual control in many high workload
flight 965 near Cali, Columbia [5] In this event, the situations. While eliminating the possibility of
crew mistakenly entered a waypoint unrelated to human intervention or limiting it severely (e.g., a
their intended route of flight. Having to look up wave-off button) may be seen as solutions to these
and enter alpha-numeric identifiers can also create concerns, for complex systems operating in
significant workload, particularly in complex, uncertain environments, this approach is not
terminal area airspace where workload is often without numerous and as yet unsolved issues [8, 9].
already high. The development of artificial systems with human-
like “common sense” and creative problem solving
When utilizing highly automated modes of
skills in real-world situations remains a “grand
operation, the crew is effectively separated from the
goal” as described by leading researchers such as
physical conduct of the flight. While they still have
Minsky [10]. From a practical perspective, systems
critical responsibilities such as monitoring for
that do not support flexible human interventions are
incipient conflicts or hazards, there is no direct
likely to remain more expensive and less robust
feedback if these functions are not performed.
than systems that do for some time to come.
Without regular involvement in the control of the
vehicle or feedback if more supervisory tasks go How then should we apply the "lessons
unperformed, it is possible for crews to fall below a learned" from late 20th century cockpit automation
safe level of situation awareness, reducing their to the design of future aircraft? How do we balance
ability to mitigate emerging hazards or respond in between exploiting increasingly powerful
time-critical situations. In addition, with no technologies and retaining authority, with clear
required physical interaction, functionally roles between humans and automation? Is there a
incapacitated crews may go undetected until the new way to conceptualize a highly-automated
situation has already deteriorated to an extreme vehicle and its operation; perhaps in the form of a
metaphor? Design metaphors transfer meaning
2
from common experiences to less familiar appointment on time, but you’re not very familiar
applications or functions. A ubiquitous example is with this park and you need to keep referring to
the desktop metaphor that has been successfully your map. The problem is that it is very difficult to
applied to many personal computer operating steer your bike and read a map at the same time, so
systems for manipulating electronic documents. you are forced to stop. As the time for your
Not all aspects of the original need be copied as a appointment draws nearer you keep thinking to
metaphor typically has plasticity such that it can, yourself — Is there a way to move through an
and should be, shaped and adapted as necessary to environment with obstacles without having to stop
support understanding. If the original reference is every time you need to do something else? How
sufficiently intact and familiar to the user, a about a Horse?! Riding a horse (Figure 2), you
metaphor facilitates easy understanding and would be able to read your map and be confident
operation of the new application. Designers and that you would not hit any trees or run into people
engineers usually have to expend a great deal of because horses instinctively avoid obstacles. Using
effort in developing and communicating their intent Haptic (manipulative touch, a combination of
not only to users, but also to others involved in the tactile and kinesthetic cues) feedback through the
development, operation, and training processes. seat of your pants and your reins, you are constantly
This communication can be improved by using a aware of what your horse is doing, even while
‘seed crystal’ such as a suitable metaphor. focusing your attention elsewhere. If the horse is
unsure about where to go, it will slow down, and
What could be an appropriate metaphor for
seek a new obstacle free path while trying to get
automated vehicles? Let’s leave technology for a
you back into the loop. The horse might also be
moment and imagine a situation in everyday life:
aware of how engaged you are and adjust its
behavior. If a dangerous situation suddenly pops
The H-Metaphor: up, it will try to react before it is too late. You can
let your horse choose its path without being out-of-
Imagine you are riding your bicycle through a
the-loop or you can take it on tight rein to reassert a
park with trees and people. You are late for an
more direct command.
appointment so you’re in a hurry. You’re trying to
avoid hitting anything and also get to your Now apply this image to an aircraft.
Imagine flying through complex airspace with other
traffic, weather, and tall obstructions while
comfortably attending to other tasks like navigation,
communication, or visually searching the
surrounding terrain. Through the physical feedback
from your haptic interface, you are constantly aware
of what your airplane is doing. If your airplane
senses a conflict, it will take appropriate action and
you can feel where it’s leading. You can let your
aircraft choose its path without being completely
out-of-the-loop, or you can reassert more direct
control by taking a tighter grip on the stick. The
airplane would be aware of how engaged you are in
the control task and adjust its behavior accordingly.
An extreme example would if the pilot became
incapacitated. In this event, the airplane would
declare an emergency and divert to an appropriate
airport. An H-Metaphor inspired aircraft like this
could make flying significantly safer and more
productive. The metaphor is perhaps best suited to
applications such as single-pilot and Unmanned
Aircraft System (UAS) operations. These
Figure 2. Horse as an autonomous vehicle
archetype
3
applications have a strong incentive for adopting to perform immediate trajectory control tasks; the
advanced technologies and are not well served by intelligence of its interaction; and its ability to
simple extrapolations from transport aircraft-like independently monitor and maintain immediate
interfaces. operational safety. The goals of this coupling are
improved overall system safety, performance, and
flexibility beyond what could be accomplished with
Overview of the H-Metaphor
conventional automation or manual control while
The H-metaphor as detailed in Flemisch, et al. significantly reducing the cost and complexity of
[11] and summarized here, structures the the needed automation and operator training.
relationship between operator and an intelligent Furthermore, the metaphor provides a path for the
vehicle as a partnership between two dissimilar, but incremental introduction of increasingly capable
cooperative agents sharing control of a vehicle (e.g., automation technologies and may be a necessary
Figure 3). This partnership is motivated by the evolutionary step toward, and in many cases an
emergence of highly automated vehicles capable of alternative to, autonomous vehicles that routinely
autonomously performing a wide variety of tasks operate in complex, safety-critical environments
with great precision and vigilance combined with without human backup.
The metaphor provides guidance on the
human-vehicle interaction, the vehicle as an
AArrbbiittrraattiioonn
ffuussiioonn autonomous agent, and multi-vehicle interaction
HHuummaann AAuuttoommaattiioonn [11]. A horse and rider system interacts through
multi-modal communication underpinned by a
strong, bi-directional haptic component. Human
iinntteenntt
haptic senses (e.g., proprioception and cutaneous
touch) are unique in that they both sense and effect
the external environment. Furthermore, as a vector
aa ss
sssseessss aacctt aasssseess cqoumanmtiutyn i(ci.aet.i,n fgo rcsep) atthieayl pcroovnicdeep tas nastuurcahl maesa nst hoef
direction of an intended or desired maneuver.
These properties enable robust and rapid, bi-
directional communication and negotiation between
EEnnvviirroonnmmeenntt
horse and rider. For example, even in the case of a
horse cart without direct bodily contact, the reins
Figure 3. Shared control between cooperative
form a bi-directional link with the horse able to
Agents
transmit information to the driver as well as receive
it. According to the German National Equestrian
Federation, turns are performed by yielding the
the recognition that their specialized intelligence
outside rein with a twist of the hand(s), not pulling
will still be relatively brittle when faced with real-
the inside one (Figure 4). There should always be a
world complexities. For example, the ability to
"soft, steady, elastic connection" between the
determine whether a sensed object is a benign
driver's hand and horse's mouth, where "the horse
empty bag or a hazardous chunk of concrete could
be important in a variety of situations but is far
from feasible in any general sense. The H-
metaphor addresses this situation by relying on the
general intelligence of a human operator. In the
partnership, the human strategizes, corroborates,
and backs up actions of the automation in the
accomplishment of operations that are beyond the
vehicle’s reliable, independent cognition. The
Figure 4. Hands and reins position for elastic
human is aided in this role by the vehicle’s
connection [12]
awareness of the routine situation elements; ability
4
seeks the contact and the driver provides it" [12]. If that the operator has regular, but relaxed, physical
the horse wants to change direction, perhaps to involvement throughout a mission, thus aiding
avoid a rut, it leads with a turn of its head. The maintenance of situation awareness. The second
rider feels a change in rein tension and can accept motivating factor is optimizing the interaction for
or reject the action by relaxing or increasing the robust communication of near-term, possibly time
tension. In addition, discrete communication and safety critical goals and actions. This near-term
elements such as an agitated jerk (possibly focus serves to minimize miscommunication and/or
accompanied with an auditory snort and visual tail protracted exchanges by keeping the language set
flick) are often combined with such continuous and range of situationally applicable behaviors
signals. Furthermore, from a lack of any tension small and with a minimum of abstraction.
the horse can infer that the driver is relatively
disengaged and adjust its expectations accordingly.
The H-mode, a Realization
Efficient bi-directional communication results
The metaphor provides a broad vision as to
in a type of shared control not present in
how an H-inspired vehicle should interact and
conventional automation. Conventional automation
behave. Achieving this vision requires better
tends to be ‘all or nothing’ where either the human
understanding the fundamental human-automation
or the machine has control. In comparison, the
interaction issues, applying and advancing the
rider of a horse can hold the locus of control while
underlying technologies for autonomous systems,
the horse remains “in-the-loop” performing inner-
and integrating this knowledge into practical
loop functions and monitoring for hazards (i.e.,
designs for specific operational applications such as
tight reins operation). Alternatively, the rider can
ground vehicles, aircraft and spacecraft. Ideally,
also explicitly or implicitly designate a clear task or
this work should result in specific, yet malleable
behavior of some duration for the horse to perform
automation design and user interaction guidelines
and allow the horse to have the locus of control
that could be applied across a wide range of
(e.g., loose reins). The rider, even in loose reins,
“intelligent” vehicles. By facilitating positive
remains in-the-loop and can flexibly control
transfer of training, knowledge, and experience
important aspects of the behavior such as what
between operational domains and reducing the
position to maintain relative to another, running
potential for confusion, we could expect significant
animal (e.g., calf roping ) or how to avoid a hazard
improvements in safety, operational effectiveness
(e.g., left, right or over).
and training productivity for both users and
It should be recognized that the H-metaphor developers.
applies to the near-term or tactical actions of the
The H-mode, a practical realization of the
vehicle. Horses, outside of TV and movies, do not
metaphor using commercially available haptic
worry about what is hours ahead relative to the
interface hardware (e.g., an active, force-feedback
goals of the rider. As part of a complete system
side-stick) and with applicability across operational
concept, a separate, but complementary planning
domains, is currently being developed by NASA
sub-system is envisioned to support longer term,
and the German Aerospace Lab, DLR. Early steps
mission planning and monitoring requirements
in this process include an exploratory study
(e.g., Schutte and Goodrich [13]). The maximum
conducted first at NASA and latter extended at the
look-ahead of an H-inspired system is at the
DLR. As described by Flemisch, Goodrich, and
discretion of the designers, but in keeping with the
Conway [14], this study investigated the general
metaphor, the human-automation interaction in
user acceptance and expectations of the metaphor
particular, would nominally project only as far as
and developed an initial, generic “language” for the
needed to support the current action and possible
interaction between a simple automated vehicle and
transitions while accounting for local hazards.
a human operator. A small but broadly
While it may seem counterintuitive to limit the
representative group of prototypical users
ability to pre-program sequences of action as far
participated in a structured interview to explore
into the future as technologically feasible, the intent
their “naïve” reaction to the metaphor and
of the limitation is two fold. The first is to ensure
expectations for, and acceptance of, an intelligent
5
H-inspired vehicle. The results of these interviews
(the questionnaire and responses are reproduced in
ref. 14), indicate overall acceptance of the metaphor
and that the general issues and capabilities of such
vehicles are intuitively understood.
The interaction investigation consisted of a
“structured exploration” to develop and evaluate a
preliminary interaction language between a simple
automated vehicle simulation and a human operator
using an active side-stick for the following generic
1-dimensional control tasks:
• Starting and Stopping the vehicle
• Maintaining a desired spacing interval behind a Figure 6. Theater system with curtain
lead vehicle. As shown in figure 5, deviations open
from the desired interval were indicated by
the subject’s stick. The technique allowed flexible
triangular zones of increasing risk and
exploration of users expectations and possible
rectangular zones of significant danger. The
design variations prior to interactions with a rapid
subjects and “H” did not necessarily have the
prototype implemented in technology. While in the
same zones, requiring a haptic negotiation to
original technique described in ref. 15, the
find the best compromise.
participant does not know about the confederate,
• Transitioning between a more automated and a here the participant knew about it from the very
less automated mode of operation (Tight beginning. The “curtain” was open in the training
Rein/Loose Rein) phases (Figure 6) and closed during the usability
assessments.

The operator performed the last two tasks in The exploration went through the following
parallel with a secondary task to mimic the divided stages:
attention likely in actual operations.
• Pre-Design of potential interaction patterns by a
A goal of the exploration was to match the
core design team in the theater.
expectations of users (e.g., “What behavior would
you expect of this artifact in that situation?”) with • Usability assessment and refinement of the pre-
the technical potential of the design space and the design with other team members.
ideas of the designer. One of the tools used to • Rapid prototyping implementation of selected
achieve this match was a “theater” system or features of the pre-design.
“Wizard of Oz” technique, based on the concept of
• Structured interview with prototypical users.
Salber and Coutaz [14]. A confederate “behind the
• Laboratory based assessment with prototypical
curtain” simulated parts of the automation via a
users.
second side-stick that was electronically linked to
o Collection of expectations directly in the
Situation “Ice”, with differing goals theater system
participant confederate/automation o “Naïve runs” (users interact with Pre-design
without any training)
o Teaching of the pre-design and comparison
to expectations
o Usability assessment of pre-design with the
theater system
Figure 5. Distance task from ref. 14 o Usability assessment of pre-design with the
rapid prototypes
6
o De-Briefing summarized in Flemisch, Schieben, and Schindler
[16] and the aviation application is described here.
• Harmonization of the pre-design with the
expectations
Technical Perspective
While a detailed description of the results goes Figure 7 illustrates the conceptual system
beyond the scope of this paper, a first draft of architecture. Major system elements include the
interaction consists of the following haptic multi-modal user interface including an active or
communication cornerstones: haptic side-stick and speed-command lever (not
shown); inner-loop control augmentation; real-time
- Vibrations: A specific scheme of vibrations that
sensors and/or data link system; and the H-inspired
combines information about danger and degree of
automation (HIA).
automation (“Alive”-vibration), combined with
Within the user interface, the active stick
- Pushing: A specific scheme of pushing back the
and speed-command lever provide a common
stick from danger, combined with
means by which both the operator and HIA issue
- Signaling: Discrete interaction “double-ticks” for
commands to the inner-loop. The haptic interaction
signaling the desire for discrete distance or speed
allows each agent to sense the others involvement
changes, combined with
(e.g., applied force) and infer an associated intent.
- Transitioning: A specific scheme of fluidly
Visual displays (i.e., a primary flight and tactical
transitioning between a higher level of
map displays) provide graphical representations of
automation (Loose Rein) and a lower level of
the external environment overlaid by basic flight
automation (Tight Rein)
parameters and the HIA’s interpretation of this
o Tight Rein (cid:198) Loose Rein by first leading the information. Since the inner-loop automation
H-vehicle into the desired maneuver and then provides full-time stability and control
(Variant 1) relaxing the hand on the stick or augmentation, presentation of basic flight
(Variant 2) a discrete “yes”-signal or (Variant parameters is primarily for awareness rather than
3) a combination of 1 and 2. control and can be significantly simpler than current
o Loose Rein (cid:198) Tight Rein by taking a firm formats. The displayed environmental information
grip, if necessary combined with a discrete includes physical elements such as terrain,
“no”-signal. obstructions, weather, and traffic and virtual
elements such as airspace boundaries, published
- Fine Tuning: A specific scheme of fine tuning
fixes and procedures. The route from a mission
reference values the automation in Loose Rein
planning system, if present, is also included.
- Arbitration: Specific schemes of prioritizing
Elements rendered on the displays are also known
between the will of the user and of the automation
to the HIA. HIA specific symbology allows the
pilot to quickly ascertain its status, motivation,
While a couple of additional design ideas center of attention, projected trajectory, and how it
surfaced and a few glitches were found during the would respond to potential directives. Finally,
usability assessment, overall the pre-design although not currently implemented in a prototype,
matched most of the expectations of the evaluation auditory interface features will be used to
subjects and the prototypes were, for this state of compliment the haptic and visual elements. In
development, quite intuitive. keeping with the H-metaphor, these
communications can be bi-directional but limited in
content and complexity (e.g., not complex
Toward a Comprehensive Aircraft
sentences or abstractions).
Implementation
The output of the stick provides the reference
DLR and NASA are currently investigating
signal acted on by the inner-loop control element,
more comprehensive applications of the H-mode to
essentially a fly-by-wire autopilot providing flight-
ground and air vehicles, respectively. The current
path oriented, control-wheel steering and envelope
status of the ground vehicle application is
protection (e.g., [17-20]). Considering the horse
7
Figure 7. H-mode based flight control system
metaphor, the inner-loop can be likened to the hind other system elements (i.e., HAI, interaction
brain, spinal cord, and voluntary neuromuscular manager) so that they can appropriately update their
system. The inner-loop simplifies the pilot-vehicle internal models.
interaction into an easily managed set of elemental
The sensor and data link capabilities allow the
actions. This simplification is accomplished by
vehicle to obtain real-time information about the
delegating management of a number of high-
local environment, functioning as the external
frequency, non-linear, but largely deterministic
senses of the metaphor. These sensors could
control processes to the inner-loop. This
include conventional instruments such as satellite
aggregation also reduces the workload and training
based navigation (e.g., GPS), weather radar, etc, as
of the operator and simplifies the design and
well has emerging technologies like multi-spectral
understanding of the outer-loop automation
machine vision for navigation and hazard avoidance
elements. Obviously, the effects of possible failure
and digital data link based state and intent sharing
modes and other “non-normals” need to be
between vehicles and air traffic control. One of the
considered on overall system behavior. The current
benefits of the H-mode concept is that new sensor
state of the art is rapidly enabling retention of
capabilities can be readily integrated into its
augmented response characteristics in the presence
framework rather than being added in a stand alone
of component failures, albeit with some
fashion with “integration” largely left to the pilot as
performance degradation, rather than reversion to a
is often done today.
completely different set of dynamics (e.g., a direct
electric link mode). Optional, advanced functions Within the HIA, relevant static information
within the inner-loop include upset recovery and stored in databases is analogous to long term
adaptive processes compensating for vehicle memory. This static data provides the HIA with
changes caused by airframe damage or other effects knowledge of nonphysical elements such as
(e.g., airframe icing) such as described by Burken, airways, static physical features such as terrain, and
et al [21]. For maximum capability, these functions databases characterizing its own capabilities.
should communicate operating envelope changes to Within the situation assessment function, real-time
8
and static data are analyzed, features identified, and extensive autonomous actions are possible design
an integrated representation of the current situation extensions but are not discussed here as they are
constructed. The situation assessment also includes perhaps best viewed outside the context of the
relevant inputs from the operator. The important metaphor.
aspects of the perceived situation are communicated
In tight reins, the pilot commands the near-
back to the operator via the interaction manager.
term actions of the vehicle through the inner-loop
When operating in loose reins mode, the HIA
control and does not communicate any sustained
creates and maintains a desired trajectory which is
intent to the HIA. In this mode, the HIA provides
followed by a guidance subsystem. In tight reins
no or limited aid unless it perceives an emerging
mode, the current aircraft implementation does not
conflict or hazard. In the event of a perceived
generate a reference trajectory and no maneuvering
conflict, the HIA brings the situation to the pilot’s
command is sent to the stick unless an imminent
attention via the visual and haptic interface
conflict is perceived. Alternatively, the DLR is
elements. If the conflict severity exceeds a defined
investigating the soft haptic presentation of a
threshold, the automation sends a force input,
desired trajectory even during tight reins operation.
effectively a maneuver command, to the stick. If
This presentation may assist the operator and
unopposed, this input safely redirects the vehicle.
increase their awareness of potential future actions
The pilot can also perform their own deconfliction
by the HIA and be particularly useful in situations
maneuver or, if for some reason they need to
where the scope of safe tight reins maneuvering is
continue toward the conflict, apply an appropriate
limited.
counter-force. The degree and duration of the
Various means of conflict avoidance are under automation’s resistance can be situationlly tailored
investigation at both organizations. Currently, by the arbitration function. For example, once the
simple reactive behaviors such as potential-field pilot has demonstrated an unambiguous and
guidance [22] appear to be an intuitive, robust and informed intent to continue toward a hazard, it may
predictable means of performing obstacle be advisable for the automation to reduce or cease
avoidance. In addition, superposition of individual its opposition. Such situations might arise due to
guidance commands (e.g., path following + erroneous or incomplete local situation awareness
avoidance) aided by simple heuristics, is being and/or knowledge of the mission objective by the
investigated as a means of effectively reacting to HIA. Limiting the strength and/or duration of
simultaneous influences. opposition provides a mechanizm through which
the pilot retains authority.
Pilot’s Perspective In loose reins, the HIA performs a well defined
task or behavior. This behavior may be initiated
The H-mode system integrates control of the
through explicit direction from the pilot or implicit
vehicle’s full range of control automation into a
in the local situation (e.g., flare before touchdown).
single consistent interface system and natural set of
Flying published procedures or a preplanned route
interaction skills. The pilot uses the same basic
are examples of actions normally performed in
control and display devices for augmented manual
loose rein. Several behaviors may be combined
control through complex behaviors such as takeoffs,
such as route following while maintaining spacing
landings, coupled procedures, and automated
from other traffic. Other loose rein behaviors
conflict/hazard avoidance.
include precise, high-bandwidth tasks such as
The H-mode emphasizes two basic modes of
takeoff and flare and landing.
operation corresponding to tight and loose reins. In
In loose reins, the pilot is relieved from
the tight reins mode, the pilot holds the locus of
continuously generating instantaneous commands
immediate control while in loose reins the aircraft
but remains actively involved in managing and
autoflight system has a relatively high degree of
transitioning between behaviors. By lightly
autonomy in satisfying longer-term directives.
touching the haptic interface elements, the pilot can
Transitions between tight and loose are managed by
feel the HIA’s instantaneous commands, providing
an “arbitration function” in the interaction manager.
a non-visual means of maintaining awareness. This
Other modes of operation supporting more
9
awareness is aided by discrete haptic signals pilot. Like a normal decision point, the pilot can
indicating the general situation status and imminent initiate the action with a simple, positive control
events such as reaching an intersection or planned input while remaining in loose reins. If not satisfied
trajectory transition (e.g., top of descent). with any of the loose reins resolution actions, the
pilot has the option of transitioning into tight reins.
The pilot also uses the haptic interface to
If the pilot does not respond, or if there is
modify key parameters within a behavior and
insufficient time to allow reaction, the HIA
manage transitions between behaviors. Within a
autonomously shifts toward a conservative posture.
behavior, pilot applied forces can be fed back to the
HIA and used to update its active reference targets Transitions between loose and tight reins are
in an operationally intuitive manner. For example, managed by an arbitration function in the human-
in a following task, pressure on the speed command machine interaction manager. The interaction
lever is fed back to adjust the separation interval. manager interprets the pilot’s inputs in the context
Similarly, in a landing behavior, light stick forces of the HIA’s perceived situation. This
can be used to adjust the aim point used in the interpretation is aided by a small set of discrete
landing guidance. The HIA’s receptiveness to elements allowing simple, but direct
these updates is situationally tailored such that only communication, similar to the interaction
safe and appropriate actions are supported in loose commonly seen between a human and a well trained
reins. The pilot’s awareness of this receptiveness is animal. These elements include signals meaning
supported via multi-modal cues including tailoring ‘yes’, ‘no’, ‘NO’, ‘I am unsure’, and ‘I want your
of the basic feel characteristics of the haptic attention’. The interaction manager typically
interface (e.g., a heavier feel as limits are encodes these multi-modally including a haptic
approached). component. Current research is exploring a variety
of generation methods from the pilot’s side,
At major transition or decision points, the HIA
including haptic signals and dedicated buttons.
can be configured to react only after the pilot has
made a positive control input indicating they are In general, if the pilot sustains a firm input
aware of the situation. The pilot typically makes while in loose reins, a transition into tight reins
this input by manually initiating the transition. This occurs. If, while operating in tight reins, the pilot
interaction helps prevent the detachment and loss of maintains the vehicle in a specified relationship
situation awareness that can occur with relative to certain external elements, the automation
conventional automation and encourages the pilot to offers or initiates the appropriate loose reins
verify the appropriateness of major transitions. If behavior. Figure 8, for example, shows the pilot
the pilot fails to initiate an expected transition, the directing the automation to enter a published
aircraft provides increasingly salient cues. If the holding pattern. First, the pilot maneuvers the
pilot still does not respond, the automation shifts airplane so that the flight path marker (FPM)
toward a more conservative (i.e., safer) posture or symbol is stabilized on a visualization of the fix
action. At decision height, for example, the pilot is associated with the hold. Rendered as a transparent
expected to make an input confirming that it is safe “pole”, the visualization allows the pilot to
to proceed and transition from the approach to the graphically specify the altitude of the procedure.
landing behavior. If this input is not sensed, the The vertical extent of the pole indicates the range of
aircraft initiates a missed approach. The pilot can appropriate altitudes and the projection of the FPM
nominally alter or reject an expected transition by on the pole, rounded to the nearest appropriate
making a light control input consistent with an increment, specifies the desired altitude. As the
alternate loose-reins action or by making a firm pilot stabilizes on the pole, the HIA highlights the
input, triggering a transition to tight reins. procedure, indicating that it can transition to loose
reins. The pilot completes the transition via a “yes”
If the HIA perceives an emerging conflict
signal and a magenta line provides a visual
during loose reins operations, it brings the situation
indication of the trajectory guiding the automation.
to the operator’s attention. If the HIA is capable of
Simultaneously, the basic feel characteristics of the
generating a safe, straightforward deconfliction
stick are changed to indicate the vehicle has the
maneuver, it will be offered as an option to the
10