Flexible robotic control via co-operation between an operator and an ai-based control system

159 Pages·2012·17.41 MB·English

by  Emmanouil Chiou

FLEXIBLE ROBOTIC CONTROL VIA CO-OPERATION
BETWEEN AN OPERATOR AND AN AI-BASED
CONTROL SYSTEM
by
EMMANOUIL CHIOU
Athesissubmittedto
TheUniversityofBirmingham
forthedegreeof
DOCTOROFPHILOSOPHY
SchoolofComputerScience
CollegeofEngineeringandPhysicalSciences
TheUniversityofBirmingham
11thApril2017
University of Birmingham Research Archive

e-theses repository

This unpublished thesis/dissertation is copyright of the author and/or third
parties. The intellectual property rights of the author or third parties in respect
of this work are as defined by The Copyright Designs and Patents Act 1988 or
as modified by any successor legislation.

Any use made of information contained in this thesis/dissertation must be in
accordance with that legislation and must be properly acknowledged. Further
distribution or reproduction in any format is prohibited without the permission
of the copyright holder.
Abstract
Thisthesisaddressestheproblemofvariableautonomyinteleoperatedmobilerobots. Vari-
ableautonomyreferstotheapproachofincorporatingseveraldifferentlevelsofautonomous
capabilities(Level(s)ofAutonomy(LOA))rangingfrompureteleoperation(humanhascom-
pletecontroloftherobot)tofullautonomy(robothascontrolofeverycapability), within
a single robot. Most robots used for demanding and safety critical tasks (e.g. search and
rescue,hazardousenvironmentsinspection),arecurrentlyteleoperatedinsimpleways,but
couldsoonstarttobenefitfromvariableautonomy. Theuseofvariableautonomywould
allowArtificialIntelligence(AI)controlalgorithmstoautonomouslytakecontrolofcertain
functionswhenthehumanoperatorissufferingahighworkload,highcognitiveload,anxiety,
orotherdistractionsandstresses. Incontrast,somecircumstancesmaystillnecessitatedirect
humancontroloftherobot. Morespecifically,thisthesisisfocusedoninvestigatingtheissues
ofdynamicallychangingLOA(i.e. duringtaskexecution)usingeitherHuman-Initiative(HI)
orMixed-Initiative(MI)control. MIreferstothepeer-to-peerrelationshipbetweentherobot
andtheoperatorintermsoftheauthoritytoinitiateactionsandLOAswitches.HIreferstothe
humanoperatorsswitchingLOAbasedontheirjudgment,withtherobothavingnocapacity
toinitiateLOAswitches. AHIandanovelexpert-guidedMIcontrollerarepresentedinthis
thesis. Thesecontrollerswereevaluatedusingamultidisciplinarysystematicexperimental
framework,thatcombinesquantifiableandrepeatableperformancedegradationfactorsfor
both the robot and the operator. The thesis presents statistically validated evidence that
variableautonomy,intheformofHIandMI,providesadvantagescomparedtoonlyusing
teleoperationoronlyusingautonomy,invariousscenarios. Lastly,analysesoftheinterac-
tionsbetweentheoperatorsandthevariableautonomysystemsarereported. Theseanalyses
highlighttheimportanceofpersonalitytraitsandpreferences,trustinthesystem,andthe
understandingofthesystembythehumanoperator,inthecontextofHRIwiththeproposed
controllers.
Acknowledgments
TheysayhavingmorethanonePhDsupervisorscanbecomplicated. Thiswascertainlynot
thecaseinmyPhD.Iwasveryfortunatetohavesuchanunderstandingandsupportiveteam
of supervisors. Their multidisciplinary guidance and support proved invaluable. Rustam
Stolkin,NickHawes,KimromShapiro;Iamverygrateful,thankyou! RustamandNickwere
alwaysthereformeandprovidedmewithtrustandfreedomonhowtomanagemytime,
ideas,andplans. Thisfreedomwasoneofthemostimportantfactorscontributingtothis
thesis. KimandthepeopleoftheVisualExperienceLabmademefeelalwayswelcomein
theirlabmeetings. Withoutthediscussionsandinsightscomingfromthosemeetingsonhow
toconductexperimentswithhumanparticipants,thisthesiswouldhavebeenimpossible.
Next,IwouldliketothankthedstltechnicalpartnerTimothyHarrisonforbelievinginthis
projectandforbeingveryhelpfultowardsovercomingthestrictdstlprocedures. Iwouldalso
liketothankJeremyBaxterforbeeninthesupervisionteamforthefirstyearofthisproject.
I am also thankful to Michael Mistry and Russell Beale for all the useful discussions and
constructivefeedbacktheyprovidedmeduringmythesisgroupmeetings. Moreover,Iam
thankful to the IRlab and ERL members for providing me with feedback on my work. In
particular to Chie Takahasi for the long discussions on the human factors aspects of my
experiments.
Iwishtoacknowledgethehelpprovidedbytheco-authorsofmy(our)papers. GodaBieksaite,
JessKerlin,andAndrewClouteryourhelpwasmuchappreciated.
Special thankstomy parents, family, andfriends. Theirsupportinsomanylevelsduring
those4yearswaspriceless.
Lastly,shoutouttoallthepeoplewhoparticipatedinmyexperiments. Theytrulymadethe
processoffindingvolunteersandrunningtheexperiments,abreeze.
This research was supported by the British Ministry of Defence and the UK Defence Sci-
ence and Technology Laboratory, under their PhD bursary scheme, contract no. DSTLX-
1000074621.
Abbreviations
• AmericanAssociationforArtificialIntelligence(AAAI)
• ArtificialIntelligence(AI)
• analysisofvariance(ANOVA)
• Electroencephalography(EEG)
• ExplosiveOrdnanceDisposal(EOD)
• Fisher’sleastsignificantdifference(LSD)
• Human-RobotInteraction(HRI)
• Human-Initiative(HI)
• Level(s)ofAutonomy(LOA)
• Mixed-Initiative(MI)
• NASATaskLoadIndex(NASA-TLX)
• OperatorControlUnit(OCU)
• reactiontime(RT)
• RobotOperatingSystem(ROS)
• SearchandRescue(SAR)
• SituationAwareness(SA)
• SimultaneousLocalizationandMapping(SLAM)
• UrbanSearchandRescue(USAR)
Contents
ListofTables vii
ListofFigures ix
1 INTRODUCTION 1
1.1 Contextofresearchandprojectsummary. . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Autonomyversusteleoperation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 Interfacesandtelepresence . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.2 VariableautonomyandMixed-Initiative . . . . . . . . . . . . . . . . . . . 7
1.3 ContributionsofthisThesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 ListofPublications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Thesisstructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 BACKGROUND 12
2.1 Human-RobotInteraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.1 Human-RobotInteractiondefinitionandcontext . . . . . . . . . . . . . . 13
2.1.2 Human-RobotInteractionawareness . . . . . . . . . . . . . . . . . . . . . 14
2.1.3 Human-RobotInteractionfieldstudies . . . . . . . . . . . . . . . . . . . . 15
2.1.4 MetricsforHuman-RobotInteractionandmobilerobots . . . . . . . . . 17
2.2 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.1 Interfacesasawaytoimproveteleoperation . . . . . . . . . . . . . . . . . 18
2.2.2 Situationawarenessandtheuseofmaps . . . . . . . . . . . . . . . . . . . 19
2.3 Variableautonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.1 Thenotionoflevelsofautonomy . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.2 Sharedcontrol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.3 Tradedcontrol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.3.4 Multiplelevelsofautonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Dynamicallyswitchinglevelsofautonomy . . . . . . . . . . . . . . . . . . . . . . 28
2.4.1 Conductingvariableautonomyexperiments . . . . . . . . . . . . . . . . . 29
2.4.2 Human-Initiativevariableautonomy . . . . . . . . . . . . . . . . . . . . . 30
2.4.3 DefinitionandtaxonomyofMixed-Initiativecontrol . . . . . . . . . . . . 32
2.4.4 Mixed-Initiativecontrolrelatedsystems . . . . . . . . . . . . . . . . . . . 33
2.4.5 Human-RobotInteractionwithLOAswitchingrobots . . . . . . . . . . . 35
2.5 Measuringandinducingcognitiveworkload . . . . . . . . . . . . . . . . . . . . . 36
2.5.1 Physiologicalmeasures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.5.2 Subjectivemeasures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.5.3 Taskperformancemeasures . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3 TOWARDSTHEPRINCIPLEDSTUDYOFVARIABLEAUTONOMY 40
3.1 Systemdescription . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Pilotexperiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2.1 Tasksandrobotarena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.2 Participantsandexperimentaldesign . . . . . . . . . . . . . . . . . . . . . 46
3.2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3 Suggestionsforfutureexperiments . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4 Conclusionandimpact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4 EXPERIMENTALANALYSISOFHUMAN-INITIATIVEVARIABLEAUTONOMY 55
4.1 Apparatusandroboticsoftware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2 Experimentaldesignandprocedure . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2.1 Experimentalsetup-operatorcontrolunitandrobottestarena . . . . . 59
4.2.2 Primaryandsecondarytasks,andexperimentaltestmodalities . . . . . 61
4.2.3 Participantsandprocedure . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.3 Human-Initiativesystemperformanceanalysis . . . . . . . . . . . . . . . . . . . 65
4.3.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.4 Human-InitiativeHuman-RobotInteractionanalysis . . . . . . . . . . . . . . . . 70
4.4.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.4.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.5 Conclusionsandimpact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5 DESIGNANDEVALUATIONOFAMIXED-INITIATIVECONTROLSYSTEM 76
5.1 Frameworkfordesigninganexpert-guidedMixed-Initiativeroboticsystem . . 78
5.2 Designingtheexpert-guidedMixed-Initiativecontrollerfornavigation . . . . . 80
5.2.1 ThresholdMixed-Initiativecontroller . . . . . . . . . . . . . . . . . . . . . 81
5.2.2 FuzzyMixed-Initiativecontroller . . . . . . . . . . . . . . . . . . . . . . . . 84
5.3 Evaluationusingasimulatedrobotandtestenvironment . . . . . . . . . . . . . 89
5.3.1 Results: tasksperformance . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.3.2 Results: Human-Robot-Interaction . . . . . . . . . . . . . . . . . . . . . . 93
5.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.4 Evaluationusingrealrobotandtestenvironment . . . . . . . . . . . . . . . . . . 98
5.4.1 Experimentalsetup-apparatus,robottestarena,andcontrolmodes . . 99
5.4.2 Tasksandperformancedegradationfactors . . . . . . . . . . . . . . . . . 102
5.4.3 Participantsandexperimentaldesign . . . . . . . . . . . . . . . . . . . . . 105
5.4.3.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.4.3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.5 Conclusionandimpact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6 CONCLUSIONSANDFUTUREWORK 116
6.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.2 Futurework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.3 Closingthoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
References 125
Appendices 140
A STATISTICS 141
A.1 Statisticalmethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
List of Tables
3.1 TheANOVAcalculationsforthemetricsusedintheexperiment. Descriptive
statistics,withtheexceptionofcollisions,areshownonlyfortheconditionsthat
asignificantstatisticaldifferencewasfound. . . . . . . . . . . . . . . . . . . . . . 51
4.1 TableshowingtheANOVAresultsandthedescriptivestatisticsforthemetrics
used. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.1 Thefuzzyrulebase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.2 Thefuzzymembershipfunctionsforlinguisticvaluesofinputvariables"error"
and"speed"(seeFig. 5.2). For"error",trapezoidmembershipfunctionshave
beenused. For"speed"membershipfunctions,twotrapezoidfor"reverse"and
"forward"andonetriangularfor"zero",wereused. Themembershipfunctions
wereheuristicallychoseninordertosmoothlyoverlapthroughouttheuniverse
ofdiscourse. Thisisacommonpracticewhendesigningfuzzycontrollers. The
membershipfunctionfor"errorlarge"waschosenbasedontheerrorthreshold
calculatedinSection5.2.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.3 TableshowingtheANOVAresultsandthedescriptivestatisticsforthemetrics
usedintheexperiment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93