A practical introduction to human-in-the-loop cyber-physical systems

307 Pages·2017·11.338 MB·English

by  Boavida, Fernando, Nunes, David, Silva, Jorge Sá

(cid:2)
APracticalIntroduction
toHuman-in-the-Loop
Cyber-PhysicalSystems
(cid:2) (cid:2)
(cid:2)
(cid:2)
A Practical Introduction to Human-in-the-Loop
Cyber-Physical Systems
DavidNunes
UniversityofCoimbra
JorgeSáSilva
UniversityofCoimbra
FernandoBoavida
UniversityofCoimbra
(cid:2) (cid:2)
(cid:2)
(cid:2)
Thiseditionfirstpublished2018
©2018JohnWiley&SonsLtd
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,or
transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,
exceptaspermittedbylaw.Adviceonhowtoobtainpermissiontoreusematerialfromthistitleisavailable
athttp://www.wiley.com/go/permissions.
TherightofDavidNunes,JorgeSáSilvaandFernandoBoavidatobeidentifiedastheauthorsofthiswork
hasbeenassertedinaccordancewithlaw.
RegisteredOffice(s)
JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,USA
JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UK
EditorialOffice
TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UK
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWileyproducts
visitusatwww.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Somecontentthat
appearsinstandardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
Whilethepublisherandauthorshaveusedtheirbesteffortsinpreparingthiswork,theymakeno
representationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthisworkand
specificallydisclaimallwarranties,includingwithoutlimitationanyimpliedwarrantiesofmerchantabilityor
fitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedbysalesrepresentatives,written
salesmaterialsorpromotionalstatementsforthiswork.Thefactthatanorganization,website,orproductis
referredtointhisworkasacitationand/orpotentialsourceoffurtherinformationdoesnotmeanthatthe
publisherandauthorsendorsetheinformationorservicestheorganization,website,orproductmayprovide
(cid:2) (cid:2)
orrecommendationsitmaymake.Thisworkissoldwiththeunderstandingthatthepublisherisnotengaged
inrenderingprofessionalservices.Theadviceandstrategiescontainedhereinmaynotbesuitableforyour
situation.Youshouldconsultwithaspecialistwhereappropriate.Further,readersshouldbeawarethat
websiteslistedinthisworkmayhavechangedordisappearedbetweenwhenthisworkwaswrittenandwhen
itisread.Neitherthepublishernorauthorsshallbeliableforanylossofprofitoranyothercommercial
damages,includingbutnotlimitedtospecial,incidental,consequential,orotherdamages.
LibraryofCongressCataloging-in-PublicationData
Names:Nunes,David,1987-author.|Silva,JorgeSá,author.|Boavida,
Fernando,1959-author.
Title:Apracticalintroductiontohuman-in-the-loopcyber-physicalsystems/
DavidNunes,JorgeSáSilva,FernandoBoavida.
Description:Firstedition.|Hoboken,NJ:JohnWiley&Sons,2018.|
Includesbibliographicalreferencesandindex.|
Identifiers:LCCN2017025006(print)|LCCN2017042126(ebook)|ISBN
9781119377801(pdf)|ISBN9781119377788(epub)|ISBN9781119377771
(cloth)
Subjects:LCSH:Cooperatingobjects(Computersystems)|Human-computer
interaction.
Classification:LCCTJ213(ebook)|LCCTJ213.N862017(print)|DDC
621.39–dc23
LCrecordavailableathttps://lccn.loc.gov/2017025006
CoverDesign:Wiley
CoverImage:©ipopba/Gettyimages
Setin10/12ptWarnockbySPiGlobal,Chennai,India
10 9 8 7 6 5 4 3 2 1
(cid:2)
(cid:2)
Tomyparents,Jorgeand
Eulália,andtomybrother,
Telmo.
DavidNunes
ToFátima,Catarina,
Pedro,Jojó,andmyparents
JorgeSáSilva
ToMariaJoãoandour
threedaughters–Susana,
(cid:2) (cid:2)
Inês,andCatarina
FernandoBoavida
(cid:2)
(cid:2)
vii
Contents
ListofFigures xi
ListofTables xvii
Foreword xix
Preface xxi
Acknowledgments xxiii
ListofAbbreviations xxv
AbouttheCompanionWebsite xxvii
1 Introduction 1
1.1 TheRiseofCyber-PhysicalSystems 1
(cid:2) (cid:2)
1.2 HumansasElementsofCyber-PhysicalSystems 4
1.3 ObjectivesandStructure 7
PartI EvolutionandTheory 9
2 EvolutionofHiTLTechnologies 11
2.1 "Things",Sensors,andtheRealWorld 11
2.2 HumanSensingandVirtualCommunities 17
2.3 InSummary... 23
3 TheoryofHiTLCPSs 25
3.1 TaxonomiesforHiTLCPSs 25
3.2 DataAcquisition 28
3.2.1 HumansasSetsofSensors 28
3.2.2 HumansasCommunicationNodes 29
3.3 StateInference 30
3.3.1 HumanNature 30
3.3.2 HumansasProcessingNodes 31
3.4 Actuation 31
3.4.1 HumansandRobotsasActuators 31
3.5 InSummary... 32
(cid:2)
(cid:2)
viii Contents
4 HITLTechnologiesandApplications 35
4.1 TechnologiesforSupportingHiTLCPS 35
4.1.1 DataAcquisition 35
4.1.2 StateInference 39
4.1.3 Actuation 42
4.2 ExperimentalProjects 45
4.2.1 HiTLinIndustryandatHome 46
4.2.2 HiTLinHealthcare 48
4.2.3 HiTLinSmartphonesandSocialNetworking 51
4.3 InSummary... 56
PartII Human-in-the-Loop:Hands-On 57
5 ASampleApp 59
5.1 ASampleBehaviorChangeInterventionApp 59
5.2 TheSampleApp’sBaseArchitecture 60
5.2.1 TheAndroidApp 60
5.2.2 TheServer 65
5.3 EnhancingtheSampleAppwithHiTLEmotion-awareness 66
5.3.1 ChoosingaMachineLearningTechnique 67
5.3.2 ImplementingEmotion-awareness 68
(cid:2) 5.4 InSummary... 71 (cid:2)
6 SettinguptheDevelopmentEnvironment 73
6.1 InstallingAndroidStudio 73
6.2 CloningtheAndroidProject 77
6.3 DeployingtheServer 85
6.3.1 InstallingtheSoftwareandCloningtheServer’sProject 85
®
6.3.2 ObtainingaFoursquare ’sClientIDandClientSecret 88
6.3.3 SettinguptheDatabase 91
6.3.4 DeployingtheServeronTomcat7 95
6.4 TestingtheSampleApp 99
6.5 InSummary... 101
7 DataAcquisition 103
7.1 CreatingtheEmotionTasker 103
7.2 ProcessingSensoryData 110
7.3 InSummary... 118
8 StateInference 121
8.1 ImplementingaNeuralNetwork 121
8.2 RequestingUserFeedback 125
8.2.1 CreatingtheEmotionFeedbackActivity 125
8.2.2 ImplementingtheEmotionSpaceView 127
8.2.3 FinishingEmotionFeedback 151
8.2.4 ShowingaFeedbackRequestNotification 158
(cid:2)
(cid:2)
Contents ix
8.3 ProcessingUserFeedback 168
8.3.1 ProcessingFeedbackontheEmotionTasker 168
8.3.2 TrainingtheNeuralNetwork 170
8.3.3 SendingEmotionalInformationtotheServer 173
8.4 InSummary... 176
9 Actuation 179
9.1 HandlingEmotionsontheServer 179
9.1.1 ParsingJSONRequests 180
9.1.2 CreatingtheWebInterface 189
9.1.3 CreatingtheServer’sBackgroundThread 190
9.1.4 ProcessingIncomingEmotions 192
9.1.5 PruningOutdatedEmotions 208
9.2 FinishingupEmotionTasker 210
9.2.1 HandlingANNOutput 212
9.2.2 PostingNewEmotionInferences 217
9.3 ProvidingPositiveReinforcement 226
9.3.1 CreatingaMotivationalDialogBox 226
9.3.2 EnablingtheEmotionHeatmaps 234
9.4 InSummary… 237
(cid:2) PartIII FutureofHuman-In-the-LoopCyber-PhysicalSystems 239 (cid:2)
10 RequirementsandChallengesforHiTLApplications 241
10.1 Resilience 241
10.2 SecurityandPrivacy 242
10.3 StandardCommunications 244
10.4 Localization 248
10.5 StateInference 248
10.6 Safety 250
10.7 InSummary… 250
11 Human-in-the-LoopConstraints 253
11.1 TechnicalLimitations 253
11.2 Ethicallimitations 256
A EmotionTasker’sfullcode 261
References 275
Index 289
(cid:2)
(cid:2)
xi
ListofFigures
2.1 In[1],booksandothercommonobjectswereaugmentedwithRFID
tagsandassociatedwithvirtualdocumentsbyPDAs. 12
2.2 Shaman[2]actedasarepresentativefortheconnectedLiteServers,
offeringJavaandHTMLinterfaces. 12
2.3 DevicewebpresenceinCooltown[3].Source:Adaptedfrom
Kindbergetal.2002. 13
2.4 JXTA[4]peerscreatedvirtualadhocnetworkswhichservedto
abstracttherealones. 14
(cid:2) (cid:2)
2.5 Workssuchas[5]and[6]usedproxiestoofferembeddeddevices’
capabilitiesthroughRESTfulwebservices. 14
2.6 TheSenseWeb[7]architecture. 16
2.7 WikiCity[8]interfacedbetweenvirtualdataandthephysicalworld
throughasemanticallydefinedformatfordataexchange. 18
2.8 Nokia6101vsiPhone6s/LGNexus5X. 20
2.9 HiTLtechnologiesevolutiontimeline. 22
3.1 Basicprocessesofhuman-in-the-loopcontrol. 26
3.2 Taxonomyofhumancontrol. 27
3.3 Taxonomyofhumanroles. 28
4.1 SenQ’squerysystemstackshownside-by-sidewiththetopologyand
componentsofAlarmNet,aprototypicalimplementationfor
assisted-living[9].Source:AdaptedfromWood2008. 37
4.2 ThearchitectureofCenceME[10],oneofMetroSense’s
implementations. 38
4.3 ThethreekeycomponentsofBCIusingsmartphones[11].Source:
AdaptedfromLathiaetal.2013. 40
4.4 SociableSensearchitecture[12].Source:AdaptedfromRachuri
2011. 41
(cid:2)
(cid:2)
xii ListofFigures
4.5 ControlarchitectureforenergysavingwithHiTL[13].Source:
AdaptedfromLiang2013. 46
4.6 ArchitectureofanHiTLHVACsystem[14].Source:Adaptedfrom
Agarwal2011. 48
4.7 DiagramshowingthemaincomponentsofCAALYX’sroaming
monitoringsystem[15].Source:AdaptedfromBoulosetal.
2007. 49
4.8 Asemi-autonomouswheelchairreceivesbrainsignalsfromtheuser
andexecutestheassociatedtasksofpathplanning,obstacle
avoidance,andlocalization[16].Source:AdaptedfromSchirner
2013. 50
4.9 AmockupofamapinterfacesimilartotheHighlight
application. 52
4.10 Overviewofthesystemproposedin[17].Source:Adaptedfrom
W.-H.RhoandS.-B.Cho2014. 54
5.1 HappyWalkHiTLcontrol. 60
5.2 HappyWalk’sarchitecture. 61
5.3 Android’sactivitylifecycle. 62
(cid:2) (cid:2)
5.4 HappyWalk’sAndroidclassstructure. 63
5.5 AnoverviewofHappyWalkAndroidapp’smainclasses. 64
5.6 AnoverviewofHappyWalkServer’smainclasses. 66
5.7 Atypicalartificialneuralnetworkarchitecture. 68
5.8 Soundsignalinthetimedomain(leftside)analyzedthrougha
Fouriertransformationtoshowitsfrequencydomain
(rightside). 69
5.9 HappyWalk’sEmotionalFeedback. 70
5.10 HappyWalk’sneuralnetworkdesign. 72
6.1 InstallingJavaSEDevelopmentKit7u79. 74
6.2 InstallingAndroidStudioandAndroidSDK. 74
6.3 Cancelingthesetupwizard. 75
6.4 OpeningtheAndroidSDKmanager. 75
6.5 InstallingAndroidAPI21. 76
6.6 OpeningthestandaloneSDKmanager. 76
6.7 InstallingAndroidSDKBuild-tools21.1.2. 77
6.8 InstallingGit#1.(a)AddingGittothePATH,onWindows
(b)ChooseCheckoutWindows-style 78
(cid:2)
(cid:2)
ListofFigures xiii
6.9 InstallingGit#2.(a)WerecommendusingMinTTY(b)Uncheck
Enablefilesystemcaching 78
6.10 ImportingHappyWalkfromGit. 79
6.11 CloningtheHappyWalkproject. 79
6.12 OpeningtheHappyWalkproject. 79
6.13 ChoosingHappyWalk’sprojectfolder. 80
6.14 DonotupgradeAndroidGradleoritsplugin. 80
6.15 RunningHappyWalk. 81
6.16 HappyWalk’sfirstlaunch. 81
6.17 ObtainingtheAndroiddebugkey. 82
6.18 CreatingaprojecttoobtainaGoogleMapsAndroidAPIkey. 82
6.19 CreatingtheGoogleMapsAndroidAPIkey. 83
6.20 ObtainingtheGoogleMapsAndroidAPIkey. 83
6.21 Changingintotheproject’sview. 84
6.22 Openingapp/debug/res/values/google_maps_api.xml. 84
(cid:2) 6.23 ChoosingPostgreSQLsuperuser’spassword. 86 (cid:2)
6.24 NoneedtolaunchStackBuilder. 86
6.25 ClonefromaURI. 87
6.26 IntroducetheURIcorrespondingtoHappyWalk’sserver. 87
6.27 Selectthemasterbranch. 88
6.28 Selectingthelocalstoragedirectory. 89
6.29 SelecttheoptionImportexistingEclipseprojects. 89
6.30 TickthecheckboxoftheHappyWalkServerproject. 90
®
6.31 CreatingaFoursquare app. 90
®
6.32 Foursquare ’sClientIDandClientSecret. 90
6.33 Navigatingintotheserver’sGlobalVariables. 91
6.34 LogintothePostgreSQL9.3server. 92
6.35 Createanewdatabase. 92
6.36 Namethenewdatabaseashappywalk. 93
6.37 SelectthecorrectSQLscript. 93
6.38 Populatingthedatabase. 94
6.39 Createanewserver. 95
(cid:2)