Alexia Briassouli · Jenny Benois-Pineau Alexander Hauptmann Editors Health Monitoring and Personalized Feedback using Multimedia Data Health Monitoring and Personalized Feedback using Multimedia Data Alexia Briassouli • Jenny Benois-Pineau Alexander Hauptmann Editors Health Monitoring and Personalized Feedback using Multimedia Data 123 Editors AlexiaBriassouli JennyBenois-Pineau CentreforResearchandTechnology,Hellas LaboratoireBordelaisdeRecherche InformationTechnologiesInstitute enInformatique/UniversityBordeaux Thermi,Thessaloniki,Greece Talence,France AlexanderHauptmann ComputerScienceDepartment CarnegieMellonUniversity Pittsburgh,PA,USA ISBN978-3-319-17962-9 ISBN978-3-319-17963-6 (eBook) DOI10.1007/978-3-319-17963-6 LibraryofCongressControlNumber:2015939176 SpringerChamHeidelbergNewYorkDordrechtLondon ©SpringerInternationalPublishingSwitzerland2015 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade. Printedonacid-freepaper SpringerInternational PublishingAGSwitzerlandispartofSpringerScience+Business Media(www. springer.com) Preface Astheworldpopulationagesandthecareratio(ratioofhealthyyoungcitizensto elderly citizens) is in decline, monitoringand care of individualswith continuous recording of medical information in electronic form, remotely or during in-site medicalvisits, is becomingmore and moreincorporatedin daily life. Multimodal technologiesareconstantlybeingdevelopedtoassistpeopleintheirdailylife,witha vastrangeofwearablesensorsnowavailableformonitoringhealthparameters(e.g., bloodpressure,sweat,bodytemperature,heartrate,etc.),lifestyle(e.g.,monitoring utilityuse,levelsofactivity,sleepquantityandquality,etc.),andaperson’sability to carry out activities of daily living. At the same time, health professionals have integratednewtechnologiesintotheirworkflow,forexamplebyusingvarioustypes ofmedicalimagerytofacilitateandsupporttheirclinicalpracticeanddiagnosis,and alsobyexaminingdatafromsensorsandhomemedicaldevices,whichallowthem to remotelycarefor theirpatients. Health recordsand databasesare now enriched with digital multimodal data on the patients, for which new methods need to be developedforaccurateandfastaccessandretrieval. OneofthestartingpointsforthisbookwastheEUfundedprojectDem@Care. InDem@Care,amulti-modalsensingplatformwasdevelopedfortheremotemulti- modalmonitoringandcareofpeoplewithdementia.Intelligentfusionanddecision supportmade its outcomesimmediatelyuseable byclinicians, informalcaregivers and also the people with dementia, via appropriate interfaces. This book covers, initsthreeparts,theabove-mentionedaspectsonthepresenceofmultimodaldata in healthcare, both for individuals (patients, family, and friends) and for medical professionals. The first part entitled “Multimedia and Multimodal Pattern Recognition for HealthcareApplications”examinestheanalysisofdifferentkindsofmedicalimages and video data for diagnosis, evaluation of the health status of an individual, and theirabilitytolivealone.Frommedicalimageanalysistomultimodalvideoanalyt- ics,thispartpresentsnewmultimediatechnologiesfromhealthcareapplications. v vi Preface The second part “Multimedia Analysis and Feedback in Medicine” presents currentsolutionstoprovidefeedbacktopatients,forsupport,training,andrehabili- tationpurposes,orformonitoringthemforsafetyandprovidingfeedbackifneeded. All its chapters present tools which directly mainly communicatewith patients in anefficientrecommendationorself-assessmentprotocols. Finally,thethirdpartcovers“MultimediaandTechnologyinMedicine,”where aspectsofaccessingmultimodalhealth-relateddata(e.g.,electronichealthrecords, medicalimages)arepresented. We hope that this book will give medical practitionersand general scientists a goodoverviewofthemostadvancedpracticesandtoolsofmultimediadataanalysis forhealthandwillhelpthemintheireverydaypractice. Thermi,Thessaloniki,Greece AlexiaBriassouli Talence,France JennyBenois-Pineau Pittsburgh,PA,USA AlexanderHauptmann Contents OverviewofMultimediainHealthcare........................................ 1 AlexiaBriassouli, JennyBenois-Pineau, andAlexanderHauptmann PartI MultimediaDataAnalysis CraniofacialImageAnalysis.................................................... 9 Ezgi Mercan, Indriyati Atmosukarto, Jia Wu, Shu Liang, andLindaG.Shapiro MammographicMassDescriptionforBreastCancerRecognition......... 31 KhalifaDjemal,ImeneCheikhrouhou,andHichemMaaref Development, Debugging, and Assessment ofPARKINSONCHECKAttributesThroughVisualisation................... 47 Vida Groznik,Martin Možina,JureŽabkar,DejanGeorgiev, IvanBratko,andAleksanderSadikov MeaningfulBagsofWordsforMedicalImageClassification andRetrieval ..................................................................... 73 AntonioFoncubiertaRodríguez,AlbaGarcíaSecodeHerrera, andHenningMüller MultimediaInformationRetrievalfromOphthalmicDigitalArchives.... 95 GwenoléQuellec,MathieuLamard,BéatriceCochener, andGuyCazuguel CharacterisationofDataQualityinElectronicHealthcareRecords....... 115 SheenaDungey,NataliaBeloff,RachaelWilliams,TimWilliams, ShivaniPuri,andA.RosemaryTate vii viii Contents PartII Multimedia Event and Activity Detection andRecognitionforHealth-RelatedMonitoring ActivityDetectionandRecognitionofDailyLivingEvents ................. 139 KonstantinosAvgerinakis,AlexiaBriassouli, andIoannisKompatsiaris Recognitionof Instrumental ActivitiesofDaily Living in EgocentricVideoforActivityMonitoringofPatientswithDementia...... 161 Iván González-Díaz, Vincent Buso, Jenny Benois-Pineau, GuillaumeBourmaud,GaelleUsseglio,RémiMégret, YannGaestel,andJean-FrançoisDartigues CombiningMultipleSensorsforEventDetectionofOlderPeople......... 179 CarlosF.Crispim-Junior,QiaoMa,BaptisteFosty, RimRomdhane,FrancoisBremond,andMoniqueThonnat PartIII Multimedia-BasedPersonalizedHealthFeedback Solutions TheUseofVisualFeedbackTechniquesinBalanceRehabilitation........ 197 VassiliaHatzitaki Recommending Video Content for Use in Group-Based ReminiscenceTherapy........................................................... 215 AdamBermingham,NiamhCaprani,RonánCollins,CathalGurrin, KateIrving,JuliaO’Rourke,AlanF.Smeaton, andYangYang UsingOntologiesforManagingUserProfiles inPersonalisedMobileServiceDelivery....................................... 245 Kerry-LouiseSkillen,ChrisNugent,MarkDonnelly, LimingChen,andWilliamBurns MonitoringandCoachingtheUseofHomeMedicalDevices............... 265 YangCai,YiYang,AlexanderHauptmann,andHowardWactlar Overview of Multimedia in Healthcare AlexiaBriassouli,JennyBenois-Pineau,andAlexanderHauptmann Astheworldpopulationagesandthecareratio(ratioofhealthyyoungcitizensto elderly citizens) is in decline, monitoringand care of individualswith continuous recording of medical information in electronic form remotely or during in-site medical visits is becoming more and more incorporatedin daily life. Multimodal technologiesareconstantlybeingdevelopedtoassistpeopleintheirdailylife,witha vastrangeofwearablesensorsnowavailableformonitoringhealthparameters(e.g. bloodpressure,sweat,bodytemperature,heartrateetc.),lifestyle(e.g.monitoring utility use, levels of activity, sleep quantity and quality etc.), a person’s ability to carry out activities of daily living. At the same time, health professionals have integratednewtechnologiesintotheirworkflow,forexamplebyusingvarioustypes ofmedicalimagerytofacilitateandsupporttheirclinicalpracticeanddiagnosis,and alsobyexaminingdatafromsensorsandhomemedicaldevices,whichallowthem to remotelycarefor theirpatients. Health recordsand databasesare now enriched with digital multimodal data on the patients, for which new methods need to be developedforaccurateandfastaccessandretrieval. A.Briassouli((cid:2)) CentreforResearchandTechnology,Hellas,InformationTechnologiesInstitute, Thermi,Thessaloniki,Greece e-mail:[email protected] J.Benois-Pineau LaboratoireBordelaisdeRechercheenInformatique/UniversityBordeaux, Talence,France e-mail:[email protected] A.Hauptmann ComputerScienceDepartment,CarnegieMellonUniversity,Pittsburgh,PA,USA e-mail:[email protected] ©SpringerInternationalPublishingSwitzerland2015 1 A.Briassoulietal.(eds.),HealthMonitoringandPersonalizedFeedback usingMultimediaData,DOI10.1007/978-3-319-17963-6_1