Visualizing and Predicting the Effects of Rheumatoid Arthritis on Hands

123 Pages·2016·5.84 MB·English

UUnniivveerrssiittyy ooff KKeennttuucckkyy
UUKKnnoowwlleeddggee
Theses and Dissertations--Computer Science Computer Science
2014
VViissuuaalliizziinngg aanndd PPrreeddiiccttiinngg tthhee EEffffeeccttss ooff RRhheeuummaattooiidd AArrtthhrriittiiss oonn
HHaannddss
Radu P. Mihail
University of Kentucky, [email protected]
RRiigghhtt cclliicckk ttoo ooppeenn aa ffeeeeddbbaacckk ffoorrmm iinn aa nneeww ttaabb ttoo lleett uuss kknnooww hhooww tthhiiss ddooccuummeenntt bbeenneefifittss yyoouu..
RReeccoommmmeennddeedd CCiittaattiioonn
Mihail, Radu P., "Visualizing and Predicting the Effects of Rheumatoid Arthritis on Hands" (2014). Theses
and Dissertations--Computer Science. 19.
https://uknowledge.uky.edu/cs_etds/19
This Doctoral Dissertation is brought to you for free and open access by the Computer Science at UKnowledge. It
has been accepted for inclusion in Theses and Dissertations--Computer Science by an authorized administrator of
UKnowledge. For more information, please contact [email protected].
SSTTUUDDEENNTT AAGGRREEEEMMEENNTT::
I represent that my thesis or dissertation and abstract are my original work. Proper attribution
has been given to all outside sources. I understand that I am solely responsible for obtaining
any needed copyright permissions. I have obtained needed written permission statement(s)
from the owner(s) of each third-party copyrighted matter to be included in my work, allowing
electronic distribution (if such use is not permitted by the fair use doctrine) which will be
submitted to UKnowledge as Additional File.
I hereby grant to The University of Kentucky and its agents the irrevocable, non-exclusive, and
royalty-free license to archive and make accessible my work in whole or in part in all forms of
media, now or hereafter known. I agree that the document mentioned above may be made
available immediately for worldwide access unless an embargo applies.
I retain all other ownership rights to the copyright of my work. I also retain the right to use in
future works (such as articles or books) all or part of my work. I understand that I am free to
register the copyright to my work.
RREEVVIIEEWW,, AAPPPPRROOVVAALL AANNDD AACCCCEEPPTTAANNCCEE
The document mentioned above has been reviewed and accepted by the student’s advisor, on
behalf of the advisory committee, and by the Director of Graduate Studies (DGS), on behalf of
the program; we verify that this is the final, approved version of the student’s thesis including all
changes required by the advisory committee. The undersigned agree to abide by the statements
above.
Radu P. Mihail, Student
Dr. Nathan Jacobs, Major Professor
Dr. Miroslaw Truszczynski, Director of Graduate Studies
VisualizingandPredictingtheEffectsofRheumatoidArthritisonHands
DISSERTATION
Adissertationsubmittedinpartial
fulfillmentoftherequirementsforthe
degreeofDoctorofPhilosophyinthe
CollegeofEngineeringatthe
UniversityofKentucky
By
RaduP.Mihail
Lexington,Kentucky
Co-Directors: Dr. NathanJacobsandDr. JudyGoldsmith
ProfessorsofComputerScience
Lexington,Kentucky2014
Copyright(cid:13)c RaduP.Mihail2014
ABSTRACTOFDISSERTATION
VisualizingandPredictingtheEffectsofRheumatoidArthritisonHands
This dissertation was inspired by difficult decisions patients of chronic diseases have
to make about about treatment options in light of uncertainty. We look at rheumatoid
arthritis (RA), a chronic, autoimmune disease that primarily affects the synovial joints of
the hands and causes pain and deformities. In this work, we focus on several parts of a
computer-based decision tool that patients can interact with using gestures, ask questions
aboutthedisease,andvisualizepossiblefutures. Weproposeahandgesturebasedinterac-
tionmethodthatiseasilysetupinadoctor’sofficeandcanbetrainedusingacustomsetof
gestures that are least painful. Our system is versatile and can be used for operations like
simple selections to navigating a 3D world. We propose a point distribution model (PDM)
thatiscapableofmodelinghanddeformitiesthatoccurduetoRAandageneralizedfitting
method for use on radiographs of hands. Using our shape model, we show novel visu-
alization of disease progression. Using expertly staged radiographs, we propose a novel
distance metric learning and embedding technique that can be used to automatically stage
anunlabeledradiograph. Givenalargesetofexpertlylabeledradiographs,ourdata-driven
approachcanbeusedtoextractdifferentmodesofdeformationspecifictoadisease.
KEYWORDS: RA,deformities,visualization,hands,xrays
RaduP.Mihail
Date: 05/02/2014
VisualizingandPredictingtheEffectsofRheumatoidArthritisonHands
By
RaduP.Mihail
Co-DirectorsofDissertation: Dr. NathanJacobsandDr. JudyGoldsmith
DirectorofGraduateStudies: Dr. MiroslawTruszczynski
Date: 05/02/2014
Formywife,Marina.
ACKNOWLEDGMENTS
This dissertation would not be possible without the help of many people throughout the
years. I was fortunate to have been advised by two amazing people, who put up with me
for the duration of my graduate studies and helped minimize failure modes. Thank you
Judy Goldsmith and Nathan Jacobs for the cool courses I took from both of you, for the
encouragementIreceived,fortheadviceyou’vegiven,fortheimmaculateprofessionalism
you showed during the years, and for the support to pursue my interests. This dissertation
wouldnothavebeenpossiblewithoutyou.
I take this opportunity to thank the rest of my committee, Ruigang Yang, Sen-ching
(Samson) Cheung, Ruigang Yang and Kristine Lohr for their invaluable comments and
feedback over the years. I would like to thank Kristine Lohr and Gustav Blomquist for
everything, especially the help with the medical IRB and the invaluable radiographs used
throughoutthiswork.
My immediate family has been there for me during the ups and downs, thanks for the
help you’ve given along the way to keep me in touch with my better self, Lili, Walter—
mayyourgentlesoulrestinpeace,BebiandAdi. ThankyouUnchiu’(Adrian)forhelpon
countlessoccasions. ThankyouRandyandLaurenforyoursupportandbelievinginme.
LastandbyfarmosttoMarina,foreverything.
iii
TABLEOFCONTENTS
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
TableofContents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
ListofFigures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
ListofTables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Chapter1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 MainContributionsandRelatedPublications . . . . . . . . . . . . . . . . 1
1.3 StructureoftheDissertation . . . . . . . . . . . . . . . . . . . . . . . . . 5
Chapter2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1 RheumatoidArthritis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 HumanComputerInteractionfortheDisabled . . . . . . . . . . . . . . . . 13
2.3 ConditionalRandomFields . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 MachineLearningPreliminaries . . . . . . . . . . . . . . . . . . . . . . . 16
Chapter3 DecisionAidsforRAPatients . . . . . . . . . . . . . . . . . . . . . . 21
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 ProbabilitiesandRisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 RATreatmentOptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4 DifficultiesinChoosingaTreatment . . . . . . . . . . . . . . . . . . . . . 25
3.5 PatientDAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.6 AvailableDAsforRATreatments . . . . . . . . . . . . . . . . . . . . . . 27
3.6.1 Methotrexate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.6.2 BiologicAgents . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Chapter4 KinectBasedHandGestureRecognition . . . . . . . . . . . . . . . . . 34
4.0.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1 AReal-TimeGestureRecognitionSystem . . . . . . . . . . . . . . . . . . 38
4.1.1 UsageScenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.1.2 DataAcquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.1.3 WorldCoordinateSystem . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 GestureRecognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2.1 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2.2 RotationInvariance . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.3 Descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.4 Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
iv
4.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3.1 TrainingData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.3.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Chapter5 ACRF-basedApproachtoFittingGeneralizedHandSkeletonModels . 52
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.2 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.3 ProblemDefinition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.3.1 ShapeModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.3.2 KeyChallenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5.4 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4.1 PotentialFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.4.2 ShapeModelPrior . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.4.3 ShapeInference . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.4.4 EstimatingCRFWeights . . . . . . . . . . . . . . . . . . . . . . . 64
5.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Chapter6 DiseaseStageMetricLearning . . . . . . . . . . . . . . . . . . . . . . 70
6.1 Background: SupervisedMetricLearning . . . . . . . . . . . . . . . . . . 70
6.2 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.3 OurSemi-SupervisedApproach . . . . . . . . . . . . . . . . . . . . . . . 73
6.4 DiseaseStageMetricLearning . . . . . . . . . . . . . . . . . . . . . . . . 73
6.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.5.1 LinearGenerativeDataModel . . . . . . . . . . . . . . . . . . . . 74
6.5.2 EmbeddingsusingSyntheticData . . . . . . . . . . . . . . . . . . 76
6.5.3 ClassificationonSyntheticData . . . . . . . . . . . . . . . . . . . 78
6.6 EmbeddingofRAhandshapes . . . . . . . . . . . . . . . . . . . . . . . . 78
6.7 ImplementationDetails . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Chapter7 VisualizingHandsAffectedbyRA . . . . . . . . . . . . . . . . . . . . 83
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
7.3 Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.3.1 DataAcquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.3.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7.5 FutureWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Chapter8 ConclusionsandFutureDirections . . . . . . . . . . . . . . . . . . . . 95
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
v
Vita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
vi