Marking Time: Increased Scope and Accuracy for Sketch Classification of the Clock Drawing Test ...

86 Pages·2015·1.75 MB·English

Marking Time: Increased Scope and Accuracy for
Sketch Classification of the Clock Drawing Test
by
Jonathan Chien
Submitted to the Department of Electrical Engineering and Computer
Science
in partial fulfillment of the requirements for the degree of
Master of Engineering in Electrical Engineering and Computer Science
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June 2015
○c Massachusetts Institute of Technology 2015. All rights reserved.
Author ................................................................
Department of Electrical Engineering and Computer Science
May 22, 2015
Certified by............................................................
Randall Davis
Professor
Thesis Supervisor
Accepted by ...........................................................
Albert R. Meyer
Chairman, Master of Engineering Thesis Committee
2
Marking Time: Increased Scope and Accuracy for Sketch
Classification of the Clock Drawing Test
by
Jonathan Chien
Submitted to the Department of Electrical Engineering and Computer Science
on May 22, 2015, in partial fulfillment of the
requirements for the degree of
Master of Engineering in Electrical Engineering and Computer Science
Abstract
In this thesis, I designed and implemented improvements to an automatic classifier
for the digitized clock drawing test, a diagnostic tool for assessing cognitive impair-
ment, which asks the patient to draw an analog clock face using a digital pen. The
classifier handles both the grouping of strokes into clock components and the sub-
sequent labeling of those groups. Despite the domain-specificity, classification is a
challenging problem because the subject often has cognitive or motor impairments.
It is thus important for the classifier to be able to handle a wide range of input with
distorted, overwritten, or missing components. I improve the robustness of the classi-
fier, particularly for messy clinical data, by incorporating intrinsic stroke properties,
developing additional symbol recognizers, and creating a global context evaluator. I
describe in this thesis properties for isolated symbol recognition, features for symbol
recognition and match scoring, as well as common sense rules based on a symbol’s
local and global context in a drawing. I combine these elements into a new system
that locally maximizes a global label assignment based on match quality and context.
I demonstrate that this system accurately recognizes a wide variety of clinical input,
improving overall classification performance.
Thesis Supervisor: Randall Davis
Title: Professor
3
4
Acknowledgments
I would like to acknowledge Professor Randall Davis for his ideas, guidance, and
feedback over the past year. Not only did I learn a tremendous amount technically,
but I also learned a lot about communication and academic research, which I hope
to continue applying in future pursuits.
I’d also like to thank the rest of my research group, my friends, and my family for
their support and encouragement throughout my time at MIT.
5
6
Contents
1 Introduction and Motivation 11
1.1 The Digital Clock Drawing Test . . . . . . . . . . . . . . . . . . . . 11
1.2 Sketch Recognition and ClockSketch . . . . . . . . . . . . . . . . . . 12
2 Definitions and Data 15
2.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.1 Data Representation . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.2 Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.3 Properties of Strokes, Symbols, or Drawings . . . . . . . . . . 17
2.1.4 Other Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Data Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3 Background and Previous Work 21
3.1 Segmentation and Grouping . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2.1 Numerals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2.2 Hands, Clockface, and Center Dot . . . . . . . . . . . . . . . . 23
3.3 Previous ClockSketch Program . . . . . . . . . . . . . . . . . . . . . 24
3.3.1 Noise and Clockface Detection . . . . . . . . . . . . . . . . . . 24
3.3.2 Inner and Outer Stroke Separation . . . . . . . . . . . . . . . 25
3.3.3 Numeral Segmentation and Labeling . . . . . . . . . . . . . . 25
3.3.4 Hands and Center Dot Labeling . . . . . . . . . . . . . . . . . 27
3.3.5 Crossout Detection and Error Correction . . . . . . . . . . . . 27
7
3.3.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4 Improvements to Previous ClockSketch Classifier 33
4.1 Clockface Identification by Shape . . . . . . . . . . . . . . . . . . . . 33
4.1.1 Previous ClockSketch Behavior . . . . . . . . . . . . . . . . . 33
4.1.2 Improvement Using Inherent Stroke Properties . . . . . . . . . 35
4.1.3 Possible Further Improvements . . . . . . . . . . . . . . . . . 38
4.2 Elliptical Clockfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2.1 Previous ClockSketch Behavior . . . . . . . . . . . . . . . . . 40
4.2.2 Using a Distance Radius Ratio . . . . . . . . . . . . . . . . . 41
4.3 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.4 Tick Mark Labeling Support . . . . . . . . . . . . . . . . . . . . . . . 42
4.4.1 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.4.2 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.4.3 Further Improvements . . . . . . . . . . . . . . . . . . . . . . 45
5 Local Classifiers and Global Evaluation System 47
5.1 Context and System Motivation . . . . . . . . . . . . . . . . . . . . . 47
5.2 Hands, Center Dot, and Clockface Scorer . . . . . . . . . . . . . . . . 49
5.2.1 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.2.2 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.3 Segmentation Repair with Perturbation . . . . . . . . . . . . . . . . . 51
5.4 Global Context Evaluator . . . . . . . . . . . . . . . . . . . . . . . . 52
5.4.1 Common Sense Rules . . . . . . . . . . . . . . . . . . . . . . . 53
5.4.2 Weighting Strategy . . . . . . . . . . . . . . . . . . . . . . . . 55
6 Final System Performance and Conclusion 57
6.1 Overall Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.2 Comparison of Results . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.3.1 Other Classes of Strokes . . . . . . . . . . . . . . . . . . . . . 59
8
6.3.2 State-based Segmentation Approach . . . . . . . . . . . . . . . 60
6.3.3 Parameter Analysis and Other Probabilistic Algorithms . . . . 61
A Clock Drawing Data Sets 63
A.1 YDU-51 Healthy Training Set . . . . . . . . . . . . . . . . . . . . . . 63
A.2 YDU-100 Healthy Testing Set . . . . . . . . . . . . . . . . . . . . . . 65
A.3 VIN-96 Clinical Test Set . . . . . . . . . . . . . . . . . . . . . . . . . 68
A.4 EGE/ORU-112 Clinical Test Set . . . . . . . . . . . . . . . . . . . . . 72
A.5 EMD-20 Clinical Test Set . . . . . . . . . . . . . . . . . . . . . . . . 75
A.6 CIN-170 Clinical Test Set . . . . . . . . . . . . . . . . . . . . . . . . 76
A.7 TICK-74 Clinical Test Set . . . . . . . . . . . . . . . . . . . . . . . . 82
9
10