Automatic Determination of Skeletal Maturity using Statistical Models of Appearance

184 Pages·2010·14.48 MB·English

Automatic Determination of
Skeletal Maturity using Statistical
Models of Appearance
A thesis submitted to the University of Manchester for the degree of
Doctor of Philosophy in the Faculty of Medical and Human Sciences
2010
Steve Adetunji Adeshina
School of Medicine
Contents
1 Introduction 21
1.1 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.2 Overview of Skeletal Maturity Estimation . . . . . . . . . . . . . . . 21
1.3 Current methods of determining skeletal maturity . . . . . . . . . . . 24
1.4 Automatic determination of skeletal maturity - System Overview . . . 24
1.5 Challenges of building the system . . . . . . . . . . . . . . . . . . . . 25
1.6 Novel contributions of the Thesis . . . . . . . . . . . . . . . . . . . . 28
1.7 Overview of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2 Background 30
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2 Clinical . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2.1 Measurement of maturity . . . . . . . . . . . . . . . . . . . . 31
2.2.2 Methods for determination of skeletal maturity . . . . . . . . 32
2.2.3 Maturity growth points in the hand and the wrist . . . . . . . 36
2.2.4 Imaging methods for skeletal maturity . . . . . . . . . . . . . 38
2.2.5 Alternative imaging techniques . . . . . . . . . . . . . . . . . 40
2.2.6 Applications of skeletal maturity . . . . . . . . . . . . . . . . 43
2.3 Computer assisted methods . . . . . . . . . . . . . . . . . . . . . . . 44
2
Contents
2.3.1 Automated methods of skeletal maturity estimation . . . . . 45
2.3.2 Concluding remarks on reviewed methods . . . . . . . . . . . 50
2.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3 Model Based Vision 53
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2 Deformable Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.2.1 Snakes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.2.2 Eigen models . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.2.3 Shape models . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3 Point Distribution Models . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4 Active Shape Models (ASM) . . . . . . . . . . . . . . . . . . . . . . 59
3.4.1 Summary of ASM . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4.2 Active Shape Model Search . . . . . . . . . . . . . . . . . . . 59
3.5 Active Appearance Models . . . . . . . . . . . . . . . . . . . . . . . . 61
3.5.1 Background to the Active Appearance Model . . . . . . . . . 61
3.5.2 Statistical Models of Appearance . . . . . . . . . . . . . . . . 62
3.5.3 Fitting Appearance Models . . . . . . . . . . . . . . . . . . . 64
3.5.4 Active Appearance Model search . . . . . . . . . . . . . . . . 66
3.5.5 AAM Model Refinement . . . . . . . . . . . . . . . . . . . . . 67
3.5.6 Initialisation . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.6 Automatic Construction of Models . . . . . . . . . . . . . . . . . . . 68
3.6.1 Review of groupwise registration . . . . . . . . . . . . . . . . . 68
3.7 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 69
4 Methods 71
Word Count: 50644 3
Contents
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Overview of complete system . . . . . . . . . . . . . . . . . . . . . . 71
4.3 Details of major parts of system . . . . . . . . . . . . . . . . . . . . . 73
4.3.1 Part based models(PBM). . . . . . . . . . . . . . . . . . . . . 73
4.3.2 Active appearance models and extensions . . . . . . . . . . . . 77
4.3.3 Estimation of skeletal maturity . . . . . . . . . . . . . . . . . 80
4.4 Training and Automatic registration techniques . . . . . . . . . . . . 82
4.4.1 Summary of Training and registration techniques . . . . . . . 82
4.4.2 Groupwise registration framework . . . . . . . . . . . . . . . . 83
4.4.3 Groupwise registration algorithm . . . . . . . . . . . . . . . . 85
4.4.4 Deformation fields . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4.5 Local normalization . . . . . . . . . . . . . . . . . . . . . . . . 87
4.5 Building Part Based Models (PBM) for groupwise registration . . . . 88
4.6 Evaluation of registration and models . . . . . . . . . . . . . . . . . . 91
4.6.1 Evaluation of registration and correspondences . . . . . . . . . 92
4.7 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 93
5 Statistical models of bones in hand radiographs 95
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.2 Model building for skeletal maturity . . . . . . . . . . . . . . . . . . . 96
5.3 Process overview of model building . . . . . . . . . . . . . . . . . . . 97
5.4 Model building methodology . . . . . . . . . . . . . . . . . . . . . . . 98
5.5 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.6.2 Part-based model experiments . . . . . . . . . . . . . . . . . . 102
Word Count: 50644 4
Contents
5.6.3 Global registration and modeling experiments . . . . . . . . . 106
5.6.4 Local annotation and modeling experiments . . . . . . . . . . 112
5.7 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 118
6 Locating bones in a radiograph using statistical models 119
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.2 Process diagram for automatic matching of hand radiograph . . . . . 120
6.3 Methods for Model Matching . . . . . . . . . . . . . . . . . . . . . . 120
6.4 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.4.1 Constrained Active Appearance Model fitting experiments . . 122
6.4.2 Local model fitting experiments . . . . . . . . . . . . . . . . . 125
6.5 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 129
7 Automatic determination of skeletal maturity. 131
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
7.2 Automatic determination of Skeletal Maturity . . . . . . . . . . . . . 131
7.3 Method for Maturity Determination . . . . . . . . . . . . . . . . . . . 133
7.3.1 Estimating Skeletal Maturity from model parameters . . . . . 133
7.3.2 Skeletal age estimation concepts . . . . . . . . . . . . . . . . . 134
7.3.3 Evaluating precision . . . . . . . . . . . . . . . . . . . . . . . 135
7.4 Preliminary experiments . . . . . . . . . . . . . . . . . . . . . . . . . 135
7.4.1 Global vs local models . . . . . . . . . . . . . . . . . . . . . . 136
7.4.2 Manual vs automatic registration . . . . . . . . . . . . . . . . 137
7.4.3 Shape, appearance and texture parameters compared . . . . . 137
7.4.4 Multiple Carpal models vs single combined model . . . . . . . 137
7.4.5 13 RUS Joint complex models vs 8 bone models . . . . . . . . 138
Word Count: 50644 5
Contents
7.4.6 Conclusion from preliminary experiments . . . . . . . . . . . . 139
7.5 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
7.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
7.6.1 Comparing Predicted Age with Chronological Age . . . . . . . 142
7.6.2 Comparing Predicted Age with expert TW3 readings. . . . . . 145
7.6.3 Comparing Predicted Age with Consensus Skeletal Age . . . . 148
7.6.4 Consolidated accuracy results . . . . . . . . . . . . . . . . . . 151
7.6.5 Precision experiments . . . . . . . . . . . . . . . . . . . . . . . 152
7.6.6 Validation experiments . . . . . . . . . . . . . . . . . . . . . . 154
7.6.7 Accuracy of manual rating . . . . . . . . . . . . . . . . . . . . 156
7.7 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 159
8 Estimating skeletal maturity for infants. 162
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.3.1 Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 169
9 Conclusions and Further Work 170
9.1 Summary of original work and results . . . . . . . . . . . . . . . . . . 170
9.1.1 AAM methods development . . . . . . . . . . . . . . . . . . . 170
9.1.2 Registration and annotation of bones . . . . . . . . . . . . . . 171
9.1.3 Automatic location of bones . . . . . . . . . . . . . . . . . . . 171
9.1.4 Skeletal maturity . . . . . . . . . . . . . . . . . . . . . . . . . 172
Word Count: 50644 6
Contents
9.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
9.2.1 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
9.2.2 Other modalities . . . . . . . . . . . . . . . . . . . . . . . . . 173
9.2.3 Part and geometry model improvement . . . . . . . . . . . . . 173
9.2.4 Skeletal age estimators . . . . . . . . . . . . . . . . . . . . . . 174
9.3 Final Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Word Count: 50644 7
List of Tables
5.1 Statistics of mean distance errors for models with different numbers of
parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.2 Statistics of mean distance errors of models for the three age-groups
from the Part Based Models (PBM) . . . . . . . . . . . . . . . . . . . 104
5.3 Statistics of point location error after dense registration for age-group
1 (mm). Distance errors below 2mm (mm) is acceptable after dense
registration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.4 Statistics of point location error after dense registration for Age-group
2 (mm). Distance errors below 2mm (mm) is acceptable after dense
registration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.5 Statistics of point location error after dense registration for Age-group
3 (mm). Distance errors below 2mm (mm) is acceptable after dense
registration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.6 Statistics of point to curve error after dense propagation for Age-group
1 (mm) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.7 Statistics of point to curve error after dense propagation for Age-group
2 (mm) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
5.8 Statistics of point to curve error after dense propagation for Age-group
3 (mm) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
5.9 Statistics of point location repeatability error after dense registration
for the three Age groups (mm) . . . . . . . . . . . . . . . . . . . . . . 115
6.1 Search error statistics. Point to point errors (mm) of the global model
fit for Initial points from TPS warp and After CAAM refinement. This
is compared with equivalent manual annotations as ground truth. . . 123
8
List of Tables
6.2 Search error statistics. Point to point errors (mm) of local models
without and with refinement with CAAM . . . . . . . . . . . . . . . 126
7.1 Mean absolute age prediction error (years) from a model of the whole
hand. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7.2 Average performance of single bone complex models, with and without
automatic registration (Mean absolute error in years). . . . . . . . . . 137
7.3 Average performance of local models - Mean absolute predictions error
(years) of RUS13 complexes and 20 bone complexes. . . . . . . . . . . 138
7.4 Mean absolute prediction error (years) for six combined bone models
and the equivalent average of 2 single RUS Complex. . . . . . . . . . 139
7.5 Mean absolute predictions error (years) using average predictions of
constituent bone models. . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.6 Mean absolute predictions error (years), root mean square (rms) error
(years) and number of images, using Chronological Age as the ground
truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.7 Mean absolute predictions error (years), root mean square (rms) error
(years) and number of images, using TW 3 expert readings as the
ground truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
7.8 Mean absolute prediction error (years), root mean square (rms) error
(years) and number of images, using Consensus Skeletal Age as the
ground truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7.9 Mean absolute prediction error (years), root mean square (rms) error
(years) and number of images. Predictions from Chronological Age,
Consensus Skeletal Age and TW3 readings respectively . . . . . . . . 151
7.10 Mean absolute predictions error (years), root mean square (rms) error
(years) and number of images. Predictions comparing Chronological
age with both Intrinsic Skeletal Age and manual TW3 readings. . . . 156
8.1 Average performance various carpal based models - Mean absolute
prediction error for female and male (years). Letters correspond to
the description in Section 8.4 and models shown in Figure 8.3 . . . . 166
9
List of Figures
1.1 The bones of the hand and the wrist and radial growth plate . . . . . 22
1.2 Process diagram for automatic matching of hand radiographs . . . . . 25
1.3 Research challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.4 a) Example of a hand radiograph of a child with no bones, b) Carpal
bones labeled and c) carpal region with two bones. . . . . . . . . . . 27
1.5 Examples of hand radiographs with varying orientation. . . . . . . . . 27
2.1 Epiphysis-metaphysis growth region . . . . . . . . . . . . . . . . . . . 33
2.2 TW2/3 stages of maturation (B to I) in Radius and ulna . . . . . . . 34
2.3 Global appearance and shape models . . . . . . . . . . . . . . . . . . 35
2.4 (a) Greulich and pyle growth points . . . . . . . . . . . . . . . . . . . 37
2.5 X-rays generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1 Global shape models . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.2 Global and local appearance models . . . . . . . . . . . . . . . . . . . 65
4.1 Process diagram for automatic matching of hand radiographs . . . . . 72
4.2 Multi resolution patch models showing best 15 matches to model built
from ‘patch’ shown in first image. Intensity indicates rank (brigh-
test=best match) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.3 Process diagram for automatic matching of hand radiographs . . . . . 85
4.4 Triangular mesh of a piecewise affine deformation. . . . . . . . . . . . 87
10