Method and apparatus for efficient three-dimensional contouring of medical images

36 Pages·2013·2.52 MB·English

US008577107B2
(12) Ulllted States Patent (10) Patent N0.: US 8,577,107 B2
Hibbard et al. (45) Date of Patent: Nov. 5, 2013
(54) METHOD AND APPARATUS FOR EFFICIENT 6,112,109 A 8/2000 D’UISO
THREE-DIMENSIONAL CONTOURING OF Q1 1 genchimtttilet at
a a Osman e .
MEDICAL IMAGES 6,262,739 B1* 7/2001 Migdalet a1. ............... .. 345/423
6,343,936 B1 2/2002 K f t l.
(75) Inventors: Lyndon S. Hibbard, St. Louis, MO 6,606,091 B2 g/2()()3 Lflalrlléngnai a
(US); Xiao Han, Chester?eld, MO (US) 6,683,933 B2 1/2004 Saito et al.
6,947,584 B1 9/2005 Avila et al.
(73) Assignee: IS MPAC 1M edCiAca l SSy stems, Inc., 77,,101100,,518634 BB22 93//22000066 YWaemeasuec ehtl
unnyva 6, (U ) 7,167,172 B2 1/2007 Kaus et a1.
_ _ _ _ _ 7,333,644 B2 2/2008 Jerebko et a1.
( * ) Not1ce: Subject to any disclaimer, the term ofth1s 7,428,334 B2 * 9/2()()g Schoisswohl et a1, ,,,,,, ,, 332/173
patent is extended or adjusted under 35 7,620,224 B2 11/2009 Matsumoto
U_S_C_ 154(1)) by 0 days_ 8,098,909 B2 1/2012 Hibbard et a1.
2005/0168461 A1 8/2005 Acosta et al.
_ 2005/0231530 A1 10/2005 Liang et al.
(21) APP1-N°-- 13/295,525 2005/0276455 A1* 12/2005 Fidrich et a1. ............... .. 382/128
(22) Filed: Nov. 14, 2011 (Continued)
OTHER PUBLICATIONS
(65) Prior Publication Data
Stefanescu, “Parallel Nonlinear Registration of Medical Images With
Us 2012/0057769 A1 Mar’ 8’ 2012 a Priori Information on Anatomy and Pathology”, PhD Thesis.
Sophia-Antipolis: University of Nice, 2005, 140 pages.
Related US. Application Data
(Continued)
(62) Division of application No. 11/848,624, ?led on Aug.
31, 2007, noW Pat. No. 8,098,909. Primary Examiner * Anand Bhatnagar
(51) I Cl (74) Attorney, Agent, or Firm * Thompson Coburn LLP
nt. .
G06K 9/00 (2006.01) (57) ABSTRACT
(52) US‘ Cl‘ A technique is disclosed for generating a neW contour and/or
US_ PC ..........._. ........_. ......... .. 382/128; 128/922; 378/4 a 3D surface such as a var1- at1- onal 1- mpl1-c 1-t surface from con
(58) Fleld of Classl?catlon Search _ tour data. In one embodiment, a point reduction operation is
USPC """""""" " 382/109’ 128’ 122130’ 131’ 132’ performed on data sets corresponding to any combination of
128/922’ 378/4T27’ 334455 /2 1585162575’ transverse, sagittal, or coronal contour data prior to process
_ _ _ ing those data sets to generate a 3D surface such as a varia
See apphcanon ?le for Complete Search hlstory' tional implicit surface. A neW contour can also be generated
(56) References Cited by the 1ntersect1on of this surface W1th an appropnately
placed and oriented plane. In this manner, the computation of
the variational implicit surface becomes suf?ciently ef?cient
U.S. PATENT DOCUMENTS
to make its use for neW contour generation practical.
5,859,891 A 1/1999 Hibbard
6,075,538 A 6/2000 Shu et al. 24 Claims, 23 Drawing Sheets
1402. Receive a plurality of input points. the input points comprising a plurality
of different initial sets of input points, wherein each initial point set is
representative of a contour in any plane (e.g., a T, S, or C contour)
1404: For each initial point set. reduce the number of points therein to generate
a plurality of different reduced sets of input points
i
1406; Generate a variational implicit surface from the reduced sets of input
points
i
1408: Clip the variational implicit surface at a desired plane to generate a
contour in the desired plane (e.g., a T contour)
l 1410: Does the generated contour match the image?
Yes No
1412: Archive the generated contour
US 8,577,107 B2
Page 2
(56) References Cited Botsch et al., “On Linear Variational Surface Deformation Methods”,
IEEE Transactions on Visualization and Computer Graphics, 2008,
U.S. PATENT DOCUMENTS pp. 213-230, vol. 14, No. 1.
Cruz et al., “A sketch on Sketch-Based Interfaces and Modeling”,
2006/0147114 A1 7/2006 Kaus et al. Graphics, Patterns and Images Tutorials, 23rd SIBGRAPI Confer
2006/0149511 A1 7/2006 Kaus et al. ence, 2010, pp. 22-33.
2006/0159322 A1 7/2006 Rinck et al. De Berg et al., “Computational Geometry: Algorithms and Applica
2006/0159341 A1 7/ 2006 Pekar et al. tions”, 1997, Chapter 5, Springer-Verlag, New York.
2006/0177133 A1 8/2006 Kee
Dice’s coef?eient, Wikipedia, 1945.
2006/0204040 A1 9/2006 Freeman et al.
Dinh et al., “Reconstructing Surfaces by Volumetric Regularization
2006/0256114 A1 11/2006 Nielsen et al.
2007/0014462 A1 1/2007 Rousson et al. Using Radial Basis Functions”, IEEE Transactions on Pattern Analy
2007/0041639 A1 2/2007 Von Berg et al. sis and Machine Intelligence, Oct. 2002, pp. 1358-1371, vol. 24, No.
2007/0092115 A1 4/2007 Usher et al. 10.
2007/0167699 A1 7/ 2007 Lathuiliere et a1. Duchon, “Splines Minimizing Rotation-Invariant SBMI-NORMS in
2009/0016612 A1 1/2009 Lobregt et a1. Soboley Spaces”, 1977, Universite Scienti?que et Medicale
2009/0190809 A1 7/2009 Han et al. Laboratoire de Mathematiques Appliques, Grenoble France.
2012/0057768 A1 3/ 2012 Hibbard et al. Gelas et al., “Variatonal Implicit Surface Meshing”, Computers and
Graphics, 2009, pp. 312-320, vol. 33.
OTHER PUBLICATIONS
Gering et al., “An Integrated Visualization System for Surgical Plan
ning and Guidance using Image Fusion and Interventional Imaging”,
Strang, “Introduction to Applied Mathematics”, 1986, Wellesley,
Int Conf Med Image Comput Assist Interv, 1999, pp. 809-819, vol. 2.
MA: Wellesley-Cambridge Press, pp. 242-262.
Girosi et al., “Priors, Stabilizers and Basis Functions: from regular
Thirion, “Image Matching as a Diffusion Process: An Analog with ization to radial, tensor and additive splines”, Massachusetts Institute
Maxwell’s Demons”, Med. Imag. Anal., 1998, pp. 243-260, vol. 2 of Technology Arti?cial Intelligence Laboratory, Jun. 1993, 28
(3) pages.
Thomas, “Numerical Partial Differential Equations: Finite Differ Ibanez et al., “The ITK Software Guide” Second Edition, 2005.
ence Methods”, Springer, NewYork, 1995. J ackowski et al., “A Computer-Aided Design System for Re?nement
Turk et al., “Shape Transformation Using Variational Implicit Func of Segmentation Errors”, MICCAI 2005, LNCS 3750, pp. 717-724.
tions”, Proceedings of SIGGRAPH 99, Annual Conference Series, Kalbe et al., “High-Quality Rendering of Varying Isosurfaces with
(Los Angeles, California), pp. 335-342, Aug. 1999. Cubic Trivariate Cl-continuous Splines”, ISVC 1, LNCS 5875, 2009,
Vemuri et al., “Joint Image Registration and Segmentation”, Geo pp. 596-607.
metric Level Set Methods in Imaging, Vision, and Graphics, S. Osher Kaus et al., “Automated 3-D PDM Construction From Segmented
and N. Paragios, Editors, 2003, Springer-Verlag, NewYork, pp. 251 Images Using Deformable Models”, IEEE Transactions on Medical
269. Imaging, Aug. 2003, pp. 1005-1013, vol. 22, No. 8.
Wahba, “Spline Models for Observational Data”, SIAM (Society for Kho et al., “Sketching Mesh Deformations”, ACM Symposium on
Industrial and Applied Mathematics), Philadelphia, PA, 1990. Interactive 3D Graphics and Games, 2005, 8 pages.
Wang et al., “Validation of an Accelerated ‘Demons’ Algorithm for Knoll et a1 ., “Fast and Robust Ray Tracing of General Implicits on the
Deformable Image Registration in Radiation Therapy”, Phys. Med. GPU”, Scienti?c Computing and Imaging Institute, University of
Biol., 2005, pp. 2887-2905, vol. 50. Utah, Technical Report No. UUSCI-2007-014, 2007, 8 pages.
Wolf et al., “ROPES: a Semiautomated Segmentation Method for Leventon et al., “Statistical Shape In?uence in Geodesic Active Con
Accelerated Analysis of Three-Dimensional Echocardiographic tours”, IEEE Conference on Computer Vision and Pattern Recogni
Data”, IEEE Transactions on Medical Imaging, 21, 1091-1 104, 2002. tion, 2000, pp. 1316-1323.
Xing et al., “Overview of Image-Guided Radiation Therapy”, Med. Nealen et al., “A Sketch-Based Interface for Detail-Preserving Mesh
Dosimetry, 2006, pp. 91-'112, vol. 31 (2). Editing”, Proceedings of ACM SIGGRAPH 2005, 6 pages, vol. 24,
Xu et al., “Image Segmentation Using Deformable Models”, Hand No. 3.
book of Medical Imaging, vol. 2, M. Sonka and J. M. Fitzpatrick, Osher et al., “Level Set Methods and Dynamic Implicit Surfaces”,
Editors, 2000, SPIE Press, Chapter 3. Chapters 11-13, 2003, Springer-Verlag, New York, NY
Yezzi et al., “A Variational Framework for Integrating Segmentation Pieper et al., “The NA-MIC Kit: ITK, VTK, Pipelines, Grids and 3D
and Registration Through Active Contours”, Med. Imag. Anal., 2003, Slicer as an Open Platform for the Medical Image Computing Com
pp. 171-185, vol. 7. munity”, Proceedings of the 3rd IEEE International Symposium on
Yoo, “Anatomic Modeling from Unstructured Samples Using Varia Biomedical Imaging: From Nano to Macro 2006, pp. 698-701, vol. 1.
tional Implicit Surfaces”, Proceedings of Medicine Meets Virtual Pohl et al., “A Bayesian model for joint segmentation and registra
Reality 2001, 594-600. tion”, NeuroImage, 2006, pp. 228-239, vol. 31.
Young et al., “Registration-Based Morphing of Active Contours for Sapiro, “Geometric Partial Differential Equations and Image Analy
Segmentation of CT Scans”, Mathematical Biosciences and Engi sis”, Chapter 8, 2001, Cambridge University Press.
neering, Jan. 2005, pp. 79-96, vol. 2 (1). Schroeder et al., “The Visualization Toolkit”, 2nd edition, Chapter 5,
Yushkevich et al., “User-Guided 3D Active Contour Segmentation of 1998, Prentice-Hall, Inc.
Anatomical Structures: Signi?cantly Improved Ef?ciency and Reli Singh et al., “Real-Time Ray-Tracing of Implicit Surfaces on the
ability”, NeuroImage 31, 1116-1128, 2006. GPU”, IEEE Transactions on Visualization and Computer Graphics,
Zagrodsky et al., “Registration-Assisted Segmentation of Real-Time 2009, pp. 261-272, vol. 99.
3-D Echocardiographic Data Using Deformable Models”, IEEE Tikhonov et al., “Solutions of III-Posed Problems”, Introduction
Trans. Med. Imag., Sep. 2005, pp. 1089-1099, vol. 24 (9). Chapter 2, 1977, John Wiley & Sons.
Zeleznik et al., “SKETCH: An Interface for Sketching 3D Scenes”, Tsai et al, “A Shape-BasedApproach to the Segmentation of Medical
Proceedings of SIGGRAPH 96, 163-170, 1996. Imagery Using Level Sets”, IEEE Transactions on Medical Imaging,
Zhong et al., “Object Tracking Using Deformable Templates”, IEEE Feb. 2003, pp. 137-154, vol. 22, No. 2.
Trans. Pall. Anal. Machine Intell., May 2000, pp. 544-549, vol. 22 Tsai et al., “An EM algorithm for shape classi?cation based on level
(5). sets”, Medical Image Analysis, 2005, pp. 491-502, vol. 9.
Of?ce Action for US. Appl. No. 13/295,494 dated Sep. 13, 2012. Notice of Allowance for US. Appl. No. 12/022,929 dated May 8,
International Search Report and Written Opinion for PCT/US2012/ 2012.
048938 dated Oct. 16, 2012. Adelson et al., “Pyramid Methods in Image Processing”, RCA Engi
Bookstein, “Principal Warps: Thin-Plate Splines and the Decompo neer, Nov/Dec. 1984, pp. 33-41, vol. 29-6.
sition of Deformations”, IEEE Transactions on Pattern Analysis and Anderson et al., “LAPACK User’s Guide”, Third Edition, SIAMi
Machine Intelligence, Jun. 1989, pp. 567-585, vol. 11, No. 6. Society for Industrial and Applied Mathematics, 1999, Philadelphia.
US 8,577,107 B2
Page 3
(56) References Cited Igarashi et al., “Teddy: A Sketching Interface for 3D Freeform
Design”, Proceedings of SIGGRAPH 1999, 409-416.
OTHER PUBLICATIONS Ijiri et al., “Seamless Integration of Initial Sketching and Subsequent
Detail Editing in Flower Modeling”, Eurographics 2006, 25, 617
Barrett et al., “Interactive Live-Wire Boundary Extraction”, Medical
624, 2006.
Image Analysis, 1, 331-341, 1997.
Jain et al., “Deformable Template Models: A Review”, Signal Proc.,
Bertalmio et al., “Morphing Active Countours”, IEEE Trans. Patt.
1998, pp. 109-129, vol. 71.
Anal. Machine Intell., 2000, pp. 733-737, vol. 22.
Jain, “Fundamentals of Digital Image Processing”, Prentice-Hall,
Bloomenthal, “An Implicit Surface PolygoniZer”, Graphics Gems IV,
New Jersey, 1989.
P Heckbert, Ed., Academic Press, New York, 1994.
Jehan-Besson et al., “Shape Gradients for Histogram Segmentation
Burnett et al., “A Deformable-Model Approach to Semi-Automatic
Segmentation of CT Images Demonstrated by Application to the Using Active Contours”, 2003, presented at the 9th IEEE Int. Conf.
Spinal Canal”, Med. Phys., Feb. 2004, pp. 251-263, vol. 31 (2). Comput. Vision, Nice, France, 8 pages.
Carr et al., “Reconstruction and Representation of 3D Objects with Kalet et al., “The Use of Medical Images in Planning and Delivery of
Radial Basis Functions”, Proceedings of SIGGRAPH 01, pp. 67-76, Radiation Therapy”, J. Am. Med. Inf. Assoc., Sep./Oct. 1997, pp.
327-339, vol. 4 (5).
2001 .
Carr et al., “Surface Interpolation with Radial Basis Functions for Karpenko et al., “Free-Form Sketching with Variational Implicit
Medical Imaging”, IEEE Transactions on Medical Imaging, 16, Surfaces”, Computer Graphics Forum, 21, 585-594, 2002.
96-107, 1997. Karpenko et al., “SmoothSketch: 3D Free-Form Shapes From Com
Cover et al., “Elements of Information Theory”, Chapter 2, 1991, plex Sketches”, Proceedings of SIGGRAPH 06, pp. 589-598.
Wiley, NewYork, 33 pages. Leyrnarie et al., “Tracking Deformable Objects in the Plane Using an
Cover et al., “Elements of Information Theory”, Chapter 8, 1991, Active Contour Model”, IEEE Trans. Patt. Anal. Machine Intell., Jun.
Wiley, New York, 17 pages. 1993, pp. 617-634, vol. 15 (6).
Davis et a1 ., “Automatic Segmentation of Intra-Treatment CT Images Lipson et al., “Conceptual Design and Analysis by Sketching”, Jour
for Adaptive Radiation Therapy of the Pro state”, presented at 8th Int.
nal ofAI in Design and Manufacturing, 14, 391-401, 2000.
Conf. MICCAI 2005, Palm Springs, CA, pp. 442-450.
Lorenson et al., “Marching Cubes: A High Resolution 3D Surface
DeBoor, “A Practical Guide to Splines”, Springer, NewYork, 2001.
Construction Algorithm”, Computer Graphics, Jul. 1987, pp. 163
Digital Imaging and Communications in Medicine (DICOM), http://
medical.nema.org/. 169, vol. 21 (4).
Lu et al., “Automatic Re-Contouring in 4D Radiotherapy”, Phys.
Dinh et al., “Reconstructing Surfaces by Volumetric RegulariZation
Med. Biol, 2006, pp. 1077-1099, vol. 51.
Using Radial Basis Functions”, IEEE Trans. Patt. Anal. Mach. Intell.,
24, 1358-1371, 2002. Lu et al., “Fast Free-Form Deformable Registration Via Calculus of
Dinh et al., “Texture Transfer During Shape Transformation”, ACM Variations”, Phys. Med. Biol., 2004, pp. 3067-3087, vol. 49.
Transactions on Graphics, 24, 289-310, 2005. Marker et al., “Contour-Based Surface Reconstruction Using
DoCarmo, “Differential Geometry of Curves and Surfaces”, Prentice Implicit Curve Fitting, and Distance Field Filtering and Interpola
Hall, New Jersey, 1976. tion”, The Eurographics Association, 2006, 9 pages.
Falcao et al., “An Ultra-Fast User-Steered Image Segmentation Para Paragios et al., “Geodesic Active Contours and Level Sets for the
digm: Live Wire on the Fly”, IEEE Transactions on Medical Imaging, Detection and Tracking of Moving Objects”, IEEE Trans. Patt. Anal.
19, 55-62, 2000. Machine Intell., Mar. 2000, pp. 266-280, vol. 22 (3).
Freedman et al., “Active Contours for Tracking Distributions”, IEEE Pekar et al., “Automated Model-Based Organ Delineation for Radio
Trans. Imag. Proc., Apr. 2004, pp. 518-526, vol. 13 (4). therapy Planning in Prostate Region”, Int. J. Radiation Oncology
Gao et al., “A Deformable Image Registration Method to Handle Biol. Phys., 2004, pp. 973-980, vol. 60 (3 ).
Distended Rectums in Prostate Cancer Radiotherapy”, Med. Phys., Pentland et al., “Closed-Form Solutions for Physically Based Shape
Sep. 2006, pp. 3304-3312, vol. 33 (9). Modeling and Recognition”, IEEE Trans. Patt. Anal. Machine Intell.,
Gering et al., “An Integrated Visualization System for Surgical Plan Jul. 1991, pp. 715-729, vol. 13 (7).
ning and Guidance Using Image Fusion and an Open MR”, Journal of Piegl et al., “The NURBS Book”, Second Edition, Springer, New
Magnetic Resonance Imaging, 13, 967-975, 2001. York, 1997.
Gering, “A System for Surgical Planning and Guidance Using Image Press et al., “Numerical Recipes in C”, Second Edition, Cambridge
Fusion and Interventional MR”, MS Thesis, MIT, 1999. University Press, 1992.
Golub et al., “Matrix Computations”, Third Edition, The Johns Rogelj et al., “Symmetric Image Registration”, Med. Imag. Anal.,
Hopkins University Press, Baltimore, 1996. 2006, pp. 484-493, vol. 10.
Han et al., “A Morphing Active Surface Model for Automatic Re Sarrut et al., “Simulation of Four-Dimensional CT Images from
Contouring in 4D Radiotherapy”, Proc. of SPIE, 2007, vol. 6512, 9 Deformable Registration Between Inhale and Exhale Breath-Hold
pages. CT Scans”, Med. Phys., Mar. 2006, pp. 605-617, vol. 33 (3).
Ho et al., “SNAP: A Software Package for User-Guided Geodesic Schmidt et al., ShapeShop: Sketch-Based Solid Modeling with Blob
Snake Segmentation”, Technical Report, UNC Chapel Hill, Apr. Trees, EUROGRAPHICS Workshop on Sketch-Based Interfaces and
2003. Modeling, 2005.
Huang et al., “Semi-Automated CT Segmentation Using Optic Flow Schroeder et al., “The Visualization Toolkit”, 2nd Edition, Kitware,
and Fourier Interpolation Techniques”, Computer Methods and Pro 2006, Ch. 5 & 13,65 pp.
grams in Biomedicine, 84, 124-134, 2006. Sethian, “Level Set Methods and Fast Marching Methods”, 2nd ed.,
Igarashi et al., “Smooth Meshes for Sketch-based Freeform Model 1999, Cambridge University Press, Chapters 1, 2 & 6, 39 pages.
ing” in ACM Symposium on Interactive 3D Graphics, (ACM
I3D’03), pp. 139-142, 2003. * cited by examiner
US. Patent
Nov. 5, 2013 Sheet 1 0f 23 US 8,577,107 B2
102 100 q
Transverse
Coronal
106
104
Figure 1
US. Patent Nov. 5, 2013 Sheet 2 0f 23 US 8,577,107 B2
Figure 2(a)
US. Patent Nov. 5, 2013 Sheet 3 0f 23 US 8,577,107 B2
CHM/1C1
OmW
G
m 0
Yw wYz w n w wr
A
VXw)
w DI “In n e W .m d Q w
/
The Window Coordinate System Sagittal
Transverse
Figure 2(b)
US. Patent Nov. 5, 2013 Sheet 4 0f 23 US 8,577,107 B2
Figure 3
US. Patent Nov. 5, 2013 Sheet 5 0f 23 US 8,577,107 B2
Figure 4
Processor
(502)
W M
Input Points for Output T
Contours Contou?s)
(500) (504)
Figure 5(a)
Processor
(502)
............................ ..> ......................>
Input Poi ms for Output 3 D
Contours Surface
(500) (506)
Figure 5(b)
US. Patent Nov. 5, 2013 Sheet 6 0f 23 US 8,577,107 B2
602: Receive a plurality of input points, the input points comprising a plurality of
different initial sets of input points, wherein each initial point set is representative
of a an S contour or a C contour
V
604: For each initial point set, reduce the number of points therein to generate a
plurality of different reduced sets of input points
T
606: Re-order the points within each reduced set with respect to a frame of
reference and a direction of rotation
608: Find the points of intersection in a T plane for the re-ordered points Within
each reduced set
610: Generate a contour in the T plane by interpolating through the points of
intersection using B-spline interpolation
t
612: Does the generated T contour match the image?
Yes No
v
614: Archive the generated contour
Figure 6
US. Patent Nov. 5, 2013 Sheet 7 0f 23 US 8,577,107 B2
700
2352230
453B Arc Length
Figure 7