Medicine Meets Virtual Reality 20

520 Pages·2013·16.663 MB·English

by  Felländer-Tsai, L., Westwood, S. W., Westwood, J. D.

MEDICINE MEETS VIRTUAL REALITY 20
Studies in Health Technology and
Informatics
This book series was started in 1990 to promote research conducted under the auspices of the EC
programmes’ Advanced Informatics in Medicine (AIM) and Biomedical and Health Research
(BHR) bioengineering branch. A driving aspect of international health informatics is that
telecommunication technology, rehabilitative technology, intelligent home technology and many
other components are moving together and form one integrated world of information and
communication media. The series has been accepted by MEDLINE/PubMed, SciVerse Scopus,
EMCare, Book Citation Index – Science and Thomson Reuters’ Conference Proceedings Citation
Index.
Series Editors:
Dr. O. Bodenreider, Dr. J.P. Christensen, Prof. G. de Moor, Prof. A. Famili, Dr. U. Fors,
Prof. A. Hasman, Prof. E.J.S. Hovenga, Prof. L. Hunter, Dr. I. Iakovidis, Dr. Z. Kolitsi,
Mr. O. Le Dour, Dr. A. Lymberis, Prof. J. Mantas, Prof. M.A. Musen, Prof. P.F. Niederer,
Prof. A. Pedotti, Prof. O. Rienhoff, Prof. F.H. Roger France, Dr. N. Rossing,
Prof. N. Saranummi, Dr. E.R. Siegel, Prof. T. Solomonides and Dr. P. Wilson
Volume 184
Recently published in this series
Vol. 183. K.L. Courtney, O. Shabestari and A. Kuo (Eds.), Enabling Health and Healthcare
through ICT – Available, Tailored and Closer
Vol. 182. A.C. Smith, N.R. Armfield and R.H. Eikelboom (Eds.), Global Telehealth 2012 –
Delivering Quality Healthcare Anywhere Through Telehealth – Selected Papers from
Global Telehealth 2012 (GT2012)
Vol. 181. B.K. Wiederhold and G. Riva (Eds.), Annual Review of Cybertherapy and
Telemedicine 2012 – Advanced Technologies in the Behavioral, Social and
Neurosciences
Vol. 180. J. Mantas, S.K. Andersen, M.C. Mazzoleni, B. Blobel, S. Quaglini and A. Moen
(Eds.), Quality of Life through Quality of Information – Proceedings of MIE2012
Vol. 179. M. García-Rojo, B. Blobel and A. Laurinavicius (Eds.), Perspectives on Digital
Pathology – Results of the COST Action IC0604 EURO-TELEPATH
Vol. 178. A.J. Maeder and F.J. Martin-Sanchez (Eds.), Health Informatics: Building a
Healthcare Future Through Trusted Information – Selected Papers from the 20th
Australian National Health Informatics Conference (HIC 2012)
Vol. 177. B. Blobel, P. Pharow and F. Sousa (Eds.), pHealth 2012 – Proceedings of the 9th
International Conference on Wearable Micro and Nano Technologies for Personalized
Health
Vol. 176. T. Kotwicki and T.B. Grivas (Eds.), Research into Spinal Deformities 8

ISSN 0926-9630 (print)
ISSN 1879-8365 (online)
Meedicinee Meeets Virrtual RReality 20
NextMMed / MMMVR20
Edited byy
Jamess D. Wesstwood
Susan WW. Westwwood MAA
LLi Fellännder-Tsai MD PhDD
RRandy S. Haluck MMD FACCS
Richarrd A. Robbb PhD
Steveen Sengerr PhD
and
Kirby GG. Vosburrgh PhD
Amstterdam • Berrlin • Tokyo •• Washington, DC
© 2013 The authors and IOS Press.
All rights reserved. No part of this book may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, without prior written permission from the publisher.
ISBN 978-1-61499-208-0 (print)
ISBN 978-1-61499-209-7 (online)
Library of Congress Control Number: 2013931909
Publisher
IOS Press BV
Nieuwe Hemweg 6B
1013 BG Amsterdam
Netherlands
fax: [phone]
e-mail: [email protected]
Distributor in the USA and Canada
IOS Press, Inc.
4502 Rachael Manor Drive
Fairfax, VA 22032
USA
fax: [phone]
e-mail: [email protected]

LEGAL NOTICE
The publisher is not responsible for the use which might be made of the following information.
PRINTED IN THE NETHERLANDS
Medicine Meets Virtual Reality 20 v
J.D. Westwood et al. (Eds.)
IOS Press, 2013
© 2013 The authors and IOS Press. All rights reserved.
Preface
James D. WESTWOOD
Aligned Management Associates, Inc.
During the past twenty years, computers have evolved from relatively cumbersome
machines, used primarily at work to create documents and do accounting, into sleek,
intuitive, and nearly omnipresent extensions of our fingers, eyes, and brains. Ready
access to the information they provide has transformed our lives. It is telling that one
hears health warnings against sleeping next to a mobile phone (the most common varie-
ty of computer) because twenty years ago, no one worried about computers being taken
to bed. It’s not just an issue of devices’ growing “sexiness.” In 1992, there wasn’t
much on a PC that one would desire in the middle of the night.
Healthcare also has been transformed by the expanded role of our data-purveying
gizmos. Physicians and scientists have unprecedented access to the knowledge of peers
around the world, enabling better decision-making. For investigators, data mining can
amplify the number of subjects in a study; crowd sourcing enlists a wider perspective.
Medical schools educate students more effectively with tools that offer greater realism,
useful repetition, and continual assessment. Even the general public can explore medi-
cal articles online to learn more about the therapies their doctors prescribe. Also at
home, devices monitor patients, report back to their caregivers, and automatically re-
mind them to take their medicine.
There are shortcomings, of course. Physicians can feel like clerks instead of hea-
lers, and patients often resent the effect that typing has on bedside manner. Electronic
health records can be spoiled when clinicians cut and paste data to save time, while
haphazard standardization and interoperability limit records’ utility. Crowd sourcing
may generate poorly vetted noise instead of useful guidance. Hackers compromise
network integrity, and insurers and industry sift through information with eyes perhaps
too focused on profits. The relentless disruption of technological change creates addi-
tional stress for caregivers and administrators. And for sick patients not in the mood for
a learning curve, weighing too many therapeutic options is confusing and frustrating.
Yet there is no feasible alternative to increased reliance upon our devices and their
data. Wealthier countries have high expectations about maintaining the health of their
rapidly aging Boomer populations, despite shrinking budgets. The developing world,
with its growing middle class, wants greater investment in wellness with measurable
outcomes as the result. The technological efficiency—meaning affordability—
described by Moore’s Law is the only way the medical community can address both
demographic challenges successfully.
Back in 1992, my colleagues founded “Medicine Meets Virtual Reality” with the
aim of using computers to advance clinical care and medical education. They recog-
nized how exponential upgrades in software and hardware would make healthcare more
efficient, precise, and personal. Over the years, many visionary ideas have become via-
ble tools: it is not uncommon to read in the general media about a medical break-
through whose basis was shared at this conference in previous years. And although
vi
“virtual reality” faded as a buzzword years ago and “NextMed” more accurately de-
scribes this conference now, the creative energy that turns information technology into
better medicine remains vibrant.
Many thanks to the Organizing Committee for its steadfast support over two dec-
ades, and to all of you who are participating in this year’s conference—the twentieth
since 1992 and a noteworthy anniversary.
vii

Surgery, Virtual Reality, and the Future
Kirby G. VOSBURGH, PhD,1,a,b Alexandra GOLBY, MD,c,b
Steven D. PIEPER, PhDd,e
aClinical Image Guidance Laboratory, Department of Radiology,

bBrigham and Women’s Hospital, Harvard Medical School, Boston, MA,
cDepartment of Neurosurgery, dSurgical Planning Laboratory,
eIsomics, Inc.
Abstract. MMVR has provided the leading forum for the multidisciplinary
interaction and development of the use of Virtual Reality (VR) techniques in
medicine, particularly in surgical practice. Here we look back at the foundations of
our field, focusing on the use of VR in Surgery and similar interventional
procedures, sum up the current status, and describe the challenges and
opportunities going forward.
Keywords. Virtual reality, Augmented reality, Surgery, Image-guided Surgery
Introduction
Richard Satava, MD has been an articulate and effective spokesman for the vision of
high-tech interventional medicine:
“…most (of the) information that a health care provider needs can be acquired in
electronic form (images, scans, vital signs, the medical record). And with the
emergence of teleoperation, we can leverage the power of the advanced information
tools of software (AI, 3D visualization and decision support), hardware (high
performance computing) and networking (the information superhighway). All this
will enhance the skills of the health care provider beyond mere physical limitations
to enable a quality of care previously considered unachievable. Better access will be
provided by remote telemedicine. Lower cost will be achieved through flexible
manufacturing, just in time inventory, and best-in-class business management.”
(Proceedings, MMVR III, 1995)
In this paper, we look back at these foundations, focusing on the use of virtual reality
(VR) in surgery and similar interventional procedures, sum up the current status, and
describe the challenges and opportunities going forward.
1. The Promise in the Early 1990s
From the beginning it was clear that progress would involve intense collaboration
between physicians, scientists, and engineers. Military medicine and the manned space
programs were often used as a model for this type of focused effort; some of the early

1 Corresponding Author: L1-050, 75 Francis Street, Boston, MA, 02115 USA; [email protected]
viii K.G. Vosburgh et al. / Surgery, Virtual Reality, and the Future
MMVR papers even seemed to adopt the goals of the medical device designers for Star
Trek.[1]
Very quickly, efforts were made to organize the information to be presented. For
example, the Visible Human Project [2] adopted the terminology and generative syntax
of anatomy. This fit well with the vision of displaying a 3D model of the patient’s body
in an augmented-reality view…the simplest concept being to “see beneath the skin.”
The anatomic descriptors also matched the essential task of surgery: to remove (or
denature) diseased tissue with minimal damage to healthy tissue and its functions.
Likewise, the creation of modeling software, which corrected for (or accommodated)
artifacts and noise in the raw data, enabled differentiation and labeling of anatomic
structures to simplify communication. As a culmination of these developments, atlases
of key organs and organ systems were made to support teaching, identify anatomic
variations in a specific patient, and serve as “strong priors” for image interpretation and
cohort analysis.
The development of patient-specific digital models provided a foundation for
procedure simulation, and ongoing studies of approaches to adapt these models so that
they accurately present conditions during a specific intervention. The two primary
goals of VR (or “immersive”) simulators are 1) to enable training in a realistic and
consistent environment, and 2) to rehearse an actual procedure so that the physician
does not “practice on the patient.” Real-time-adapted models were focused on
providing augmented reality data support intraprocedurally.

Figure 1. The opportunities surrounding the surgeon in the early 1990s
.

In 1992, the surgeon was surrounded by opportunity (Figure 1). We were at the
peak of a golden age of medical imaging technology established by CT, MRI, and the
ongoing conversion to digital image data; the following decade would see an explosion
of advanced systems and concepts in several modalities, and a wide range of new
diagnostic instruments and approaches. This was matched by comparable potential in
telemedicine, robotics, surgical simulation, and minimally invasive surgery:
Master-Slave Robotics. The development of surgical robotics and surgical
telepresence proceeded in parallel, since optimal operation of the instruments was best
done sitting at a console connected only electronically to the activity at the surgical site.
In 1990, the ROBODOC system for hip replacement had been developed to the
prototype stage, Hap Paul, DVM, did the first animal model studies, and IBM infused
$3M into the concept to create Integrated Surgical Systems.[3] At the same time, Philip
K.G. Vosburgh et al. / Surgery, Virtual Reality, and the Future ix
Green and colleagues at the Stanford Research Institute (now SRI International) were
completing the configuration of underlying technology for the DaVinci surgical robot.
Other early experiments, particularly in neurosurgery, were occurring around the
world.[4]
Surgical Simulation and Training. At the beginning of the MMVR era, the
development of increasingly accurate and sophisticated digital models and real-time
interfaces to interact with them was envisioned to provide capabilities for improving
training, the capability to evaluate operator performance in a standardized fashion, and
the ability to do surgical planning and, eventually, rehearsal.
Minimally Invasive Surgery. In 1990, flexible endoscopy and rigid endoscopes
(laparoscopes) were in clinical use, but large incisions still characterized surgical
practice. Flexible scopes were used (as they are today) primarily for diagnostic studies;
the use of laparoscopes in surgery, pioneered by Semm, was generally restricted to
gynecology. Improvements in instrumentation, which would stimulate broader use by
general surgeons [5] and other surgical specialties, were just being introduced. Of
particular importance for visualization was the introduction of video cameras
positioned independently from the scopes (generally attributed to C. Nezhat), enabling
“operating off the monitor.”
Also, 1990 marked the end of a period when intracranial surgery was a dangerous
and unpredictable intervention. Imaging support (Figure 2) to guide procedures was
limited, and difficult to utilize in the OR.

Figure 2. Neurosurgical Guidance Display in the early 2000s.

2. Twenty Years of Accomplishment
Although there have been no significant new imaging modalities for surgery guidance
(with the possible exception of Cone-beam CT), imaging technology has improved
across the board: Resolution is higher, and contrast has been improved by new
intralumenal and particularly “molecular targeted” parenchymal contrast agents. In
addition to the well-established optical tracking devices, small footprint
electromagnetic tracking systems, both stand-alone (Ascension Technologies, Northern
Digital), and integrated into guidance systems (St. Jude, Biosense-Webster, Brainlab,
GE Logiq E9) have made it possible to track the motion of instruments and anatomic
targets with high accuracy. Magnetic actuation has recently demonstrated potential for
x K.G. Vosburgh et al. / Surgery, Virtual Reality, and the Future
complementing or conducting therapeutic procedures in workspaces such as the
peritoneal cavity.
As impressive as hardware development has been, the advancement of image-
based software has arguably had greater impact. Imaging system vendors all supply
capable workstations for image analysis and display. Several commercial systems bring
specialized and ever-more expanding capabilities, while software packages originating
in academic labs, such as Richard Robb’s team at the Mayo Clinic, provide support to
research as well as clinical applications.[6] An open-source system for image analysis
and presentation, 3DSlicer,[7] has thousands of worldwide users. Due to its flexibility
and large base of active users, the 3D Slicer community is in the vanguard of
converting off-line analytic tools into real-time actions for direct surgical support.[8,9]
Over these two decades, surgical practice evolved through the broad acceptance of
minimally invasive techniques, particularly laparoscopy. This change was not driven
by improvement in long-term clinical outcomes, but rather by less-traditional criteria
such as shorter recovery time and, to some extent, cosmesis, primarily through pressure
from patients. The process was markedly unsystematic, as noted by Cuschieri,[10] but
it spawned a significant effort to implement training protocols and certification criteria
to ensure patient safety, which is still ongoing through organizations such as the
Society of American Gastrointestinal and Endoscopic Surgeons and the American
College of Surgeons.[11]
Attempts to build on the success of laparoscopic surgery have had less spectacular
results. Following a logical train that, if a few small skin incisions are better than open
surgery, then only one incision is better yet, “single port” systems have proliferated. It
is not yet clear that the cosmetic advantages will outweigh the difficulties of
accomplishing both surgery and imaging through a single aperture. Likewise, the
extension of the image display from 2D to 3D has moved slowly, at best.
Orthopedic surgery, and particularly hip replacement, was an early target for
robotic techniques because it appeared that higher precision would lead to better long-
term functionality (fewer joint displacements) and less-traumatic procedures, leading to
faster recovery. Both ROBODOC[3] and HipNav[12] were developed extensively, but
have not displaced existing, less-refined approaches. The “lesson learned,” in attempts
to automate hip (and knee) prosthetic surgery, is that the body adapts remarkably well
to changes in joint structure, thus limiting the benefits of these more accurate
procedures.
By contrast, the DaVinci surgical system has had significant commercial success.
While the current market for this massive system is driven by patient demand for
prostate cancer resection, it may be that DaVinci’s success is due to its configuration as
a surgical platform with many potential applications, so it could be adapted to serve a
market need that was identified well after product introduction.[13]
For both minimally invasive and robotic approaches, the trend toward minimal
access implies that VR will be even more helpful, because the surgeon’s natural view is
more constricted. The past decade has seen a steady improvement in the realism of
simulated anatomic features for training and surgical rehearsal, and all of us now access
YouTube videos routinely to better understand surgical procedures and techniques.