Artificial Intelligence and Deep Learning in Pathology PDF
Preview Artificial Intelligence and Deep Learning in Pathology
Artificial Intelligence and Deep Learning in Pathology Edited by Stanley Cohen, M.D Emeritus Chair of Pathology & Emeritus Founding Director Center for Biophysical Pathology Rutgers-New Jersey Medical School, Newark, NJ, United States Adjunct Professor of Pathology Perelman Medical School University of Pennsylvania, Philadelphia, PA, United States Adjunct Professor of Pathology Kimmel School of Medicine Jefferson University, Philadelphia, PA, United States Elsevier Radarweg29,POBox211,1000AEAmsterdam,Netherlands TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom 50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates Copyright©2021ElsevierInc.Allrightsreserved. Nopartofthispublicationmaybereproducedortransmittedinanyformorbyany means,electronicormechanical,includingphotocopying,recording,oranyinformation storageandretrievalsystem,withoutpermissioninwritingfromthepublisher.Detailson howtoseekpermission,furtherinformationaboutthePublisher’spermissionspolicies andourarrangementswithorganizationssuchastheCopyrightClearanceCenterandthe CopyrightLicensingAgency,canbefoundatourwebsite:www.elsevier.com/permissions. Thisbookandtheindividualcontributionscontainedinitareprotectedundercopyright bythePublisher(otherthanasmaybenotedherein). Notices Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchand experiencebroadenourunderstanding,changesinresearchmethods,professional practices,ormedicaltreatmentmaybecomenecessary. Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgein evaluatingandusinganyinformation,methods,compounds,orexperimentsdescribed herein.Inusingsuchinformationormethodstheyshouldbemindfuloftheirownsafety andthesafetyofothers,includingpartiesforwhomtheyhaveaprofessional responsibility. Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,or editors,assumeanyliabilityforanyinjuryand/ordamagetopersonsorpropertyasa matterofproductsliability,negligenceorotherwise,orfromanyuseoroperationofany methods,products,instructions,orideascontainedinthematerialherein. LibraryofCongressCataloging-in-PublicationData AcatalogrecordforthisbookisavailablefromtheLibraryofCongress BritishLibraryCataloguing-in-PublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN:978-0-323-67538-3 ForinformationonallElsevierpublicationsvisitourwebsiteat https://www.elsevier.com/books-and-journals Publisher:DoloresMeloni AcquisitionsEditor:MichaelHouston EditorialProjectManager:SamW.Young ProjectManager:NiranjanBhaskaran CoverDesigner:AlanStudholme TypesetbyTNQTechnologies This book is dedicated to the American Society for Investigative Pathology, and especially Mark Sobel and Martha Furie, for supportingmyeffortstoproselytizeonbehalfofartificialintelligence inpathology.ItisalsodedicatedtothemembersoftheWarrenAlpert CenterforDigitalandComputationalPathologyatMemorialSloan- Kettering, under the leadership of David Klimstra and Meera Hameed, for nourishing my enthusiasm for this revolution in pathology.IamespeciallyindebtedtobothDeanChristinePalusand the Departments of Computer Science and Mathematics at Villanova Universityforenablingmetogetuptospeedrapidlyinthisrelatively new interestof minewhile beingso accommodating andfriendly toa student older than his teachers. Most important, it is also dedicated to my wonderful wife and scientific colleague Marion, children (Laurie, Ron, and Ken), kids-in-law (Ron, Helen, and Kim respectively), and grandkids (Jessica, Rachel, Joanna, Julie, Emi, Risa, Brian, Seiji, and Ava). With natural intelligence like that, who needs artificial intelligence? Contributors Tanishq Abraham Department ofBiomedical Engineering, University of California,Davis,CA, United States (cid:1) Ognjen Arandjelovic,M.Eng. (Oxon), Ph.D. (Cantab) School ofComputerScience, University ofSt Andrews, StAndrews, United Kingdom Peter D. Caie, BSc,MRes,PhD School ofMedicine, QUAD Pathology,University ofSt Andrews, StAndrews, United Kingdom Stanley Cohen, MD EmeritusChairofPathology&EmeritusFoundingDirector,CenterforBiophysical Pathology,Rutgers-New JerseyMedical School, Newark,NJ, United States; AdjunctProfessorofPathology,PerelmanMedicalSchool,Universityof Pennsylvania,Philadelphia,PA,UnitedStates;AdjunctProfessorofPathology, KimmelSchoolofMedicine,JeffersonUniversity,Philadelphia,PA,UnitedStates Neofytos Dimitriou,B.Sc School ofComputerScience, University ofSt Andrews, StAndrews, United Kingdom Michael Donovan Department ofPathology,Icahn SchoolofMedicine atMount Sinai, New York, NY,UnitedStates Gerardo Fernandez Department ofPathology,Icahn SchoolofMedicine atMount Sinai, New York, NY,UnitedStates Rajarsi Gupta, MD, PhD AssistantProfessor,BiomedicalInformatics,StonyBrookMedicine,StonyBrook, NY,UnitedStates RichardLevenson,MD ProfessorandViceChair,PathologyandLaboratoryMedicine,UCDavisHealth, Sacramento, CA, United States Bahram Marami Department ofPathology,Icahn SchoolofMedicine atMount Sinai, New York, NY,UnitedStates BenjaminR.Mitchell, BA, MSE, PhD AssistantProfessor,Department ofComputingSciences, VillanovaUniversity, Villanova,PA,UnitedStates xiii xiv Contributors Daniel A.Orringer,MD AssociateProfessor,DepartmentofSurgery,NYULangoneHealth,NewYorkCity, NY,United States Anil V.Parwani, MD, PhD,MBA Professor,PathologyandBiomedical Informatics, The Ohio State University, Columbus, OH, United States Marcel Prastawa Department ofPathology,Icahn School of Medicine atMount Sinai,New York, NY,United States Abishek SainathMadduri Department ofPathology,Icahn School of Medicine atMount Sinai,New York, NY,United States Joel Haskin Saltz, MD, PhD Professor,Chair,Biomedical Informatics, Stony Brook University,Stony Brook, NY,United States Richard Scott Department ofPathology,Icahn School of Medicine atMount Sinai,New York, NY,United States Austin Todd UTHealth,SanAntonio,TX, United States JohnE. Tomaszewski,MD SUNY Distinguished Professor and Chair, Pathology and Anatomical Sciences, University atBuffalo, State University ofNew York, Buffalo, NY,United States JackZeineh Department ofPathology,Icahn School of Medicine atMount Sinai,New York, NY,United States Preface It is now well-recognized that artificial intelligence (AI) represents an inflection point for society at least as profound as was the Industrial Revolution. It plays increasingrolesinindustry,education,recreation,andtransportation,aswellassci- enceandengineering.Asoneexampleofitsubiquity,arecentheadlineproclaimed that “NovakDjokovic used AI to Train for Wimbledon!” In particular, AI has become important for both biomedical research and clin- ical practice. Part of the reason for this is the sheer amount of data currently obtainable by modern techniques. In the past, we would study one or a few en- zymes or transduction pathways at a time. Now we can sequence the entire genome, develop gene expression arrays that allow us to dissect out critical net- works and pathways, study the entire microbiome instead of a few bacterial spe- cies in isolation, and so on. We can also use AI to look for important patterns in themassivedatathathavealreadybeencollected,i.e.,dataminingbothindefined datasetsandinnaturallanguagesources.Also,wecantakeadvantageofthepower ofAIinpatternrecognitiontoanalyzeandclassifypictorialdatanotonlyforclas- sificationtasksbutalsotoextractinformationnotobvioustoanindividuallimited to direct observations through the microscope, a device that has never received FDA approval. RadiologistshavebeenamongthefirstclinicalspecialiststoembraceAItoassist in the processing and interpretation of X-ray images. Pathologists have lagged, in partbecausethevisualcontentoftheimagesthatpathologistsuseisatleastanorder ofmagnitudegreater than thoseof radiology. Another factor has been the delay of themajorindustrialplayerstoenterthisarena,aswellaslimitedhospitalresources based on perceived differences in hospital profitability between radiology and pa- thology practices. However, more and more pathology departments are embracing digitalpathologyviawholeslide imaging(WSI), andWSIprovidestheinfrastruc- ture both for advanced imaging modalities and the archiving of annotated datasets needed for AI training and implementation. An ever-increasing number of articles utilizing AI are appearing in the literature, and a number of professional societies devoted to this field have emerged. Existing societies and journals in pathology have beguntocreate subsections specifically devotedtoAI. Unfortunately,existingtextsonmachinelearning,deeplearning,andAI(terms that are distinct, butloosely overlap) tend to be either very general and superficial overviewsforthelaymanorhighlytechnicaltomesassumingabackgroundinlinear algebra, multivariate calculus, and programming languages. They do not therefore serve as a useful introduction for most pathologists, as well as physicians in other xv xvi Preface disciplines.Thepurposeofthismonographistoprovideadetailedunderstandingof boththeconceptsandmethodsofAI,aswellasitsuseinpathology.Inbroadgeneral terms, the book covers machine learning and introduction to data processing, the integrationofAIandpathology,anddetailedexamplesofstate-of-the-artAIappli- cations in pathology. It focuses heavily on diagnostic anatomic pathology, and thankstoadvancesinneuralnetworkarchitecturewenowhavethetoolsforextrac- tion ofhigh-levelinformation fromimages. We begin with an opening chapter on the evolution of machine learning, including both a historical perspective and some thoughts as to its future. The nextfourchaptersprovidetheunderlyingframeworkofmachinelearningwithspe- cial attention to deep learning and AI, building from the simple to the complex. Althoughquitedetailedandintensive,theserequirenoknowledgeofcomputerpro- gramming or of mathematical formalism. Wherever possible, concepts are illus- trated by examples from pathology. The next two chapters deal with the integration of AI with digital pathology and its application not only for classifica- tion, prediction, and inference but also for the manipulation of the images them- selves for extraction of meaningful information not readily accessible to the unaidedeye.Followingthisisachapteronprecisionmedicineindigitalpathology that demonstrates the power of traditional machine learning paradigms as well as deep learning, The reader will find a certain amount of overlap in these chapters for the following reasons; each author needs to review some basic aspects of AI as they specifically relate to his or her assigned topic, each presents their own perspectiveontheunderpinningsoftheworktheydescribe,andeachtopicconsid- eredcanbestbeunderstoodinthelightofparallelworkinotherareas.Thenexttwo chaptersdealwithtwoimportanttopicsrelatingtocancerinwhichtheauthorsuti- lizeAItogreateffectanddiscusstheirfindingsinthecontextofstudiesinotherlab- oratories.Takentogether,theselatterchaptersprovidecomplementaryviewsofthe current state ofthe art based upon different perspectivesand priorities. Thebookconcludeswithanoverviewthatputsallthisinperspectiveandmakes the argument that AI does not replace us, but rather serves as a digital assistant. While it is still too early to determine the final direction in which pathology will evolve, it is nottoo early torecognize thatit will grow well beyondits current ca- pabilitiesandwilldosowiththeaidofAI.However,until“generalAI”isattained that can fully simulate conscious thought (which may never happen), biological brains will remain indispensable. Thus, for the foreseeable future, human patholo- gistswillnotbeobsolete.Instead,thepartnershipbetweenartificialandnaturalin- telligence will continue to expand to form a true partnership. The machine will Preface xvii generateknowledge,andthehumanwillprovidethewisdomtoproperlyapplythat knowledge. The machine will provide the knowledge that a tomato is a fruit. The human will provide the wisdom that it does not belong in a fruit salad. Together theywilldetermine that ketchup is nota smoothie. Stanley Cohen, MD Emeritus Chair of Pathology &EmeritusFounding Director Center for Biophysical Pathology Rutgers-NewJerseyMedical School,Newark, NJ, United States Adjunct Professor ofPathology Perelman Medical School UniversityofPennsylvania, Philadelphia, PA, United States Adjunct Professor ofPathology Kimmel School ofMedicine JeffersonUniversity, Philadelphia, PA, United States CHAPTER 1 The evolution of machine learning: past, present, and future Stanley Cohen, MD1,2,3 1EmeritusChairofPathology&EmeritusFoundingDirector,CenterforBiophysicalPathology, Rutgers-NewJerseyMedicalSchool,Newark,NJ,UnitedStates;2AdjunctProfessorofPathology, PerelmanMedicalSchool,UniversityofPennsylvania,Philadelphia,PA,UnitedStates;3Adjunct ProfessorofPathology,KimmelSchoolofMedicine,JeffersonUniversity,Philadelphia,PA,United States Introduction The first computers were designed solely to perform complexcalculations rapidly. Although conceived in the 1800s, and theoretical underpinnings developed in the early 1900s, it was not until ENIAC was built that the first practical computer can be said to have come into existence. The acronym ENIAC stands for Electronic NumericalIntegratorandCalculator.Itoccupieda20(cid:1)40footroom,andcontained over18,000vacuumtubes.Fromthattimeuntilthemid-20thcentury,theprograms thatranonthedescendantsofENIACwererulebased,inthatthemachinewasgiven a set of instructions to manipulate data based upon mathematical, logical, and/or probabilistic formulas. The difference between an electronic calculator and a computer is that the computer encodes not only numeric data but also the rules for the manipulation ofthese data (encoded as number sequences). Therearethreebasicpartstoacomputer:corememory,whereprogramsanddata are stored, a central processing unit that executes the instructions and returns the output of that computation to memory for further use, and devices for input and outputofdata.High-levelprogramminglanguagestakeasinputprogramstatements and translate them into the (numeric) information that a computer can understand. Additionally,therearethreebasicconceptsthatareintrinsictoeveryprogramming language: assignment, conditionals, and loops. In a computer, x¼5 is not a state- ment about equality. Instead, it means that the value 5 is assigned to x. In this context, X¼Xþ2 though mathematically impossible makes perfect sense. We add 2 towhatever is in X (in this case, 5) and the new value (7) now replaces that old value (5) in X. It helps to think of the¼sign as meaning that the quantity on theleftisreplacedbythequantityontheright.Aconditionaliseasiertounderstand. Weaskthecomputertomakeatestthathastwopossibleresults.Ifoneresultholds, thenthecomputerwilldosomething.Ifnot,itwilldosomethingelse.Thisistypi- callyanIF-THENstatement.IFyouarewearingagreensuitwithyellowpolkadots, 1 ArtificialIntelligenceandDeepLearninginPathology.https://doi.org/10.1016/B978-0-323-67538-3.00001-4 Copyright©2021ElsevierInc.Allrightsreserved.
The list of books you might like
