Multiscale Multimodal Medical Imaging: Third International Workshop, MMMI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 22, 2022, Proceedings

139 Pages·2022·21.789 MB·English

by  Xiang Li, Jinglei Lv, Yuankai Huo, Bin Dong, Richard M. Leahy, Quanzheng Li

Lecture Notes in Computer Science 13594
FoundingEditors
GerhardGoos
KarlsruheInstituteofTechnology,Karlsruhe,Germany
JurisHartmanis
CornellUniversity,Ithaca,NY,USA
EditorialBoardMembers
ElisaBertino
PurdueUniversity,WestLafayette,IN,USA
WenGao
PekingUniversity,Beijing,China
BernhardSteffen
TUDortmundUniversity,Dortmund,Germany
MotiYung
ColumbiaUniversity,NewYork,NY,USA
Moreinformationaboutthisseriesathttps://link.springer.com/bookseries/558
· · · ·
Xiang Li Jinglei Lv Yuankai Huo Bin Dong
·
Richard M. Leahy Quanzheng Li (Eds.)
Multiscale Multimodal
Medical Imaging
Third International Workshop, MMMI 2022
Held in Conjunction with MICCAI 2022
Singapore, September 22, 2022
Proceedings
Editors
XiangLi JingleiLv
MassachusettsGeneralHospital UniversityofSydney
Boston,MA,USA Sydney,Australia
YuankaiHuo BinDong
VanderbiltUniversity PekingUniversity
Nashville,TN,USA Beijing,China
RichardM.Leahy QuanzhengLi
UniversityofSouthernCalifornia MassachusettsGeneralHospital
LosAngeles,CA,USA Boston,MA,USA
ISSN 0302-9743 ISSN 1611-3349 (electronic)
LectureNotesinComputerScience
ISBN 978-3-031-18813-8 ISBN 978-3-031-18814-5 (eBook)
https://doi.org/10.1007/978-3-031-18814-5
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicense
toSpringerNatureSwitzerlandAG2022
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartofthe
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodologynow
knownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbookare
believedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsortheeditors
give a warranty, expressed or implied, with respect to the material contained herein or for any errors or
omissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictionalclaimsin
publishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Onbehalfoftheorganizingcommittee,wewelcomeyoutothe3rdInternationalWork-
shop on Multiscale Multimodal Medical Imaging (MMMI 2022), held in conjunction
withtheInternationalConferenceonMedicalImageComputingandComputerAssisted
Intervention (MICCAI 2022) in Singapore. The workshop was organized through the
combinedeffortsoftheMassachusettsGeneralHospitalandHarvardMedicalSchool,
theUniversityofSouthernCalifornia,PekingUniversity,VanderbiltUniversity,andthe
UniversityofSydney.
This series of MMMI workshops aims to develop the state of the art in acquiring
andanalyzingmedicalimagesatmultiplescalesandfrommultiplemodalities.Topics
oftheworkshopincludealgorithmdevelopment,implementationofmethodologies,and
experimental studies. The workshop also aims to facilitate more communication and
interchangeofideasbetweenresearchersinthefieldsofclinicalstudy,medicalimage
analysis,andmachinelearning.
Since the first edition in 2019 (Shenzhen, China), the MMMI workshop has been
wellreceivedbytheMICCAIcommunity.Thisyear,theworkshop’sthemewasnovel
methodology development for multi-modal fusion. MMMI 2022 received a total of
18 submissions. All submissions underwent a double-blind peer-review process, each
beingreviewedbyatleasttwoindependentreviewersandoneProgramCommittee(PC)
member.Finally,12submissionswereacceptedforpresentationattheworkshop,which
areincludedinthisproceedings,basedonthePCreviewscoresandcomments.Thetime
andeffortsofallthePCmembersandreviewers,whichensuredthattheMMMIworkshop
wouldfeaturehigh-qualityandvaluableworksinthefield,arehighlyappreciated.
With the increasing application of multi-modal, multi-scale imaging in medical
researchstudiesandclinicalpractice,weenvisionthattheMMMIworkshopwillcon-
tinuetoserveasaninternationalplatformforpresentingnovelworks,discussingessential
challenges,andpromotingcollaborationswithinthecommunity.Wewouldliketothank
everyoneforthehardwork,andseeyounextyear!
September2022 Xiang Li
Jinglei Lv
Yuankai Huo
Bin Dong
Richard M. Leahy
Quanzheng Li
Organization
WorkshopChairs
XiangLi MassachusettsGeneralHospital,USA
JingleiLv UniversityofSydney,Australia
YuankaiHuo VanderbiltUniversity,USA
BinDong PekingUniversity,China
RichardM.Leahy UniversityofSouthernCalifornia,USA
QuanzhengLi MassachusettsGeneralHospital,USA
ProgramCommittee
DakaiJin PAIIInc.,USA
DonghaoZhang MonashUniversity,Australia
EungJooLee MassachusettsGeneralHospitalandHarvard
MedicalSchool,USA
GeorgiosAngelis UniversityofSydney,Australia
HuiRen MassachusettsGeneralHospitalandHarvard
MedicalSchool,USA
JiangHu MassachusettsGeneralHospitalandHarvard
MedicalSchool,USA
LeiBi UniversityofSydney,Australia
LiyueShen StanfordUniversity,USA
MarianoCabezas UniversityofSydney,Australia
ShijieZhao NorthwesternPolytechnicalUniversity,China
ZhoubingXu SiemensHealthineers,USA
Contents
M2F: A Multi-modal and Multi-task Fusion Network for Glioma
DiagnosisandPrognosis ................................................ 1
ZilinLu,MengkangLu,andYongXia
VisualModalitiesBasedMultimodalFusionforSurgicalPhaseRecognition ... 11
BogyuPark,HyeongyuChi,BokyungPark,JiwonLee,SunghyunPark,
WooJinHyung,andMin-KookChoi
Cross-Scale Attention Guided Multi-instance Learning for Crohn’s
DiseaseDiagnosiswithPathologicalImages ............................... 24
RuiningDeng, CanCui, LucasW.Remedios, ShunxingBao,
R.MichaelWomick, SophieChiron, JiaLi, JosephT.Roland,
KenS.Lau,QiLiu,KeithT.Wilson,YaohongWang,LoriA.Coburn,
BennettA.Landman,andYuankaiHuo
VesselSegmentationviaLinkPredictionofGraphNeuralNetworks .......... 34
HaoYu,JieZhao,andLiZhang
A Bagging Strategy-Based Multi-scale Texture GLCM-CNN Model
for Differentiating Malignant from Benign Lesions Using Small
PathologicallyProvenDataset ........................................... 44
ShuZhang, JinruWu, SigangYu, RuoyangWang, EnzeShi,
YongfengGao,andZhengrongLiang
LiverSegmentationQualityControlinMulti-sequenceMRStudies ........... 54
Yi-QingWangandGiovanniPalma
PatternAnalysisofSubstantiaNigrainParkinsonDiseasebyFifth-Order
TensorDecompositionandMulti-sequenceMRI ........................... 63
HayatoItoh, TaoHu, MasahiroOda, ShinjiSaiki, KojiKamagata,
NobutakaHattori,ShigekiAoki,andKensakuMori
Gabor Filter-Embedded U-Net with Transformer-Based Encoding
forBiomedicalImageSegmentation ...................................... 76
AbelA.Reyes,SidikePaheding,MakarandDeo,andMichelAudette
Learning-Based Detection of MYCN Amplification in Clinical
NeuroblastomaPatients:APilotStudy ................................... 89
XiangXiang,ZihanZhang,XuehuaPeng,andJianboShao
viii Contents
Coordinate Translator for Learning Deformable Medical Image
Registration .......................................................... 98
YihaoLiu, LianruiZuo, ShuoHan, YuanXue, JerryL.Prince,
andAaronCarass
Towards Optimal Patch Size in Vision Transformers for Tumor
Segmentation ......................................................... 110
RamtinMojtahedi, MohammadHamghalam, RichardK.G.Do,
andAmberL.Simpson
Improved Multi-modal Patch Based Lymphoma Segmentation
withNegativeSampleAugmentationandLabelGuidanceonPET/CTScans ... 121
LiangchenLiu, JianfeiLiu, ManasKumarNag, NavidHasani,
SeungYeonShin, SriramS.Paravastu, BabakSaboury, JingXiao,
LingyunHuang,andRonaldM.Summers
AuthorIndex ......................................................... 131
M2F: A Multi-modal and Multi-task
Fusion Network for Glioma Diagnosis
and Prognosis
B
Zilin Lu1,2, Mengkang Lu2, and Yong Xia1,2( )
1 Ningbo Institute of Northwestern Polytechnical University, Ningbo 315048, China
2 National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big
Data Application Technology, School of Computer Science and Engineering,
Northwestern Polytechnical University, Xi’an 710072, China
[email protected]
Abstract. Clinical decision of oncology comes from multi-modal infor-
mation, such as morphological information from histopathology and
molecularprofilesfromgenomics.Mostoftheexistingmulti-modallearn-
ing models achieve better performance than single-modal models. How-
ever, these multi-modal models only focus on the interactive informa-
tion between modalities, which ignore the internal relationship between
multiple tasks. Both survival analysis task and tumor grading task can
provide reliable information for pathologists in the diagnosis and prog-
nosis of cancer. In this work, we present a Multi-modal and Multi-task
Fusion (M2F) model to make use of the potential connection between
modalities and tasks. The co-attention module in multi-modal trans-
former extractor can excavate the intrinsic information between modal-
ities more effectively than the original fusion methods. Joint training of
tumorgradingbranchandsurvivalanalysisbranch,insteadofseparating
them,canmakefulluseofthecomplementaryinformationbetweentasks
to improve the performance of the model. We validate our M2F model
ongliomadatasetsfromtheCancerGenomeAtlas(TCGA).Experiment
resultsshowourM2Fmodelissuperiortoexistingmulti-modalmodels,
which proves the effectiveness of our model.
· · ·
Keywords: Multi-modal learning Multi-task Survival analysis
Tumor grading
1 Introduction
Gliomas are the most common primary malignant brain tumors, which account
for 80% of cases [26]. Clinical diagnosis and prognosis of gliomas comes from
multi-modal information, such as pathological images and genomics data [7].
Pathological images contain the structural and morphological information of
tumor cells, and genomics data provide molecular profiles.
(cid:2)c TheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerlandAG2022
X.Lietal.(Eds.):MMMI2022,LNCS13594,pp.1–10,2022.
https://doi.org/10.1007/978-3-031-18814-5_1