Fish4Knowledge: Collecting and Analyzing Massive Coral Reef Fish Video Data

321 Pages·2016·13.98 MB·English

by  Robert B. Fisher

Intelligent Systems Reference Library 104
Robert B. Fisher
Yun-Heh Chen-Burger
Daniela Giordano
Lynda Hardman
Fang-Pang Lin Editors
Fish4Knowledge:
Collecting and
Analyzing Massive
Coral Reef Fish
Video Data
Intelligent Systems Reference Library
Volume 104
Series editors
Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland
e-mail: [email protected]
Lakhmi C. Jain, Bournemouth University, Fern Barrow, Poole, UK, and
University of Canberra, Canberra, Australia
e-mail: [email protected]
About this Series
The aim of this series is to publish a Reference Library, including novel advances
and developments in all aspects of Intelligent Systems in an easily accessible and
well structured form. The series includes reference works, handbooks, compendia,
textbooks,well-structuredmonographs,dictionaries,andencyclopedias.Itcontains
well integrated knowledge and current information in the field of Intelligent Sys-
tems. The series covers the theory, applications, and design methods of Intelligent
Systems. Virtually all disciplines such as engineering, computer science, avionics,
business, e-commerce, environment, healthcare, physics and life science are
included.
More information about this series at http://www.springer.com/series/8578
Robert B. Fisher Yun-Heh Chen-Burger
(cid:129)
Daniela Giordano Lynda Hardman
(cid:129)
Fang-Pang Lin
Editors
Fish4Knowledge: Collecting
and Analyzing Massive Coral
Reef Fish Video Data
123
Editors
RobertB. Fisher Lynda Hardman
Schoolof Informatics Centrum Wiskunde& Informatica (CWI)
University of Edinburgh Amsterdam
Edinburgh TheNetherlands
UK
Fang-Pang Lin
Yun-HehChen-Burger National Center of HighPerformance
Heriot-Watt University Computing
Edinburgh Hsinchu
UK Taiwan
Daniela Giordano
Universita Degli StudiDi Catania
Catania
Italy
ISSN 1868-4394 ISSN 1868-4408 (electronic)
Intelligent Systems Reference Library
ISBN978-3-319-30206-5 ISBN978-3-319-30208-9 (eBook)
DOI 10.1007/978-3-319-30208-9
LibraryofCongressControlNumber:2016932519
©SpringerInternationalPublishingSwitzerland2016
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilarmethodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained
hereinorforanyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAGSwitzerland
To Mies, Hannah, Phoebe and Lars—with
thanks for their patience
To Albert and Benjamin for their love and
support
To Alberto, Robin, and Karin for their loving
support and encouragement
To my parents Esin and Cengiz for their
endless support
To Li-Chun and Matthew
Preface
This book was conceived near the end of the Fish4Knowledge project as a way of
communicating the achievements of the project to the scientific community. Many
scientificprojectsareverysuccessfulwithjournalandconferencepublications,but
it is rare to find an extended document that gives a full overview of a project,
describing not only the original contributions but also the necessary infrastructure
aspects. This book belongs to the latter category—it gives a brief introduction to
almost all aspects of the project in a series of 18 short chapters. This exposes a
range of topics, and also a view of how all of the topics fit together into the full
project.It isnota“popularscience”account oftheproject,i.e.,it doesnotgointo
the personalities, motivations, and challenges behind the project. Instead, it is a
technical book describing the scientific and engineering of the project. But by
keeping the chapters short, we have tried to make the content accessible to the
broader scientific public, particularly for the biological and computer science
communities. Each chapter cites more extensive descriptions of the content from
the more than 50 technical publications arising from the project.
The idea for the Fish4Knowledge project started to germinate from discussions
and visits between the Edinburgh and Taiwan partners, originally as an ecological
monitoringprojectbasedonvideodatacapturedoffthecoastofTaiwan.Later,we
sawtheEuropean UnionFrameworkSeventhProgrammecall forproposalsonthe
topic of Digital Libraries and Content. This call came at the start of the scientific
community’s widespread interest in “big data.” As a consequence of this conver-
gence, we conceived of a project that would combine computer vision, large
datasets and databases, supercomputer processing, and intelligent information
presentationmethods.Clearly,theproposalwassuccessfulandtheresultingproject
ranfromOctober1,2010throughSeptember30,2013.Intheend,werecordedand
analyzed about 90,000 hours (90 TB) of video from nine cameras off the coast of
Taiwan,detectingandtrackingover1billion fish,andrecordingtheirdetails inan
SQLdatabaseapproaching500GBinsize.Theresultsareanalyzableandviewable
bymarineecologistsusingthefacet-baseduserinterfacethattheprojectdeveloped.
vii
viii Preface
Youmightwonderif theauthorsaregoingtogetrich fromtheroyaltiesarising
from this book? With more than ten authors to share the royalties, we thought that
eachauthor’ssharewouldhardlybemorethanenoughtotaketheirpatientfriends
andfamilyoutforanicedinnertocelebratethebook.So,instead,wedecidedthat
theroyaltiesshouldbedonatedtotheFishBaseproject(FishBase.org)thatprovided
uswithmuchuseful,andfree,backgroundknowledgeaboutthespecies.Wegreatly
appreciate this excellent and free resource.
I (Bob) would like to make a final, personal comment—although I was the
coordinator of the Fish4Knowledge project, it was more a first amongst equals
situation.Therewasagreatdealofenthusiasmbyallprojectmembers,bothsenior
andearlycareerresearchers,andgreatcooperationandcollaborationbyall.Itmade
iteasytocoordinatetheproject(andthisbook).Itwasalsoafunproject,wherethe
consortium meetingsrotatedaround thedifferent partners’locations.Thisincluded
two meetings in Taiwan—great food, a bouncy boat trip to LanYu Island (one
oftherecordingsites)andabitofteamsnorkelingaroundthefishthatwehadseen
somuchofinthevideos.Itwasarealpleasureworkingwith everyoneonthisfun
and scientifically interesting project!
Edinburgh, UK Robert B. Fisher
October 2015 Yun-Heh Chen-Burger
Acknowledgments
First,wethankthefundersoftheresearchthatisdescribedinthisbook.Muchofthe
Fish4Knowledge project was funded by the European Union Seventh Framework
Programme[FP7/2007-2013]undergrantagreement257024,addressingObjective
ICT-2009.4.3: Intelligent Information Management, Challenge 4: Digital Libraries
and Content. Taiwan’s NARL/NCHC was funded by the Taiwan National Science
CouncilundergrantagreementNSC101-2923-I-492-002-MY2.Wealsothankthe
Taiwan Power Company, Taiwan Ocean Research Institute, and Kenting National
Parkforsharingthevideodatasetscollectedfromtheirunderwaterobservatoriesat
Nanwan, Lanyu and Houbi Lake of Taiwan, which enabled this research. The
underwaterobservationprojectwasfundedbyTaiwanPowerCompany.Wethank
theThirdNuclearPowerPlantofTaipowerforlogisticalsupport.Also,wethankthe
marine ecologists, led by T.Y. Fan, from theNational Museum of Marine Biology
and Aquarium, Taiwan for camera deployments and maintenance at the research
sites.
We thank National Center for High-performance Computing (NCHC), Taiwan
and Academic Sinica, Taiwan, for their efforts in capturing those valuable under-
water marine life videos and their tireless endeavors to combat regular typhoons
and open sea conditions in maintaining the high quality of videos, to assist us
achievethebestpossibleprocessingresults.Weextend ourgratitude tothestaffin
the System Administration Division of NCHC, especially Yin-Yu Shig, for his
great help and patience for our rule-breaking requests to use the supercomputing
platform, ALPS.
The CWI team is grateful for the valuable contributions of Elvira Arslanova;
Prof. Shao, and his colleagues from Academia Sinica; Prof. Fan, Dr. Hai Chiang
and their colleagues from the National Museum of Marine Science & Technology
of Taiwan; Dr.Ir. Nagelkerke, Dr. Tulp and their colleagues from the IMARES
Research Institute of Wageningen University; Dr. Lavaleye, Dr. Duineveld and
theircolleaguesfromtheRoyalInstituteforSeaResearch(NIOZ);TizianoPerrucci
and Martin van Harmelen.
ix
x Acknowledgments
We thank the OpenVCE.net project for the building shell and the Virtual
University of Edinburgh (Vue) for co-sponsoring the cost of the virtual land in
Second Life and OpenSimulator which host copies of our gallery.
TheUniversityofEdinburghteamwouldliketothankEPCCandIainRaefrom
theSchoolofInformaticsforassistancewithOracle(previouslySun)GridEngine.
Also, we greatly appreciate the help of Omer F. Rana (Cardiff University) and
Rafael Tolosana-Calasanz (University of Zaragoza) who collaborated on the
workflow performance evaluation.
The University of Edinburgh and University of Catania teams would like to
thank the many known and unknown people who contributed to the ground truth
datagenerationforfishdetection, fishtracking,andfishspeciesclassification.Bob
thankstheInstituteforAdvancedStudiesattheUniversityofWesternAustraliafor
the support during a sabbatical visit, which gave him space to finish the assembly
of the book, and especially Mohammed Bennamoun for hosting him and all the
helpful discussions.
We acknowledge the great support from the external people involved with the
project, including the project’s Scientific Advisory Board:
Kwang-Tsao Shao (Biodiversity Research Center, Academica Sinica)
Steffen Staab (University of Koblenz and Landau)
Konstantinos Stergiou (Aristotle University of Thessaloniki)
Monique Thonnat (INRIA)
We thank the EU Project Officer Stefano Bertolo, who was great—good sug-
gestions, good encouragement, minimizing bureaucracy, and the EU scientific
reviewers: Jenny Benois-Pineau, Anna Bosch Rue, and Rafael Garcia for their
support and advice.
Finally, we thank the Springer team, and especially Prof. Lakhmi Jain,
University of South Australia, for his support in developing the opportunity to
publish this book with Springer.