Smart Algorithms for Multimedia and Imaging(2021)[Rychagov et al][9783030667412]

439 Pages·2021·20.888 MB·English

by  Rychagov

Signals and Communication Technology
Michael N. Rychagov
Ekaterina V. Tolstaya
Mikhail Y. Sirotenko  Editors
Smart
Algorithms
for Multimedia
and Imaging
Signals and Communication Technology
SeriesEditors
Emre Celebi, Department of Computer Science, University of Central Arkansas,
Conway,AR,USA
JingdongChen,NorthwesternPolytechnicalUniversity,Xi'an,China
E. S. Gopi, Department of Electronics and Communication Engineering, National
InstituteofTechnology,Tiruchirappalli,TamilNadu,India
AmyNeustein,LinguisticTechnologySystems,FortLee,NJ,USA
H. Vincent Poor, Department of Electrical Engineering, Princeton University,
Princeton,NJ,USA
This series is devoted to fundamentals and applications of modern methods of
signal processing and cutting-edge communication technologies. The main topics
are information and signal theory, acoustical signal processing, image processing
and multimedia systems, mobile and wireless communications, and computer and
communicationnetworks.Volumesintheseriesaddressresearchersinacademiaand
industrial R&D departments. The series is application-oriented. The level of
presentation of each individual volume, however, depends on the subject and can
rangefrompracticaltoscientific.
**Indexing:Allbooksin“SignalsandCommunicationTechnology”areindexedby
ScopusandzbMATH**
For general information about this book series, comments or suggestions, please
contact Mary James at [email protected] or Ramesh Nath Premnath at
[email protected].
Moreinformationaboutthisseriesathttp://www.springer.com/series/4748
(cid:129) (cid:129)
Michael N. Rychagov Ekaterina V. Tolstaya
Mikhail Y. Sirotenko
Editors
Smart Algorithms for
Multimedia and Imaging
Editors
MichaelN.Rychagov EkaterinaV.Tolstaya
NationalResearchUniversityofElectronic AramcoInnovationsLLC
Technology(MIET) Moscow,Russia
Moscow,Russia
MikhailY.Sirotenko
GoogleResearch
NewYork,NY,USA
ISSN1860-4862 ISSN1860-4870 (electronic)
SignalsandCommunicationTechnology
ISBN978-3-030-66740-5 ISBN978-3-030-66741-2 (eBook)
https://doi.org/10.1007/978-3-030-66741-2
©SpringerNatureSwitzerlandAG2021
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartofthe
materialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors, and the editorsare safeto assume that the adviceand informationin this
bookarebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Over the past decades, people have produced vast amounts of multimedia content,
includingtext,audio,images,animations,andvideo.Thesubstanceofthiscontent
belongs, in turn, to various areas, including entertainment, engineering, medicine,
business,scientificresearch,etc.Thiscontentshouldbereadilyprocessed,analysed,
anddisplayedbynumerousdeviceslikeTVs,mobiledevices,VRheadsets,medical
devices, media players, etc., without losing its quality. This brings researchers and
engineers to the problem of the fast transformation and processing of
multidimensionalsignals,wheretheymustdealwithdifferentsizesandresolutions,
processingspeed,memory,andpowerconsumption.Inthisbook,wedescribesmart
algorithms applied both for multimedia processing in general and in imaging
technologyinparticular.
Inthefirstbookofthisseries,AdaptiveImageProcessingAlgorithmsforPrinting
by I.V. Safonov, I.V. Kurilin, M.N. Rychagov, and E.V. Tolstaya, published by
Springer Nature Singapore in 2018, several algorithms were considered for the
image processing pipeline of photo-printer and photo-editing software tools that
wehaveworkedonatdifferenttimesforprocessingstillimagesandphotos.
Thesecondbook,DocumentImageProcessingforScanningandPrintingbythe
same authors, published by Springer Nature Switzerland in 2019, dealt with docu-
mentimageprocessingforscanningandprinting.Acopyingtechnologyisneededto
make perfect copies from extremely varied originals; therefore, copying is not in
practiceseparablefromimageenhancement.Fromatechnicalperspective,itisbest
toconsiderdocumentcopyingjointlywithimageenhancement.
Thisbookisdevotedtomultimediaalgorithmsandimaging,anditisdividedinto
fourmaininterconnectedparts:
(cid:129) ImageandVideoConversion
(cid:129) TVandDisplayApplications
(cid:129) MachineLearningandArtificialIntelligence
(cid:129) MobileAlgorithms
v
vi Preface
ImageandVideoConversionincludesfivechaptersthatcoversolutionsonsuper-
resolution using a multi-frame-based approach as well as machine learning-based
super-resolution. They also cover the processing of 3D signals, namely depth
estimationandcontrol,andsemi-automatic2Dto3Dvideoconversion.Acompre-
hensivereviewofvisuallosslesscolourcompressiontechnologyconcludesthispart.
TV and Display Applications includes three chapters in which the following
algorithms are considered: video editing, real-time sports episode detection by
videocontentanalysis,andthegenerationandreproductionofnaturaleffects.
Machine Learning and Artificial Intelligence includes four chapters, where the
followingtopicsarecovered:imageclassificationasaservice,mobileuserprofiling,
andautomaticviewplanninginmagneticresonanceimaging,aswellasdictionary-
basedcompressedsensingMRI(magneticresonanceimaging).
Finally, Mobile Algorithms consists of four chapters where the following algo-
rithmsandsolutionsimplementedformobiledevicesaredescribed:adepthcamera
based on a colour-coded aperture, the animated graphical abstract of an image, a
motionphoto,andapproachesandmethodsforirisrecognitionformobiledevices.
The solutions presented in the first two books and in the current one have been
includedindozensofpatentsworldwide,presentedatinternationalconferences,and
realized in the firmware of devices and software. The material is based on the
experienceofbotheditorsandtheauthorsofparticularchaptersinindustrialresearch
andtechnologycommercialization.Theauthorshaveworkedonthedevelopmentof
algorithms for different divisions of Samsung Electronics Co., Ltd, including the
Printing Business, Visual Display Business, Health and Medical Equipment Divi-
sion,andMobileCommunicationBusinessformorethan15years.
We should especially note that this book in no way pretends to present an
in-depth review of the achievements accumulated to date in the field of image and
videoconversion,TVanddisplayapplications,ormobilealgorithms.Instead,inthis
book,themainresultsofthestudiesthatwehaveauthoredaresummarized.Wehope
that the main approaches, optimization procedures, and heuristic findings are still
relevantandcanbeusedasabasisfornewintelligentsolutionsinmultimedia,TV,
andmobileapplications.
Howcanalgorithmscapableofbeingadaptivetoimagecontentbedeveloped?In
many cases, inductive or deductive inference can help. Many of the algorithms
include lightweight classifiers or other machine-learning-based techniques, which
havelowcomputationalcomplexityandmodelsize.Thismakesthemdeployableon
embeddedplatforms.
As we have mentioned, the majority of the described algorithms were
implemented as systems-on-chip firmware or as software products. This was a
challenge because, for each industrial task, there are always strict specification
requirements, and, as a result, there are limitations on computational complexity,
memory consumption, and power efficiency. In this book, typically, no device-
dependentoptimizationtricksaredescribed,thoughtheideasforeffectivemethods
fromanalgorithmicpointofviewareprovided.
This book is intended for all those who are interested in advanced multimedia
processing approaches, including applications of machine learning techniques for
Preface vii
thedevelopmentofeffectiveadaptivealgorithms.Wehopethatthisbookwillserve
asausefulguideforstudents,researchers,andpractitioners.
Itistheintentionoftheeditorsthateachchapterbeusedasanindependenttext.In
thisregard,atthebeginningofalargefragment,themainprovisionsconsideredin
the preceding text are briefly repeated with reference to the appropriate chapter or
section.Referencestotheworksofotherauthorsanddiscussionsoftheirresultsare
giveninthecourseofthepresentationofthematerial.
WewouldliketothankourcolleagueswhoworkedwithusbothinKoreaandat
theSamsungR&DInstituteRus,Moscow,onthedevelopmentandimplementation
of the technologies mentioned in the book, including all of the authors of the
chapters: Sang-cheon Choi, Yang Lim Choi, Dr. Praven Gulaka, Dr. Seung-Hoon
Hahn, Jaebong Yoo, Heejun Lee, Kwanghyun Lee, San-Su Lee, B’jungtae O,
Daekyu Shin, Minsuk Song, Gnana S. Surneni, Juwoan Yoo, Valery
V. Anisimovskiy, Roman V. Arzumanyan, Andrey A. Bout, Dr. Victor V. Bucha,
Dr. Vitaly V. Chernov, Dr. Alexey S. Chernyavskiy, Dr. Aleksey B. Danilevich,
AndreyN.Drogolyub,YuriS.Efimov,MartaA.Egorova,Dr.VladimirA.Eremeev,
Dr. Alexey M. Fartukov, Dr. Kirill A. Gavrilyuk, Ivan V. Glazistov, Vitaly
S. Gnatyuk, Aleksei M. Gruzdev, Artem K.Ignatov, Ivan O.Karacharov, Aleksey
Y. Kazantsev, Dr. Konstantin V. Kolchin, Anton S. Kornilov, Dmitry
A. Korobchenko, Mikhail V. Korobkin, Dr. Oxana V. Korzh (Dzhosan), Dr. Igor
M. Kovliga, Konstantin A. Kryzhanovsky, Dr. Mikhail S. Kudinov, Artem
I. Kuharenko, Dr. Ilya V. Kurilin, Vladimir G. Kurmanov, Dr. Gennady
G. Kuznetsov, Dr. Vitaly S. Lavrukhin, Kirill V. Lebedev, Vladislav A. Makeev,
Vadim A.Markovtsev,Dr.MstislavV.Maslennikov,Dr.Artem S.Migukin,Gleb
S. Milyukov, Dr. Michael N. Mishourovsky, Andrey K. Moiseenko, Alexander
A. Molchanov, Dr. Oleg F. Muratov, Dr. Aleksei Y. Nevidomskii, Dr. Gleb
A. Odinokikh, Irina I. Piontkovskaya, Ivan A. Panchenko, Vladimir
P. Paramonov, Dr. Xenia Y. Petrova, Dr. Sergey Y. Podlesnyy, Petr Pohl,
Dr. Dmitry V. Polubotko, Andrey A. Popovkin, Iryna A. Reimers, Alexander
A. Romanenko, Oleg S. Rybakov, Associate Prof., Dr. Ilia V. Safonov, Sergey
M. Sedunov, Andrey Y. Shcherbinin, Yury V. Slynko, Ivan A. Solomatin, Liubov
V. Stepanova (Podoynitsyna), Zoya V. Pushchina, Prof., Dr.Sc. Mikhail
K. Tchobanou, Dr. Alexander A. Uldin, Anna A. Varfolomeeva, Kira
I. Vinogradova, Dr. Sergey S. Zavalishin, Alexey M. Vil’kin, Sergey
Y. Yakovlev, Dr. Sergey N. Zagoruyko, Dr. Mikhail V. Zheludev, and numerous
volunteerswhotookpartinthecollectionoftestdatabasesandtheevaluationofthe
qualityofouralgorithms.
Contributionsfromourpartnersatacademicandinstitutionalorganizationswith
whomweareassociatedthroughjointpublications,patents,andcollaborativework,
i.e., Prof. Dr.Sc. Anatoly G. Yagola, Prof. Dr.Sc. Andrey S. Krylov, Dr. Andrey
V.Nasonov,andDr.ElenaA.PavelyevafromMoscowStateUniversity;Academi-
cian RAS, Prof., M.D. Sergey K. Ternovoy, Prof., M.D. Merab A. Sharia, and
M.D. Dmitry V. Ustuzhanin from the Tomography Department of the Cardiology
Research Center (Moscow); Prof., Dr.Sc. Rustam K. Latypov, Dr. Ayrat
F. Khasyanov, Dr. Maksim O. Talanov, and Irina A. Maksimova from Kazan
viii Preface
StateUniversity;AcademicianRAS,Prof.,Dr.Sc.EvgeniyE.Tyrtyshnikovfromthe
MarchukInstituteofNumericalMathematicsRAS;AcademicianRAS,Prof.,Dr.Sc.
SergeiV.Kislyakov,CorrespondingMemberofRAS,Dr.Sc.MaximA.Vsemirnov,
and Dr. Sergei I. Nikolenko from the St. Petersburg Department of Steklov Math-
ematical Institute of RAS; Corresponding Member of RAS, Prof., Dr.Sc. Rafael
M.Yusupov,Prof.,andProf.,Dr.Sc.VladimirI.GorodetskifromtheSt.Petersburg
InstituteforInformaticsandAutomationRAS;Prof.,Dr.Sc.IgorS.Gruzmanfrom
Novosibirsk State Technical University; and Prof., Dr.Sc. Vadim R. Lutsiv from
ITMOUniversity(St.Petersburg),arealsodeeplyappreciated.
Duringalltheseyearsandthroughoutthedevelopmentofthesetechnologies,we
receivedcomprehensive assistanceand activetechnical supportfromSRR General
DirectorsDr.YoungminLee,Dr.Sang-YoonOh,Dr.KimHyoGyu,andJong-Sam
Woo; the members of the planning R&D team: Kee-Hang Lee, Sang-Bae Lee,
Jungsik Kim, Seungmin (Simon) Kim, and Byoung Kyu Min; the SRR IP Depart-
ment, Mikhail Y. Silin, Yulia G. Yukovich, and Sergey V. Navasardyan from
General Administration. All of their actions were always directed toward finding
themostoptimalformsofR&Dworkbothformanagersandengineers,generating
new approaches to create promising algorithms and SW, and ultimately creating
solutionsofhighquality.Atanytime,wereliedontheirparticipationandassistance
inresolvingissues.
Moscow,Russia MichaelN.Rychagov
NewYork,NY,USA EkaterinaV.Tolstaya
MikhailY.Sirotenko
Acknowledgment
Proofreading of all pages of the manuscript was performed by PRS agency (http://
www.proof-reading-service.com).
ix