Introduction to Visual Effects. A Computational Approach

243 Pages·2023·32.431 MB·English

by  Bruno Madeira, Luiz Velho

Introduction to Visual Effects
Introduction to Visual Effects: A Computational Approach is the first single introduction to
computational and mathematical aspects of visual effects incorporating both computer vision
and graphics. The book provides the readers with the source code to a library, enabling them
to follow the chapters directly and build up a complete visual effects platform. The book covers
the basic approaches to camera pose estimation, global illumination, and image-based lighting,
and includes chapters on the virtual camera, optimization and computer vision, path tracing,
and more.
Key features include:
• Introduction to projective geometry, image-based lighting (IBL), global illumination
solved by the Monte Carlo method (pathtracing), an explanation of a set of optimiza-
tion methods, and the techniques used for calibrating one, two, and many cameras,
including how to use the RANSAC algorithm in order to make the process robust, and
providing code to be implemented using the Gnu Scientific Library.
• C/C++ code using the OpenCV library to be used in the process of tracking points
on a movie (an important step for the matchmove process) and in the construction of
modeling tools for visual effects.
• A simple model of the bidirectional reflectance distribution function (BRDF) of sur-
faces and the differential rendering method allowing the reader to generate consistent
shadows, supported by a code that can be used in combination with a software like
Luminance HDR.
Bruno Madeira is a computer engineer from the Military Engineering Institute (IME). He
earned his master’s and DSc degrees in mathematics with emphasis on computer graphics and
vision at IMPA. He has more than 15 years of experience teaching computer graphics at IME
and is a member of the Simulators Group at CTEx, Brazil.
Luiz Velho is a senior researcher, professor, and a VISGRAF Laboratory leading scientist at the
National Institute for Pure and Applied Mathematics (IMPA), Brazil. His academic background
includes a BE in industrial design from ESDI-UERJ, an MS in computer graphics from MIT
Media Lab, and a PhD in computer science from the University of Toronto. His experience in
computational media spans the fields of modeling, rendering, imaging, and animation.

Introduction to Visual Effects
A Computational Approach
Bruno Madeira
Luiz Velho
First edition published 2023
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2023 Luiz Velho and Bruno Madeira
CRC Press is an imprint of Taylor & Francis Group, LLC
Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have
attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders
if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please
write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized
in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying,
microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]
Trademark notice:Product or corporate names may be trademarks or registered trademarks and are used only for
identification and explanation without intent to infringe.
ISBN: 978-1-032-07230-2 (hbk)
ISBN: 978-1-032-06124-5 (pbk)
ISBN: 978-1-003-20602-6 (ebk)
DOI: 10.1201/9781003206026
Typeset in Latin Modern font
by KnowledgeWorks Global Ltd.
Publisher’s note:This book has been prepared from camera-ready copy provided by the authors.
To our loved ones.

Contents
Preface xiii
Chapter 1(cid:4) Introduction 1
1.1 CAMERACALIBRATION 1
1.2 HISTORICALOVERVIEWOFTRACKING 2
1.3 GLOBALILLUMINATION 3
1.4 IMAGE-BASEDLIGHTING 4
1.5 MATHEMATICALNOTATIONS 4
1.6 PROJECTIVEGEOMETRYCONCEPTS 5
1.6.1 Projective Space 5
1.6.2 Projective Transforms 5
1.6.3 Projective Geometry on This Book 6
1.6.4 Parallelism and Ideal Points 7
1.7 ABOUTTHECODE 9
Chapter 2(cid:4) Virtual Camera 11
2.1 BASICMODEL 12
2.1.1 Camera in the Origin 12
2.1.2 Camera in Generic Position 13
2.1.3 Digital Camera 13
2.1.4 Intrinsic Parameters 13
2.1.5 Dimension of the Space of Virtual Cameras 13
2.2 CAMERAFORIMAGESYNTHESIS 14
2.2.1 Terminologies 14
2.2.2 Clipping and Visibility 14
2.3 TRANSFORMATIONOFVISUALIZATION 15
2.3.1 Positioning the Camera 15
2.3.2 Transformation of Normalization 16
vii
viii (cid:4) Contents
2.3.3 Perspective Projection 16
2.3.4 Device Coordinates 16
2.4 COMPARISONWITHTHEBASICMODEL 17
2.4.1 Intrinsic Parameters 17
2.4.2 Dimension 18
2.4.3 Advantages over the Basic Model 18
2.5 CAMERAFORPATHTRACING 18
2.6 VISIBILITYANDRAYCASTING 18
2.7 CAMERASFORCALIBRATION 19
2.7.1 Projective Model 20
2.7.2 Projective Notation for Cameras 20
2.7.3 Generic Projective Camera 20
2.8 MAPPINGACALIBRATEDCAMERAINTOTHES3DLIBRARY 21
2.8.1 Specification of Extrinsic Parameters 21
2.8.2 Specification of Intrinsic Parameters 21
2.9 API 22
2.9.1 MatchMove Software Functions 22
2.9.2 Render Software Functions 23
2.10 CODE 24
2.10.1 Code in the MatchMove Software 24
2.10.2 Code in the Render Software 25
Chapter 3(cid:4) Optimization Tools 29
3.1 MINIMIZEAFUNCTIONDEFINEDONANINTERVAL 29
3.2 LEASTSQUARES 31
3.3 NON-LINEARLEASTSQUARES 32
3.3.1 Gauss-Newton Method 32
3.3.2 Levenberg-Marquardt Algorithm 33
3.4 MINIMIZETHENORMOFALINEARFUNCTIONOVERASPHERE 33
3.5 TWOSTAGESOPTIMIZATION 34
3.6 ROBUSTMODELESTIMATION 35
3.6.1 RANSAC Algorithm 35
3.6.2 Example of Using the RANSAC Algorithm 35
Contents (cid:4) ix
Chapter 4(cid:4) Estimating One Camera 37
4.1 CALIBRATIONINRELATIONTOASETOF3DPOINTS 37
4.1.1 Calibration Using Six Matches 37
4.1.2 Calibration Using More Than Six Matches 38
4.2 NORMALIZATIONOFTHEPOINTS 38
4.3 ISOLATIONOFCAMERAPARAMETERS 38
4.4 CAMERAFORIMAGESYNTHESIS 41
4.5 CALIBRATIONBYRESTRICTEDOPTIMIZATION 42
4.5.1 Adjusting the Levenberg-Marquardt to the Problem 42
4.5.2 Parameterization of Rotations 43
4.5.3 Parameterization of the Camera Space 43
4.6 PROBLEMPOINTSOFPARAMETERIZATION 44
4.7 FINDINGTHEINTRINSICPARAMETERS 44
4.8 CALIBRATIONUSINGAPLANARPATTERN 45
4.9 API 48
4.10 CODE 50
4.11 SINGLECAMERACALIBRATIONPROGRAM 59
4.12 SIXPOINTSSINGLECAMERACALIBRATIONPROGRAM 61
Chapter 5(cid:4) Estimating Two Cameras 63
5.1 REPRESENTATIONOFRELATIVEPOSITIONING 63
5.2 RIGIDMOVEMENT 64
5.3 OTHERPROJECTIVEMODEL 64
5.4 EPIPOLARGEOMETRY 64
5.4.1 Essential Matrix 64
5.5 FUNDAMENTALMATRIX 65
5.6 THE8-POINTSALGORITHM 65
5.6.1 Calculation of F 66
5.6.2 Using More Than 8 Points 66
5.6.3 Calculation of F˜ 66
5.7 NORMALIZED8-POINTSALGORITHM 67
5.8 FINDINGTHEEXTRINSICPARAMETERS 67
5.8.1 Adding Clipping to the Model 68
5.8.2 Three-Dimensional Reconstruction 69