Remote Sensing Image Fusion

328 Pages·2015·9.949 MB·English

by  Luciano Alparone, Bruno Aiazzi, Stefano Baronti, Andrea Garzelli

Remote Sensing
Image Fusion
K20347_FM.indd 1 2/10/15 12:53 PM
Signal and Image Processing of
Earth Observations Series
Series Editor
C.H. Chen
Published Titles
Remote Sensing Image Fusion
Luciano Alparone, Bruno Aiazzi, Stefano Baronti, and Andrea Garzelli
K20347_FM.indd 2 2/10/15 12:53 PM
Remote Sensing
Image Fusion
Luciano Alparone • Bruno Aiazzi
Stefano Baronti • Andrea Garzelli
K20347_FM.indd 3 2/10/15 12:53 PM
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2015 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20150113
International Standard Book Number-13: 978-1-4665-8750-2 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copy-
right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222
Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro-
vides licenses and registration for a variety of users. For organizations that have been granted a photo-
copy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
List of Figures xi
List of Tables xv
Foreword xvii
Preface xix
Symbol Description xxi
List of Acronyms xxiii
1 Instructions for Use 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Aim and Scope of the Book . . . . . . . . . . . . . . . . . . . 2
1.3 Organization of Contents . . . . . . . . . . . . . . . . . . . . 3
1.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 8
2 Sensors and Image Data Products 11
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Definition of Terms . . . . . . . . . . . . . . . . . . . . 14
2.2.2 Spatial Resolution . . . . . . . . . . . . . . . . . . . . 15
2.2.2.1 Point spread function . . . . . . . . . . . . . 17
2.2.2.2 Modulation transfer function . . . . . . . . 18
2.2.2.3 Classification of sensors on spatial resolution 21
2.2.3 Radiometric Resolution . . . . . . . . . . . . . . . . . 21
2.2.4 Spectral Resolution. . . . . . . . . . . . . . . . . . . . 22
2.2.5 Temporal Resolution . . . . . . . . . . . . . . . . . . . 23
2.3 Acquisition Strategy . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.1 Whisk Broom Sensors . . . . . . . . . . . . . . . . . . 25
2.3.2 Push Broom Sensors . . . . . . . . . . . . . . . . . . . 26
2.4 Optical Sensors . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.1 Reflected Radiance Sensors . . . . . . . . . . . . . . . 31
2.4.1.1 Reflection . . . . . . . . . . . . . . . . . . . . 31
2.4.2 HR and VHR Sensors . . . . . . . . . . . . . . . . . . 32
2.4.2.1 IKONOS . . . . . . . . . . . . . . . . . . . . 33
v
vi Contents
2.4.2.2 QuickBird . . . . . . . . . . . . . . . . . . . 35
2.4.2.3 WorldView . . . . . . . . . . . . . . . . . . . 36
2.4.2.4 GeoEye . . . . . . . . . . . . . . . . . . . . . 38
2.4.2.5 Pl´eiades . . . . . . . . . . . . . . . . . . . . . 39
2.4.2.6 FormoSat-2 and KoMPSat . . . . . . . . . . 39
2.4.3 Thermal Imaging Sensors . . . . . . . . . . . . . . . . 40
2.5 Active Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.5.1 Radar and Synthetic Aperture Radar . . . . . . . . . 41
2.5.1.1 Across-track spatial resolution . . . . . . . . 42
2.5.1.2 Along-track spatial resolution. . . . . . . . . 44
2.5.1.3 Synthetic aperture radar (SAR) . . . . . . . 44
2.5.1.4 SAR images and speckle . . . . . . . . . . . 45
2.5.2 SAR Sensors . . . . . . . . . . . . . . . . . . . . . . . 47
2.5.3 LiDAR . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 50
3 Quality Assessment of Fusion 51
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.2 Quality Definition for Pansharpening . . . . . . . . . . . . . 52
3.2.1 Statistical Quality/Distortion Indices. . . . . . . . . . 54
3.2.1.1 Indices for scalar valued images . . . . . . . 54
3.2.1.2 Indices for vector valued images . . . . . . . 56
3.2.2 Protocols Established for Pansharpening . . . . . . . . 57
3.2.2.1 Wald’s protocol . . . . . . . . . . . . . . . . 58
3.2.2.2 Zhou’s protocol . . . . . . . . . . . . . . . . 59
3.2.2.3 QNR protocol . . . . . . . . . . . . . . . . . 60
3.2.2.4 Khan’s protocol . . . . . . . . . . . . . . . . 61
3.2.3 Extension to Hyperspectral Pansharpening . . . . . . 61
3.2.4 Extension to Thermal V-NIR Sharpening . . . . . . . 62
3.3 Assessment of Optical and SAR Fusion . . . . . . . . . . . . 62
3.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 64
4 Image Registration and Interpolation 67
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2 Image Registration . . . . . . . . . . . . . . . . . . . . . . . 69
4.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2.2 Geometric Corrections . . . . . . . . . . . . . . . . . . 70
4.2.2.1 Sources of geometric distortions . . . . . . . 70
4.2.3 Point Mapping Methods: Image Transformations . . . 73
4.2.3.1 2-D polynomials . . . . . . . . . . . . . . . . 73
4.2.3.2 3-D polynomials . . . . . . . . . . . . . . . . 76
4.2.4 Resampling . . . . . . . . . . . . . . . . . . . . . . . . 77
4.2.5 Other Registration Techniques . . . . . . . . . . . . . 78
4.3 Image Interpolation . . . . . . . . . . . . . . . . . . . . . . . 78
4.3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . 79
Contents vii
4.3.2 Theoretical Fundamentals of Digital Interpolation . . 79
4.3.2.1 Interpolation by continuous-time kernels . . 79
4.3.2.2 Interpolation by digital kernels . . . . . . . . 80
4.3.3 Ideal and Practical Interpolators . . . . . . . . . . . . 81
4.3.4 Piecewise Local Polynomial Kernels . . . . . . . . . . 83
4.3.5 Derivation of Piecewise Local Polynomial Kernels. . . 84
4.3.5.1 Continuous-time local polynomial kernels . . 84
4.3.5.2 Odd and even local polynomial kernels . . . 85
4.3.6 Interpolation of MS Data for Pansharpening . . . . . 89
4.3.6.1 One-dimensional case . . . . . . . . . . . . . 89
4.3.6.2 Two-dimensional case . . . . . . . . . . . . . 90
4.3.7 Interpolation Assessment for Pansharpening . . . . . . 92
4.3.7.1 Evaluation criteria . . . . . . . . . . . . . . . 92
4.3.7.2 Dataset . . . . . . . . . . . . . . . . . . . . . 92
4.3.7.3 Results and discussion . . . . . . . . . . . . . 93
4.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 99
5 Multiresolution Analysis for Image Fusion 101
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.2 Multiresolution Analysis . . . . . . . . . . . . . . . . . . . . 102
5.2.1 Orthogonal Wavelets . . . . . . . . . . . . . . . . . . . 104
5.2.2 Biorthogonal Wavelets . . . . . . . . . . . . . . . . . . 105
5.3 Multilevel Unbalanced Tree Structures . . . . . . . . . . . . 105
5.3.1 Critically Decimated Schemes . . . . . . . . . . . . . . 106
5.3.2 Translation Invariant Schemes . . . . . . . . . . . . . 106
5.4 2-D Multiresolution Analysis . . . . . . . . . . . . . . . . . . 108
5.4.1 2-D Undecimated Separable Analysis . . . . . . . . . . 109
5.4.2 A`-Trous Analysis . . . . . . . . . . . . . . . . . . . . . 111
5.5 Gaussian and Laplacian Pyramids . . . . . . . . . . . . . . . 112
5.5.1 Generalized Laplacian Pyramid . . . . . . . . . . . . . 115
5.6 Nonseparable MRA . . . . . . . . . . . . . . . . . . . . . . . 117
5.6.1 Curvelets . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.6.2 Contourlets . . . . . . . . . . . . . . . . . . . . . . . . 118
5.7 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 121
6 Spectral Transforms for Multiband Image Fusion 123
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.2 RGB-to-IHS Transform and Its Implementations . . . . . . . 124
6.2.1 Linear IHS Cylindric Transforms . . . . . . . . . . . . 125
6.2.2 Nonlinear IHS Transforms . . . . . . . . . . . . . . . . 128
6.2.2.1 Nonlinear IHS triangle transform. . . . . . . 129
6.2.2.2 Nonlinear HSV hexcone transform . . . . . . 130
6.2.2.3 Nonlinear HSL bi-hexcone transform. . . . . 131
6.2.3 Generalization of IHS Transforms for MS Image Fusion 131
6.2.3.1 Linear IHS . . . . . . . . . . . . . . . . . . . 132
viii Contents
6.2.3.2 Nonlinear IHS . . . . . . . . . . . . . . . . . 134
6.3 PCA Transform . . . . . . . . . . . . . . . . . . . . . . . . . 136
6.3.1 Decorrelation Properties of PCA . . . . . . . . . . . . 136
6.3.2 PCA Transform for Image Fusion . . . . . . . . . . . . 137
6.4 Gram-Schmidt Transform . . . . . . . . . . . . . . . . . . . . 139
6.4.1 Gram-Schmidt Orthogonalization Procedure. . . . . . 139
6.4.2 Gram-Schmidt Spectral Sharpening . . . . . . . . . . 140
6.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 142
7 Pansharpening of Multispectral Images 143
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.2 Classification of Pansharpening Methods . . . . . . . . . . . 145
7.3 A Critical Review of Pansharpening Methods . . . . . . . . . 147
7.3.1 Component Substitution . . . . . . . . . . . . . . . . . 148
7.3.1.1 Generalized Intensity-Hue-Saturation . . . . 151
7.3.1.2 Principal Component Analysis . . . . . . . . 152
7.3.1.3 Gram-Schmidt orthogonalization . . . . . . . 152
7.3.2 Optimization of CS Fusion Methods . . . . . . . . . . 153
7.3.3 Multiresolution Analysis . . . . . . . . . . . . . . . . . 155
7.3.4 Optimization of MRA Based on Instrument MTF . . 158
7.3.4.1 ATW . . . . . . . . . . . . . . . . . . . . . . 161
7.3.4.2 GLP. . . . . . . . . . . . . . . . . . . . . . . 161
7.3.4.3 MRA-optimized CS . . . . . . . . . . . . . . 163
7.3.5 Hybrid Methods . . . . . . . . . . . . . . . . . . . . . 164
7.4 Simulation Results and Discussion . . . . . . . . . . . . . . . 165
7.4.1 Fusion of IKONOS Data . . . . . . . . . . . . . . . . . 165
7.4.2 Fusion of QuickBird Data . . . . . . . . . . . . . . . . 172
7.4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 174
7.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 175
8 Pansharpening of Hyperspectral Images 177
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.2 Multispectral to Hyperspectral Pansharpening . . . . . . . . 179
8.3 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . 181
8.3.1 BDSD-MMSE Fusion . . . . . . . . . . . . . . . . . . 181
8.3.2 Fusion with a Constrained Spectral Angle . . . . . . . 183
8.4 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . 184
8.4.1 BDSD-MMSE Algorithm . . . . . . . . . . . . . . . . 186
8.4.2 CSA Algorithm . . . . . . . . . . . . . . . . . . . . . . 186
8.4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 188
8.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 191
Contents ix
9 E(cid:11)ects of Aliasing and Misalignments on Pansharpening 193
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
9.2 Mathematical Formulation . . . . . . . . . . . . . . . . . . . 195
9.2.1 CS-Based Methods . . . . . . . . . . . . . . . . . . . . 195
9.2.2 MRA-Based Methods . . . . . . . . . . . . . . . . . . 195
9.3 Sensitivity to Aliasing . . . . . . . . . . . . . . . . . . . . . . 196
9.3.1 CS-Based Methods . . . . . . . . . . . . . . . . . . . . 196
9.3.2 MRA-Based Methods . . . . . . . . . . . . . . . . . . 197
9.3.2.1 ATW-based fusion . . . . . . . . . . . . . . . 197
9.3.2.2 GLP-based fusion . . . . . . . . . . . . . . . 197
9.3.3 Results and Discussion . . . . . . . . . . . . . . . . . . 198
9.3.3.1 CS vs. MRA-ATW. . . . . . . . . . . . . . . 199
9.3.3.2 MRA-GLP vs. MRA-ATW . . . . . . . . . . 200
9.4 Sensitivity to Spatial Misalignments . . . . . . . . . . . . . . 203
9.4.1 MRA-Based Methods . . . . . . . . . . . . . . . . . . 205
9.4.2 CS-Based Methods . . . . . . . . . . . . . . . . . . . . 205
9.4.3 Results and Discussion . . . . . . . . . . . . . . . . . . 206
9.4.3.1 Misregistration . . . . . . . . . . . . . . . . . 206
9.4.3.2 Interpolation shifts . . . . . . . . . . . . . . 210
9.5 Sensitivity to Temporal Misalignments . . . . . . . . . . . . 212
9.5.1 Results and Discussion . . . . . . . . . . . . . . . . . . 216
9.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 221
10 Fusion of Images from Heterogeneous Sensors 223
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
10.2 Fusion of Thermal and Optical Data . . . . . . . . . . . . . . 224
10.2.1 Background and Literature Review . . . . . . . . . . . 225
10.2.2 Fusion of ASTER Data . . . . . . . . . . . . . . . . . 226
10.2.2.1 ASTER imaging sensor . . . . . . . . . . . . 227
10.2.2.2 A Fusion scheme for TIR ASTER data . . . 227
10.2.2.3 Interchannel correlation . . . . . . . . . . . . 228
10.2.2.4 Results and discussion . . . . . . . . . . . . . 230
10.3 Fusion of Optical and SAR Data . . . . . . . . . . . . . . . . 233
10.3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . 234
10.3.2 Literature Review . . . . . . . . . . . . . . . . . . . . 235
10.3.3 Quality Issues . . . . . . . . . . . . . . . . . . . . . . . 236
10.3.4 Fusion of Landsat 7 ETM+ and ERS SAR Data . . . 237
10.3.4.1 Fusion scheme . . . . . . . . . . . . . . . . . 237
10.3.4.2 Generalized intensity modulation. . . . . . . 239
10.3.4.3 Simulation results . . . . . . . . . . . . . . . 240
10.3.4.4 Summary and discussion . . . . . . . . . . . 244
10.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 245