Mathematical Foundations for Signal Processing, Communications, and Networking

852 Pages·2012·12.123 MB·\852

by  Erchin Serpedin (Editor), Thomas Chen (Editor), Dinesh Rajan (Editor)

K12570_cover_final_rev 12/2/11 11:13 AM Page 1
C M Y CM MY CY CMY K
ELECTRICAL ENGINEERING Serpedin
• Mathematical Foundations
Mathematical Foundations Chen
for •
for
Rajan
SIGNAL PROCESSING, COMMUNICATIONS,
AND NETWORKING S SIGNAL PROCESSING,
I
G
N
Mathematical Foundations for Signal Processing, Communications, and Networking COMMUNICATIONS,
A
describes mathematical concepts and results important in the design, analysis, and
L
optimization of signal processing algorithms, modern communication systems, and

networks. Helping readers master key techniques and comprehend the current PM
AND NE TWORKING
research literature, the book offers a comprehensive overview of methods and Ra
applications from linear algebra, numerical analysis, statistics, probability, stochastic Ot
A h
processes, and optimization.
NCe
Em
D
From basic transforms to Monte Carlo simulation to linear programming, the text S
a
covers a broad range of mathematical techniques essential to understanding the NSt
i
concepts and results in signal processing, telecommunications, and networking. Ic
EN
a
Along with discussing mathematical theory, each self-contained chapter presents
TGl
examples that illustrate the use of various mathematical concepts to solve different
W F
,
applications. Each chapter also includes a set of homework exercises and pointers o
to further readings for additional topics and applications. OCu
ROn
This text helps readers understand fundamental and advanced results as well as KMd
a
recent research trends in the interrelated fields of signal processing, IMt
N i
telecommunications, and networking. It provides all the necessary mathematical o
GU
background to prepare students for more advanced courses and train specialists n
working in these areas. Ns

I f
C o
r
A
T
I
O
N
S
,
Edited by
K12570
Erchin Serpedin • Thomas Chen • Dinesh Rajan
6000 Broken Sound Parkway, NW
Suite 300, Boca Raton, FL 33487
711 Third Avenue
an informa business New York, NY 10017
www.crcpress.com 2 Park Square, Milton Park www.crcpress.com
Abingdon, Oxon OX14 4RN, UK
Composite
(cid:105) (cid:105)
“main˙def” — 2011/11/21 — 10:05 —
(cid:105) (cid:105)
Mathematical Foundations
for
SIGNAL PROCESSING,
COMMUNICATIONS,
AND NETWORKING
(cid:105) (cid:105)
(cid:105) (cid:105)
TThhiiss ppaaggee iinntteennttiioonnaallllyy lleefftt bbllaannkk
(cid:105) (cid:105)
“main˙def” — 2011/11/21 — 10:05 —
(cid:105) (cid:105)
Mathematical Foundations
for
SIGNAL PROCESSING,
COMMUNICATIONS,
AND NETWORKING
Edited by
Erchin Serpedin • Thomas Chen • Dinesh Rajan
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Group, an informa business
(cid:105) (cid:105)
(cid:105) (cid:105)
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2011 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20111212
International Standard Book Number-13: 978-1-4398-5514-0 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
(cid:105) (cid:105)
“main˙def” — 2011/11/21 — 10:05 —
(cid:105) (cid:105)
For Zara, Nisa, Aisha and Nesrin (ES)
For Robin and Kayla (TC)
In memory of Dilip Veeraraghavan (DR)
“Mathematics is the queen of the sciences.”
Carl Friedrich Gauss
(cid:105) (cid:105)
(cid:105) (cid:105)
TThhiiss ppaaggee iinntteennttiioonnaallllyy lleefftt bbllaannkk
(cid:105) (cid:105)
“main˙def” — 2011/11/21 — 10:05 —
(cid:105) (cid:105)
Contents
List of Figures xxi
List of Tables xxvii
Preface xxix
Editors xxxi
List of Contributors xxxiii
List of Acronyms xxxv
Notations and Symbols xxxix
1 Introduction 1
2 Signal Processing Transforms 5
Serhan Yarkan and Khalid A. Qaraqe
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Basic Transformations . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Fourier Series and Transform. . . . . . . . . . . . . . . . . . . . . 6
2.3.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.2 Fourier Series. . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3.3 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4.1 Impulse-Train Sampling . . . . . . . . . . . . . . . . . . . 12
2.4.2 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5 Cosine and Sine Transforms . . . . . . . . . . . . . . . . . . . . . 14
2.5.1 Cosine Transform . . . . . . . . . . . . . . . . . . . . . . 14
2.5.2 Sine Transform . . . . . . . . . . . . . . . . . . . . . . . . 16
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“main˙def” — 2011/11/21 — 10:05 —
(cid:105) (cid:105)
viii CONTENTS
2.6 Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6.1 Properties of Laplace Transform . . . . . . . . . . . . . . 20
2.7 Hartley Transform. . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.7.1 Properties of Hartley Transform . . . . . . . . . . . . . . 22
2.8 Hilbert Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.8.1 Properties of Hilbert Transform . . . . . . . . . . . . . . 24
2.9 Discrete-Time Fourier Transform . . . . . . . . . . . . . . . . . . 25
2.10 The Z-Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.10.1 Properties of Z-Transform. . . . . . . . . . . . . . . . . . 29
2.11 Conclusion and Further Reading . . . . . . . . . . . . . . . . . . . 30
2.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3 Linear Algebra 35
Fatemeh Hamidi Sepehr and Erchin Serpedin
3.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1.1 Subspaces and Direct Sums . . . . . . . . . . . . . . . . . 36
3.1.2 Spanning and Linear Independency . . . . . . . . . . . . 37
3.1.3 Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.1.4 Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1.5 Ordered Basis . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1.6 Norms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1.7 Inner-Products . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1.8 Induced Norms . . . . . . . . . . . . . . . . . . . . . . . . 40
3.1.9 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . 40
3.1.10 Gram-Schmidt Orthogonalization . . . . . . . . . . . . . 41
3.2 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Range and Nullspace of a Linear Transformation . . . . . 42
3.2.2 Composition and Invertibility. . . . . . . . . . . . . . . . 42
3.2.3 Matrix Representation of Linear Transformations . . . . 42
3.2.4 Projection Operators . . . . . . . . . . . . . . . . . . . . 43
3.2.5 Linear Functionals and Dual Spaces . . . . . . . . . . . . 44
3.2.6 Adjoint of a Linear Transformation . . . . . . . . . . . . 45
3.2.7 Four Fundamental Subspaces . . . . . . . . . . . . . . . . 47
3.3 Operator Norms and Matrix Norms . . . . . . . . . . . . . . . . . 47
3.4 Systems of Linear Equations . . . . . . . . . . . . . . . . . . . . . 49
3.5 Determinant, Adjoint, and Inverse of a Matrix . . . . . . . . . . . 50
3.6 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.7 Unitary and Orthogonal Operators and Matrices. . . . . . . . . . 51
3.8 LU Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.9 LDL and Cholesky Decomposition . . . . . . . . . . . . . . . . . . 53
3.10 QR Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 55
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“main˙def” — 2011/11/21 — 10:05 —
(cid:105) (cid:105)
CONTENTS ix
3.11 Householder and Givens Transformations . . . . . . . . . . . . . . 55
3.11.1 Orthogonal Reduction . . . . . . . . . . . . . . . . . . . . 58
3.12 Best Approximations and Orthogonal Projections . . . . . . . . . 59
3.13 Least Squares Approximations . . . . . . . . . . . . . . . . . . . . 59
3.14 Angles Between Subspaces . . . . . . . . . . . . . . . . . . . . . . 61
3.14.1 Principal Angles Between Subspaces . . . . . . . . . . . . 61
3.15 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . 62
3.15.1 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . 62
3.16 Schur Factorization and Spectral Theorem . . . . . . . . . . . . . 63
3.17 Singular Value Decomposition (SVD) . . . . . . . . . . . . . . . . 64
3.18 Rayleigh Quotient . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.19 Application of SVD and Rayleigh Quotient: Principal Component
Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.20 Special Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.20.1 Block Matrices . . . . . . . . . . . . . . . . . . . . . . . . 68
3.20.2 Circulant Matrices . . . . . . . . . . . . . . . . . . . . . . 69
3.20.3 Toeplitz Matrices . . . . . . . . . . . . . . . . . . . . . . 70
3.20.4 Hankel Matrices . . . . . . . . . . . . . . . . . . . . . . . 71
3.20.5 Vandermonde Matrices . . . . . . . . . . . . . . . . . . . 71
3.20.6 Normal Matrices . . . . . . . . . . . . . . . . . . . . . . . 72
3.20.7 Stochastic Matrices . . . . . . . . . . . . . . . . . . . . . 72
3.20.8 Positive and Negative Definite Matrices . . . . . . . . . . 72
3.20.9 Matrix Condition Number . . . . . . . . . . . . . . . . . 73
3.20.10 Sherman-Morrison-Woodbury Identity . . . . . . . . . . . 74
3.20.11 Schur Complement. . . . . . . . . . . . . . . . . . . . . . 75
3.20.12 Generalized Inverses . . . . . . . . . . . . . . . . . . . . . 75
3.21 Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.21.1 Kronecker Product. . . . . . . . . . . . . . . . . . . . . . 78
3.21.2 Hadamard Product . . . . . . . . . . . . . . . . . . . . . 78
3.21.3 Dot Product . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.21.4 Direct Sum . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.21.5 Differentiation of Matrix and Vector Functions . . . . . . 80
3.22 References and Further Studies . . . . . . . . . . . . . . . . . . . 82
3.23 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4 Elements of Galois Fields 93
Tolga Duman
4.1 Groups, Rings, and Fields . . . . . . . . . . . . . . . . . . . . . . 94
4.1.1 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.1.2 Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.1.3 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
(cid:105) (cid:105)
(cid:105) (cid:105)