Digital signal and image processing using MATLAB®. Volume 3, Advances and applications : the Stochastic case

336 Pages·2015·18.02 MB·English

Contents
Cover
Title
Copyright
Foreword
Notations and Abbreviations
1: Mathematical Concepts
1.1 Basic concepts on probability
1.2 Conditional expectation
1.3 Projection theorem
1.4 Gaussianity
1.5 Random variable transformation
1.6 Fundamental statistical theorems
1.7 Other important probability distributions
2: Statistical Inferences
2.1 Statistical model
2.2 Hypothesis tests
2.3 Statistical estimation
3: Monte-Carlo Simulation
3.1 Fundamental theorems
3.2 Stating the problem
3.3 Generating random variables
3.4 Variance reduction
4: Second Order Stationary Process
4.1 Statistics for empirical correlation
4.2 Linear prediction of WSS processes
4.3 Non-parametric spectral estimation of WSS processes
5: Inferences on HMM
5.1 Hidden Markov Models (HMM)
5.2 Inferences on HMM
5.3 Gaussian linear case: the Kalman filter
5.4 Discrete finite Markov case
6: Selected Topics
6.1 High resolution methods
6.2 Digital Communications
6.3 Linear equalization and the Viterbi algorithm
6.4 Compression
7: Hints and Solutions
H1 Mathematical concepts
H2 Statistical inferences
H3 Monte-Carlo simulation
H4 Second order stationary process
H5 Inferences on HMM
H6 Selected Topics
8: Appendices
A1 Miscellaneous functions
A2 Statistical functions
Bibliography
Index
End User License Agreement
List of Tables
2: Statistical Inferences
Table 2.1. – Growth in mm under the action of fertilizers 1 and 2
Table 2.2. – H: height in cm, W: weight in kg.
Table 2.3. – Observed pairs (x ,y )
n n
Table 2.4. —Cavity figures given as thousands. The fluoride level is given in ppm
(parts per million).
Table 2.5. – Temperature in degrees Fahrenheit and state of the joint: 1 signifies the
existence of a fault and 0 the absence of faults
Table 2.6. – Survival period Y in number of months and state c of the patient. A value
of c = 1 indicates that the patient is still living, hence their survival period is greater
than value Y. A value of c = 0 indicates that the patient is deceased
3: Monte-Carlo Simulation
Table 3.1. – The initial probabilities P{X = i} and the transition probabilities P
0
{X =j |X =i} of a three-state homogeneous Markov chain. We verify that, for all i,
n+1 n
4: Second Order Stationary Process
Table 4.1. – Correlation vs. partial correlation
List of Illustrations
1: Mathematical Concepts
Figure 1.1 – Orthogonality principle: the point X which is the closest to X in is
0
such that X−X is orthogonal to
0
Figure 1.2 – The conditional expectation is the orthogonal projection of X
onto the set of measurable functions of Y. The expectation is the orthogonal
projection of X onto the set of constant functions. Clearly,
2: Statistical Inferences
Figure 2.1 – The two curves represent the respective probability densities of the test
function Φ(X) under hypothesis H (m = 0) and under hypothesis H (m = 2). For a
0 0 1 1
threshold value η, the light gray area represents the significance level or probability
of false alarm, and the dark gray area represents the power or probability of
detection.
Figure 2.2 – ROC curve. The statistical model is , where m {0,
1}. The hypothesis H = {0} is tested by the likelihood ratio (2.9). The higher the
0
value of n, the closer the curve will come to the ideal point, with coordinates (0, 1).
The significance level α is interpreted as the probability of a false alarm. The power
β is interpreted as the probability of detection.
Figure 2.3 – Diagram showing the calculation of the GLRT
Figure 2.4 – Least squares method for a non-linear model given by equation (2.34).
The points represent observed value pairs (Table 2.3)
Figure 2.5 – Percentage of dental cavities observed as a function of fluoride levels in
drinking water in ppm (parts per million).
Figure 2.6 – Multimodal distribution
Figure 2.7 – Step equal to 1/N for the five values of the series, ranked in increasing
order
Figure 2.8 – Cross-validation: a block is selected as the test base, for example block
no. 4 in this illustration, and the remaining blocks are used as a learning base,
before switching over.
3: Monte-Carlo Simulation
Figure 3.1 – Direct calculation (circles) and calculation using a Monte-Carlo
method (crosses), showing that the Monte-Carlo method presents faster convergence
in terms of sample size. The error ε represented in the lower diagram is random in
MC
nature. In the case of direct calculation, the error is monotone and decreasing
Figure 3.2 – Typical form of a cumulative function. We choose a value in a uniform
manner between 0 and 1, then deduce the realization t = F(−1)(u)
Figure 3.3 – The greater the value of M, the more samples need to be drawn before
accepting a value. The value chosen for M will therefore be the smallest possible
value such that, for any x, Mq (x) ≥ p (x). The curve noted MU (x) in the figure
q
corresponds to a draw of the r.v. U from the interval 0 to 1. For this draw, the only
accepted values of Y are those for which p (x) ≥ MU (x)
q
4: Second Order Stationary Process
Figure 4.1 – Forward/backward predictions: is the linear space spanned by X
n−1,N n
, …, X . The dotted arrows represent the forward prediction error and the
−1 n−N
backward prediction error . The two errors are orthogonal to and to
.
Figure 4.2 – Modulus of the lattice analysis filter
Figure 4.3 – The lattice analysis filter
Figure 4.4 – The lattice synthesis filter
Figure 4.5 – Application of a smoothing window to the spectrum
Figure 4.6 – Spectral estimation. Full line: spectrum estimated using a smoothed
periodogram. Dotted line: theoretical spectrum
Figure 4.7 – Integrated MSE (expression (4.63)) as a function of the size M of the
smoothing window. The signal is a second order auto-regressive process. The
number of samples is N = 4096. The points represent the values of the MSE over 20
draws. The full line shows the mean for these 20 draws
5: Inferences on HMM
Figure 5.1 – The Directed Acyclic Graph associated with an HMM takes the form of
a tree
Figure 5.2 – Lattice associated with S = 3 states for a sequence of length N = 6
6: Selected Topics
Figure 6.1 – Calculation of the coefficients of based on P = GGH
Figure 6.2 − DTFT of the signal x (n) = cos(2πf n) + 0.2cos(2πf n) + b (n) with f =
1 2 1
0.2, f = 0.21, N = 60. The signal-to-noise ratio is equal to 20 dB
2
Figure 6.3 – Performances of the MUSIC algorithm. Signal x (n) = cos(2pf n) + 0.02
1
cos(2pf n) + b (n) with f = 0.2, f = 0.21, N = 60. Square root of the square deviation
2 1 2
of the estimation, for each of the two frequencies, evaluated for a length 300
simulation, for several values of the signal-to-noise ratio in dB
Figure 6.4 – Linear antenna with three sensors and P = 1 distant source assumed to
be far away. The dashed lines represent the equiphase lines
Figure 6.5 – Angular references
Figure 6.6 – Baseband modulation and demodulation
Figure 6.7 – Carrier frequency modulation and demodulation
Figure 6.8 – 8-PSK state constellation
Figure 6.9 – 8-PSK modulation signal
Figure 6.10 – An example of 8-PAM modulation
Figure 6.11 – HDB3 coding: processing the four consecutive zeros and updating the
bipolar violation bits
Figure 6.12 – Diagram of the receiver
Figure 6.13 – A raised cosine function satisfying the Nyquist criterion
Figure 6.14 – PAM-2: eye pattern without noise
Figure 6.15 – Constructing the signal transmitted based on the symbols and the
polyphase components of the emission filter in the case where N = 4
T
Figure 6.16 – The eye pattern for M = 4. The noise is null Spectrum support B = 300
Hz, rate 1,000 bps
Figure 6.17 – Eye pattern for M = 4. The noise is null. Spectrum support B = 500 Hz,
rate 1,000 bps
Figure 6.18 – Eye pattern for M = 4. The signal-to-noise ratio is equal to 15 dB, the
spectrum support is B = 300 Hz and the rate 1,000 bps
Figure 6.19 – Eye pattern for M = 4. The signal-to-noise ratio is equal to 15 dB, the
spectrum support is B = 500 Hz and the rate 1,000 bps
Figure 6.20 – Transition probability graphs for the four states depending on the
input symbol
Figure 6.21 – Comparing the error probabilities as functions of the signal-to-noise
ratio in dB. Plot (‘×’): zero forcing linear equalization. Plot (‘o’): Viterbi algorithm.
Results obtained through a simulation with a length of 5,000. The channel filter has
the finite impulse response (1 − 1.4 0.8)
Figure 6.22 – Two partitions of the plane in 16 regions (four bits are used to code
each representative element of the corresponding region)
Figure 6.23 – Partition R of the plane
Figure 6.24 – partition by the perpendicular bisectors
Figure 6.25 – Original image on the left and the result of the LBG compression with
m = 16 and b = 3
Figure 6.26 – LBG compression: (m = 4, b = 2) on the left (m = 4, b = 4) on the right
7: Hints and Solutions
Figure H2.1 – Top left: the N points (x , y ) and in the space Ox y. The points
n,1 n 1
(’.’) are observed values and (’x’) the predicted values. Bottom left: the N points
(x , y ) and (x , ) in the space Ox y. Right: the N points (x , x , y ) and (x ,
n,2 n n,2 2 n,1 n,2 n n,1
x , ) in the space Ox x y
n,2 1 2
Figure H2.2 – Prediction errors as a function of the supposed order of the model: ‘x-’
for the learning base, ‘o-’ for the test base. The observation model is of the form
, where Z includes p = 10 column vectors. As the number of regressors
increases above and beyond the true value, we “learn” noise; this is known as
overtraining
Figure H4.1 – Variations of the mean square integrated bias, the mean integrated
variance and the mean integrated MSE for 20 trajectories as a function of M. The
signal is made up of 4096 samples, and smoothing is carried out using the Hamming
window
Figure H5.1 – Results for the study of the filtering
Figure H6.1 – Gray code for an 8 state phase modulation
Figure H6.2 – The real and imaginary parts of the complex envelope and the signal
Figure H6.3 – Constellation and Gray code
Figure H6.4 – Noised signal, the signal-to-noise ratio is equal to 20 dB; signal
containing the ISI caused by the FIR filter, h = 1 and h = −1.6; signal after
0 1
equalization
Figure H6.5 – Comparison for a minimum phase channel
Figure H6.6 – Comparison for a non-minimum phase channel
Figure H6.7 – Signals at different points of the communication line
Figure H6.8 – Symbol-by-symbol detection of a Zero Forcing equalizer’s output for a
binary transmission. The equivalent channel has the coefficients g = 1, g = −1.4, g
0 1 2
= 0.8. The x’s indicate the probabilities obtained through 5,000 simulations for
different values of the signal-to-noise ratio in dB
Figure H6.9 – Error probability plotted against the signal-to-noise ratio in dB after
equalizing with the Wiener filter (‘o’) and with the Zero Forcing filter (‘ x’). The
results are obtained through simulation using 5,000 symbols. The channel filter has
the finite impulse response (1 – 1.4 0.8)
8: Appendices
Figure A2.1 – Distribution (left) and cumulative distribution (right) for values of k 1,
3, 5 and 10. The limiting Gaussian is shown alongside with the distributions
Figure A2.2 – Application of Newton’s method to find t from T = 0.8 and k = 3
Figure A2.3 – A few probability densities and cumulative distributions of the χ2
distribution
Figure A2.4 – Use of the mychi2inv function to generate a random sequence,
histogram and theoretical pdf
Figure A2.5 – pdf and cdf of the Fisher distribution for k = 5 and k = 7
1 2
Digital Signal and Image Processing
using MATLAB®
Advances and Applications: The Stochastic Case
Revised and Updated 2nd Edition
Volume 3
Gérard Blanchet
Maurice Charbit
First published 2015 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the
Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any
means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the
terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at
the undermentioned address:
ISTE Ltd
27-37 St George’s Road
London SW19 4EU
UK
www.iste.co.uk
John Wiley & Sons, Inc.
111 River Street
Hoboken, NJ 07030
USA
www.wiley.com
© ISTE Ltd 2015
The rights of Gérard Blanchet and Maurice Charbit to be identified as the authors of this work have been asserted by them in
accordance with the Copyright, Designs and Patents Act 1988.
Library of Congress Control Number: 2015948073
British Library Cataloguing-in-Publication Data
A CIP record for this book is available from the British Library
ISBN 978-1-84821-795-9
MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the
accuracy of the text or exercises in this book. The book’s use or discussion of MATLAB® software does not constitute
endorsement or sponsorship by The MathWorks of a particular pedagogical approach or use of the MATLAB® software.