Table Of ContentRenewable Energy Series 17
Nonlinear Optimization in EN
no
gn
i
nl
Electrical Engineering with i
en
ee
ra
Applications in MATLAB® inr
gO
p
w
t
itim
h
Aiz
a
p
t
pio
lin
Nonlinear Optimization in Electrical Engineering with Applications in MATLAB® Mohamed Bakr is an Associate c
provides an introductory course on nonlinear optimization in electrical engineering, Professor at the Department of a i
with a focus on applications such as the design of electric, microwave, and Electrical and Computer Engineering, tn
photonic circuits, wireless communications, and digital filter design. McMaster University, Canada, io
where his research interests include E
Basic concepts are introduced using a step-by-step approach and illustrated with optimization methods, computer- nle
MATLAB® codes that the reader can use and adapt. Topics covered include: aided design and modelling of s
c
(cid:127) classical optimization methods mapipcrloicwataiovnes c, isrcmuiatsrt, anneaulryasli sn eotfw orks intr
(cid:127) one dimensional optimization microwave circuits and efficient ic
(cid:127) unconstrained and constrained optimization optimization using time domain Ma
(cid:127)(cid:127) gsploabcael mopatpimpiinzagt ioopntimization simulation methods. Al
(cid:127) adjoint variable methods T Nonlinear Optimization in
L
Nonlinear Optimization in Electrical Engineering with Applications in MATLAB® is A
essential reading for advanced students in electrical engineering.
B
®
Electrical Engineering with
Applications in MATLAB®
B
a
k
r
Mohamed Bakr
The Institution of Engineering and Technology
www.theiet.org
ISBN 978-1-84919-543-0
Nonlinear Optimization.indd 1 23/08/2013 09:19:27
Nonlinear Optimization in
Electrical Engineering with
‡
Applications in MATLAB
Nonlinear Optimization in
Electrical Engineering with
‡
Applications in MATLAB
Mohamed Bakr
The Institution of Engineering andTechnology
PublishedbyTheInstitutionofEngineeringandTechnology,London,UnitedKingdom
TheInstitutionofEngineeringandTechnologyisregisteredasaCharityinEngland&
Wales(no.211014)andScotland(no.SC038698).
†TheInstitutionofEngineeringandTechnology2013
Firstpublished2013
ThispublicationiscopyrightundertheBerneConventionandtheUniversalCopyright
Convention.Allrightsreserved.Apartfromanyfairdealingforthepurposesofresearch
orprivatestudy,orcriticismorreview,aspermittedundertheCopyright,Designsand
PatentsAct1988,thispublicationmaybereproduced,storedortransmitted,inany
formorbyanymeans,onlywiththepriorpermissioninwritingofthepublishers,orin
thecaseofreprographicreproductioninaccordancewiththetermsoflicencesissued
bytheCopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethose
termsshouldbesenttothepublisherattheundermentionedaddress:
TheInstitutionofEngineeringandTechnology
MichaelFaradayHouse
SixHillsWay,Stevenage
Herts,SG12AY,UnitedKingdom
www.theiet.org
Whiletheauthorandpublisherbelievethattheinformationandguidancegiveninthis
workarecorrect,allpartiesmustrelyupontheirownskillandjudgementwhenmaking
useofthem.Neithertheauthornorpublisherassumesanyliabilitytoanyoneforany
lossordamagecausedbyanyerrororomissioninthework,whethersuchanerroror
omissionistheresultofnegligenceoranyothercause.Anyandallsuchliabilityis
disclaimed.
Themoralrightsoftheauthortobeidentifiedasauthorofthisworkhavebeen
assertedbyhiminaccordancewiththeCopyright,DesignsandPatentsAct1988.
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisproductisavailablefromtheBritishLibrary
ISBN978-1-84919-543-0(hardback)
ISBN978-1-84919-544-7(PDF)
TypesetinIndiabyMPSLimited
PrintedintheUKbyCPIGroup(UK)Ltd,Croydon
To my wife Mahetab, my children Jannah, Omar, and Youssef, and
to my parents to whom I am indebted for as long as I live
Contents
Preface xi
Acknowledgments xv
1 Mathematical background 1
1.1 Introduction 1
1.2 Vectors 1
1.3 Matrices 3
1.4 The solution of linear systems of equations 6
1.5 Derivatives 11
1.5.1 Derivative approximation 11
1.5.2 The gradient 12
1.5.3 The Jacobian 14
1.5.4 Second-order derivatives 15
1.5.5 Derivatives of vectors and matrices 16
1.6 Subspaces 18
1.7 Convergence rates 20
1.8 Functionsand sets 20
1.9 Solutions of systems of nonlinear equations 22
1.10 Optimization problem definition 25
References 25
Problems 25
2 Anintroduction to linear programming 29
2.1 Introduction 29
2.2 Examples of linear programs 29
2.2.1 Afarming example 29
2.2.2 Aproductionexample 30
2.2.3 Power generation example 31
2.2.4 Wireless communication example 32
2.2.5 Abattery charging example 32
2.3 Standard form of an LP 33
2.4 Optimality conditions 37
2.5 The matrix form 39
2.6 Canonical augmented form 40
2.7 Moving from one basic feasible solution to another 42
2.8 Cost reduction 45
2.9 The classical Simplex method 46
2.10 Starting the Simplex method 49
2.10.1 Endless pivoting 51
2.10.2 The big M approach 51
2.10.3 The two-phase Simplex 52
viii NonlinearoptimizationinelectricalengineeringwithapplicationsinMATLAB(cid:2)
2.11 Advanced topics 55
A2.1 Minimax optimization 55
A2.1.1Minimax problem definition 55
A2.1.2Minimax solution usinglinear programming 57
A2.1.3Amicrowave filter example 59
A2.1.4The design of coupled microcavities optical filter 61
References 65
Problems 65
3 Classical optimization 69
3.1 Introduction 69
3.2 Single-variable Taylor expansion 69
3.3 Multidimensional Taylor expansion 71
3.4 Meaningof the gradient 73
3.5 Optimality conditions 76
3.6 Unconstrained optimization 76
3.7 Optimization with equality constraints 78
3.7.1 Method of direct substitution 79
3.7.2 Method of constrained variation 80
3.8 Lagrange multipliers 84
3.9 Optimization with inequality constraints 86
3.10 Optimization with mixed constraints 92
A3.1 Quadratic programming 92
A3.2 Sequential quadratic programming 95
References 99
Problems 99
4 One-dimensionaloptimization-Line search 101
4.1 Introduction 101
4.2 Bracketing approaches 102
4.2.1 Fixed line search 103
4.2.2 Accelerated line search 104
4.3 Derivative-free line search 105
4.3.1 Dichotomousline search 105
4.3.2 The interval-halving method 106
4.3.3 The Fibonacci search 108
4.3.4 The Golden Sectionmethod 111
4.4 Interpolationapproaches 112
4.4.1 Quadratic models 113
4.4.2 Cubic interpolation 116
4.5 Derivative-based approaches 119
4.5.1 The classical Newton method 119
4.5.2 Aquasi-Newtonmethod 121
4.5.3 The Secant method 122
4.6 Inexact line search 123
A4.1 Tuning of electric circuits 124
Contents ix
A4.1.1Tuning of a current source 125
A4.1.2Coupling of nanowires 127
A4.1.3Matching of microwave antennas 128
References 129
Problems 130
5 Derivative-free unconstrainedtechniques 131
5.1 Why unconstrained optimization? 131
5.2 Classification of unconstrained optimization techniques 131
5.3 The random jump technique 132
5.4 The random walk method 133
5.5 Grid search method 134
5.6 The univariate method 135
5.7 The pattern search method 137
5.8 The Simplex method 140
5.9 Responsesurface approximation 143
A5.1 Electrical application: impedance transformers 146
A5.2 Electrical application: the designof photonic devices 149
References 151
Problems 152
6 First-order unconstrainedoptimization techniques 153
6.1 Introduction 153
6.2 The steepest descent method 153
6.3 The conjugate directions method 156
6.3.1 Definition of conjugacy 157
6.3.2 Powell’smethod of conjugate directions 158
6.4 Conjugate gradient methods 162
A6.1 Solution of large systems of linear equations 164
A6.2 The designof digital FIRfilters 169
References 173
Problems 173
7 Second-order unconstrainedoptimization techniques 175
7.1 Introduction 175
7.2 Newton’s method 175
7.3 The Levenberg–Marquardt method 178
7.4 Quasi-Newton methods 179
7.4.1 Broyden’s rank-1 update 180
7.4.2 The Davidon–Fletcher–Powell (DFP)formula 182
7.4.3 The Broyden–Fletcher–Goldfarb–Shanno method 184
7.4.4 The Gauss–Newton method 185
A7.1 Wireless channel characterization 188
A7.2 The parameter extractionproblem 189
A7.3 Artificial neural networks training 193
References 201
Problems 201
