Symmetries and Groups in Signal Processing: An Introduction

163 Pages·2010·2.148 MB·English

by  Prof. Virendra P. Sinha (auth.)

SIGNALSANDCOMMUNICATIONTECHNOLOGY
Forothertitlespublishedinthisseries,goto
http://www.springer.com/series/4748
Virendra P. Sinha
Symmetries and Groups
in Signal Processing
An Introduction
123
Prof.VirendraP.Sinha
DhirubhaiAmbaniInstitute
ofInformationandComm.Tech.
NearIndrodaCircle
382007Gandhinagar,Gujarat,India
vp [email protected]
ISBN978-90-481-9433-9 e-ISBN978-90-481-9434-6
DOI10.1007/978-90-481-9434-6
SpringerDordrechtHeidelbergLondonNewYork
LibraryofCongressControlNumber:2010931379
(cid:2)c SpringerScience+BusinessMediaB.V.2010
Nopartofthisworkmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformor
byanymeans,electronic,mechanical,photocopying,microfilming,recordingorotherwise,without
writtenpermissionfromthePublisher,withtheexceptionofanymaterialsuppliedspecificallyforthe
purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserof
thework.
Coverdesign: SPiPublisherServices
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
To my grandsons
Rohan, Ishaan, and Shloak,
for a whiff of algebra
Preface
Thefieldofsignalprocessing,asitstandstoday,aboundsinvariedgeneralizations
ofsystemtheoreticconceptsthatcanbesaidtorestonthenotionofsymmetry,and
ongrouptheoreticmethodsofexploitingsymmetries.
Awiderangeofsuchgeneralizationsanddevelopmentsrelycentrallyonatran-
sitionfrom the classical Fourier theory to the modern theory of non-commutative
harmonic analysis, with its roots in the representation theory of groups. In the
frameworkthatemergesthroughthistransition,allthebasicnotions—transforms,
convolutions,spectra,andsoon,carryoverinaformthatallowsawidevarietyof
interpretations,subsumingtheoldonesandadmittingnewones.
Thisbookisanintroductorytreatmentofaselectionoftopicsthattogetherserve
toprovideinmyviewabackgroundforaproperunderstandingofthetheoreticalde-
velopmentswithinthisframework.Addressedprimarilytobeginninggraduatestu-
dentsinelectricalandcommunicationengineering,itismeanttoserveasabridge
between what they know from their undergraduate years, and what lies ahead for
them in their graduate studies, be it in the area of signal processing, or in related
areassuchasimageprocessingandimageunderstanding,codingtheory,faultdiag-
nostics,andthetheoryofalgorithmsandcomputation.
I assume that the reader is familiar with the theory of linear time–invariant
continuous–timeanddiscrete–timesystemsasitis generallytaughtina basicun-
dergraduatecourseonsignalsandsystems.Therearenomathematicalprerequisites
beyondwhattheywouldhavelearntintheirundergraduateyears.Familiaritywith
rudimentsoflinearalgebrawouldbehelpful,buteventhatisnotnecessary;what-
everofitisneededinthebook,theycanpickupontheirownastheygoalong.
Apointaboutpedagogy.Inteachingmathematicalconceptstoengineeringstu-
dents, a plan of action that is commonly followed is to separate what is regarded
asmathematicspersefromitsapplications,andtointroducethetwoseparatelyin
alternation.Thusonefirstintroducesthemtodifferentialequations,linearormod-
ern algebra, or discrete mathematics, on abstract lines as they would appear in a
mathematics text, and then one turns to their applications in solving engineering
problems.
This plan works well, perhaps just about, when the students are fresh to their
engineeringstudies.Butatastagewhentheyhavealreadyhadtheirfirstexposure
to basic engineering principles, it has an inhibiting influence, both on their pace
of learning and on their motivation for it. Faced with a new abstract concept at
vii
viii Preface
thisstage,theyinstinctivelybegintolookforapatterninwhichthenewwillfitin
smoothly,andthroughanalogiesandmetaphors,withwhattheyalreadyintuitively
knowoftheirmainsubjects.Theylookforthesortofexperiencethat,forinstance,
they had at the time they learnt their elements of Euclidean geometry, when they
saw how the theorem on triangle inequality, logically derived from the axioms,
agreedwithwhattheyknewallalongabouttrianglesastheydrewthemonpaper.
It is the same experience whichthey had whilelearning elementsof graph theory
concurrently with network analysis. More generally, they look for a backdrop of
intuitionagainstwhichtheywouldliketheabstractionstobesetandtounfold.
Study of new mathematical structures becomes, as result, an easier and more
pleasant task for such students if the abstractions are presented seamlessly with
theirconcreteengineeringinterpretations.Ihavetriedtokeepthispointinmindin
mypresentationinthisbook.
The contents of the book are organized as follows. Chapter 1 is devoted to an
overviewofbasicsignalprocessingconceptsinanalgebraicsetting.Verybroadly,
it is an invitation to the reader to revisit these concepts in a manner that places
in viewtheir algebraic and structural foundations.The specific questionthat I ex-
amine is the following: How should system theoretic concepts be formulated or
characterized so that they are, in the first instance, independent of details such as
whetherthesignalsofinteresttousarediscrete,discretefinite,one–dimensional,or
multi–dimensional.Implicitinthisquestionisafinerquestionaboutrepresentation
of signals that I discuss first, focussing attention on the distinction between what
signals are physically, and the models by which they are represented. Next I dis-
cuss those aspects of linearity,translation–invariance,causality,convolutions,and
transforms,thataregermanetotheirgeneralizations,inthecontextofdiscretesig-
nals.Chapter2presentsinanutshellthosebasicalgebraicconceptsthatarerelied
uponinagrouptheoreticinterpretationoftheconceptofsymmetry.InChapter3,
thepointsmadeinChapter1aboutthechoiceofmathematicalmodelsistakenup
again.Chapter4isaboutsymmetryanditsalgebraicformalization.Representation
theoryoffinitegroupsisintroducedinChapter5.Chapter6givesafinallookatthe
roleofgrouprepresentationtheoryinsignalprocessing.
Acknowledgements
Thisbookhasgrownoutofnoteswrittenforatransitioncourseofferedtobeginning
graduate students in the Department of Electrical Engineering of IIT Kanpur, and
alsoforasimilarcourseatDA-IICTGandhinagar.Creatingthesecourseshasbeen
a very fruitfullearning experiencefor me,and I have benefitted enormouslyfrom
discussionsandinteractionswiththeparticipatingstudentsatboththeplaces.Ican
notthank themenough for their activeinvolvement.My specialthanks are due to
RatnikGandhiandPratikShah,bothcurrentlyworkingfortheirdoctoraldegreesat
DA-IICT,fortheirregularinteractionsandfeedback.Ratnikhasbeenmyknow-all
Preface ix
manforthesubtletiesoflatex,andhasactedasasoundingboardformeatvarious
stagesofwriting.
Writinghasitslows,whenoneisheldbackbyboutsofperfectionism.Constant
nudging ‘to get on with the job’ is in such times a pragmatic antidote. My deep
appreciation for that to Dr. A.P. Kudchadker, former Director of DA-IICT, and to
Dr.S.C.Sahasrabudhe,thepresentDirector.
Amongstcolleagues,andformerstudents,therearemanywhohavedirectlyand
criticallyinfluencedmythoughtprocessesthathavepromptedthistext.Igratefully
acknowledge receiving constructive inputs from Drs. S.D. Agashe, S. Chatterji,
S.K. Mullick, P. Ramakrishna Rao, K.R. Sarma, M.U. Siddiqi, V.R. Sule, and
K.S.Venkatesh.
FromthetimeIfirstputforthmybookproposaltoSpringerinNovember2008,
ithasbeenapleasureinteractingwithEditorMarkdeJongh.Mythanksgotohim,
andtoMrs.CindyZitter,hisSeniorAssistant,forbenignlyputtingupwithdelays
inmyself-imposeddeadlines,andforallthemeticuloussupport.
Finally,tomywife,Meera,anddaughters,ShubhraandShalini,Iamimmeasur-
ablygratefulforbeingatonewithmeinnegotiatingtherhythmsofacademiclife.
Gandhinagar VirendraP.Sinha
April,2010
Contents
Preface vii
1 SignalsandSignalSpaces:AStructuralViewpoint 1
1.1 WhatIsaSignal? . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 SpacesandStructures . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 SignalSpacesandSystems . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Linearity,Shift–InvarianceandCausality. . . . . . . . . . . . . . . 12
1.4.1 Linearity . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4.2 Shift–Invariance . . . . . . . . . . . . . . . . . . . . . . . 15
1.4.3 Causality . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.4.4 Characterization . . . . . . . . . . . . . . . . . . . . . . . 17
1.5 ConvolutionalAlgebraandtheZ–Transform . . . . . . . . . . . . . 20
1.6 Shifts,TransformsandSpectra . . . . . . . . . . . . . . . . . . . . 24
1.6.1 Shift–InvarianceonFiniteIndexSets . . . . . . . . . . . . 26
1.6.2 TransformsandSpectra . . . . . . . . . . . . . . . . . . . 31
2 AlgebraicPreliminaries 43
2.1 What’sinaDefinition? . . . . . . . . . . . . . . . . . . . . . . . . 43
2.2 SetTheoreticNotation . . . . . . . . . . . . . . . . . . . . . . . . 44
2.3 RelationsandOperations . . . . . . . . . . . . . . . . . . . . . . . 47
2.3.1 EquivalenceRelationsandPartitions . . . . . . . . . . . . . 47
2.3.2 Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.4 Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.4.1 GroupsWithinGroups . . . . . . . . . . . . . . . . . . . . 51
2.4.2 GroupMorphisms . . . . . . . . . . . . . . . . . . . . . . 52
2.4.3 GroupsandGeometry . . . . . . . . . . . . . . . . . . . . 53
2.5 VectorSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.5.1 MatricesofVectorsandLinearTransformations. . . . . . . 54
2.5.2 DirectSumsofSubspaces . . . . . . . . . . . . . . . . . . 57
2.6 Posets,Lattices,andBooleanAlgebras . . . . . . . . . . . . . . . . 61
2.6.1 FromPosetstoLattices . . . . . . . . . . . . . . . . . . . . 61
2.6.2 ComplementedandDistributiveLattices . . . . . . . . . . . 63
2.6.3 LatticeofSubspacesofaVectorSpace . . . . . . . . . . . 64
2.7 ClosingRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
xi
xii Contents
3 Measurement,Modeling,andMetaphors 69
3.1 ArchimedesandtheTortoise . . . . . . . . . . . . . . . . . . . . . 69
3.2 TheRepresentationalApproach . . . . . . . . . . . . . . . . . . . 70
3.2.1 MeasuringLengths . . . . . . . . . . . . . . . . . . . . . . 70
3.2.2 FromMeasurementtoModeling . . . . . . . . . . . . . . . 72
3.2.3 TimeandSpace . . . . . . . . . . . . . . . . . . . . . . . . 74
3.2.4 ModelsinGeneral . . . . . . . . . . . . . . . . . . . . . . 76
3.3 Metaphors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4 Symmetries,AutomorphismsandGroups 81
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.2 SymmetriesandAutomorphisms . . . . . . . . . . . . . . . . . . . 83
4.3 GroupsofAutomorphisms . . . . . . . . . . . . . . . . . . . . . . 90
4.4 SymmetriesofLinearTransformations . . . . . . . . . . . . . . . . 93
4.4.1 SymmetriesandSymmetryOperations . . . . . . . . . . . 93
4.4.2 TranslationOperators. . . . . . . . . . . . . . . . . . . . . 94
4.5 SymmetryBasedDecompositions . . . . . . . . . . . . . . . . . . 96
4.5.1 Block–DiagonalizabilityandInvariantSubspaces . . . . . . 97
4.5.2 TransformationGroupsandTheirInvariantSubspaces . . . 98
4.5.3 TransformationswithSymmetries . . . . . . . . . . . . . . 100
5 RepresentationsofFiniteGroups 105
5.1 TheNotionofRepresentation. . . . . . . . . . . . . . . . . . . . . 105
5.2 MatrixRepresentationsofGroups . . . . . . . . . . . . . . . . . . 105
5.3 AutomorphismsofaVectorSpace . . . . . . . . . . . . . . . . . . 108
5.4 GroupRepresentationsinGL(V) . . . . . . . . . . . . . . . . . . . 108
5.5 ReducibleandIrreducibleRepresentations . . . . . . . . . . . . . . 112
5.6 ReducibilityofRepresentations. . . . . . . . . . . . . . . . . . . . 115
5.7 Schur’sLemmaandtheOrthogonalityTheorem . . . . . . . . . . . 119
5.8 CharactersandTheirProperties. . . . . . . . . . . . . . . . . . . . 122
5.9 ConstructingIrreducibleRepresentations. . . . . . . . . . . . . . . 126
5.10 CompleteReductionofRepresentations . . . . . . . . . . . . . . . 130
5.11 FurtheronReduction . . . . . . . . . . . . . . . . . . . . . . . . . 139
6 SignalProcessingandRepresentationTheory 143
6.1 SignalsasFunctionsonGroups. . . . . . . . . . . . . . . . . . . . 144
6.2 SymmetriesofLinearEquations . . . . . . . . . . . . . . . . . . . 145
6.3 FastDiscreteSignalTransforms . . . . . . . . . . . . . . . . . . . 147
A Parentheses,TheirProperPairing,andAssociativity 151
A.1 ProperPairingofParentheses . . . . . . . . . . . . . . . . . . . . . 151
A.2 ParenthesesandtheAssociativeLaw . . . . . . . . . . . . . . . . . 153
Index 157
Chapter 1
Signals and Signal Spaces:
A Structural Viewpoint
1.1 What Is a Signal?
The notionof a signal, like that of weight or temperature, is a two-sided one. We
commonly think and speak of signals as functions of some sort, with numerical
valuesbothfortheirdomainandfortheirrange.Yet,signalstobeginwithhavetodo
withwhatweperceiveofobjectsandeventsaroundusthroughoursenses.Wecould
thus,tobemoreexplicit,saythatthetermsignalcarrieswithinittwoconnotations,
one empirical and one formal. The former refers to the physical world in which
statements about its objects and events are true or false in the sense that they are
empiricallyobserved to be so. The latter, on the other hand, refers to abstractions
that are members of a formally defined mathematical world in which statements
about a class of its members are true or false in the sense that they do or do not
logicallyfollowfromtheinitialdefinitionsandaxiomsgoverningthatclass.
Tomakethepointclearer,considerthefamiliarnotionofheavinessofanobject
forinstance.Howheavyanobjectis,wedescribebymeansofitsweightgivenasa
number.Thefeelingofheavinessis,however,somethingempirical,andisassessed
qualitatively.Itisamatterofempiricalverificationthatoneveryoccasionthatwe
check,wefindtwoobjectsputtogethertobeheavierthaneitherofthemindividu-
ally,andthereforeweinductivelyinferthistobeauniversalruleaboutallobjects.
Aboutnumbers,ontheotherhand,itisamatterofdeductiveinferencefromthe
axiomsofarithmeticthatthesumofanytwopositivenumbersisgreaterthaneither
of the two. It makes sense to quantify the perception of heaviness by assigning
numbers to objects as their weights because we have at our disposal a practical
procedure,basedonadevicesuchasapanbalance,forassigningnumberstoobjects
inaveryspecialway.Theresultingassignmentissuchthatthereisamatchbetween
what we find to be empirically true about objects regarding their heaviness, and
whatislogicallytrueabouttheirnumericalweightsonaccountoftheirarithmetic
properties.
V.P.Sinha,SymmetriesandGroupsinSignalProcessing,SignalsandCommunicationTechnology, 1
DOI10.1007/978-90-481-9434-6 1,(cid:2)c SpringerScience+BusinessMediaB.V.2010