Theory and Computation of Tensors: Multi-Dimensional Arrays

138 Pages·2016·2.15 MB·English

by  Yimin Wei

Theory and
Computation
of Tensors
Theory and
Computation
of Tensors
Multi-Dimensional Arrays
WEIYANG DING
YIMIN WEI
AMSTERDAM (cid:129) BOSTON (cid:129) HEIDELBERG (cid:129) LONDON
NEW YORK (cid:129) OXFORD (cid:129) PARIS (cid:129) SAN DIEGO
SAN FRANCISCO (cid:129) SINGAPORE (cid:129) SYDNEY (cid:129) TOKYO
Academic Press is an imprint of Elsevier
AcademicPressisanimprintofElsevier
125LondonWall,LondonEC2Y5AS,UK
525BStreet,Suite1800,SanDiego,CA92101-4495,USA
50HampshireStreet,5thFloor,Cambridge,MA02139,USA
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UK
Copyright©2016ElsevierLtd.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronic
ormechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,
withoutpermissioninwritingfromthepublisher.Detailsonhowtoseekpermission,further
informationaboutthePublisher’spermissionspoliciesandourarrangementswithorganizationssuchas
theCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:
www.elsevier.com/permissions.
Thisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythe
Publisher(otherthanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperience
broadenourunderstanding,changesinresearchmethods,professionalpractices,ormedicaltreatment
maybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluating
andusinganyinformation,methods,compounds,orexperimentsdescribedherein.Inusingsuch
informationormethodstheyshouldbemindfuloftheirownsafetyandthesafetyofothers,including
partiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assume
anyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability,
negligenceorotherwise,orfromanyuseoroperationofanymethods,products,instructions,orideas
containedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN978-0-12-803953-3(print)
ISBN978-0-12-803980-9(online)
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.elsevier.com/
Publisher:GlynJones
AcquisitionEditor:GlynJones
EditorialProjectManager:Anna Valutkevich
ProductionProjectManager:DebasishGhosh
Designer:Nikki Levy
TypesetbySPiGlobal,India
Contents
Preface v
I GenerTahle ory 1
1 Introductiaonnd Preliminaries 3
1.1 WhatA rcT ensor.s ? 3
1.2 BasiOcp erations 6
1.3 TensoDre compositions. 8
1.4 TensoEri genvalPureo blems !O
2 GeneralizTeedn sorE igenvaluPer oblems 11
2.1 A UnifiFerda mework. 11
2.2 BasiDce finitions 13
2.3 SevenBtlr lsPirco preties 14
2.3.1N umbero fE igel1va.l ues 14
2.3.2S pectrRa.la di.u s 15
2.3.3D iagonalizTaebnlseo Pra irs 15
2.3.4G crshgorni Circle Theorem 16
2.3.5B ackwarEdr roArn alysis 19
2.4 ReaTle nsoPra i.r s 20
2.4.1T heC rawforNdu mber 21
2.4.2S ymmetric-DefTiennistoeP rai sr 22
2.5 Sign-ComplSepxe ctrRaald ius 26
2.5.1D efinitions 26
2.5.2C ollaWtize-lallFdotr mula 27
2.5.3P ropertfioersS inglTee nsors 29
2.5.4T heC omponentwiDsies tantcoeS ingula.r ity 31
2.5.5B auer-FiTkhee Ol'eul 33
2.6 An IllustrEaxtaimvpel e 34
II HankelT ensors 37
3 FastT ensor-VectPorro ducts 39
3.1 HankeTle nsors 39
3.2 Exponential Data Fit,ting 40
3.2.1T heO ne-Dimensional Case 40
3.2.2T hel\ lultidimensCiaoslel al 42
3.3 Anti-CircTuelnasnotr s 45
3.3.1D iagonalization 46
3.3.2S ingulVaari lles 47
viii CONTENTS
3.3.3B locTke llso.r s 49
3.4 FastH ankeTle nsor-VecPtroord uct 50
3.5 NumericEaxla mples 53
4 InheritanPcreo perties 59
4.1 InheritaPnrcoep erties 59
4.2 TheF irsltu hcl'itParnocpee rotfyH aukeTle nso.r s 61
4.2.1A COllvolutFioornm ula . 61
4.2.2L ower-OrdIemrp liHeisg her-Order 63
4.2.3S OS Dccomposit.oifoS ltlr onHga nkeTle nso.r s 65
11.3 The SecoInndh eritaPnrcoep erotfyH ankeTle nsors 66
4.3.1S tronHga nkeTle nsor.s . 66
4.3.2A GeneraVla ndermollDdecc ompositoifoH na nkei\ll atric6e8 s
4.3.3A n AugmcnteVda ndcnnonde DccompoosfiH tainokne l
Tensors 71
4.3.4T he SeconIdn herit.aPnrcoep erot.fyH ankeTle nsors 75
4.4 TheT hirdI nheritanPcreo pertoyfH ankeTle nsors 77
III M-Tensors 79
5 Definitionasn d BasicP roperties 81
5.1P reliminaries 81
5.l.!N onnegatiTveen sor 81
5.1.2F romA tf-l\ftaotM l-'Tiexn sol' 82
5.2 SpeclrPRr]o perto.fiM e-sT enso.r s 83
5.3 Semi-Positivity 84
5.3.1D efinitions 84
5.3.2S emi-Positive Z-Tensol's 85
5.3.3P roooff T heorem5. 7 87
5.3.4G eneraMl- Tensors 89
5.4 l\lonotonicity 90
5.4.1D efiuitions 90
5.4.2P roperties 90
5.4.3A Count.eErx ample 93
5.4.4A Nontrivl\ilaoll lotZo-nTee nsor 93
5.5 An ExtensioofnM -Tensors 93
5.6 SummRtion 95
6 MultilineSayrs temsw ithM -Tensors 97
6.1 i\lotivat.ions 97
6.2 TrianguElqaura tio.n s 99
6.3 M-EquatioalnldsB eyond 102
6.3.1M -Equations 102
6.3.2N onpositive R.igShitd-eH and 104
6.3.3N onhomoegneouLse ft-HaSniedle 105
6.3.4A bsoluMt-eE quations 106
6.3.5B andedM -Equatio.n 107
6.4 Iterati\ilveet hofdosrM -Equatio.n s 108
6.4.1T he ClassiIctaelr ations 109
CONTENTS ix
6.4.T2h eN ewtoMne thofdo Sry mmetrMi-cE quiaot.lls 111
6.'1.N3u mericTaels ts 112
6.5 PerturbatAinoanl ysoifsM -Eql.lations ]]4
6.5,B]a ckwaErrrrlo.or fsT rianguMl-aErq uations 115
6.5.C2o nditiNounm bers 116
6.6 InverIstee ration 118
Bibliography 125
SubjecItn dex 135
Preface
Thisbookisdevotedtothetheoryandcomputationoftensors,alsocalledhyper-
matrices. Our investigation includes theories on generalized tensor eigenvalue
problems and two kinds of structured tensors, Hankel tensors and M-tensors.
Both theoretical analyses and computational aspects are discussed.
We begin with the generalized tensor eigenvalue problems, which are re-
garded as a unified framework of different kinds of tensor eigenvalue problems
arising from applications. We focus on the perturbation theory and the error
analysis of regular tensor pairs. Employing various techniques, we extend sev-
eral classical results from matrices or matrix pairs to tensor pairs, such as the
Gershgorin circle theorem, the Collatz-Wielandt formula, the Bauer-Fike the-
orem, the Rayleigh-Ritz theorem, backward error analysis, the componentwise
distance of a nonsingular tensor to singularity, etc.
Inthesecondpart,wefocusonHankeltensors. Wefirstproposeafastalgo-
rithmforHankeltensor-vectorproductsbyintroducingaspecialclassofHankel
tensors that can be diagonalized by Fourier matrices, called anti-circulant ten-
sors. ThenweobtainafastalgorithmforHankeltensor-vectorproductsbyem-
beddingaHankeltensorintoalargeranti-circulanttensor. Thecomputational
complexityisreducedfromO(nm)toO(m2nlogmn). Next,weinvestigatethe
spectral inheritance properties of Hankel tensors by applying the convolution
formulaofthefastalgorithmandanaugmentedVandermondedecompositionof
strong Hankel tensors. We prove that if a lower-order Hankel tensor is positive
semidefinite,thenahigher-orderHankeltensorwiththesamegeneratingvector
has no negative H-eigenvalues, when (i) the lower order is 2, or (ii) the lower
order is even and the higher order is its multiple.
The third part is contributed to M-tensors. We attempt to extend the
equivalentdefinitionsofnonsingularM-matrices,suchassemi-positivity,mono-
tonicity, nonnegative inverse, etc., to the tensor case. Our results show that
the semi-positivity is still an equivalent definition of nonsingular M-tensors,
whilethemonotonicityisnot. Furthermore, thegeneralizationofthe“nonneg-
ative inverse” property inspires the study of multilinear system of equations.
We prove the existence and uniqueness of the positive solutions of nonsingular
M-equations with positive right-hand sides, and also propose several iterative
methods for computing the positive solutions.
We would like to thank our collaborator Prof. Liqun Qi of the Hong Kong
Polytechnic University, who leaded us to the research of tensor spectral theory
and always encourages us to explore the topic. We would also like to thank
Prof. Eric King-wah Chu of Monash University and Prof. Sanzheng Qiao of
McMasterUniversity,whoreadthisbookcarefullyandprovidedfeedbackduring
the writing process.
This work was supported by the National Natural Science Foundation of
China under Grant 11271084, School of Mathematical Sciences and Key Labo-
ratory of Mathematics for Nonlinear Sciences, Fudan University.
Chapter 1
Introduction and
Preliminaries
We first introduce the concepts and sources of tensors in this chapter. Several
essentialandfrequentlyusedoperationsinvolvingtensorsarealsoincluded. Fur-
thermore, two basic topics, tensor decompositions and tensor eigenvalue prob-
lems, are briefly discussed at the end of this chapter.
1.1 What Are Tensors?
The term tensor or hypermatrix in this book refers to a multiway array. The
numberofthedimensionsofatensoriscalleditsorder,thatis,A=(a )
i1i2...im
is an mth-order tensor. Particularly, a scalar is a 0th-order tensor, a vector is
a 1st-order tensor, and a matrix is a 2nd-order tensor. As other mathematical
concepts, tensor or hypermatrix is abstracted from real-world phenomena and
other scientific theories. Where do the tensors arise? What kinds of properties
do we care most? How many different types of tensors do we have? We will
briefly answer these questions employing several illustrative examples in this
section.
Example 1.1. As we know, a table is one of the most common realizations of
a matrix. We can also understand tensors or hypermatrices as complex tables
with multivariables. For instance, if we record the scores of 4 students on 3
subjects for both the midterm and final exams, then we can design a 3rd-order
tensor S of size 4×3×2 whose (i,j,k) entry s denotes the score of the i-th
ijk
student on the j-th subjects in the k-th exam. This representation is natural
and easily understood, thus it is a convenient data structure for construction
andquery. However,whenweneedtoprinttheinformationonapieceofpaper,
the 3D structure is apparently not suitable for 2D visualization. Thus we need
tounfoldthecubictensorintoamatrix. Thefollowingtwodifferentunfoldings
of the same tensor both include all the information in the original complex
table. We can see from the two tables that their entries are the same up to a
permutation. Actually, there are many different ways to unfold a higher-order
tensorintoamatrix,andthelinkagesbetweenthemarepermutationsofindices.
(cid:84)(cid:104)(cid:101)(cid:111)(cid:114)(cid:121)(cid:32)(cid:97)(cid:110)(cid:100)(cid:32)(cid:67)(cid:111)(cid:109)(cid:112)(cid:117)(cid:116)(cid:97)(cid:116)(cid:105)(cid:111)(cid:110)(cid:32)(cid:111)(cid:102)(cid:32)(cid:84)(cid:101)(cid:110)(cid:115)(cid:111)(cid:114)(cid:115)(cid:46) 3
(cid:104)(cid:116)(cid:116)(cid:112)(cid:58)(cid:47)(cid:47)(cid:100)(cid:120)(cid:46)(cid:100)(cid:111)(cid:105)(cid:46)(cid:111)(cid:114)(cid:103)(cid:47)(cid:49)(cid:48)(cid:46)(cid:49)(cid:48)(cid:49)(cid:54)(cid:47)(cid:66)(cid:57)(cid:55)(cid:56)(cid:45)(cid:48)(cid:45)(cid:49)(cid:50)(cid:45)(cid:56)(cid:48)(cid:51)(cid:57)(cid:53)(cid:51)(cid:45)(cid:51)(cid:46)(cid:53)(cid:48)(cid:48)(cid:48)(cid:49)(cid:45)(cid:48)
Copyright © (cid:50)(cid:48)(cid:49)(cid:54)(cid:32)(cid:69)(cid:108)(cid:115)(cid:101)(cid:118)(cid:105)(cid:101)(cid:114)(cid:32)(cid:76)(cid:116)(cid:100)(cid:46)(cid:32)(cid:65)(cid:108)(cid:108)(cid:32)(cid:114)(cid:105)(cid:103)(cid:104)(cid:116)(cid:115)(cid:32)(cid:114)(cid:101)(cid:115)(cid:101)(cid:114)(cid:118)(cid:101)(cid:100)(cid:46)
4 CHAPTER 1. INTRODUCTION AND PRELIMINARIES
Sub. 1 Sub. 2 Sub. 3
Mid Final Mid Final Mid Final
Std. 1 s s s s s s
111 112 121 122 131 132
Std. 2 s s s s s s
211 212 221 222 231 232
Std. 3 s s s s s s
311 312 321 322 331 332
Std. 4 s s s s s s
411 412 421 422 431 432
Table 1.1: The first way to print S.
Mid Final
Sub. 1 Sub. 2 Sub. 3 Sub. 1 Sub. 2 Sub. 3
Std. 1 s s s s s s
111 121 131 112 122 132
Std. 2 s s s s s s
211 221 231 212 222 232
Std. 3 s s s s s s
311 321 331 312 322 332
Std. 4 s s s s s s
411 421 431 412 422 432
Table 1.2: The second way to print S.
Example1.2. Anotherimportantrealizationoftensorsarethestorageofcolor
imagesandvideos. Ablack-and-whiteimagecanbestoredasagreyscalematrix,
whoseentriesarethegreyscalevaluesofthecorrespondingpixels. Colorimages
are often built from several stacked color channels, each of which represents
value levels of the given channel. For example, RGB images are composed of
threeindependentchannelsforred, green, andblueprimarycolor components.
Wecanapplya3rd-ordertensorP tostoreanRGBimage,whose(i,j,k)entry
denotesthevalueofthek-thchannelinthe(i,j)position. (k=1,2,3represent
the red, green, and blue channel, respectively.) In order to store a color video,
we may need an extra index for the time axis. That is, we employ a 4th-order
tensor M = (m ), where M(:,:,:,t) stores the t-th frame of the video as a
ijkt
color image.
Example 1.3. Denote x = (x ,x ,...,x )(cid:62) ∈ Rn. As we know, a degree-1
1 2 n
polynomialp (x)=c x +c x +···+c x canberewrittenintop (x)=x(cid:62)c,
1 1 1 2 2 n n 1
wherethevectorc=(c ,c ,...,c )(cid:62). Similarly,adegree-2polynomialp (x)=
1 2 n 2
(cid:80)n c x x ,thatis,aquadraticform,canbesimplifiedintop (x)=x(cid:62)Cx,
i,j=1 ij i j 2
where the matrix C = (c ). By analogy, if we denote an mth-order tensor
ij
C = (c ) and apply a notation, which will be introduced in the next
i1i2...im
section, then the degree-m homogeneous polynomial
n n n
(cid:88) (cid:88) (cid:88)
p (x)= ··· c x x ...x
m i1i2...im i1 i2 im
i1=1i2=1 im=1
can be rewritten as
p (x)=Cxm.
m
Moreover, x(cid:62)c = 0 is often used to denote a hyperplane in Rn. Similarly,
Cxm =0canstandforandegree-mhypersurfaceinRn. WeshallseeinSection
1.2 that the normal vector at a point x on this hypersurface is n =Cxm−1.
0 x0 0