Linear Algebra for Computational Sciences and Engineering

472 Pages·2016·8.5 MB·English

by  Ferrante Neri (auth.)

Ferrante Neri
Linear Algebra
for Computational
Sciences and Engineering
Linear Algebra for Computational
Sciences and Engineering
Ferrante Neri
Linear Algebra
for Computational
Sciences and Engineering
Foreword by Alberto Grasso
123
Ferrante Neri
Centrefor Computational Intelligence
De MontfortUniversity
Leicester
UK
and
University of Jyväskylä
Jyväskylä
Finland
ISBN978-3-319-40339-7 ISBN978-3-319-40341-0 (eBook)
DOI 10.1007/978-3-319-40341-0
LibraryofCongressControlNumber:2016941610
©SpringerInternationalPublishingSwitzerland2016
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
foranyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAGSwitzerland
We can only see a short distance ahead, but
we can see plenty there that needs to be done
Alan Turing
Foreword
Linear Algebra in Physics
The history of linear algebra can be viewed within the context of two important
traditions.
Thefirsttradition(withinthehistoryofmathematics)consistsoftheprogressive
broadening of the concept of number so to include not only positive integers, but
also negative numbers, fractions, algebraic and transcendental irrationals.
Moreover, the symbols in the equations became matrices, polynomials, sets, per-
mutations. Complex numbers and vector analysis belong to this tradition. Within
thedevelopmentofmathematics,theonewasconcernednotsomuchaboutsolving
specificequations,butmostlyaboutaddressinggeneralandfundamentalquestions.
The latter were approached by extending the operations and the properties of sum
and multiplication from integers to other linear algebraic structures. Different
algebraic structures (Lattices and Boolean algebra) generalized other kinds of
operationsthusallowingtooptimizesomenon-linearmathematicalproblems.Asa
first example, Lattices were generalizations of order relations on algebraic spaces,
suchassetinclusioninsettheoryandinequalityinthefamiliarnumbersystems(N,
Z,Q,andR).Asasecondexample,Booleanalgebrageneralizedtheoperationsof
intersection and union and the Principle of Duality (De Morgan’s Relations),
already valid in set theory, to formalize the logic and the propositions’ calculus.
ThisapproachtologicasanalgebraicstructurewasmuchsimilarastheDescartes’
algebraapproachtothegeometry.Settheoryandlogichavebeenfurtheradvanced
in the past centuries. In particular, Hilbert attempted to build up mathematics by
using symbolic logicin a way that could prove its consistency. On theother hand,
Gödelprovedthatinanymathematicalsystemtherewillalwaysbestatementsthat
can never be proven either true or false.
The second tradition (within the history of physical science) consists of the
search for mathematical entities and operations that represent aspects of the
physicalreality.ThistraditionplayedaroleintheGreekgeometry’sbearingandits
following application to physical problems. When observing the space around us,
vii
viii Foreword
wealwayssupposetheexistenceofareferenceframe,identifiedwithanideal“rigid
body”, in the part of the universe in which the system we want to study evolves
(e.g.athreeaxes’systemhavingtheSunastheiroriginanddirectversusthreefixed
stars). This is modelled in the so called “Euclidean affine space”. A reference
frame’s choice is purely descriptive, at a purely kinematic level. Two reference
frameshavetobeintuitivelyconsidereddistinctifthecorrespondent“rigidbodies”
are in relative motion. Therefore, it is important to fix the links (Linear
Transformations)betweenthekinematicentitiesassociatedtothesamemotionbut
related to two different reference frames (Galileo’s Relativity).
In the XVII and XVIII centuries, some physical entities needed a new repre-
sentation. This necessity made the above-mentioned two traditions converged by
adding quantities as velocity, force, momentum and acceleration (vectors) to the
traditionalquantitiesasmassandtime(scalars).Importantideasledtothevectors’
major systems: the forces’ parallelogram concept by Galileo, the situations geom-
etry and calculus concepts by Leibniz and by Newton and the complex numbers’
geometricalrepresentation.Kinematicsstudiesthemotionofbodiesinspaceandin
timeindependentlyonthecauseswhichprovokeit.Inclassicalphysics,theroleof
timeisreducedtothatofaparametricindependentvariable.Itneedsalsotochoose
a model for the body (or bodies) whose motion one wants to study. The funda-
mental and simpler model is that of point (useful only if the body’s extension is
smaller than the extension of its motion and of the other important physical
quantitiesconsideredinaparticularproblem).Themotionofapointisrepresented
by a curve in the tridimensional Euclidean affine space. A second fundamental
modelisthe“rigidbody”one,adoptedforthoseextendedbodieswhosecomponent
particles do not change mutual distances during the motion.
Later developments in Electricity, Magnetism, and Optics further promoted the
use of vectors in mathematical physics. The XIX century marked the development
of vector space methods, whose prototypes were the three-dimensional geometric
extensive algebra by Grassmann and the algebra of quaternions by Hamilton to
respectivelyrepresentorientationandrotationofabodyinthreedimensions.Thus,
itwasalreadyclearhowasimplealgebrashouldmeettheneedsofthephysicistsin
order to efficiently describe objects in space and in time (in particular, their
Dynamical Symmetries and the corresponding Conservation Laws) and the prop-
erties of space-time itself. Furthermore, the principal characteristic of a simple
algebrahadtobeitsLinearity(oratmostitsmulti-Linearity).Duringthelatterpart
oftheXIXcentury,Gibbsbasedhisthreedimensionalvectoralgebraonsomeideas
by Grassmann and by Hamilton, while Clifford united these systems into a single
geometric algebra (direct product of quaternions’ algebras). After, the Einsteins
description of the four-dimensional continuum space-time (Special and General
Relativity Theories) required a Tensor Algebra. In 1930s, Pauli and Dirac intro-
duced Clifford algebra’s matrix representations for physical reasons: Pauli for
describingtheelectronspin,whileDiracforaccommodatingboththeelectronspin
and the special relativity.
Each algebraic system is widely used in Contemporary Physics and is a fun-
damentalpart ofrepresenting,interpreting, andunderstandingthenature.Linearity
Foreword ix
in physics is principally supported by three ideas: Superposition Principle,
Decoupling Principle, and Symmetry Principle.
Superposition Principle. Let us suppose to have a linear problem where each
Ok isthefundamentaloutput(linearresponse)ofeachbasicinputIk.Then,bothan
arbitrary input as its own response can be written as a linear combination of the
basic ones, i.e. I ¼c1I1þ...þckIk and O¼c1O1þ...þckOk.
Decoupling Principle. If a system of coupled differential equations (or differ-
ence equations) involves a diagonalizable square matrix A, then it is useful to
consider new variables x0k ¼Uxk with ðk 2N;1(cid:2)k(cid:2)nÞ, where U is an Unitary
matrix and x0 is an orthogonal eigenvectors set (basis) of A. Rewriting the equa-
k
tions in terms of the x0, the one discovers that each eigenvectors evolution is
k
independent on the others and that the form of each equation depends only on the
corresponding eigenvalue of A. By solving the equations so to get each x0 as a
k
function of time, it is also possible to get xk as a function of time (xk ¼U(cid:3)1x0k) .
When A is not diagonalizable (not normal), the resulting equations for x are not
completely decoupled (Jordan canonical form), but are still relatively easy (of
course,ifonedoesnottakeintoaccountsomedeepproblemsrelatedtothepossible
presence of resonances).
Symmetry Principle. If A is a diagonal matrix representing a linear transfor-
mation of a physical system’s state and x0k its eigenvectors set, each unitary
transformationsatisfyingthematrixequationUAU(cid:3)1 ¼A(orUA¼AU)iscalled
“Symmetry Transformation” for the considered physical system. Its deep meaning
is to eventually change each eigenvector without changing the whole set of
eigenvectors and their corresponding eigenvalues.
Thus, special importance in computational physics is assumed by the standard
methods for solving systems of linear equations: the procedures suited for sym-
metric real matrices and the iterative methods converging fast when applied to
matrix having its non-zero elements concentrated near the main diagonal
(Diagonally Dominated Matrix).
Physics has a very strong tradition about tending to focus on some essential
aspectswhileneglectingothersimportantissues.Forexample,Galileofoundedthe
Mechanics neglecting friction, despite its important effect on mechanics. The
statementofGalileo’sInertiaLaw(Newton’sFirstLaw,i.e.“Anobjectnotaffected
by forces moves with constant velocity”) is a pure abstraction and it is approxi-
mately valid. While modelling, a popular simplification has been for centuries the
search of a linear equation approximating the nature. Both Ordinary and Partial
Linear Differential Equations appear through classical and quantum physics and
even where the equations are non-linear, Linear Approximations are extremely
powerful.Forexample,thankstoNewton’sSecondLaw,muchofclassicalphysics
isexpressedintermsofsecondorderordinarydifferentialequations’systems.Ifthe
forceisaposition’slinearfunction,theresultingequationsarelinear(md2x¼(cid:3)Ax,
dt2
where A matrix not depending on x). Every solution may be written as a linear
combination of the special solutions (oscillation’s normal modes) coming from
eigenvectors of the A matrix. For nonlinear problems near equilibrium, the force
x Foreword
can always be expanded as a Taylor’s series and the leading (linear) term is
dominantforsmalloscillations.Adetailedtreatmentofcoupledsmalloscillationsis
possiblebyobtainingadiagonalmatrixofthecoefficientsinN coupleddifferential
equations by finding the eigenvalues and the eigenvectors of the Lagrange’s
equations for coupled oscillators. In classical mechanics, another example of lin-
earisation consists of looking for the principal moments and principal axes of a
solid body through solving the eigenvalues’ problem of a real symmetric matrix
(Inertia Tensor). In the theory of continua (e.g. hydrodynamics, diffusion and
thermal conduction, acoustic, electromagnetism), it is (sometimes) possible to
convert a partial differential equation into a system of linear equations by
employing the finite difference formalism. That ends up with a Diagonally
Dominated coefficients’ Matrix. In particular, Maxwell’s equations of electromag-
netism have an infinite number of degrees offreedom (i.e. the value of the field at
eachpoint)buttheSuperpositionPrincipleandtheDecouplingPrinciplestillapply.
The response to an arbitrary input is obtained as the convolution of a continuous
basis of Dirac δ functions and the relevant Green’s function.
Even without the differential geometry’s more advanced applications, the basic
concepts of multilinear mapping and tensor are used not only in classical physics
(e.g.inertiaandelectromagneticfieldtensors),butalsoinengineering(e.g.dyadic).
In particle physics, it was important to analyse the problem of Neutrino
Oscillations, formally related both to the Decoupling and the Superposition
Principles.Inthiscase,theThreeNeutrinosMassesMatrixisnotdiagonalandnot
normal in the so called Gauge States’ basis. However, through a bi-unitary trans-
formation (one unitary transformation for each “parity” of the gauge states), it is
possible to get the eigenvalues and their own eigenvectors (Mass States) which
allow to render it diagonal. After this transformation, it is possible to obtain the
Gauge States as a superposition (linear combination) of Mass States.
Schrödinger’s Linear Equation governs the non relativistic quantum mechanics
andmanyproblemsarereducedtoobtainadiagonalHamiltonianoperator.Besides,
when studying the quantum angular momentum’s addition one considers
Clebsch-Gordon coefficients related to an unitary matrix that changes a basis in a
finite-dimensional space.
In experimental physics and statistical mechanics (Stochastic methods’ frame-
work) researchers encounter symmetric, real positive definite and thus diagonaliz-
able matrices (so-called covariance or dispersion matrix). The elements of a
covariance matrix in the i, j positions are the covariances between ith and jth
elements of a random vector (i.e. a vector of random variables, each with finite
variance).Intuitively,thevariance’snotionissogeneralizedtomultipledimension.
The geometrical symmetry’s notion played an essential part in constructing
simplified theories governing the motion of galaxies and the microstructure of
matter(quarks’motionconfinedinsidethehadronsandleptons’motion).Itwasnot
until the Einstein’s era that the discovery of the space-time symmetries of the
fundamental laws and the meaning of their relations tothe conservation laws were
fully appreciated, for example Lorentz Transformations, Noether’s Theorem and
Weyl’s Covariance. An object with a definite shape, size, location and orientation
Foreword xi
constitutes a state whose symmetry properties are to be studied. The higher its
“degree ofsymmetry” (and thenumberof conditions defining thestate isreduced)
the greater is the number of transformations that leave the state of the object
unchanged.
While developing some ideas by Lagrange, by Ruffini and by Abel (among
others), Galois introduced important concepts in group theory. This study showed
thatanequationofordern(cid:4)5cannot,ingeneral,besolvedbyalgebraic methods.
Hedidthisbyshowingthatthefunctionalrelationsamongtherootsofanequation
have symmetries under the permutations of roots. In 1850s, Cayley showed that
every finite group is isomorphic to a certain permutation group (e.g. the crystals’
geometrical symmetries are described in finite groups’ terms). Fifty years after
Galois, Lie unified many disconnected methods of solving differential equations
(evolved over about two centuries) by introducing the concept of continuous
transformationofagroupinthetheoryofdifferentialequations.Inthe1920s,Weyl
and Wigner recognized that certain group theory’s methods could be used as a
powerfulanalyticaltoolinQuantumPhysics.Inparticular,theessentialroleplayed
by Lie’s groups, e.g. rotation isomorphic groups SOð3Þ and SUð2Þ, was first
emphasizedbyWigner.Theirideashavebeenusedinmanycontemporaryphysics’
branches which range from the Theory of Solids to Nuclear Physics and Particle
Physics. In Classical Dynamics, the invariance of the equations of motion of a
particle under the Galilean transformation is fundamental in Galileo’s relativity.
The search for a linear transformation leaving “formally invariant” the Maxwell’s
equations of Electromagnetism led to the discovery of a group of rotations in
space-time (Lorentz transformation).
Frequently, it is important to understand why a symmetry of a system is
observedtobebroken. Inphysics,twodifferent kindsofsymmetrybreakdownare
considered. If two states of an object are different (e.g. by an angle or a simple
phase rotation) but they have the same energy, one refers to “Spontaneous
SymmetryBreaking”.Inthissense, theunderlying laws ofasystem maintaintheir
form(Lagrange’sEquationsareinvariant)underasymmetrytransformation,butthe
system as a whole changes under such transformation by distinguishing between
two or more fundamental states. This kind of symmetry breaking (for example)
characterizes the ferromagnetic and the superconductive phases, where the
Lagrange function (or the Hamiltonian function, representing the energy of the
system) is invariant under rotations (in the ferromagnetic phase) and under a
complex scalar transformation (in the superconductive phase). On the contrary, if
the Lagrange function is not invariant under particular transformations, the
so-called “Explicit Symmetry Breaking” occurs. For example, this happens when
an external magnetic field is applied to a paramagnet (Zeeman’s Effect).
Finally, by developing the determinants through the permutations’ theory and
therelatedLevi-Civitasymbolism,onegainsanimportantandeasycalculationtool
for modern differential geometry, with applications in engineering as well as in
modern physics. This is the case in general relativity, quantum gravity, and string
theory.