Asymmetry : the foundation of information

174 Pages·2007·2.323 MB·English

by  Muller, Scott J

t h e f r o n t i e r s c o l l e c t i o n
t h e f r o n t i e r s c o l l e c t i o n
SeriesEditors:
A.C.Elitzur M.P.Silverman J.Tuszynski R.Vaas H.D.Zeh
Thebooksinthiscollectionaredevotedtochallengingandopenproblemsattheforefront
ofmodernscience,includingrelatedphilosophicaldebates.Incontrasttotypicalresearch
monographs, however, they strive to present their topics in a manner accessible also to
scientificallyliteratenon-specialistswishingtogaininsightintothedeeperimplicationsand
fascinatingquestionsinvolved.Takenasawhole,theseriesreflectstheneedforafundamental
andinterdisciplinaryapproachtomodernscience.Furthermore,itisintendedtoencourage
active scientists in all areas to ponder over important and perhaps controversial issues
beyond their own speciality. Extending from quantum physics and relativity to entropy,
consciousnessandcomplexsystems–theFrontiersCollectionwillinspirereaderstopush
backthefrontiersoftheirownknowledge.
InformationandItsRoleinNature TheThermodynamic
ByJ.G.Roederer MachineryofLife
ByM.Kurzynski
RelativityandtheNatureofSpacetime
ByV.Petkov TheEmergingPhysics
ofConsciousness
QuoVadisQuantumMechanics?
EditedbyJ.A.Tuszynski
EditedbyA.C.Elitzur,S.Dolev,
N.Kolenda
WeakLinks
StabilizersofComplexSystems
Life–AsaMatterofFat
fromProteinstoSocialNetworks
TheEmergingScienceofLipidomics
ByP.Csermely
ByO.G.Mouritsen
Quantum–ClassicalAnalogies Mind,MatterandtheImplicateOrder
ByD.DragomanandM.Dragoman ByP.T.I.Pylkkänen
KnowledgeandtheWorld QuantumMechanicsattheCrossroads
ChallengesBeyondtheScienceWars NewPerspectivesfromHistory,
EditedbyM.Carrier,J.Roggenhofer, PhilosophyandPhysics
G.Küppers,P.Blanchard ByJ.Evans,A.S.Thomdike
Quantum–ClassicalCorrespondence ParticleMetaphysics
ByA.O.Bolivar ACriticalAccountofSubatomicReality
ByB.Falkenburg
Mind,MatterandQuantumMechanics
ByH.Stapp
ThePhysicalBasisoftheDirection
ofTime
QuantumMechanicsandGravity
ByH.D.Zeh
ByM.Sachs
ExtremeEventsinNatureandSociety Asymmetry:TheFoundation
EditedbyS.Albeverio,V.Jentsch, ofInformation
H.Kantz ByS.J.Muller
Scott J. Muller
ASYMMETRY:
THE FOUNDATION
OF INFORMATION
With33Figures
123
ScottJ.Muller
BernoulliSystems
Suite145
NationalInnovationCentre
AustralianTechnologyPark
Eveleigh,NSW1430
Australia
email:[email protected]
SeriesEditors:
AvshalomC.Elitzur RüdigerVaas
Bar-IlanUniversity, UniversityofGießen,
UnitofInterdisciplinaryStudies, CenterforPhilosophyandFoundationsofScience
52900Ramat-Gan,Israel 35394Gießen,Germany
email:[email protected] email:[email protected]
MarkP.Silverman H.DieterZeh
DepartmentofPhysics,TrinityCollege, UniversityofHeidelberg,
Hartford,CT06106,USA InstituteofTheoreticalPhysics,
email:[email protected] Philosophenweg19,
69120Heidelberg,Germany
JackTuszynski email:[email protected]
UniversityofAlberta,
DepartmentofPhysics,Edmonton,AB,
T6G2J1,Canada
email:[email protected]
Coverfigure:ImagecourtesyoftheScientificComputingandImagingInstitute,
UniversityofUtah(www.sci.utah.edu).
LibraryofCongressControlNumber:2007922925
ISSN 1612-3018
ISBN 978-3-540-69883-8 SpringerBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerial
isconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broad-
casting,reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthis
publicationorpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawof
September9,1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
ViolationsareliableforprosecutionundertheGermanCopyrightLaw.
SpringerisapartofSpringerScience+BusinessMedia
springer.com
©Springer-VerlagBerlinHeidelberg2007
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnot
imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective
lawsandregulationsandthereforefreeforgeneraluse.
Typesetting:Datasuppliedbytheauthor
Production:LE-TEXJelonek,Schmidt&VöcklerGbR,Leipzig
Coverdesign:KünkelLopka,WerbeagenturGmbH,Heidelberg
Printedonacid-freepaper SPIN11783350 57/3100/YL-543210
Preface
Objects have the capacity to distinguish themselves from other objects
and from themselves at different times. The interaction of objects, to-
gether with the process of making distinctions, results in the transfer
of a quantity that we call information. Some objects are capable of dis-
tinguishing themselves in more ways than others. These objects have a
greater information capacity. The quantification of how objects distin-
guish themselves and the relationship of this process to information is
the subject of this book.
As individual needs have arisen in the fields of physics, electrical
engineering and computational science, diverse theories of information
have been developed to serve as conceptual instruments to advance
each field. Based on the foundational Statistical Mechanical physics of
Maxwell and Boltzmann, an entropic theory of information was devel-
oped by Brillouin, Szilard and Schr¨odinger. In the field of Communica-
tionsEngineering,Shannonformulatedatheoryofinformationusingan
entropyanalogue.Incomputersciencea“shortestdescriptor”theoryof
information was developed independently by Kolmogorov, Solomonoff
and Chaitin.
The considerations presented in this book are an attempt to illumi-
nate the common and essential principles of these approaches and to
proposeaunifying,non-semantic theory of information by demonstrat-
ing that the three current major theories listed above can be unified
under the concept of asymmetry, by deriving a general equation of in-
formation through the use of the algebra of symmetry, namely Group
Theory and by making a strong case for the thesis that information is
grounded in asymmetry.
The book draws on examples from a number of fields including
chemistry, physics, engineering and computer science to develop the
VI Preface
notions of information and entropy and to illustrate their interrela-
tion. The work is intended for readers with a some background in
science or mathematics, but it is hoped the overarching concepts are
general enough and their presentation sufficiently clear to permit the
non-technical reader to follow the discussion.
Chapter 1 provides an introduction to the topic, defines the scope
of the project and outlines the way forward. The technical concepts
of entropy and probability are developed in Chapter 2 by surveying
current theories of information. Distinguishability and its relationship
to information is presented in Chapter 3 along with numerous illus-
trative examples. Chapter 4 introduces symmetry and Group Theory.
This chapter demonstrates the connections between information, en-
tropy and symmetry and shows how these can unify current informa-
tion theories. Finally Chapter 5 summarises the project and identifies
some open questions.
This book represents a first step in developing a theory that may
serve as a general tool for a number of disciplines. I hope that it will
be of some use to researchers in fields that require the development of
informatic metrics or are concerned with the dynamics of information
generation ordestruction.Extendingthis,Iwouldliketoseethegroup-
theoretic account of information develop into an algebra of causation
by the quantification of transferred information.
A large portion of this research was conducted as part of my PhD
dissertation at the University of Newcastle, Australia. I would like to
express my deep gratitude to Cliff Hooker and John Collier for in-
valuable advice and guidance and to George Willis for assistance with
GroupTheory,inparticularTopologicalGroups.Earlydiscussionswith
Jim Crutchfield at the Santa Fe Institute were usefulin clarifying some
initial ideas. I would also like to thank Chris Boucher, Ellen Watson,
Jamie Pullen, Lesley Roberts and Melinda Stokes for much support
and inspiration. Finally, I would also like to thank my parents, Jon
and Lyal.
Sydney, April 2007 Scott Muller
Contents
1 Introduction ......................................... 1
1.1 Structure.......................................... 3
2 Information .......................................... 5
2.1 Scope of Information ............................... 5
2.2 A Survey of Information Theories .................... 6
2.2.1 Thermodynamic Information Theory ............ 7
2.2.2 Information (Communication) Theory ........... 32
2.2.3 Algorithmic Information Theory................ 34
2.2.4 Signpost..................................... 54
2.3 Probability ........................................ 56
2.3.1 Subjective Probability......................... 57
2.3.2 Frequency Probability......................... 57
2.3.3 Dispositional Probability ...................... 63
2.4 Signpost .......................................... 65
3 Information and Distinguishability ................... 67
3.1 Distinguishability .................................. 67
3.2 Information: A Foundational Approach ............... 76
4 Information and Symmetry ......................... 79
4.1 Symmetry......................................... 79
4.2 Symmetry and Group Theory........................ 81
4.2.1 Subgroups and Special Groups ................. 87
4.2.2 Group Theory and Information................. 89
4.3 Symmetry and Information .......................... 96
4.3.1 Information Generation ....................... 97
4.3.2 Extrinsic and Intrinsic Information ............. 99
4.4 Information and Probability ......................... 100
VIII Contents
4.4.1 Maximum Entropy Principle ................... 100
4.5 Information and Statistical Mechanics ................ 112
4.5.1 Distinguishability and Entropy ................. 112
4.5.2 Demonic Information.......................... 116
4.6 Information and Physical Thermodynamics............ 118
4.6.1 Symmetry and Physical Entropy................ 118
4.6.2 Symmetry and the Third Law .................. 120
4.6.3 Information and The Gibbs Paradox ............ 122
4.7 Quantum Information .............................. 124
4.7.1 Quantum Information and Distinguishability ..... 125
4.8 Symmetries and Algorithmic Information Theory....... 132
4.8.1 Symmetry and Kolmogorov Complexity ......... 132
4.8.2 Memory and Measurement..................... 132
4.8.3 Groups and Algorithmic Information Theory ..... 133
4.8.4 Symmetry and Randomness.................... 137
4.8.5 A Final Signpost ............................. 141
5 Conclusion ........................................... 143
A Burnside’s Lemma ................................... 147
B Worked Examples.................................... 149
B.1 Clocks ............................................ 149
B.1.1 Case 1 ...................................... 149
B.1.2 Case 2 ...................................... 150
B.1.3 Case 3 ...................................... 152
B.2 Binary String...................................... 153
References............................................... 155
Index.................................................... 161
1
Introduction
Information is a primal concept about which we have deep intuitions.
It forms part of our interface to the world. Thus is seems somewhat
odd that it is only in the last one hundred years or so that attempts
have been made to create mathematically rigorous definitions for in-
formation. Perhaps this is due to a tendency to cast information in an
epistemological or semantic light, thus rendering the problem difficult
to describe using formal analysis. Yet physical objects1 are endowed
with independent, self-descriptive capacity. They have innate discern-
able differences that may be employed to differentiate them from oth-
ers or to differentiate one state of an object from another state. These
objects vary in complexity, in the number of ways that they can dis-
tinguish themselves.
Recent attempts to quantify information have come at the problem
with the perspective and toolkits of several specific research areas. As
individual needs have arisen in such fields as physics, electrical engi-
neering and computational science, theories of information have been
developed to serve as conceptual instruments to advance that field.
These theories were not developed totally in isolation. For example,
Shannon [72] in communications engineering was aware of the work
done by Boltzmann, and Chaitin [21], in computational science, was
aware of Shannon’s work. Certain concepts, such as the use of the
frequency concept of probability, are shared by different information
theories, and some terminology, such as ‘entropy’, is used in common,
thoughoften withdivergentmeanings.However forthemostpartthese
theoriesofinformation,whileostensiblydescribingthesamething,were
developed for specific local needs and only partially overlap in scope.
1 This can also include representations of abstract objects such as numbers and
laws.