Table Of Content

t h e f r o n t i e r s c o l l e c t i o n t h e f r o n t i e r s c o l l e c t i o n SeriesEditors: A.C.Elitzur M.P.Silverman J.Tuszynski R.Vaas H.D.Zeh Thebooksinthiscollectionaredevotedtochallengingandopenproblemsattheforefront ofmodernscience,includingrelatedphilosophicaldebates.Incontrasttotypicalresearch monographs, however, they strive to present their topics in a manner accessible also to scientificallyliteratenon-specialistswishingtogaininsightintothedeeperimplicationsand fascinatingquestionsinvolved.Takenasawhole,theseriesreflectstheneedforafundamental andinterdisciplinaryapproachtomodernscience.Furthermore,itisintendedtoencourage active scientists in all areas to ponder over important and perhaps controversial issues beyond their own speciality. Extending from quantum physics and relativity to entropy, consciousnessandcomplexsystems–theFrontiersCollectionwillinspirereaderstopush backthefrontiersoftheirownknowledge. InformationandItsRoleinNature TheThermodynamic ByJ.G.Roederer MachineryofLife ByM.Kurzynski RelativityandtheNatureofSpacetime ByV.Petkov TheEmergingPhysics ofConsciousness QuoVadisQuantumMechanics? EditedbyJ.A.Tuszynski EditedbyA.C.Elitzur,S.Dolev, N.Kolenda WeakLinks StabilizersofComplexSystems Life–AsaMatterofFat fromProteinstoSocialNetworks TheEmergingScienceofLipidomics ByP.Csermely ByO.G.Mouritsen Quantum–ClassicalAnalogies Mind,MatterandtheImplicateOrder ByD.DragomanandM.Dragoman ByP.T.I.Pylkkänen KnowledgeandtheWorld QuantumMechanicsattheCrossroads ChallengesBeyondtheScienceWars NewPerspectivesfromHistory, EditedbyM.Carrier,J.Roggenhofer, PhilosophyandPhysics G.Küppers,P.Blanchard ByJ.Evans,A.S.Thomdike Quantum–ClassicalCorrespondence ParticleMetaphysics ByA.O.Bolivar ACriticalAccountofSubatomicReality ByB.Falkenburg Mind,MatterandQuantumMechanics ByH.Stapp ThePhysicalBasisoftheDirection ofTime QuantumMechanicsandGravity ByH.D.Zeh ByM.Sachs ExtremeEventsinNatureandSociety Asymmetry:TheFoundation EditedbyS.Albeverio,V.Jentsch, ofInformation H.Kantz ByS.J.Muller Scott J. Muller ASYMMETRY: THE FOUNDATION OF INFORMATION With33Figures 123 ScottJ.Muller BernoulliSystems Suite145 NationalInnovationCentre AustralianTechnologyPark Eveleigh,NSW1430 Australia email:[email protected] SeriesEditors: AvshalomC.Elitzur RüdigerVaas Bar-IlanUniversity, UniversityofGießen, UnitofInterdisciplinaryStudies, CenterforPhilosophyandFoundationsofScience 52900Ramat-Gan,Israel 35394Gießen,Germany email:[email protected] email:[email protected] MarkP.Silverman H.DieterZeh DepartmentofPhysics,TrinityCollege, UniversityofHeidelberg, Hartford,CT06106,USA InstituteofTheoreticalPhysics, email:[email protected] Philosophenweg19, 69120Heidelberg,Germany JackTuszynski email:[email protected] UniversityofAlberta, DepartmentofPhysics,Edmonton,AB, T6G2J1,Canada email:[email protected] Coverfigure:ImagecourtesyoftheScientificComputingandImagingInstitute, UniversityofUtah(www.sci.utah.edu). LibraryofCongressControlNumber:2007922925 ISSN 1612-3018 ISBN 978-3-540-69883-8 SpringerBerlinHeidelbergNewYork Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerial isconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broad- casting,reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthis publicationorpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawof September9,1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer. ViolationsareliableforprosecutionundertheGermanCopyrightLaw. SpringerisapartofSpringerScience+BusinessMedia springer.com ©Springer-VerlagBerlinHeidelberg2007 Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnot imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective lawsandregulationsandthereforefreeforgeneraluse. Typesetting:Datasuppliedbytheauthor Production:LE-TEXJelonek,Schmidt&VöcklerGbR,Leipzig Coverdesign:KünkelLopka,WerbeagenturGmbH,Heidelberg Printedonacid-freepaper SPIN11783350 57/3100/YL-543210 Preface Objects have the capacity to distinguish themselves from other objects and from themselves at different times. The interaction of objects, to- gether with the process of making distinctions, results in the transfer of a quantity that we call information. Some objects are capable of dis- tinguishing themselves in more ways than others. These objects have a greater information capacity. The quantification of how objects distinguish themselves and the relationship of this process to information is the subject of this book. As individual needs have arisen in the fields of physics, electrical engineering and computational science, diverse theories of information have been developed to serve as conceptual instruments to advance each field. Based on the foundational Statistical Mechanical physics of Maxwell and Boltzmann, an entropic theory of information was developed by Brillouin, Szilard and Schro¨dinger. In the field of Communica- tionsEngineering,Shannonformulatedatheoryofinformationusingan entropyanalogue.Incomputersciencea“shortestdescriptor”theoryof information was developed independently by Kolmogorov, Solomonoff and Chaitin. The considerations presented in this book are an attempt to illumi- nate the common and essential principles of these approaches and to proposeaunifying,non-semantic theory of information by demonstrat- ing that the three current major theories listed above can be unified under the concept of asymmetry, by deriving a general equation of information through the use of the algebra of symmetry, namely Group Theory and by making a strong case for the thesis that information is grounded in asymmetry. The book draws on examples from a number of fields including chemistry, physics, engineering and computer science to develop the VI Preface notions of information and entropy and to illustrate their interrela- tion. The work is intended for readers with a some background in science or mathematics, but it is hoped the overarching concepts are general enough and their presentation sufficiently clear to permit the non-technical reader to follow the discussion. Chapter 1 provides an introduction to the topic, defines the scope of the project and outlines the way forward. The technical concepts of entropy and probability are developed in Chapter 2 by surveying current theories of information. Distinguishability and its relationship to information is presented in Chapter 3 along with numerous illus- trative examples. Chapter 4 introduces symmetry and Group Theory. This chapter demonstrates the connections between information, entropy and symmetry and shows how these can unify current information theories. Finally Chapter 5 summarises the project and identifies some open questions. This book represents a first step in developing a theory that may serve as a general tool for a number of disciplines. I hope that it will be of some use to researchers in fields that require the development of informatic metrics or are concerned with the dynamics of information generation ordestruction.Extendingthis,Iwouldliketoseethegroup- theoretic account of information develop into an algebra of causation by the quantification of transferred information. A large portion of this research was conducted as part of my PhD dissertation at the University of Newcastle, Australia. I would like to express my deep gratitude to Cliff Hooker and John Collier for in- valuable advice and guidance and to George Willis for assistance with GroupTheory,inparticularTopologicalGroups.Earlydiscussionswith Jim Crutchfield at the Santa Fe Institute were usefulin clarifying some initial ideas. I would also like to thank Chris Boucher, Ellen Watson, Jamie Pullen, Lesley Roberts and Melinda Stokes for much support and inspiration. Finally, I would also like to thank my parents, Jon and Lyal. Sydney, April 2007 Scott Muller Contents 1 Introduction ......................................... 1 1.1 Structure.......................................... 3 2 Information .......................................... 5 2.1 Scope of Information ............................... 5 2.2 A Survey of Information Theories .................... 6 2.2.1 Thermodynamic Information Theory ............ 7 2.2.2 Information (Communication) Theory ........... 32 2.2.3 Algorithmic Information Theory................ 34 2.2.4 Signpost..................................... 54 2.3 Probability ........................................ 56 2.3.1 Subjective Probability......................... 57 2.3.2 Frequency Probability......................... 57 2.3.3 Dispositional Probability ...................... 63 2.4 Signpost .......................................... 65 3 Information and Distinguishability ................... 67 3.1 Distinguishability .................................. 67 3.2 Information: A Foundational Approach ............... 76 4 Information and Symmetry ......................... 79 4.1 Symmetry......................................... 79 4.2 Symmetry and Group Theory........................ 81 4.2.1 Subgroups and Special Groups ................. 87 4.2.2 Group Theory and Information................. 89 4.3 Symmetry and Information .......................... 96 4.3.1 Information Generation ....................... 97 4.3.2 Extrinsic and Intrinsic Information ............. 99 4.4 Information and Probability ......................... 100 VIII Contents 4.4.1 Maximum Entropy Principle ................... 100 4.5 Information and Statistical Mechanics ................ 112 4.5.1 Distinguishability and Entropy ................. 112 4.5.2 Demonic Information.......................... 116 4.6 Information and Physical Thermodynamics............ 118 4.6.1 Symmetry and Physical Entropy................ 118 4.6.2 Symmetry and the Third Law .................. 120 4.6.3 Information and The Gibbs Paradox ............ 122 4.7 Quantum Information .............................. 124 4.7.1 Quantum Information and Distinguishability ..... 125 4.8 Symmetries and Algorithmic Information Theory....... 132 4.8.1 Symmetry and Kolmogorov Complexity ......... 132 4.8.2 Memory and Measurement..................... 132 4.8.3 Groups and Algorithmic Information Theory ..... 133 4.8.4 Symmetry and Randomness.................... 137 4.8.5 A Final Signpost ............................. 141 5 Conclusion ........................................... 143 A Burnside’s Lemma ................................... 147 B Worked Examples.................................... 149 B.1 Clocks ............................................ 149 B.1.1 Case 1 ...................................... 149 B.1.2 Case 2 ...................................... 150 B.1.3 Case 3 ...................................... 152 B.2 Binary String...................................... 153 References............................................... 155 Index.................................................... 161 1 Introduction Information is a primal concept about which we have deep intuitions. It forms part of our interface to the world. Thus is seems somewhat odd that it is only in the last one hundred years or so that attempts have been made to create mathematically rigorous definitions for information. Perhaps this is due to a tendency to cast information in an epistemological or semantic light, thus rendering the problem difficult to describe using formal analysis. Yet physical objects1 are endowed with independent, self-descriptive capacity. They have innate discern- able differences that may be employed to differentiate them from others or to differentiate one state of an object from another state. These objects vary in complexity, in the number of ways that they can distinguish themselves. Recent attempts to quantify information have come at the problem with the perspective and toolkits of several specific research areas. As individual needs have arisen in such fields as physics, electrical engineering and computational science, theories of information have been developed to serve as conceptual instruments to advance that field. These theories were not developed totally in isolation. For example, Shannon [72] in communications engineering was aware of the work done by Boltzmann, and Chaitin [21], in computational science, was aware of Shannon’s work. Certain concepts, such as the use of the frequency concept of probability, are shared by different information theories, and some terminology, such as ‘entropy’, is used in common, thoughoften withdivergentmeanings.However forthemostpartthese theoriesofinformation,whileostensiblydescribingthesamething,were developed for specific local needs and only partially overlap in scope. 1 This can also include representations of abstract objects such as numbers and laws.