ebook img

Scale Invariance and Symmetry Relationships In Non-Extensive Statistical Mechanics PDF

0.21 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Scale Invariance and Symmetry Relationships In Non-Extensive Statistical Mechanics

5 0 0 2 n Scale Invariance and Symmetry Relationships a J In Non-Extensive Statistical Mechanics 2 1 Mark Fleischer ] h InformationOperationsGroup c e JohnsHopkinsUniversity m AppliedPhysicsLaboratory - 11100JohnsHopkinsRoad t a Laurel, Maryland20723-6099 t s email: [email protected] . t a (cid:13)c 2004by MarkFleischer m - December 10,2004 d n o c Abstract [ Thisarticleextends resultsdescribed inarecent articledetailingastructural 1 scaleinvariancepropertyofthesimulatedannealing(SA)algorithm.Theseexten- v sionsarebasedongeneralizationsoftheSAalgorithmbasedonTsallisstatistics 3 andanon-extensiveformofentropy. Thesescaleinvariancepropertiesshowhow 9 arbitraryaggregationsofenergylevelsretaincertainmathematicalcharacteristics. 2 In applying the non-extensive forms of statistical mechanics to illuminate these 1 scale invariance properties, an interesting energy transformation is revealed that 0 hasanumberofpotentiallyusefulapplications. Thisenergytransformationfunc- 5 0 tionalsorevealsanumberofsymmetryproperties. Furtherextensionsofthisre- / searchindicatehowthisenergytransformationfunctionrelatestopowerlawdistri- t a butionsandpotentialapplicationforovercomingtheso-called“brokenergodicity” m problemprevalentinmanycomputersimulationsofcriticalphenomena. - d n 1 Introduction o c Recentadvancesinstatisticalmechanicshavehelpedtoexplainthebehavioroflarge : v ensemblesofinteractingsystems. SincethetimeofBoltzmann,therehasbeenagreat Xi deal of effort in attempting to understand the behavior of such systems. In his day, agreatdealofprogresswasmadeusingaformofentropythatseemedtoexplainthe r a behavioroftheso-called“idealgas”model,amodelpremisedontheverylimitedform ofinteractionimpliedbyelasticityassumptions.Notwithstandingtheseassumptions,it wasabletopredict,quiteaccurately,manyattributesofthebehaviorofgasesinthermal equilibrium. 1 Despiteitsenormoussuccessfortheseidealizedsystems,theBoltzmannversionof statisticalmechanicshashadlimitedsuccessinexplainingthebehaviorofmorecom- plexsystemswherethecomponentsinteractinwaysthataredistinctlyinelastic—that is to say, the particles do not obey simple Newtonian mechanics. In these models, the particles themselves may absorb energy during collisions in forms other than by changesintheirvelocity. Theassumptionsoftheidealgasmodelthattheseparticles haveno internalstructurethereforecannotbe assumed. Or, it may be that these par- ticles exhibit attractive or repulsive forces between them. In any event, their gross behavior is not well predicted in systems governedby the Boltzmann-Gibbsentropy formula: S = −k W π lnπ where the π represents the probabilityof the ith BG i=1 i i i energy partition and W represents the size (number of energy states) of the system. P Thesecomplexsystemsobeyvariouspowerlawsandcanexhibitphasetransitionsand relatedcriticalphenomenawheretherearedrasticshiftsintheirenergyconfigurations. Thesepowerlawsandcriticalitypropertiesseemtobequitediverseandubiquitous in nature. To help explainthese behaviors, Tsallis [10, 11] developeda new entropy expressionthatformsthebasisofanon-extensiveformofthermodynamics: k 1− W pq i=1 i S = (1) q (cid:16) qP−1 (cid:17) wherekisaconstantandS istheentropyparameterizedbytheentropicparameterq. q Inclassicalstatisticalmechanics,entropyfallsintoaclassofvariablesthatarereferred toasextensivebecausetheyscalewiththesizeofthesystem. Intensivevariables,such astemperature,donotscale withthesize ofthesystem1. Tsallis’formofentropyis non-extensivebecausetheentropyoftheunionoftwoindependentsystemsisnotequal tothesumoftheentropiesofeachsystem. Thatis,forindependentsystemsAandB, (1−q)S (A)S (B) S (A+B)=S (A)+S (B)+ q q (2) q q q k andTsallispointsoutthatq =1recoverstheextensivitypropertiesoftheBoltzmann- Gibbs entropy formula and further, that limq→1Sq = SBG, hence can be seen as a generalization of Boltzmann-Gibbs [10, 11]. Moreover, the stationary probabilities p (t)ofthesystembeinginsomeifurtherillustratesthegeneralizationoftheclassic i entropyforminthat limp (t)=π (t) (3) i i q→1 forallt>0[10,p.483]. Theentropicparameterqcontrolsimportantaspectsoftheprobabilitydistribution oftheenergylevelsmakingitispossibletomarkedlychangethenatureofthethermo- dynamicsystem beingmodeled. Forexample,whenq > 1thestationaryprobability distributionshiftsfromanexponentialformtoonewithheavytailsthatgivesrisetoa 1Combinetwovesselsofgaseachwiththesamevolumeandpressureintoanothervesseloftwicethe volume, andthe pressure andtemperature ofthe combined gas will bethe sameas before. Energyand entropy,however,areexamplesofextensivevariablesinthatcombiningseveralsourcesofeitherenergyor entropyandyouincreasethetotalenergyorentropy. 2 powerlawdistribution,orconversely,onecanincreasethestationaryprobabilityoflow energystateswhenq <1,somethingquiteusefulinoptimization(seee.g.,[1,12]). Oneofthebestwaysforstudyingtheimplicationsofthisnon-extensiveformofsta- tisticalmechanicsisthroughtheuseofcomputersimulations.Thesimulatedannealing (SA)algorithmprovidesthesebasicsimulationtoolsasitis,atheart,asimulationofa thermodynamicsystemalthoughithasbeenusedprincipallyforsolvingoptimization problems. Tsallisdevelopedageneralizedsimulatedannealing(GSA)algorithm[12] basedonmaximizingtheentropyin(1). Giventhesedevelopments,itseemsquiteappropriateandusefultoexaminecertain scaleinvarianceproperties[3]oftheclassicSAalgorithminlightofthenon-extensive formofentropyandtheGSA.Thesescaleinvariancepropertiesshedlightonanumber ofbehaviorsandillustratesomecuriouseffectsontheconfigurationspaceitself.Anen- ergytransformationfunctionisidentifiedandusedtoillustrateanumberofsymmetry propertiesandpotentialapplications.Inaddition,thebehaviorofparallelformsofSA, also exhibitscale invariancepropertiesandcan beused to illustratecertainanalogies inthebehavioroflargeensemblesofinteractingparticlesorsystems,abasicaspectof non-extensivestatisticalmechanicalsystems. Thisarticleisorganizedasfollows:Section2providessomebackgroundintonon- extensivethermodynamics,simulatedannealing,andSAsscaleinvarianceproperties. Section 3 develops the scale invariance properties of the non-extensive form of SA. These scale invariance properties indicate the existence of an energy transformation function. Section5describescertainasymptoticpropertiesofthisenergytransforma- tion function that suggest how it turns an exponential distribution into a power-law distribution. Section 6 providessome discussion on the implicationsof the scale in- variance properties and the energy landscape transformation describes extensions of thisworkanddirectionsforfutureresearchinvolvingMarkovChainMonteCarlosim- ulationofcomplexsystems. 2 Background 2.1 Non-Extensive Thermodynamics andGeneralized SA TheGSAdevelopedbyTsallis[12],entailsadifferentformfortheacceptanceproba- bilitiesandstationaryprobabilitiesbecauseoftheformof(1), i.e.,thestationarydis- tributionisbasedonmaximizingthevalueofS ratherthanS . Thelatterleadsto q BG thewell-knownBoltzmann-Gibbsdistribution e−fi/t π (t)= (4) i Z BG where ZBG = Wj=1e−fj/t is the normalizing Boltzmann Partition Function. Note thatbecausewewilloftenbereferringtotheSAalgorithmandtheMetropolisAlgo- P rithm,thevalueofthe“energy”functionwillbedenotedbyf whichistypicallyused i inoptimizationproblemstodenoteanobjectivefunctionvalue,butthiscanalsodenote someenergyvalue. 3 Maximizingthenon-extensiveentropysubjecttocertainconstraints,describedbe- low,givesrisetoastationaryprobabilitydistribution 1 1+ q−1 f 1−q p (t)= t i (5) i Z (cid:2) (cid:0) q(cid:1) (cid:3) 1 where Z = W 1+ q−1 f 1−q . Note that p (t) will be used throughout to q j=1 t j i representthestationaryprobabilityinthe non-extensivecaseandπ (t) willrepresent i P (cid:2) (cid:0) (cid:1) (cid:3) thestationaryprobabilityintheclassic,extensivecase. Thedefinitionofthestationaryprobabilityp (t)in(5)isbasedontheparticularset i ofconstraintsusedindefiningathermodynamicsystem.Threedistinctandnoteworthy sets of constraints have been studied all of which employ the standard normalizing constraint p (t) = 1. Whatdistinguishesthese constraintsets isthe relationships i i theydefinebetweenprobabilitiesandenergyorobjectivefunctionvalues. Tsallis[13] P reportsthatdifferentformsoftheseconstraintshaveanumberofimplicationsforthe non-extensiveformoftheprobabilityp (t). Here,theformdefinedbyTsallis’Type2 i constraint[14]forGSAisusedwhere, W pqf =U(2), aconstant, (6) i i i=1 X whereasin his originalpaper, the constraint p˜f = U was used. This later con- i i i straintwasreferredtoastheType1constraintin[13,p. 537]andleadstoadifferent P stationaryprobabilitydenotedherebyp˜. Itbearsemphasisthatthelimitin(3)holds i forallthedifferentstationaryprobabilitiesthatarisefromtheuseofthedifferentcon- straints [13]. Initially, the notions of scale invariant structures will be based on the Type2 constraintand thenlater, in a moreusefulcontext,the scale invarianceof the Type1constraintwillbeexamined. 2.2 ClassicalSA Scale Invariance Among the many interesting aspects of the SA and Metropolis algorithms is a scale invariancepropertyassociatedwiththestationaryprobabilitiesofvariousstates. This scale invariance property is manifest in the identical mathematical forms of certain quantitiesinvolvingindividualstates, aggregatedstates ofthesolutionspace, andthe aggregationof states associated with multiple processorsin an expandedstate space. See[3,4]foracompletedescription. Oneaspectofthisscaleinvarianceinvolvestheratechangeofthestationaryprob- abilityofastateiwithrespecttotemperaturet: ∂π (t) π (t) i = i [f −hfi(t)] (7) ∂t t2 i wherehfi(t)istheexpectedobjectivefunctionvalueattemperaturet. Foraggregated statesA={i ,i ,...,i },define 1 2 m π (t)f π (t)≡ π (t), f (t)≡ i∈A i i, (8) A i A π (t) i∈A P A X 4 theconditionalexpectationoftheobjectivefunctionvaluegiventhatthecurrentstate isinsetA(see[3,p.224]foramorecompletetreatment).Itthenfollowsthat ∂π (t) π (t) A = A [f (t)−hfi(t)] (9) ∂t t2 A wherethesimilarityof(7)and(9)indicatesaformofscaleinvariance. Scaleinvariancealsoextendstosecondmoments.ForstandardSA, ∂hfi(t) ∂f (t) σ2(t) = Ω = Ω (10) ∂t ∂t t2 where σ2(t) represents the variance over the entire solution space at temperature t. Ω Scaleinvarianceisexhibitedbythefactthat ∂f (t) σ2(t) A = A (11) ∂t t2 andσ2(t)isthevarianceofobjectivefunctionvaluesattemperaturetconditionedon A thecurrentstatebeinginsetA. See[3,p.232-33]fordetails. 3 SA Scale Invariance with Non-Extensive Entropy 3.1 The BasisofScaleInvariance Todemonstratescaleinvariancebasedonaggregatedstatesinthenon-extensivecase, a basisformakingcomparisonsmustbe established. To thatend,the followingtem- peraturederivativeofp (t)iscalculatedforthenon-extensiveSAcase. Fornotational i 1 convenience,simplicityandtoensurepositivity,letN (t)≡ 1+ q−1 f 1−q (here- i t i inafterwewilldropthe(t)fromN (t)tofurthersimplifytheexpressions)andtaking i (cid:2) (cid:0) (cid:1) (cid:3) thederivativeof(5), ∂p (t) Z ∂ N −N ∂ Z i = q∂t i i∂t q. (12) ∂t (Z )2 q Notethat ∂Ni = ∂ 1+ q−1 f 1−1q = fiNiq, (13) ∂t ∂t t i t2 (cid:20) (cid:18) (cid:19) (cid:21) Z = N ,and p (t)=N /Z andhence q j j i i q P ∂ ∂ ∂Nj fjNjq Z = N = = . ∂t q ∂t j ∂t t2 j j j X X X Substitutingthisand(13)into(12)yields N fiNiq − Ni f Nq ∂pi(t) j j t2 t2 j j j = ∂t (cid:16)P (cid:17)( jNj)2 P P 5 = pi(t)fiNiq−1 − pi(t) jfjNjq t2 t2 N Pj j = pi(t) f Nq−1− jPfjNjq (14) t2 i i N " P j j # Tofurthersimplifyandclarify,define P fˆ(f ,q,t)≡f Nq−1 = fi (15) i i i i 1+ q−1 f t i andwilloftenbedenotedsimplybyfˆ orfˆ(t)where(cid:0)itis(cid:1)understoodtoinvolvef ,q i i i andt. Substituting(15)into(14)andfurthersimplifyingyields ∂p (t) p (t) i = i fˆ(t)−hfˆi(t) (16) ∂t t2 i h i and which interestingly enough is analogousto (7). Notice that for q = 1, fˆ(t) = i fi, ∀t > 0. Tsallis[10,11]establishedthatlimq→1pi(t) = πi(t),hence,inthelimit asq → 1,(16)→(7). Thesymbolfˆ(t)willbereferredtoastheq-transformationof i f . Lateron,someinterestingpropertiesofthistransformationwillbeexamined. i 3.2 ScaleInvariancefrom theAggregationofStates Insimilarfashionasin[3],definep (t)= p (t)forA⊂ Ω,whereΩistheset A i∈A i ofmicrostatesandwhereW =|Ω|and P p (t)fˆ(t) fˆ (t)= i∈A i i , (17) A p (t) P A the conditional expectation of fˆ(t) conditioned on the current state being in set A. i Takingthederivative, ∂p (t) ∂ ∂p (t) A = p (t)= i . (18) i ∂t ∂t ∂t i∈A i∈A X X Substituting(16)and(17)into(18)yields ∂p (t) p (t) A = i fˆ(t)−hfˆi(t) ∂t t2 i Xi∈A h i p (t)fˆ(t) p (t)hfˆi(t) i i i = − t2 t2 i∈A i∈A X X p (t) p (t)fˆ(t) p (t)hfˆi(t) A i i A = − t2 p (t) t2 i∈A A X p (t) = A fˆ (t)−hfˆi(t) (19) t2 A h i 6 where for aggregatedstates, (19) has a similar mathematicalstructure as (16), hence exhibits a scale invariance property the foundation of which is based on the energy transformationfunctionfˆ(f ,q,t). i i 3.2.1 ScaleInvarianceinSecondMoments Scaleinvarianceinthe non-extensiveformofSA forsecondmomentsisindicatedin the following where again the q-transformation of f and f (t) is used. First, the i A parallelstotheclassiccaseisillustrated. Thus, ∂hfˆi(t) ∂fˆ (t) ∂ = Ω = p (t)fˆ(t) i i ∂t ∂t ∂t "i∈Ω # X ∂ = p (t)fˆ(t) i i ∂t Xi∈Ω h i ∂p (t) ∂fˆ(t) = i fˆ(t)+p (t) i . (20) i i ∂t ∂t i∈Ω" # X Substituting(16)intothefirstpartof(20)andsimplifyingyields ∂hfˆi(t) p (t)fˆ2(t) fˆ (t)p (t)fˆ(t) ∂fˆ(t) = i i − Ω i i + p (t) i . (21) ∂t t2 t2 i ∂t i∈Ω i∈Ω i∈Ω X X X Notingtheformofthefirsttwotermsin(21)andthefactthatinthethirdterm ∂fˆ(t) 2 q−1 i = fˆ(t) (22) ∂t i t2 (cid:16) (cid:17) (cid:18) (cid:19) and substituting into (21) and dropping the (t) for notational clarity and adding the symbolΩtodenoteexpectationsovertheentirestatespaceyields ∂hfˆi hfˆ2i −hfˆi2 +(q−1)hfˆ2i Ω = Ω Ω Ω ∂t t2 σˆ2 (q−1)hfˆ2i = Ω + Ω (23) t2 t2 whereσˆ2 representsthevarianceoftheq-transformedobjectivefunctionvaluesover Ω the entire objective function space (at temperature t). Obviously, this expression is slightlydifferentfromtheclassicSAcaseasindicatedin(10). Notethatlimq→1σˆΩ2 = σ2 sothattheexpressionsin(23)and(10)becomeequivalent. Ω The expression in (23) quite clearly shows the effects of the value of q—values greater (less) than 1 increase (decrease) the rate of change of the expected objective function values (q-transformedvalues). Eq. (23) also providesthe basis for another form of scale invariance. Thus, after going through similar steps as in (20) through (23)weget ∂fˆA = i∈Apifˆi2 − fˆA2 + q−1 i∈Apifˆi2. (24) ∂t t2p t2 t2 p P A (cid:18) (cid:19)P A 7 Noting that the first and third terms indicate conditionalexpectationsconditionedon thecurrentstatebeinginsetA,then(24)canbere-writtenintheconvenientnotation ∂fˆ hfˆ2i −hfˆi2 +(q−1)hfˆ2i A = A A A ∂t t2 σˆ2 (q−1)hfˆ2i = A + A (25) t2 t2 where (25) is clearly analogous to (23) and so exhibits a form of scale invariance. Again,inthelimitasq →1bothoftheseequationscorrespondtothescaleinvariance ofstandard(Boltzmann-Gibbs)SA as indicatedin (10) and(11). What isinteresting howeveristhefactthatthetermsinvolving(q−1)alsoscalewiththeaggregatedset Ainthenon-extensivecase. 3.3 ScaleInvariancefrom theAggregationofProcessors 3.3.1 TheClassicSACase Fleischer[5,6,7]describesanotherformofscaleinvariancebasedontheaggregation parallel and independent processes each running the SA algorithm independently of oneanother. Thistypeofaggregationcontrastssharplywiththeaggregationofstates and hence directly affects the nature of the scale invariance and how the algorithm itselffunctions. Inthisrespect,onemayviewtheSAalgorithmasasingleprocessor orparticlethatprobabilisticallyvisitsasinglestateateachiterationofthealgorithm. Thus,whenasetofstatesisaggregatedandfixed,theparticlevisitstheaggregatedstate probabilisticallyandinawaythatdictateshowthestationaryprobabilityandobjective function value should reasonably be defined, i.e., based on the sum of the stationary probabilitiesandthe conditionalexpectationof the objectivefunctionrespectivelyas givenin(8). In aggregatingp independentprocessorshowever, the state space Ω becomes Ωp andthestationaryprobabilityandobjectivefunctionvaluesassociatedwiththesetof processors must be defined differently and requires the following definitions. First, defineastateintheproductspacespannedbypprocessorsasi i ...i . InclassicSA 1 2 p scaleinvariance p π (t)= π (t) (26) i1i2...ip im m=1 Y andisbasedonthe conceptofthe jointprobabilityofindependentprocessors. Simi- larly,theobjectivefunctionassociatedwith asetofp independentparticlesissimply thesumofeachparticle’sobjectivefunctionvalueasopposedtotheconditionalexpec- tation,anddefinedby p f ≡ f . i1i2...ip im m=1 X Scaleinvarianceisindicatedby e−fi1i2...ip/t π (t)= i1i2...ip ··· e−fj1j2...jp/t jp j1 P P 8 (see[3,6])whichhasthesameformas(4)andthefactthatthetemperaturederivative isproportionaltothedifferencebetweenafunctionoftheobjectivefunctionvalueand itsexpectation: ∂π (t) π (t) i1i2...ip = i1i2...ip f −hfi (t) (27) ∂t t2 i1i2...ip Ωp (cid:2) (cid:3) [3,p.222]wheretheexpectationisoverallthestatesoftheproductspace. 3.3.2 TheNon-ExtensiveSACase Takingcuesfromthisearlierwork,definethestationaryprobabilityofstatei i ...i 1 2 p spannedbypindependentprocessorsandbasedontheTsallisentropyby p p (t)≡ p (t). i1i2...ip im m=1 Y Simplifyingusing the case of p = 2 (the following expressionsare readily extended to the more general case) and letting a = (q −1)/t for notational clarity, the joint probabilityofastatei i is 1 2 1 1 p (t) = [1+afi1]1−q[1+afi2]1−q i1i2 Z2 q 1 = [1+a(fi1 +fi2 +afi1fi2)]1−q Z2 q = [1+af˜i1i2(t)]1−1q (28) Z2 q wheref˜ (t)≡f +f +af f andwhereitisalsothecasethat i1i2 i1 i2 i1 i2 2 W 1 Zq2 = [1+afi]1−q ! i=1 X W W 1 = 1+af˜ (t) 1−q . i1i2 iX1=1iX2=1h i Thus, the form of p (t) in the productspace closely follows that of p (t) with the i1i2 i advantagethatthenon-extensivitypropertyisexemplifiedbyf˜ (t)whichissome- i1i2 whatsimilartotheentropyrelationshipsin(2). Notethatforq =1, f˜ (t)=f = i1i2 i1i2 f +f forallt.Thus,inthelimitasq →1,(28)isequivalenttotheBoltzmann-Gibbs i1 i2 casein(26)(recall(3)). Examiningthederivativeofthestationaryprobabilityusing(16)alsoreflectsascale invariancepropertyandyields ∂p (t) ∂ i1i2 = [p (t)p (t)] ∂t ∂t i1 i2 9 p (t) = p (t) i2 fˆ −hfˆi i1 t2 i2 Ω ph(t) i +p (t) i1 fˆ −hfˆi i2 t2 i1 Ω p (t) h i = i1i2 fˆ −hfˆi t2 i1i2 Ω2 h i andingeneralitfollowsthat ∂pi1∂··t·ip(t) = pi1·t··2ip(t) fˆi1···ip −hfˆiΩp (29) h i wherefˆi1···ip = pm=1fˆim andhfˆiΩp =phfˆiΩ andequalstheexpectationoffˆi1···ip. Note that (29) again has a factor equal to the difference between a function of the P objectivefunctionvaluesanditsexpectationandhasthesamestructureasin(7),(9), (16),(19)and(27). Finally,letusexaminethetemperaturederivativeofhfˆi andseehowitcompares Ωp to(23)and(25). Thus, ∂hfˆi ∂ ∂tΩp = ∂t pi1···ip(t)fˆi1···ip(t). i1X···ip Usingtheresultsfrom(20)through(22)andnotingthatfˆi1···ip(t)isasumofterms, ∂fˆi1···ip = q−1 p fˆ2 = q−1 fˆ2 ∂t t2 im t2 i1···ip (cid:18) (cid:19)m=1 (cid:18) (cid:19) X yields ∂hfˆi hfˆ2i −hfˆi2 +(q−1)hfˆ2i Ωp = Ωp Ωp Ωp ∂t t2 σˆ2 (q−1)hfˆ2i = Ωp + Ωp (30) t2 t2 where σˆΩ2p represents the variance of the sums fˆi1···ip(t) (again, bear in mind that the dependence on t is not indicated). Eq. (30) has the same form as (20) and (25) indicatingaformofscaleinvariance.Whatisremarkablehoweveristhatinaggregating processors,theobjectivefunctionisdefinedasthesumoftheobjectivefunctionvalues ofeachprocessorwhereasinaggregatingstates,theobjectivefunctionisdefinedasthe conditionalexpectationofobjectivefunctions,yettheyhavethesameformandinthe limitasq →1areequivalenttotheaggregationofprocessorsinclassicSA. 3.4 Consistency Fleischer [3] describes the concept of consistency in the scale invariant structure of SAbasedonthedefinitionoftheaggregatedobjectivefunctionf . Thisconsistency A 10

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.