Statistical mechanics: Entropy, Order parameters and complexity

298 Pages·2006·5.04 MB·English

by  James P. Sethna

Statistical Mechanics:
Entropy, Order Parameters
and Complexity
James P. Sethna, Physics, Cornell University, Ithaca, NY
(cid:1)cApril 19, 2005
B
b
Q
F
x
SYSTEM
Alcohol
E
Two-Phase Mix
δ
E+ E
Water Oil
Electronic version of text available at
http://www.physics.cornell.edu/sethna/StatMech/book.pdf
Contents
1 Why Study Statistical Mechanics? 3
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.1 Quantum Dice. . . . . . . . . . . . . . . . . . . . . 7
1.2 Probability Distributions. . . . . . . . . . . . . . . 8
1.3 Waiting times. . . . . . . . . . . . . . . . . . . . . 8
1.4 Stirling’s Approximation and Asymptotic Series. . 9
1.5 Random Matrix Theory. . . . . . . . . . . . . . . . 10
2 Random Walks and Emergent Properties 13
2.1 Random Walk Examples: Universality and Scale Invariance 13
2.2 The Diffusion Equation . . . . . . . . . . . . . . . . . . . 17
2.3 Currents and External Forces. . . . . . . . . . . . . . . . . 19
2.4 Solving the Diffusion Equation . . . . . . . . . . . . . . . 21
2.4.1 Fourier . . . . . . . . . . . . . . . . . . . . . . . . 21
2.4.2 Green . . . . . . . . . . . . . . . . . . . . . . . . . 22
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1 Random walks in Grade Space. . . . . . . . . . . . 24
2.2 Photon diffusion in the Sun. . . . . . . . . . . . . . 24
2.3 Ratchet and Molecular Motors. . . . . . . . . . . . 24
2.4 Solving Diffusion: Fourier and Green. . . . . . . . 26
2.5 Solving the Diffusion Equation. . . . . . . . . . . . 26
2.6 Frying Pan . . . . . . . . . . . . . . . . . . . . . . 26
2.7 Thermal Diffusion. . . . . . . . . . . . . . . . . . . 27
2.8 Polymers and Random Walks. . . . . . . . . . . . 27
3 Temperature and Equilibrium 29
3.1 The MicrocanonicalEnsemble . . . . . . . . . . . . . . . . 29
3.2 The MicrocanonicalIdeal Gas . . . . . . . . . . . . . . . . 31
3.2.1 Configuration Space . . . . . . . . . . . . . . . . . 32
3.2.2 Momentum Space . . . . . . . . . . . . . . . . . . 33
3.3 What is Temperature? . . . . . . . . . . . . . . . . . . . . 37
3.4 Pressure and Chemical Potential . . . . . . . . . . . . . . 40
3.5 Entropy, the Ideal Gas, and Phase Space Refinements . . 44
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.1 Escape Velocity. . . . . . . . . . . . . . . . . . . . 47
3.2 Temperature and Energy. . . . . . . . . . . . . . . 47
i
ii CONTENTS
3.3 Hard Sphere Gas . . . . . . . . . . . . . . . . . . . 47
3.4 Connecting Two Macroscopic Systems. . . . . . . . 47
3.5 Gauss and Poisson. . . . . . . . . . . . . . . . . . . 48
3.6 Microcanonical Thermodynamics . . . . . . . . . . 49
3.7 Microcanonical Energy Fluctuations. . . . . . . . . 50
4 Phase Space Dynamics and Ergodicity 51
4.1 Liouville’s Theorem . . . . . . . . . . . . . . . . . . . . . 51
4.2 Ergodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.1 The Damped Pendulum vs. Liouville’s Theorem. . 58
4.2 Jupiter! and the KAM Theorem . . . . . . . . . . 58
4.3 Invariant Measures. . . . . . . . . . . . . . . . . . 60
5 Entropy 63
5.1 Entropy as Irreversibility: Engines and Heat Death . . . . 63
5.2 Entropy as Disorder . . . . . . . . . . . . . . . . . . . . . 67
5.2.1 Mixing: Maxwell’s Demon and Osmotic Pressure . 67
5.2.2 Residual Entropy of Glasses: The Roads Not Taken 69
5.3 Entropy as Ignorance: Information and Memory . . . . . 71
5.3.1 Nonequilibrium Entropy . . . . . . . . . . . . . . . 72
5.3.2 Information Entropy . . . . . . . . . . . . . . . . . 73
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.1 Life and the Heat Death of the Universe. . . . . . 77
5.2 P-V Diagram. . . . . . . . . . . . . . . . . . . . . . 77
5.3 Carnot Refrigerator. . . . . . . . . . . . . . . . . . 78
5.4 Does Entropy Increase? . . . . . . . . . . . . . . . 78
5.5 Entropy Increases: Diffusion. . . . . . . . . . . . . 80
5.6 Information entropy. . . . . . . . . . . . . . . . . . 80
5.7 Shannon entropy. . . . . . . . . . . . . . . . . . . . 80
5.8 Entropy of Glasses. . . . . . . . . . . . . . . . . . . 81
5.9 Rubber Band.. . . . . . . . . . . . . . . . . . . . . 82
5.10 Deriving Entropy. . . . . . . . . . . . . . . . . . . 83
5.11 Chaos, Lyapunov, and Entropy Increase.. . . . . . 84
5.12 Black Hole Thermodynamics. . . . . . . . . . . . . 84
5.13 Fractal Dimensions. . . . . . . . . . . . . . . . . . 85
6 Free Energies 87
6.1 The Canonical Ensemble . . . . . . . . . . . . . . . . . . . 88
6.2 Uncoupled Systems and Canonical Ensembles . . . . . . . 92
6.3 Grand Canonical Ensemble . . . . . . . . . . . . . . . . . 95
6.4 What is Thermodynamics? . . . . . . . . . . . . . . . . . 96
6.5 Mechanics: Friction and Fluctuations. . . . . . . . . . . . 100
6.6 Chemical Equilibrium and Reaction Rates . . . . . . . . . 101
6.7 Free Energy Density for the Ideal Gas . . . . . . . . . . . 104
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
6.1 Two–state system. . . . . . . . . . . . . . . . . . . 107
6.2 Barrier Crossing. . . . . . . . . . . . . . . . . . . . 107
Tobepub. OxfordUP,∼Fall’05 www.physics.cornell.edu/sethna/StatMech/
CONTENTS iii
6.3 Statistical Mechanics and Statistics. . . . . . . . . 108
6.4 Euler, Gibbs-Duhem, and Clausius-Clapeyron. . . 109
6.5 Negative Temperature. . . . . . . . . . . . . . . . . 110
6.6 Laplace. . . . . . . . . . . . . . . . . . . . . . . . . 110
6.7 Lagrange. . . . . . . . . . . . . . . . . . . . . . . . 111
6.8 Legendre. . . . . . . . . . . . . . . . . . . . . . . . 111
6.9 Molecular Motors: Which Free Energy? . . . . . . 111
6.10 Michaelis-Menten and Hill . . . . . . . . . . . . . . 112
6.11 Pollen and Hard Squares. . . . . . . . . . . . . . . 113
7 Quantum Statistical Mechanics 115
7.1 Mixed States and Density Matrices . . . . . . . . . . . . . 115
7.2 Quantum Harmonic Oscillator. . . . . . . . . . . . . . . . 120
7.3 Bose and Fermi Statistics . . . . . . . . . . . . . . . . . . 120
7.4 Non-Interacting Bosons and Fermions . . . . . . . . . . . 121
7.5 Maxwell-Boltzmann “Quantum” Statistics . . . . . . . . . 125
7.6 Black Body Radiation and Bose Condensation . . . . . . 127
7.6.1 Free Particles in a Periodic Box . . . . . . . . . . . 127
7.6.2 Black Body Radiation . . . . . . . . . . . . . . . . 128
7.6.3 Bose Condensation . . . . . . . . . . . . . . . . . . 129
7.7 Metals and the Fermi Gas . . . . . . . . . . . . . . . . . . 131
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.1 Phase Space Units and the Zero of Entropy. . . . . 133
7.2 Does Entropy Increase in Quantum Systems? . . . 133
7.3 Phonons on a String. . . . . . . . . . . . . . . . . . 134
7.4 Crystal Defects. . . . . . . . . . . . . . . . . . . . 134
7.5 Density Matrices. . . . . . . . . . . . . . . . . . . . 134
7.6 Ensembles and Statistics: 3 Particles, 2 Levels. . . 135
7.7 Bosons are Gregarious: Superfluids and Lasers . . 135
7.8 Einstein’s A and B . . . . . . . . . . . . . . . . . . 136
7.9 Phonons and Photons are Bosons. . . . . . . . . . 137
7.10 Bose Condensation in a Band. . . . . . . . . . . . 138
7.11 Bose Condensation in a Parabolic Potential. . . . . 138
7.12 Light Emission and Absorption.. . . . . . . . . . . 139
7.13 Fermions in Semiconductors. . . . . . . . . . . . . 140
7.14 White Dwarves, Neutron Stars, and Black Holes. . 141
8 Calculation and Computation 143
8.1 What is a Phase? Perturbationtheory. . . . . . . . . . . . 143
8.2 The Ising Model . . . . . . . . . . . . . . . . . . . . . . . 146
8.2.1 Magnetism . . . . . . . . . . . . . . . . . . . . . . 146
8.2.2 Binary Alloys . . . . . . . . . . . . . . . . . . . . . 147
8.2.3 Lattice Gas and the Critical Point . . . . . . . . . 148
8.2.4 How to Solve the Ising Model. . . . . . . . . . . . 149
8.3 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . 150
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
8.1 The Ising Model. . . . . . . . . . . . . . . . . . . . 154
8.2 Coin Flips and Markov Chains. . . . . . . . . . . . 155
(cid:2)cJamesP.Sethna,April19,2005 Entropy,OrderParameters,andComplexity
iv CONTENTS
8.3 Red and Green Bacteria . . . . . . . . . . . . . . . 155
8.4 Detailed Balance. . . . . . . . . . . . . . . . . . . . 156
8.5 Heat Bath, Metropolis, and Wolff. . . . . . . . . . 156
8.6 Stochastic Cells. . . . . . . . . . . . . . . . . . . . 157
8.7 The Repressilator. . . . . . . . . . . . . . . . . . . 159
8.8 Entropy Increases! Markov chains. . . . . . . . . . 161
8.9 Solving ODE’s: The Pendulum . . . . . . . . . . . 162
8.10 Small World Networks. . . . . . . . . . . . . . . . 165
8.11 Building a PercolationNetwork. . . . . . . . . . . 167
8.12 Hysteresis Model: Computational Methods. . . . . 169
9 Order Parameters, Broken Symmetry, and Topology 171
9.1 Identify the Broken Symmetry . . . . . . . . . . . . . . . 172
9.2 Define the Order Parameter . . . . . . . . . . . . . . . . . 172
9.3 Examine the Elementary Excitations . . . . . . . . . . . . 176
9.4 Classify the Topological Defects . . . . . . . . . . . . . . . 178
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
9.1 Topological Defects in the XY Model. . . . . . . . 183
9.2 Topological Defects in Nematic Liquid Crystals. . 184
9.3 Defect Energetics and Total Divergence Terms. . . 184
9.4 Superfluid Order and Vortices. . . . . . . . . . . . 184
9.5 Landau Theory for the Ising model. . . . . . . . . 186
9.6 Bloch walls in Magnets. . . . . . . . . . . . . . . . 190
9.7 Superfluids: Density Matrices and ODLRO. . . . . 190
10 Correlations, Response, and Dissipation 195
10.1 Correlation Functions: Motivation . . . . . . . . . . . . . 195
10.2 Experimental Probes of Correlations . . . . . . . . . . . . 197
10.3 Equal–Time Correlations in the Ideal Gas . . . . . . . . . 198
10.4 Onsager’s RegressionHypothesis and Time Correlations . 200
10.5 Susceptibility and the Fluctuation–Dissipation Theorem . 203
10.5.1 Dissipation and the imaginary part χ(cid:1)(cid:1)(ω) . . . . . 204
10.5.2 Static susceptibility χ(cid:1) (k) . . . . . . . . . . . . . . 205
0
10.5.3 χ(r,t) and Fluctuation–Dissipation . . . . . . . . . 207
10.6 Causality and Kramers Kro¨nig . . . . . . . . . . . . . . . 210
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
10.1 Fluctuations in Damped Oscillators. . . . . . . . . 212
10.2 Telegraph Noise and RNA Unfolding. . . . . . . . 213
10.3 Telegraph Noise in Nanojunctions. . . . . . . . . . 214
10.4 Coarse-GrainedMagnetic Dynamics. . . . . . . . . 214
10.5 Noise and Langevin equations. . . . . . . . . . . . 216
10.6 Fluctuations, Correlations, and Response: Ising . . 216
10.7 Spin Correlation Functions and Susceptibilities. . . 217
11 Abrupt Phase Transitions 219
11.1 Maxwell Construction. . . . . . . . . . . . . . . . . . . . . 220
11.2 Nucleation: Critical Droplet Theory. . . . . . . . . . . . . 221
11.3 Morphology of abrupt transitions. . . . . . . . . . . . . . 223
Tobepub. OxfordUP,∼Fall’05 www.physics.cornell.edu/sethna/StatMech/
CONTENTS 1
11.3.1 Coarsening. . . . . . . . . . . . . . . . . . . . . . . 223
11.3.2 Martensites.. . . . . . . . . . . . . . . . . . . . . . 227
11.3.3 Dendritic Growth. . . . . . . . . . . . . . . . . . . 227
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
11.1 van der Waals Water. . . . . . . . . . . . . . . . . 228
11.2 Nucleation in the Ising Model. . . . . . . . . . . . 229
11.3 Coarsening and Criticality in the Ising Model.. . . 230
11.4 Nucleation of Dislocation Pairs. . . . . . . . . . . . 231
11.5 OragamiMicrostructure.. . . . . . . . . . . . . . . 232
11.6 Minimizing Sequences and Microstructure.. . . . . 234
12 Continuous Transitions 237
12.1 Universality. . . . . . . . . . . . . . . . . . . . . . . . . . . 239
12.2 Scale Invariance. . . . . . . . . . . . . . . . . . . . . . . . 246
12.3 Examples of Critical Points. . . . . . . . . . . . . . . . . . 253
12.3.1 Traditional Equilibrium Criticality: Energy versus Entropy.253
12.3.2 Quantum Criticality: Zero-point fluctuations versus energy.253
12.3.3 Glassy Systems: Random but Frozen. . . . . . . . 254
12.3.4 Dynamical Systems and the Onset of Chaos. . . . 256
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
12.1 Scaling: Critical Points and Coarsening. . . . . . . 257
12.2 RG Trajectories and Scaling. . . . . . . . . . . . . 257
12.3 Bifurcation Theory and Phase Transitions. . . . . 257
12.4 Onset of Lasing as a Critical Point.. . . . . . . . . 259
12.5 Superconductivity and the Renormalization Group.260
12.6 RG and the Central Limit Theorem: Short. . . . . 262
12.7 RG and the Central Limit Theorem: Long. . . . . 262
12.8 Period Doubling. . . . . . . . . . . . . . . . . . . . 264
12.9 Percolationand Universality. . . . . . . . . . . . . 267
12.10 Hysteresis Model: Scaling and Exponent Equalities.269
A Appendix: Fourier Methods 273
A.1 Fourier Conventions . . . . . . . . . . . . . . . . . . . . . 274
A.2 Derivatives, Convolutions, and Correlations . . . . . . . . 276
A.3 Fourier Methods and Function Space . . . . . . . . . . . . 277
A.4 Fourier and Translational Symmetry . . . . . . . . . . . . 279
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
A.1 Fourier for a Waveform. . . . . . . . . . . . . . . . 281
A.2 Relations between the Fouriers. . . . . . . . . . . . 281
A.3 Fourier Series: Computation. . . . . . . . . . . . . 281
A.4 Fourier Series of a Sinusoid. . . . . . . . . . . . . . 282
A.5 Fourier Transforms and Gaussians: Computation. 282
A.6 Uncertainty. . . . . . . . . . . . . . . . . . . . . . . 284
A.7 White Noise. . . . . . . . . . . . . . . . . . . . . . 284
A.8 Fourier Matching. . . . . . . . . . . . . . . . . . . 284
A.9 Fourier Series and Gibbs Phenomenon.. . . . . . . 284
(cid:2)cJamesP.Sethna,April19,2005 Entropy,OrderParameters,andComplexity
2 CONTENTS
Tobepub. OxfordUP,∼Fall’05 www.physics.cornell.edu/sethna/StatMech/
Why Study Statistical
Mechanics?
1
Manysystemsinnaturearefartoocomplextoanalyzedirectly. Solving
for the motion of all the atoms in a block of ice – or the boulders in
an earthquake fault, or the nodes on the Internet – is simply infeasible.
Despite this, suchsystems oftenshow simple, strikingbehavior. We use
statisticalmechanicstoexplainthesimplebehaviorofcomplexsystems.
Statisticalmechanicsbringstogetherconceptsandmethodsthatinfil-
trate many fields of science, engineering, and mathematics. Ensembles,
entropy, phases, Monte Carlo, emergent laws, and criticality – all are
concepts and methods rootedin the physics and chemistry of gases and
liquids, but have become important in mathematics, biology, and com-
puter science. In turn, these broaderapplications bring perspective and
insight to our fields.
Let’s start by briefly introducing these pervasive concepts and meth-
ods.
Ensembles: Thetrickofstatisticalmechanicsisnottostudyasingle
system, but a large collection or ensemble of systems. Where under-
standing a single system is often impossible, calculating the behavior of
anenormouscollectionofsimilarlypreparedsystemsoftenallowsoneto
answer most questions that science can be expected to address.
Forexample,considertherandomwalk(figure1.1). (Youmightimag-
ine it as the trajectory of a particle in a gas, or the configuration of a
polymer in solution.) While the motion of any given walk is irregular
(left) and hard to predict, simple laws describe the distribution of mo-
tions of an infinite ensemble of random walks starting from the same
initial point (right). Introducing and deriving these ensembles are the
themes of chapters 3, 4, and 6.
Entropy: Entropyis themostinfluentialconceptarisingfromstatis-
ticalmechanics(chapter5). Entropy,originallyunderstoodasathermo-
dynamic property of heat engines that could only increase, has become
science’sfundamentalmeasureofdisorderandinformation. Althoughit
controlsthe behaviorof particularsystems,entropy canonly be defined
within a statistical ensemble: it is the child of statistical mechanics,
with no correspondence in the underlying microscopic dynamics. En-
tropy now underlies our understanding of everything from compression
algorithms for pictures on the Web to the heat death expected at the
end of the universe.
Phases. Statisticalmechanicsexplainstheexistenceandpropertiesof
3
4 Why Study Statistical Mechanics?
Fig. 1.1 Random Walks. The motion of molecules in a gas, or bacteria in a
liquid,orphotons inthe Sun, isdescribedbyan irregulartrajectory whosevelocity
rapidly changes in direction at random. Describing the specific trajectory of any
givenrandomwalk(left)isnotfeasibleoreveninteresting. Describingthestatistical
average propertiesof alargenumber of randomwalks isstraightforward; atrightis
shownthe endpoints ofrandom walksall startingatthe center. Thedeep principle
underlyingstatisticalmechanicsisthatitisofteneasiertounderstandthebehavior
ofensemblesofsystems.
phases. The three common phases of matter (solids, liquids, and gases)
have multiplied into hundreds: from superfluids and liquid crystals, to
vacuum states of the universe just after the Big Bang, to the pinned
and sliding ‘phases’ of earthquake faults. Phases have an integrity or
stability to small changes in external conditions or composition1 – with
deep connections to perturbation theory, section 8.1. Phases often have
a rigidity or stiffness, which is usually associated with a spontaneously
broken symmetry. Understanding what phases are and how to describe
their properties, excitations, and topological defects will be the themes
2Chapter 7 focuses on quantum sta- of chapters 7,2 and 9.
tistical mechanics: quantum statistics, Computational Methods: Monte–Carlo methods use simple rules
metals, insulators, superfluids, Bose
toallowthecomputertofindensembleaveragesinsystemsfartoocom-
condensation, ...Tokeepthepresenta-
plicatedtoallowanalyticalevaluation. Thesetools,inventedandsharp-
tionaccessibletoabroadaudience,the
rest of the text is not dependent upon ened in statistical mechanics, are used everywhere in science and tech-
knowingquantum mechanics. nology – from simulating the innards of particle accelerators,to studies
oftrafficflow,todesigningcomputercircuits. Inchapter8,weintroduce
the Markov–chainmathematics that underlies Monte–Carlo.
Emergent Laws. Statistical mechanics allows us to derive the new
1Water remainsaliquid,withonlyperturbativechanges initsproperties,asone
changes the temperature or adds alcohol. Indeed, it is likely that all liquids are
connected to one another, and indeed to the gas phase, through paths inthe space
ofcompositionandexternalconditions.
Tobepub. OxfordUP,∼Fall’05 www.physics.cornell.edu/sethna/StatMech/
5
Fig. 1.2 Temperature: the Ising
model at the critical temperature.
Traditional statistical mechanics fo-
cusesonunderstandingphasesofmat-
ter, and transitions between phases.
These phases – solids, liquids, mag-
nets,superfluids–areemergentprop-
erties of many interacting molecules,
spins, or other degrees of free-
dom. Pictured here is a simple
two-dimensional model at its mag-
netic transition temperature Tc. At
higher temperatures, the system is
non-magnetic: the magnetization is
on average zero. At the temperature
shown, the system is just deciding
whethertomagnetizeupward(white)
or downward (black). While predict-
ing the time dependence of all these
degrees of freedom is not practical or
possible, calculating the average be-
haviorofmanysuchsystems(astatis-
ticalensemble)isthejobofstatistical
mechanics.
lawsthatemergefromthecomplexmicroscopicbehavior. Theselawsbe-
comeexactonlyincertainlimits. Thermodynamics–the studyofheat,
temperature,andentropy– becomesexactinthe limit oflargenumbers
ofparticles. Scalingbehaviorandpowerlaws–bothatphasetransitions
and more broadly in complex systems – emerge for large systems tuned
(or self–organized) near critical points. The right figure 1.1 illustrates
the simple law (the diffusion equation) that describes the evolution of
the end-to-end lengths of random walks in the limit where the number
ofsteps becomes large. Developing the machineryto express andderive
these new laws are the themes of chapters 9 (phases), and 12 (critical
points). Chapter 10 systematically studies the fluctuations about these
emergenttheories,andhowtheyrelatetotheresponsetoexternalforces.
Phase Transitions. Beautiful spatial patterns arise in statistical
mechanics at the transitions between phases. Most of these are abrupt
phase transitions: ice is crystalline andsolid until abruptly (atthe edge
of the ice cube) it becomes unambiguously liquid. We study nucleation
andtheexoticstructuresthatevolveatabruptphasetransitionsinchap-
ter 11.
Other phase transitions are continuous. Figure 1.2 shows a snapshot
of the Ising model at its phase transition temperature T . The Ising
c
modelis a lattice ofsites that cantake one of two states. Itis usedas a
simple model for magnets (spins pointing up or down), two component
crystallinealloys(AorBatoms),ortransitionsbetweenliquidsandgases
(occupied and unoccupied sites).3 All of these systems, at their critical
3The Ising model has more far-flung applications: the three–dimensional Ising
modelhasbeenusefulinthestudyofquantum gravity.
(cid:2)cJamesP.Sethna,April19,2005 Entropy,OrderParameters,andComplexity