Chapter 10. Postulates of statistical mechanics

Thermodynamics puts constraints on the behavior of macroscopic systemswithout referencing the underlying microscopic properties. In particular, it does notprovide a quantitative connection to the origin of its fundamental quantities U and S.For U, this is less of a problem because we know from mechanics that

U =12

mivi2

i∑ +V (xi ) ,

and the macroscopic formula arises by integrating over most coordinates andvelocities. Somehow the thermal motions end up as TS, and the mechanical andelectrical motions end up as terms such as –PV+µn.

Statistical mechanics makes the macro-micro connection and provides aquantitative description of U and S is terms of microscopic quantities. For largesystems (except near the critical point), its results are in agreement withthermodynamics: one can derive thermodynamic postulates 0 – 3 from statisticalmechanics. For systems undergoing large fluctuations (small systems or thosesystems near a critical point), its prediction are different and more accurate.

In addition as the ‘mechanics’ implies, statistical mechanics can deal with time-varying systems and systems out of equilibrium. Averages over x(t) and p(t)=mv(t)of the microscopic particles are done, but not in such a way that all time-dependentinformation is lost, as in thermodynamics.

Unlike mechanics, statistical mechanics is not intended to discuss the time-dependence of an isolated particle. Rather, the time-dependent (e.g. diffusioncoefficient) and time independent properties of whole systems of particles, and theaveraged properties of whole ensembles of such systems, are of interest.

We begin with an introduction to important facts from mechanics and statistics,then proceed to the postulates of statistical mechanics, consider in detail equilibriumsystems, and finally non-equilibrium systems.

Goal of statistical mechanics:- have a system of many particles with positions xi and velocities xi (or

wavefunctions Ψ(xi ) .- want values A(t) of any observable A(xi , xi ) as particles move about,

averaged over all microstates of the system consistent with constraints,such as energy=U=constant, or V=constant.

Goal of thermodynamics: find relations among extensive observables X and theirderivatives at equilibrium only.

The postulates of statistical mechanics and connection to thermodynamics:

Postulate I: Extension of microscopic lawsHamiltonian dynamics applies to the density operator ρi of any finite closed system;fully specified by its extensive constrainnt parameters and the Hamiltonian.

Postulate II: Principle of equal probabilitiesThe principle of equal probabilities holds in its ensemble (weak) form and is assumedin it strong (time) form

i) weak form: all W microscopic realizations of a system satisfying Ihave equal probability. The ensemble density matrix is therefore

given by ρ =1W

ρii=1

W

∑ . The ensemble of these W systems is the

microcanonical ensemble.

ii) strong form: for any ensemble satisfying i) at equilibrium,ρi t = ρi e = ρ (ergodic principle). This states that averaging over

time is equivalent to averaging over the ensemble of W microstates.

Postulate III: entropyThe entropy of a n ensemble of systems satisfying postuilates I and II.i) is given by

S = −Tr ρ ln ρ{ }

Before we use them, these postulates require some explanation.

I) This is a strong statement; the system i usually has a >1020-dimensional phasespace, and we assume that the dynamics are the same as for a few degrees offreedom! Classically, ρi corresponds to a specific trajectory; quantummechanically, to a specific initial condition of the system. Among the extensivevariables fixed in a closed finite system: U (always by postulate I). Otherconstrained variables: V (or L, A, Ni=particle number … depending on thesystem).

Note that if H is independent of time, the system is closed, and U is thereforeconstant (as it needs to be for ‘full’ specification of the system), P1 ofthermodynamics is automatically satisfied.

II) This is the postulate that lets us perform macroscopic averages over theindividual density matrices, so we can derive properties for the energy-conserving (microcanonical) ensemble.

i. Classically, this says that as long as a trajectory stratifies the constraints in I(has specific energy U), we can combine it with equal weight with all other suchtrajectories to obtain ρ(qi , pi ,t) , the classical density function.

Quantum-mechanically, this means that all the linearly independent pure densitymatrices ρi characterizing a system with the same extensive parameters (i.e. allthe members of the microcanonical ensemble) can be averaged with equalweights to obtain the ensemble density matrix.

Example: consider a state of energy U that can be realized in W ways (W-folddegenerate or W microstates). One set of initial conditions ρi would be

ρ1 =1 0 00 0 00 0 0

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟ , ρ2 =

0 0 00 1 00 0 0

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟ , ...

These are pure states. All of these are equally likely because they have thesame energy (and volume, etc.), so

ρ =1W

ρii=1

W

∑ =1 /W 0

0 1 /W

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟

This is a ‘mixed’ sate of constant energy U.Note that there is a potentially embarrassing problem with this: a finite

quantum system (e.g. particle in a box) for which all extensive parameters (e.g.U, or L for particles in a 1-D box) have been specified has as discrete energyspectrum given by H ϕ i = Ei ϕ i . For a large system, the level spacing may

be very narrow, but it is nonetheless discrete. Thus, at some every U we pick,there is likely to be no state, so we have nothing to average!

In practice, this is resolved by having an energy window δU , and byconsidering all W levels within it. As discussed in more detail in III below, asthe number of degrees of freedom N=6n of the system approaches infinity, thesize of δU rigorously has no effect on the result.

ii. This says we could take a single trajectory, or a single initial condition ρi (0) ,propagate it in time, and all the possible microscopic states will also be visitedin turn to yield again ρ(qi , pi ,t) (classically) or ρ(t) (quantum mechanically).This is a much stronger statement than i): the full ensemble of W microstates bydefinition includes all realizations i of the macroscopic system compatible withH and the constraints; on the other hand ii) says a single microstate will, in time,evolve to explore all the others, or at least come arbitrarily close to them. Thisproperty is know as ‘ergodicity.’ In practice, ergodicity cannot really besatisfied, but we can use ii) for ‘all practical purposes.’

Example showing why ergodicity cannot be satisfied:

We will use a discrete system to illustrate. Consider a box with M =VV0

cells,

filled with N M particles of volume V0 . The dynamics is that the particleshop randomly to unoccupied neighboring cells at each time step Δt . This

model is called a lattice-ideal gas. The number of arrangements for N identical

particles is W =M !

(M − N )!⋅1N !

Large factorials n! (or gamma functions Γ(n −1) =N!) can be approximated byStirling’s formula

n! ~ nne−n .Thus,

W =M !

(M − N )!⋅1N !

≈ MM (M − N )N −M N −N ≈MN

⎛⎝⎜

⎞⎠⎟N

.

Let us plug realistic numbers into this:

V0 = 10 A, V = 1 cm3 ⇒ M =

VV0

= 1023.

For N = 1019 gas molecules (~1 atm) ⇒ MN

= 104

vgas ≈U0

m⎛⎝⎜

⎞⎠⎟

1/2

≈ 300 m / s (O2 at R.T.) ⇒ Δt ~ L0

V−V0

1/3

v~ 10−12 s = 1 ps

Wpossible ~ 1018 s10−12 s

~ 1030

Lifetime of universe: <1011 a ~ 1018 s

Wactual ≈ (104 )1019= 1 boogol 1030

The possible W that can be visited during the lifetime of the universe is a mere1030, negligible compared to the actual number of microstates W actual at constantenergy.

Clearly, not even a warm gas, a system about as random as conceivable, eventouches the microcanonical degeneracy W. Although the a priori probability ofmicrostates (classically: of trajectories) may be the same (i), they simply cannotall be sampled in finite time.* as seen in III, this provides a practical solution tothe quantum-dilemma outlined in I.

Why assume ii) at all? In real life ρi (t) is always observed, but it is difficult to

compute. W or ρ(t) are often much easier to compute. Although ii) fails by afactor boogol, surprisingly it sill works in most situations: most microstates inthe ensemble of Wpossible microstates are indistinguishable (e.g. the gas atoms inthe room rigth now vs. 10 seconds from now), so leaving many of them out ofthe average still yields the same average; sampling only one in 10-27 boogol stillgives the same result as true ensemble averaging.

There are cases where this reasoning fails: in glasses, members of the ensemblecan be so slowly interconnecting and so different from one another, that ρ is not

at all like ρi t unless very special care is taken.

III) This definition of the entropy was made plausible in our mathematical review,on grounds of information content: a system with many microstates has agreater potential for disorder than a system of a few microstates. But instead ofmeasuring disorder multiplicatively, we want an additive (extensive) quantity.This postulate proves the microscopic definition for thermodynamic entropy( ρ = ρeq & N →∞ ), just as energy is microscopically defined as

U = H e where H =12

pj2

mjj∑ +V (xi )

soSeq ≡ S = −kBTr ρeq ln ρeq{ }

gives the thermodynamic entropy S in terms of the equilibrium density matrix.

We must have Tr ρeq{ } = 1, [ρeq ,H ] = 0 , and by postulate II.i), all elements of

ρeq must be of equal size if we are in the microcanonical (constant energy U)

ensemble. This is satisfied only by

ρeq =

1W 0

0 1W

⎛

⎝

⎜⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟⎟

Where ρ is a diagonal W×W matrix. Inserting into S and evaluating the trace in

the eigenfunction basis of H (and ρ ), which we can call j :

S = −kB j ρeq ln ρeq jj=1

W

∑ = −kB1W

ln 1Wj=1

W

∑⇒ S = kB lnW where kB ≈ 1.38 ⋅10−23 J / K is Boltzmann's constant.

This is Boltzmann’s famous formula for the entropy. Postulate III is moregeneral, but at equilibrium Boltzmann’s formula holds. It secures for S all theproperties in postulates 2 and 3 of thermodynamics, and provides a microscopicinterpretation for S:

W specifies disorder in a system: the more possible microstates correspond tothe same macrostate, the more disorder a system has. For two independentsystems, Wtot =W1 ⋅W2 . However, thermodynamic entropy has the property of

additivity: Stot = S1 + S2 . The function that uniquely effects the transformationfrom multiplication to addition is the log function (within a constant factor)⇒ Si = lnWi must be true so that both relations at the beginning of this

paragraph are satisfied. The constant factor kB is provided to match the energyand temperature scales, which were independently defined in the early 19th

century when the equivalence of temperature and average energy was notunderstood.

Consider a system divided into subsystems by constraints, with W=W0. Whenthe constraints are removed at t=0, then by II.ii) the system now exploresadditional ensemble members as time goes on. Thus W(t > 0) = W1 > W0. Ifmacroscopic equilibrium is reached Weq >W1 >W0 ⇒ Seq > S0 .

Thus S in stat mech postulate III satisfies all requirements of postulate P2 ofthermodynamics, which is a simple consequence of the fact that microscopicdegrees of freedom tend to explore all available states (= all the available phasespace in classical mechanics).

Also, because W is monotonic in U (at higher energy U, there are always morequantum states in a multidimensional system) and because S is monotonic in W(property of the ln function), S is monotonic in U. finally, we shall see in detail

later that when ∂U∂S

⎛⎝⎜

⎞⎠⎟V ,N

= T , only the ground state is populated, so

W → 1⇒ limT→0

S = kB ln(1) = 0 . Thus, the third postulate is also satisfied as long

as the ground state is singly degenerate and the system can get to it during theexperiment. (Glasses again would be a problem here!)

The error of thermodynamics: it identifies the most probable value of a quantitywith its average, by assuming the spread is negligible. We will derive examplesof this spread later on. Thermodynamic limit: N goes to infinity but N/V or anyother ratio of extensive quantities remains constant.

To conclude this chapter, we turn to the problem of computing W in the quantumcase. A closed finite quantum system has a discrete spectrum Ei. The figure belowshows the number of states below energy U as a function of U. Because of quantummechanics, the density of states

Ω(U ) = ∂ Ω

∂Uis discontinuous, and the integrated density of states (= total number of states up toenergy U) has steps in it:

Ω = Step(U − Ej )j∑ ⇒Ω = δ (U − Ej )

j∑

At any randomly picked U, Ω is mostly likely zero, so W = 0 also!However, because Ω increases so enormously rapidly with energy, the states are

very (understatement!) closely spaced in energy for any system with even just a fewparticles. If a system is observed for a finite time δt , the states are broadened by theuncertainty principle:

δE ~

2δt⇒Ω = Li (U − Ei ,δt)

i∑

where L indicates a broadened profile of finite width that replaces the delta function.

L still counts a single state, so dU Li (U − Ei ,δt) = 1; often Li0

∞

∫ is taken as a Lorentzian

Li =1π

δU(U − Ei )

2 + δU 2 . Thus, Ω can be taken as a smooth function and its value

Ω(E,δt) W (U ) tells us how many states contribute to the degeneracy at energy U.

a

Ω

U

....

Ω

U

....

U

Ωi(U,δt)

δU ≈ /δtobs

∫ Li dU = 1

....U

Ω(U,δt) ≈ W....

δU

Ω(U)δU≈

Ω(U,δt)

Figure 10.1 The density of states Ω quantum-mechanically is a sum of delta functions

located at the energies Ei of the quantum system. Integrating it, we obtain Ω , thesum of all states between energy U=0 up to U. Because of the uncertainty principle,we can never define the energies perfectly, as shown by the ‘broadened’ version of Ωbelow. Integrating this, we obtain a smoothed version of Ω ≈ W to extremely high

accuracy. Note that Ω rises so fast (the figure does not do justice), that the numberof states contained in an tiny energy interval at energy U, of width given by theuncertainty principle, is essentially the same as the number of all states up to thatenergy.

It is clear from the above that if δU | Ej − Ei |, (the broadening is greater than

the spacing of adjacent levels), then Ω (U) is indeed independent of the choice ofδU or δt . This is guaranteed by the astronomical number of states for a macroscopicsystem (see example in II.ii)). Because Ω in the above figure grows so fast,Ω (U)δU ≈ Ω (U), as illustrated in the bottom right panel of the figure

Another way to look at it is in state space (classically: action space). It has Ncoordinates for N degrees of freedom. Ω is the number of states under the surface U= constant. If U U0 (where U0 is the average characteristic energy step for onedegree of freedom) then

Ω1 ~UU0

⎛⎝⎜

⎞⎠⎟

N

Letting δU now be an uncertainty in U instead of in individual energy levels, thenumber of states in the interval (U − δU,U ) is

Ω(U ) − Ω(U − δU ) ~ UU0

⎛⎝⎜

⎞⎠⎟

N

−U − δUU0

⎛⎝⎜

⎞⎠⎟

N

≈UU0

⎛⎝⎜

⎞⎠⎟

N

1− 1− δUU

⎡⎣⎢

⎤⎦⎥

N⎧⎨⎪

⎩⎪

⎫⎬⎪

⎭⎪≈

UU0

⎛⎝⎜

⎞⎠⎟

N

Because N~1020, as long as δU <U (even if only a small amount!), the number ofstates in any width shell δU is the same as the total number of states up to U:

In a hyperspace of 1020 dimensions, all states lie near the surface

Thus Ω(U ) ≈ Ω(U,δU ) ≈W (U ) to extreme precision. This topic will be taken up

once more in the examples of microcanonical calculations given in the next chapter.