User:Molybdenumus/statistical ensemble/generalized

Principal ensembles of statistical thermodynamics

Visual representation of five statistical ensembles.

The study of thermodynamics is concerned with systems that have many microscopic degrees of freedom (the motion of their internal parts), but which can be described simply by a set of macroscopically observable variables and appear to be static to human perception. These systems can be described by statistical ensembles that depend on a few observable parameters, and which are in statistical equilibrium. Gibbs noted that different macroscopic constraints lead to different types of ensembles, with particular statistical characteristics. Three important thermodynamic ensembles were defined by Gibbs:

• Microcanonical ensemble or NVE ensemble—a statistical ensemble where the total energy of the system and the number of particles in the system are each fixed to particular values; each of the members of the ensemble are required to have the same total energy and particle number. The system must remain totally isolated (unable to exchange energy or particles with its environment) in order to stay in statistical equilibrium.
• Canonical ensemble or NVT ensemble—a statistical ensemble where the energy is not known exactly but the number of particles is fixed. In place of the energy, the temperature is specified. The canonical ensemble is appropriate for describing a closed system which is in, or has been in, weak thermal contact with a heat bath. In order to be in statistical equilibrium, the system must remain totally closed (unable to exchange particles with its environment) and may come into weak thermal contact with other systems that are described by ensembles with the same temperature.
• Grand canonical ensemble or μVT ensemble—a statistical ensemble where neither the energy nor particle number are fixed. In their place, the temperature and chemical potential are specified. The grand canonical ensemble is appropriate for describing an open system: one which is in, or has been in, weak contact with a reservoir (thermal contact, chemical contact, radiative contact, electrical contact, etc.). The ensemble remains in statistical equilibrium if the system comes into weak contact with other systems that are described by ensembles with the same temperature and chemical potential.

The calculations that can be made using each of these ensembles are explored further in their respective articles. Other thermodynamic ensembles can be also defined, corresponding to different physical requirements, for which analogous formulae can often similarly be derived.

Generalized ensemble

A generalized ensemble, sometimes called the Gibbs canonical ensemble, can be defined by immersing the system in a large heat bath and allowing some of the extensive state variables to be shared between them. In the case of the canonical ensemble, the energy $E$ is shared, while its conjugate intensive variable, the temperature $T$, is held fixed. For the grand canonical ensemble, both the energy and the particle number $N$ are shared, while the chemical potential $\mu$, the intensive variable conjugate to particle number, is fixed with the temperature. We collect the shared extensive variables into a vector ${\vec {X}}$, except the energy, which we treat separately because of its privileged role in thermodynamics. Collecting the intensive variables conjugate to ${\vec {X}}$ in ${\vec {J}}$, we can write down the probability $P_{\nu }$ of finding the system in any completely specified state $\nu$, called a microstate. We also find the generalized version of the famous partition function $Z$.

$P_{\nu }={\frac {1}{Z}}e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}$where $Z=\sum _{\nu }e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}$

The partition function arises as a normalization factor, constant across all microstates, for the probabilities. Because it can be used to find the probabilities, it captures all of the information about the statistical ensemble as a probability distribution over the microstates. This probability distribution can be used to calculate important information like the expected values of physical observables. For an observable $O$, we have

$\langle O\rangle ={\frac {1}{Z}}\sum _{\nu }Oe^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}$.

When an analytical expression of the partition function can be found, analytical expressions for the expected values of some of these observables can be found by expressing the expectation values as derivatives of $Z$.

Derivation of the generalized partition function

Denoting the system and bath by subscripts, we know that the extensive variables they share should be conserved, i.e. ${\vec {X}}={\vec {X_{s}}}+{\vec {X_{b}}}$ and $E=E_{s}+E_{b}$. Now, we suppose that the system is in microstate $\nu$ which completely describes the microscopic configuration of the system. We want to find $\Omega _{\nu }$, the total number of possible microstates of the universe, i.e. the system and bath considered together, when the system is fixed in state $\nu$. Note that this can be found by counting the number of bath microstates compatible with system microstate $\nu$. As number of bath microstates is a functions of the extensive variables $(E_{b},{\vec {X_{b}}})$, the compatible states are those characterized by $(E_{b},{\vec {X_{b}}})$ satisfying the constraints ${\vec {X_{b}}}={\vec {X}}-{\vec {X_{\nu }}}$ and $E_{b}=E-E_{\nu }$. In other words, $\Omega _{\nu }=\Omega _{bath}(E-E_{\nu },{\vec {X}}-{\vec {X_{\nu }}})$.

The probability $P_{\nu }$of finding the system in state $\nu$ is given by dividing $\Omega _{\nu }$ by the total number of microstates of the universe, $\Omega$,

$P_{\nu }={\frac {\Omega _{\nu }}{\Omega }}={\frac {1}{\Omega }}e^{\ln(\Omega _{bath}(E-E_{\nu },{\vec {X}}-{\vec {X_{\nu }}}))}$.

Here we can use the assumption that the thermal bath is much larger than the system. As extensive variables scale with system size, this implies ${\vec {X}}\gg {\vec {X_{\nu }}}$ and $E\gg E_{\nu }$, so we can Taylor expand about small ${\vec {X_{\nu }}}$ and $E_{\nu }$. To first-order,

$\ln(\Omega _{bath}(E-E_{\nu },{\vec {X}}-{\vec {X_{\nu }}}))\approx \ln(\Omega _{bath}(E,{\vec {X}}))-E{\frac {\partial }{\partial E_{\nu }}}\ln(\Omega _{bath}(E,{\vec {X}}))-{\vec {X_{\nu }}}\cdot {\frac {\partial }{\partial {\vec {X_{\nu }}}}}\ln(\Omega _{bath}(E,{\vec {X}}))$

Recognizing that $\ln(\Omega _{bath}({\vec {X}}))$ is the natural definition of the entropy $S$ of the bath, we can resolve this expression by using the fundamental thermodynamic identity $dE=TdS+{\vec {J}}\cdot d{\vec {X}}$ where ${\vec {J}}$ is the vector of intensive variables conjugate to the extensive variables ${\vec {X}}$ and $k_{b}$ is the Boltzmann constant. (Note: this relation displays the privileged role energy plays, and is the reason we must treat it separately from the other extensive variables - it is the only extensive variable that appears without it's conjugate, the inverse temperature. Instead, the temperature is naturally associated with every shared extensive variable, which can be see by dividing by $T$.) By rearranging the differentials, we see

${\frac {\partial S}{\partial {\vec {X}}}}{\Bigr |}_{E}=-{\frac {1}{k_{b}T}}{\vec {J}}$ and ${\frac {\partial S}{\partial E}}{\Bigr |}_{\vec {X}}={\frac {1}{k_{b}T}}$

which implies

$\ln(\Omega _{bath}(E-E_{\nu },{\vec {X}}-{\vec {X_{\nu }}}))\approx \ln(\Omega _{bath}(E,{\vec {X}}))-{\frac {1}{k_{b}T}}E_{\nu }+{\frac {1}{k_{b}T}}{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}}$.

Substituting this result into our expression for the probability $P_{\nu }$,

$P_{\nu }={\frac {1}{\Omega }}e^{\ln(\Omega _{bath}(E-E_{\nu },{\vec {X}}-{\vec {X_{\nu }}}))}\approx {\frac {\Omega _{bath}(E,{\vec {X}})}{\Omega }}e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}\equiv {\frac {1}{Z}}e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}$,

where we have noticed that we can factor out the portion of the expression that depends on the specific microstate $\nu$; this is the generalized Boltzmann factor. The rest of functions as a normalization factor, constant for all microstates, that we have renamed ${\frac {1}{Z}}$. Considering all of the microstates of the system, we must have their respective probabilities add up to 100%. This means

$1=\sum _{\nu }P_{\nu }=\sum _{\nu }{\frac {1}{Z}}e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}={\frac {1}{Z}}\sum _{\nu }e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}\implies Z=\sum _{\nu }e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}$

which is the result we sought.

Partition functions of the Gibbs ensembles

For the microcanonical ensemble, the trivial case where no extensive variables are shared between the system, including the energy, we have $Z=\sum _{\nu }1=\Omega$, while for the canonical ensemble, only the energy is exchanged between the system and the heat bath, so $Z=\sum _{\nu }e^{\frac {-E}{k_{b}T}}$. For the grand canonical ensemble, both the energy and particle number are shared, so $Z=\sum _{\nu }e^{{\frac {-1}{k_{b}T}}(E-\mu N)}$. In this way, these ideas can be naturally extended to the isobaric ensemble, which allows the volume of the system to change, ensembles with multiple species of ideal gas, and even more complex systems and constraints.

Example solving a system in the microcanonical and canonical ensembles

We will consider a system of $N$ particles that can each be in one of two states, with energies $E=0,\epsilon$ respectively. The energy of this two-level system is given by

$E=\epsilon \sum _{i}^{N}n_{i}$

where $i$ is an index labeling the particles and $n_{i}$ are the occupations numbers, with $n_{i}=0$ for an particle in the ground state and $n_{i}=1$ when in the excited state. In the microcanonical ensemble, the energy is fixed, which fixes the number of particles in the excited state at $E/\epsilon$. The total number of accessible microstates is given by the number of ways to choose which particles are in the excited state,

$\Omega ={\frac {N!}{(E/\epsilon )!(N-E/\epsilon )!}}$.

In the canonical ensemble, the energy is allowed to vary, while the temperature is fixed. This gives us the partition function, which we immediately separate to find an analytical solution,

$Z=\sum _{\nu }e^{\frac {-E_{\nu }}{k_{b}T}}=\sum _{\{n_{i}\}}e^{{\frac {-\epsilon }{k_{b}T}}\sum _{i=1}^{N}n_{i}}=\left(\sum _{n_{1}=0}^{1}e^{{\frac {-\epsilon }{k_{b}T}}n_{1}}\right)...\left(\sum _{n_{N}=0}^{1}e^{{\frac {-\epsilon }{k_{b}T}}n_{N}}\right)=\left(1+e^{\frac {-\epsilon }{k_{b}T}}\right)^{N}$

where in the last step we have explicitly evaluated each of the $N$ sums. Now we can calculate

$\langle E\rangle =-{\frac {\partial }{\partial (1/k_{b}T)}}\ln(Z)={\frac {N\epsilon e^{\frac {-\epsilon }{k_{b}T}}}{1-e^{\frac {-\epsilon }{k_{b}T}}}}$.

From this we see that in the low-temperature limit, $T\rightarrow 0$, we have $e^{\frac {-\epsilon }{k_{b}T}}\rightarrow 0$ and $\langle E\rangle \rightarrow 0$, which means all of the particles go into the ground state. Conversely, in the high-temperature limit, $T\rightarrow \infty$, we have we have $e^{\frac {-\epsilon }{k_{b}T}}\rightarrow \infty$ and $\langle E\rangle \rightarrow N\epsilon$, i.e. every particle goes into the excited state. By this mechanism, the temperature determines the expected value of the energy and number of excited particles. Intuitively, in the canonical ensemble a system has access to all of the microstates it would in the microcanonical ensemble with the same expected value of the energy and more. This is because, in the canonical ensemble, $E$ can fluctuate about $\langle E\rangle$. In other words, a system with a given expected value of the energy has more entropy in the canonical ensemble than the microcanonical ensemble.

Example using a generalized ensemble

Main article: Ising model

A common example of the utility of the generalizedensemble is a lattice of $N$ magnetic dipoles. Each dipole $\sigma _{i}$ has two possible orientations, and so are described with Ising variables $\{\sigma _{i}=\pm 1\}$. We can then define an extensive variable, the net magnetization, as $M=\mu _{0}\sum _{i=1}^{N}\sigma _{i}$. Now we can define an ensemble by allowing $M$ to vary while keeping its intensive conjugate, the magnetic field $B$, constant throughout the system and its surroundings using an external source.

Assuming there are no interactions between spins ($E(\{\sigma _{i}\})=0)$, we can immediately write down and separate the partition function

$Z=\sum _{\nu }e^{\frac {BM_{\nu }}{k_{b}T}}=\sum _{\{\sigma _{i}\}}e^{{\frac {B}{k_{b}T}}\mu _{0}\sum _{i=1}^{N}\sigma _{i}}=\left(\sum _{\sigma _{1}=-1}^{1}e^{{\frac {B}{k_{b}T}}\mu _{0}\sigma _{1}}\right)...\left(\sum _{\sigma _{N}=-1}^{1}e^{{\frac {B}{k_{b}T}}\mu _{0}\sigma _{N}}\right)$

where each sum over each math>\sigma_i</math> only takes on the values math>\pm 1</math>. Substituting in these values, we see each sum evaluates to $e^{\frac {-\mu _{0}B}{k_{b}T}}+e^{\frac {+\mu _{0}B}{k_{b}T}}$. We therefore have the analytical solution

$Z=\left(e^{\frac {-\mu _{0}B}{k_{b}T}}+e^{\frac {+\mu _{0}B}{k_{b}T}}\right)^{N}=\left(2\cosh \left({\frac {\mu _{0}B}{k_{b}T}}\right)\right)^{N}$.

We can use this result to find the expected value of useful quantities like the net magnetism. Note that

$\langle M\rangle ={\frac {1}{Z}}\sum _{\nu }M_{\nu }P_{\nu }={\frac {1}{Z}}\sum _{\nu }Me^{\frac {BM_{\nu }}{k_{b}T}}=k_{b}T{\frac {d}{dB}}\ln(Z)$

Plugging in our analytical solution, we see

$\langle M\rangle =Nk_{b}T{\frac {d}{dB}}\ln \left(2\cosh \left({\frac {\mu _{0}B}{k_{b}T}}\right)\right)=\mu _{0}N\tanh \left({\frac {\mu _{0}B}{k_{b}T}}\right)$.

Formal relationship between the Gibbs ensembles

Each statistical ensemble can be understood as some Legendre transformation of the microcanonical ensemble. In statistical mechanics, a Legendre transform is a mathematical process used to substitute an extensive variable in an expression for its intensive conjugate. The microcanonical ensemble is the ensemble that holds every extensive variable fixed, while the other ensembles substitute some of these extensive variables to hold their intensive conjugates fixed instead. For example, to create the canonical ensemble, the fixed energy is substituted for it's intensive conjugate, the inverse temperature, which is then held fixed by a large heat bath. These transformations usually create more physical ensembles, as it's more physically natural to fix intensive variables between a system and its surroundings than extensive one.

In statistical mechanics, Legendre transforms are also used to create the free energies. As an extensive function, the energy is most readily expressed in terms of the extensive variables, $E=E(S,{\vec {X}})$. To perform the Legendre transform, conjugate pairs of variables are subtracted from the energy, which naturally substitutes the extensive one for it's intensive partner, creating the free energy. For example, the Helmholtz free energy is $F=F(T,{\vec {X}})$, given by $F=E-TS$. Because of this, for an ensemble created by a given Legendre transform of the microcanonical ensemble, it's free energy, $-k_{b}Tln(Z)$, is given by the equivalent Legendre transform of the energy.

Free energies corresponding to each ensemble

	microcanonical	canonical	grand canoncial	isobaric
Legendre transform	-	fixed $E\rightarrow$ fixed $1/T$	fixed $E\rightarrow$ fixed $1/T$ fixed $N\rightarrow$ fixed $\mu$	fixed $E\rightarrow$ fixed $1/T$ fixed $V\rightarrow$ fixed $-P$
Partition function	$Z=\sum _{\nu }1=\Omega$	$Z=\sum _{\nu }e^{\frac {-E}{k_{b}T}}$	$Z=\sum _{\nu }e^{{\frac {-1}{k_{b}T}}(E-\mu N)}$	$Z=\sum _{\nu }e^{{\frac {-1}{k_{b}T}}(E+PV)}$
Free energy	the internal energy $E(S,{\vec {X}})=E$	the Helmholtz free energy $F(T,{\vec {X}})=E-TS$	$A(T,\mu ,{\vec {X}})=E-TS-\mu N$	the Gibbs free energy $A(T,P,{\vec {X}})=E-TS+PV$

Derivation of the free energy of the generalizedensemble

To show that the free energy of a generalizedensemble is equal to the equivalent Legendre transform of the energy, we will use the Shannon entropy from information theory which gives an alternate form the thermodynamic entropy.  We have,

$S=-k_{b}\sum _{\nu }P_{\nu }\ln(P_{\nu })$.

Plugging the expression for $P_{\nu }$ of a microstate $\nu$ in a generalizedensemble immediately leads to a useful relation. We have,

${\begin{alignedat}{2}S&=-k_{b}\sum _{\nu }P_{\nu }\ln \left({\frac {1}{Z}}e^{{\frac {-1}{k_{b}T}}(E-{\vec {J_{\nu }}}\cdot {\vec {X_{\nu }}})}\right)=-k_{b}\sum _{\nu }P_{\nu }\left(-{\frac {E}{k_{b}T}}+{\frac {{\vec {J}}\cdot {\vec {X}}}{k_{b}T}}-\ln(Z)\right)\\&=-k_{b}\left(-{\frac {E}{k_{b}T}}\sum _{\nu }P_{\nu }E+{\frac {\vec {J}}{k_{b}T}}\cdot \sum _{\nu }P_{\nu }{\vec {X}}-\ln(Z)\sum _{\nu }P_{\nu }\right)\\\end{alignedat}}$

Here, notice that weighting an observable by the probability of a microstate and summing over all microstates evaluates the expected value of that observable. We then have

$S=-k_{b}\left({\frac {-1}{k_{b}T}}\langle E\rangle +{\frac {\vec {J}}{k_{b}T}}\cdot \langle {\vec {X}}\rangle -\ln(Z)\right)$

Multiplying by T and rearranging, we have

$-k_{b}T\ln(Z)=\langle E(S,{\vec {X}})\rangle -{\vec {J}}\cdot \langle {\vec {X}}\rangle -TS$

The left-hand side of this equation is the free energy $F$ in the ensemble, while the right-hand side of this equation the Legendre transformation of the energy that replaces the entropy $S$ with $T$ and ${\vec {X}}$ with ${\vec {J}}$. The action of the Legendre transform can be seen in differential form,

$d(E-TS-{\vec {J}}\cdot {\vec {X}})=dE-SdT-TdS-{\vec {J}}\cdot d{\vec {X}}-{\vec {X}}\cdot d{\vec {J}}$,

where, after substituting in the fundamental identity of thermodynamics for $dE$ we have

$d(E-TS-{\vec {J}}\cdot {\vec {X}})=TdS+{\vec {J}}\cdot d{\vec {X}}-SdT-TdS-{\vec {J}}\cdot d{\vec {X}}-{\vec {X}}\cdot d{\vec {J}}=-SdT-{\vec {X}}\cdot d{\vec {J}}$

This implies $F=F(T,{\vec {J}})$, so the Legendre transform has naturally replaced the variables $S$ and ${\vec {X}}$ to create a free energy that is a function of the intensive variables $S$ and ${\vec {J}}$.