Jump to content

How can information (Shannon) entropy decrease ?


studiot

Recommended Posts

On 3/22/2022 at 2:31 AM, joigus said:

... for the 

P(α) to be well-defined probabilities they should be some fixed frequency ratios to be checked in an infinitely-many-times reproducible experiment. So this P(α)lnP(α) already has to be the sought-for entropy, and not some kind of weird thing P(α,t)lnP(α,t) .

IOW, how can you even have anything like time-dependent probabilities?

The book that has been mentioned by @studiot earlier (Lemons, Don S.. A Student's Guide to Entropy) has a paragraph about time-dependent entropy:

Quote

... a broader interpretation of the relative entropy according to which the multiplicity Ω is that of an instantaneous macrostate as a system evolves from an initial low entropy reference macrostate toward an equilibrium macrostate. This interpretation assumes that the system always occupies a macrostate at every instant of its evolution. In this case, S(Ω) where (2.18)  is the instantaneous entropy of a system, not necessarily in equilibrium, relative to the entropy S(Ωo) of the system in a reference macrostate. Generally, the evolution of an isolated system is in the direction of increasing entropy. ... the instantaneous macrostate multiplicity Ω and hence the instantaneous entropy S(Ω) fluctuate as the system occupies different, not quite optimal, macrostates.

 

Edited by Genady
Link to comment
Share on other sites

On 3/22/2022 at 5:17 PM, Genady said:

I don't know how deep it is, but I doubt that I'm overthinking this. I just follow the Weinberg's derivation of the H-theorem, which in turn follows the Gibbs' one, which in turn is a generalization of the Boltzmann's one.

The first part of this theorem is that dH/dt ≤ 0. [...]

Actually, I don't think you're overthinking this at all. It does go deep. The \( P\left(\alpha,t\right) \)'s must be a priori probabilities or propensities and do not correspond to equilibrium states at all.

On 3/25/2022 at 6:39 PM, Genady said:

The book that has been mentioned by @studiot earlier (Lemons, Don S.. A Student's Guide to Entropy) has a paragraph about time-dependent entropy:

Quote

I'm not familiar with this way of dealing with the problem. The way I'm familiar with is this:

These P(α,t) 's can be hypothesized by applying the evolution equations (classical or quantum) to provide you with quantities that play the role of evolving probabilities. This is developed in great detail, e.g., in Balescu. You attribute time-independent probabilities to the (e.g. classical mechanics) initial conditions, and then you feed those initial conditions into the equations of motion to produce a time flow:

\[q_{0},p_{0}\overset{\textrm{time map}}{\mapsto}q_{t},p_{t}=q\left(q_{0},p_{0};t\right),p\left(q_{0},p_{0};t\right)\]

That induces time map onto dynamical functions:

\[A\left(q_{0},p_{0}\right)\overset{\textrm{time map}}{\mapsto}A\left(q_{t},p_{t}\right) \]

So dynamics changes dynamical functions, not probabities.

But here's the clever bit: We now do a tradeoff between averages of time-evolving variables weighed against a fixed probability density of initial conditions, and averages of fixed variables weighed against a time-evolving probability density on phase space. All of this is done by introducing a so-called Liouville operator that can be written in terms of the time-translation generator, the Hamiltonian, through the Poisson bracket. This Liouville operator produces the evolution of dynamical functions:

\[\left\langle A\right\rangle \left(t\right)=\int dq_{0}dp_{0}\rho\left(q_{0},p_{0}\right)e^{-L_{0}t}A\left(q_{0},p_{0}\right)\]

Because of the properties of this Liouville operator, you can swap its action by using integration by parts on this first-order Liouville differential operator and get,

\[\int dq_{0}dp_{0}e^{L_{0}t}\rho\left(q_{0},p_{0}\right)A\left(q_{0},p_{0}\right)=\int dq_{0}dp_{0}\rho\left(q_{0},p_{0}\right)e^{-L_{0}t}A\left(q_{0},p_{0}\right)\]

You can think of \(e^{L_{0}t}\rho\left(q_{0},p_{0}\right)\) a time-dependent probability density; while \(e^{-L_{0}t}A\left(q_{0},p_{0}\right)\) can be thought of as time-dependent dynamical functions.

For all of this to work, the system should be well-behaved; meaning that it must be ergodic. Ergodic systems are insensitive to initial conditions.

I'm sure my post leaves a lot to be desired, but I've been kinda busy lately.

 

 

 

 

 

Edited by joigus
Latex editing
Link to comment
Share on other sites

3 hours ago, joigus said:

Actually, I don't think you're overthinking this at all. It does go deep. The P(α,t) 's must be a priori probabilities or propensities and do not correspond to equilibrium states at all.

I'm not familiar with this way of dealing with the problem. The way I'm familiar with is this:

These P(α,t) 's can be hypothesized by applying the evolution equations (classical or quantum) to provide you with quantities that play the role of evolving probabilities. This is developed in great detail, e.g., in Balescu. You attribute time-independent probabilities to the (e.g. classical mechanics) initial conditions, and then you feed those initial conditions into the equations of motion to produce a time flow:

 

q0,p0time mapqt,pt=q(q0,p0;t),p(q0,p0;t)

 

That induces time map onto dynamical functions:

 

A(q0,p0)time mapA(qt,pt)

 

So dynamics changes dynamical functions, not probabities.

But here's the clever bit: We now do a tradeoff between averages of time-evolving variables weighed against a fixed probability density of initial conditions, and averages of fixed variables weighed against a time-evolving probability density on phase space. All of this is done by introducing a so-called Liouville operator that can be written in terms of the time-translation generator, the Hamiltonian, through the Poisson bracket. This Liouville operator produces the evolution of dynamical functions:

 

A(t)=dq0dp0ρ(q0,p0)eL0tA(q0,p0)

 

Because of the properties of this Liouville operator, you can swap its action by using integration by parts on this first-order Liouville differential operator and get,

 

dq0dp0eL0tρ(q0,p0)A(q0,p0)=dq0dp0ρ(q0,p0)eL0tA(q0,p0)

 

You can think of eL0tρ(q0,p0)  a time-dependent probability density; while eL0tA(q0,p0) can be thought of as time-dependent dynamical functions.

For all of this to work, the system should be well-behaved; meaning that it must be ergodic. Ergodic systems are insensitive to initial conditions.

I'm sure my post leaves a lot to be desired, but I've been kinda busy lately.

 

 

 

 

 

Yes, it goes deep ... and into the parts of mechanics with which I'm mostly unfamiliar. The main point is, H does not correspond to an equilibrium state until it reaches the minimum, and thus its decreasing does not correspond to the thermodynamic entropy increase.

However, the initial value of H corresponds to an equilibrium state before constraints are removed. The final value of H corresponds to a new equilibrium state, after the decrease of H. This difference between the initial and the final values of H corresponds to a difference in thermodynamic entropy between the two equilibrium states. Thus the inconsistency is removed. Banzai!

Link to comment
Share on other sites

23 hours ago, Genady said:

Yes, it goes deep ... and into the parts of mechanics with which I'm mostly unfamiliar. The main point is, H does not correspond to an equilibrium state until it reaches the minimum, and thus its decreasing does not correspond to the thermodynamic entropy increase.

However, the initial value of H corresponds to an equilibrium state before constraints are removed. The final value of H corresponds to a new equilibrium state, after the decrease of H. This difference between the initial and the final values of H corresponds to a difference in thermodynamic entropy between the two equilibrium states. Thus the inconsistency is removed. Banzai!

Yeah, that's a beautiful idea, and I think it works. You can sub-divide also reversible processes into infinitesimally small irreversible sub-processes, and prove it for them too. I think it was very soon that people realised that entropy must be a state function, and the arguments were very similar.

I tend to look at these things more from the point of view of statistical mechanics. Boltzmann's H theorem has suffered waves of criticism through the years, but it seems to me to be quite robust.

https://en.wikipedia.org/wiki/H-theorem#Criticism_and_exceptions

I wouldn't call these criticisms "moot points," but I think all of them rest on some kind of oversimplification of physical systems.

Note that some members may find interesting: I've been watching a lecture by John Baez dealing with Shannon entropy, and the second principle of thermodynamics in the context of studying biodiversity.

It seems that people merrily use Shannon entropy to run ecosystem simulations.

Baez says there is generally no reason to suppose that biodiversity-related Shannon entropy reaches a maximum, but there are interesting cases where this is true. Namely, when there is a dominant, mutually interdependent cluster of species in the whole array of initial species.

 

Link to comment
Share on other sites

29 minutes ago, joigus said:

Yeah, that's a beautiful idea, and I think it works. You can sub-divide also reversible processes into infinitesimally small irreversible sub-processes, and prove it for them too. I think it was very soon that people realised that entropy must be a state function, and the arguments were very similar.

I tend to look at these things more from the point of view of statistical mechanics. Boltzmann's H theorem has suffered waves of criticism through the years, but it seems to me to be quite robust.

https://en.wikipedia.org/wiki/H-theorem#Criticism_and_exceptions

I wouldn't call these criticisms "moot points," but I think all of them rest on some kind of oversimplification of physical systems.

Note that some members may find interesting: I've been watching a lecture by John Baez dealing with Shannon entropy, and the second principle of thermodynamics in the context of studying biodiversity.

It seems that people merrily use Shannon entropy to run ecosystem simulations.

Baez says there is generally no reason to suppose that biodiversity-related Shannon entropy reaches a maximum, but there are interesting cases where this is true. Namely, when there is a dominant, mutually interdependent cluster of species in the whole array of initial species.

 

Thank you. I'm glad you like the idea.

OTOH, I'm sorry, but I can't watch a 35 min video because of my APD. If it exists in writing somewhere...

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.