[Fis] Probabalistic Entropy

Colleagues,

I appreciate the opportunity of this forum to consider some of the
fascinating and stimulating ideas that have been expressed recently in
postings. It�s a pleasure for a scientist to discuss with both
scientists and nonscientists these interesting notions.
Loet wrote that �the Shannon entropy is the more interesting part� and
�the Shannon entropy is more general because it can be applied to
systems other than the physical one� and �the mathematical richness is
available because Shannon chose H and the algorithms of physics can thus
be used as a heuristic for the computation in other systems�. Also,
�these forms of organization (and self-organization) can be studied in
terms of their probabilistic entropy.�
Thanks, Loet, for making the argument so forcefully. I agree. I think
all the forms of entropy used by scientists can be expressed,
fundamentally, as probabilities. Even the seemingly primitive form of
Clausius� version, Delta-Q / T, I believe, rests on a probabilistic
foundation. Because heat is really microscopic motion, it�s always more
probable that heat will flow spontaneously from the higher to the lower
temperature. And I understand Boltzmann�s concept of entropy, in terms
of available phase space, to really be about the PROBABILITY that
physical systems, when unconstrained, are likely to move into the entire
range of motion available to them. I understand that Boltzmann also
thought of his description in terms of it�s probability.
Von Neumann derived an equation for quantum mechanics identical to
Shannon�s. I expect many of you know that quantum mechanics speaks only
to the probability of the outcome of physical events; that the
particular outcome of a given observation is not knowable except as a
probability for one or another result. Yet, we physicists have found
that quantum mechanics describes all the known physical universe we see,
from elementary microscopic particles to cosmological structures.
Aleks wrote that �Clausius� (thermodynamic) entropy is very different
from Shannon�s entropy,� and �Thermodynamic entropy is something one can
measure. Shannon�s entropy is instead a general one.� If I understand
you, Aleks, I disagree. I�m convinced that the Shannon-von Neumann
formula for entropy may be compared to the results of laboratory
experiments also. It�s usual to do that for elementary particle
experiments, like those at the CERN accelerator in Switzerland, for
example.
Now, after decades of matching experimental observations, from
elementary particles, from molecular measurements and solid-state
phenomena, electronic devices and macroscopic interference effects, even
from innumerable astronomical observations, to the probabilistic
predictions of quantum mechanics, I believe that the probabilistic
formulation of entropy is the general one. It works for heat engines
also, as Szilard�s engine exemplifies.
Broadly speaking, I think that it�s the tendency of the physical
universe to move toward its most probable state which accounts for the
heat exhausted by machines, the persistent problems of cleaning up after
ourselves, the understanding that our universe must have been more
�orderly� at its conception, and all those other observations
unidirectional in time.
So, I�ve got to disagree with the sentiment expressed in Bob�s wonderful
posting that �thermodynamics rests on a phenomenological foundation that
is far more solid than any atomic �hypothesis.� It�s my view, Bob, that
entropy is pertinent to all physical phenomena, and it can be of
fundamental importance for all of them. Pertinent to the microscopic,
ordinarily observable, and even cosmological events we describe, as well
as the engines, and such, envisioned by Carnot and Maxwell.
I expect, Loet, that by systems other than the physical one, you mean
those systems not traditionally described by physics. You mention
circulation of money in an economy, the division of classes in a school,
the characters in a Boltzmann machine at temperatures above zero K, and
living systems. I usually conceive of physical systems as tangible,
observable ones; systems that impress our human senses. So, I tend to
categorize all the systems you�ve mentioned as physical ones. I trust I
understand your idea, Loet.
According to von Neumann�s prescription, the mathematical model of
quantum mechanics meets the observable physical universe through
comparison of an ensemble of prepared experiments with the probabilistic
predictions of the mathematics. I understand all of physics to operate
in exactly that way. That we make models of everything we observe, and
then evaluate those models experimentally. Is it true that all of
science follows that scheme?
I expect, Aleks, that you mean much the same thing when you say that
Boltzmann�s model for entropy is �just a paradigm (model, not measure).
It was tested, and it was proved that the paradigm is not misguided.� I
know that physicists must spend what seems an inordinate amount of time
in university laboratories, trying to become physicists, so, essentially
all the ones I�ve known, have a palpable, ponderable, almost visceral
respect for the distinction between theoretical models of nature and
those validating observations.
We divide ourselves, usually, into theoretical or experimental
physicists. But all of us, in my experience, understand that measurement
is the criterion by which theories are judged, and that observations are
never, ever, as clean, elegant, or precise as their mathematical model.
I�ve found it often a frustrating, unsanitary business trying to make
measurement equipment give up some useful numbers. Sound familiar?
Thanks, Aleks, for the direction to Denis Evans work. I�ll read it. It�s
usual, though, isn�t it, to expect fluctuations away from the most
probable state to decrease with the increase in system size? So that�s
not new, is it? And Boltzmann�s H theorem says that thermodynamic
systems evolve toward equilibrium. So, that�s not novel either, is it?
Zurek�s paper on the Szilard engine claims to solve any fluctuation
problems with the engine. I believe him. Do you disagree with his analysis?
Thanks for the stimulating ideas.
Cordially,

Michael Devereux (dbar_x@cybermesa.com)

_______________________________________________
fis mailing list
fis@listas.unizar.es
http://webmail.unizar.es/mailman/listinfo/fis
Received on Fri Apr 16 07:13:27 2004