Re: [Fis] Re: Shannon Entropy

Fully agree with Professor Ebeling:
"entropy is one of the most difficult concepts of science! "
It is admirable at least to recognize the problem.

Any good overviews? Please contribute to our journal ENTROPY
at http://www.mdpi.net/entropy/, a journal I just mentioned as
an example in the 3rd conference Wizards of Open Society in Berlin
(http://wizards-of-os.org/).

Shu-Kun

Prof.Dr.Werner Ebeling wrote:

>Dear Michel,
>let me try to answer your complicated questions:
>
>On Saturday 12 Jun 2004 10:57 pm, you wrote:
>
>
>>Dear Werner,
>>
>>Thanks for your helpful, informative message. May I use this avenue to
>>reply now? I can?t make a posting to the FIS discussion until next week.
>>I regret I missed an acknowledgment of your communication in my latest
>>epistle.
>>
>>So far, we?ve heard in our FIS discussion that Shannon entropy is pure
>>mathematics
>>
>>
>Thats true, it is a mathematical expression which can be applied to many
>systems having probability dostributions.
>and therefore completely separate from the entropy described
>
>
>>by Boltzmann et al.
>>
>>
>For me not completely separated, but realted !!!
>
>
>>Also, that some form of ?Shannon entropy? employed
>>in research outside physics may actually decrease with increasing
>>freedom degrees, and that in some probability analyses of biological
>>systems, the probability values in the distribution may become negative.
>>
>>
>
>Negative probabilities are of course not possible, but Shannon entropies
>are necessarily positive only for discrete cases, if the undelying
>space is continuous, there are some difficulties and we need
>a more general formalism.
>
>
>
>>I interpret all this as a misappropriation of the name Shannon entropy
>>for analyses which do not, in fact, depict the sort of genuine
>>probability distributions that physical systems exhibit. I?ve argued,
>>from Shannon?s own derivation of the equation, that if the analyzed
>>property is not monotonically increasing with N, it cannot be Shannon
>>entropy.
>>
>>
>Applying Shannons formula to physical systems not necessarily gives
>thermodyanmic entropy, we have to take distributions on the space of
>coordinates and momenta (as Boltzmann actually did) or
>more general densities of quantum states or
>still more general quantum density operators (as did von Neumann).
>There are applications to physical systems as e.g. to
>electrical systems or to nonlinear dynamics, which lead to some
>entropy but not to the measurable thermodynamic entropy.
>
>
>>I do recognize that Shannon entropy is an appropriate name for
>>probability distributions of nonphysical systems (perhaps linguistics,
>>and economics and such) which satisfy Shannon?s three axioms for such
>>distributions. But, Shannon?s derivation of the equation actually
>>described a physical system; the electrical signals in a communication
>>cable. And would you agree that one might just as appropriately utilize
>>Boltzmann?s H equation for nonphysical distributions of a suitable type?
>>
>>
>
>The H-theorem of Boltzmann does not apply to any system, its true only
>for special probabilistic (stochastic) systems. However for probabilistic
>systems with Markov character
>one can find some generalization of the H-theorem which is valid
>for the so-called Kullback-Leibler entropy (this is to be found
>in many books on stochastic theory, I could give references).
>
>
>
>>Are you familiar with Szilard?s engine? I continue to believe it?s the
>>best model for demonstrating the direct connection between Clausius?
>>form of entropy and information.
>>
>>
>Yes Szilard worked at my University.
>My interpretation is, that any exchange of information betrween two systems
>is connected with some exchange of thermodynamic entropy.
>
>
>>Very best wishes,
>>
>>Michael Devereux
>>
>>
>By the way I do not claim to know the full truth,
>entropy is one of the most difficult concepts of science !!!!
>In reply best wishes
>Werner Ebeling
>
>

-- 
Dr. Shu-Kun Lin
Molecular Diversity Preservation International (MDPI)
Matthaeusstrasse 11, CH-4057 Basel, Switzerland
Tel. +41 61 683 7734 (office)
Tel. +41 79 322 3379 (mobile)
Fax +41 61 302 8918
E-mail: lin@mdpi.org
http://www.mdpi.org/lin/
_______________________________________________
fis mailing list
fis@listas.unizar.es
http://webmail.unizar.es/mailman/listinfo/fis