Re: [Fis] FIS / introductory text / 5 April 2004

Dear Loet,

You mean entropy S is equal to information (I), or almost equal. Can you
still give
a little bit of sympathy to the relation that Delta S > - Delta
(information) and make some
comments on this different relation? If we can agree on the relation that
Delta S > - Delta (information),
then we are ready to ask
"why information loss is related to entropy", a question asked by
physicists at the
http://www.lns.cornell.edu/spr/2000-12/msg0030047.html website,
and try to answer.

Michel, thank you for your introduction.

Shu-Kun

Loet Leydesdorff wrote:

>Dear Michel,
>
>The relation between thermodynamic entropy and the information is
>provided by the Szilard-Brillouin relation as follows:
>
>Delta S >= k(B) Delta H
>
>(W. Ebeling. Chaos, Ordnung und Information. Frankfurt a.M.: Harri
>Deutsch Thun, 1991, at p. 60.)
>
>k(B) in this formula is the Boltzmann constant. Thus, a physical change
>of the system can provide an information, but it does not have to.
>Unlike the thermodynamic entropy, probabilistic entropy has no
>dimensionality (because it is mathematically defined). The Boltmann
>constant takes care of the correction in the dimensionality in the
>equation.
>
>When applied as a statistics to other systems (e.g., biological ones)
>one obtains another (specific) theory of communication in which one can
>perhaps find another relation between the (in this case biological)
>information and the probabilistic entropy. This can be elaborated for
>each specific domain.
>
>With kind regards,
>
>
>Loet
>
>
>
> _____
>
>Loet Leydesdorff
>Science & Technology Dynamics, University of Amsterdam
>Amsterdam School of Communications Research (ASCoR)
>Kloveniersburgwal 48, 1012 CX Amsterdam
>Tel.: +31-20-525 6598; fax: +31-20-525 3681
> <mailto:loet@leydesdorff.net> loet@leydesdorff.net;
><http://www.leydesdorff.net/> http://www.leydesdorff.net
>
>
>-----Original Message-----
>From: fis-bounces@listas.unizar.es [mailto:fis-bounces@listas.unizar.es]
>On Behalf Of Michel Petitjean
>Sent: Monday, April 05, 2004 9:15 AM
>To: fis@listas.unizar.es
>Subject: [Fis] FIS / introductory text / 5 April 2004
>
>
>2004 FIS session introductory text.
>
>Dear FISers,
>
>I would like to thank Pedro Marijuan for his kind invitation
>to chair the 2004 FIS session. The session is focussed on "Entropy and
>Information". It is vast, that I am afraid to be able only to evoke some
>general aspects, discarding specific technical developments.
>
> Entropy and Information: two polymorphic concepts.
>
>Although these two concepts are undoubtly related, they have different
>stories.
>
>Let us consider first the Information concept.
>There was many discussions in the FIS list about the meaning
>of Information. Clearly, there are several definitions.
>The information concept that most people have in mind is outside the
>scope of this text: is it born with Computer Sciences, or is it born
>with Press, or does it exist since a so long time that nobody could date
>it? Neglecting the definitions from the dictionnaries (for each language
>and culture), I would say that anybody has his own concept. Philosophers
>and historians have to look. The content of the FIS archives suggests
>that the field is vast.
>
>Now let us look to scientific definitions. Those arising from
>mathematics are rigorous, but have different meanings. An example is the
>information concept emerging from information theory (Hartley, Wiener,
>Shannon, Renyi,...). This concept, which arises from probability theory,
>has little connections with the Fisher information, which arises also
>from probability theory. The same word is used, but two rigorous
>concepts are defined. One is mostly related to coding theory, and the
>other is related to estimation theory. One deals mainly with non
>numerical finite discrete distributions, and the other is based on
>statistics from samples of parametrized family of distributions. Even
>within the framework of information theory, there are several
>definitions of information (e.g. see the last chapter of Renyi's book on
>Probability Theory). This situation arises often in mathematics: e.g.,
>there are several concepts of "distance", and, despite the basic axioms
>they all satisfy, nobody would say that they have the same meaning, even
>when they are defined on a common space.
>
>Then, mathematical tools are potential (and sometimes demonstrated)
>simplified models for physical phenomenons. On the other hand,
>scientists may publish various definitions of information for physical
>situations. It does not mean that any of these definitions should be
>confused between themselves and confused with the mathematical ones. In
>many papers, the authors insist on the analogies between their own
>concepts and those previously published by other authors: this attitude
>may convince the reviewers of the manuscript that the work has interest,
>but contribute to the general confusion, particularly when the confusing
>terms are recorded in the bibliographic databases. Searching in
>databases with the keyword "information" would lead to a considerable
>number of hits: nobody would try it without constraining the search with
>other terms (did some of you tried?).
>
>We consider now the Entropy concepts. The two main ones are the
>informational entropy and the thermodynamical entropy. The first one has
>non ambiguous relations with information (in the sense of information
>theory), since both are defined within the framework of a common theory.
>Let us look now to the thermodynamical entropy, which was defined by
>Rudolf Clausius in 1865. It is a physical concept, usually introduced
>from the Carnot Cycle. The existence of entropy is postulated, and it is
>a state function of the system. Usual variables are temperature and
>pressure. Entropy calculations are sometimes made discarding the
>implicit assumptions done for an idealized Carnot Cycle. Here come
>difficulties. E.g., the whole universe is sometimes considered as a
>system for which the the entropy is assumed to have sense. Does the
>equilibrium of such a system has sense? Does thermodynamical state
>functions make sense here? And what about "the" temperature? These
>latter variable, even when viewed as a function of coordinates and/or
>time, has sense only for a restricted number of situations. These
>difficulties appear for many other systems. At other scales, they may
>appear for microscopic systems, and for macroscopic systems unrelated to
>thermochemistry.
>In fact, what is often implicitly postulated is that the thermodynamical
>entropy theory could work outside thermodynamics.
>
>Statistical mechanics creates a bridge between microscopic and
>macroscopic models, as evidenced from the work of Boltzmann. These two
>models are different. One is a mathematical model for an idealized
>physical situation (punctual balls, elastic collisions, distribution of
>states, etc..), and the other is a simplified physical model, working
>upon a restricted number of conditions. The expression of the entropy,
>calculated via statistical mechanics methods, is formally similar to the
>informational entropy. This latter has appeared many decades after the
>former. Thus, the pioneers of information theory (Shannon, von Neumann)
>who retain the term "entropy", are undoubtly responsible of the
>historical link between <<Entropy>> and <<Information>> (e.g. see
>http://www.bartleby.com/64/C004/024.html).
>
>Although "entropy" is a well known term in information theory, and used
>coherently with the term "information" in this area, the situation is
>different in science. I do not know what is "information" in
>theermodynamics (does anybody know?). However, "chemical information" is
>a well known area of chemistry, which covers many topics, including data
>mining in chemical data bases. In fact, chemical information was
>reognized as a major field when the ACS decided in 1975 to rename one of
>its journals "Journal of Chemical Information and Computer Sciences": it
>was previously named the "Journal of Chemical Documentation". There are
>little papers in this journal which are connected with entropy
>(thermodunamical of informational). An example is the 1996 paper of
>Shu-Kun Lin, relating entropy with similarity and symmetry. Similarity
>is itself a major area in chemical information, but I consider that the
>main area of chemical information is related to chemical databases, such
>that the chemical information is represented by the nodes and edges
>graph associated to a structural formula. Actually, mathematical tools
>able to work on this kind of chemical information are lacking,
>particulary for statistics (did anyone performed statistics on graphs?).
>
>In 1999, the links between information sciences and entropy were again
>recognized, when Shu-Kun lin created the open access journal "Entropy":
><<An International and Interdisciplinary Journal of Entropy and
>Information Studies>>. Although most pluridisciplinary journals are at
>the intersection of two areas, Shu-Kun Lin is a pionneer in the field of
>transdisciplinarity, permitting the publication in a single journal of
>works related to entropy and/or information theory, originating from
>mathematics, physics, chemistry, biology, economy, and philosophy.
>
>The concept of information exists in other sciences for which the term
>entropy is used. Bioinformation is a major concept in bioinformatics,
>for which I am not specialist. Thus I hope that Pedro Marijuan would
>like to help us to understand what are the links between bioinformation
>and entropy. Entropy and information are known from economists and
>philosophers. I also hope they add their voice to those of scientists
>and mathematicians, to enlight our discussions during the session.
>
>Now I would like to draw some provocative conclusions. Analogies between
>concepts or between formal expressions of quantities are useful for the
>spirit, for the quality of the papers, and sometimes they are used by
>modellers to demonstrate why their work merit funds (does anybody never
>do that?). The number of new concepts in sciences (includes mathematics,
>economy, humanities, and so on) is increasing, and new terms are picked
>in our natural language: the task of the teachers becomes harder and
>harder. Entropy and Information are like the "fourth dimension", one
>century ago: they offer in common the ability to provide exciting topics
>to discuss. Unfortunately, Entropy and Information are much more
>difficult to handle.
>
>Michel Petitjean Email: petitjean@itodys.jussieu.fr
>Editor-in-Chief of Entropy entropy@mdpi.org
>ITODYS (CNRS, UMR 7086) ptitjean@ccr.jussieu.fr
>1 rue Guy de la Brosse Phone: +33 (0)1 44 27 48 57
>75005 Paris, France. FAX : +33 (0)1 44 27 68 14
>http://www.mdpi.net http://www.mdpi.org
>http://petitjeanmichel.free.fr/itoweb.petitjean.html
>http://petitjeanmichel.free.fr/itoweb.petitjean.freeware.html
>_______________________________________________
>fis mailing list
>[email protected] http://webmail.unizar.es/mailman/listinfo/fis
>
>
>
>
>

-- 
Dr. Shu-Kun Lin
Molecular Diversity Preservation International (MDPI)
Matthaeusstrasse 11, CH-4057 Basel, Switzerland
Tel. +41 61 683 7734 (office)
Tel. +41 79 322 3379 (mobile)
Fax +41 61 302 8918
E-mail: lin@mdpi.org
http://www.mdpi.org/lin/
_______________________________________________
fis mailing list
fis@listas.unizar.es
http://webmail.unizar.es/mailman/listinfo/fis