[Fis] FIS / introductory text / 5 April 2004

2004 FIS session introductory text.

Dear FISers,

I would like to thank Pedro Marijuan for his kind invitation
to chair the 2004 FIS session. The session is focussed on
"Entropy and Information". It is vast, that I am afraid
to be able only to evoke some general aspects, discarding
specific technical developments.

     Entropy and Information: two polymorphic concepts.

Although these two concepts are undoubtly related, they have
different stories.

Let us consider first the Information concept.
There was many discussions in the FIS list about the meaning
of Information. Clearly, there are several definitions.
The information concept that most people have in mind is
outside the scope of this text: is it born with Computer Sciences,
or is it born with Press, or does it exist since a so long time
that nobody could date it? Neglecting the definitions from
the dictionnaries (for each language and culture), I would say
that anybody has his own concept. Philosophers and historians
have to look. The content of the FIS archives suggests that
the field is vast.

Now let us look to scientific definitions. Those arising from
mathematics are rigorous, but have different meanings. An example
is the information concept emerging from information theory (Hartley,
Wiener, Shannon, Renyi,...). This concept, which arises from probability
theory, has little connections with the Fisher information, which
arises also from probability theory. The same word is used, but two
rigorous concepts are defined. One is mostly related to coding theory,
and the other is related to estimation theory. One deals mainly
with non numerical finite discrete distributions, and the other
is based on statistics from samples of parametrized family of
distributions. Even within the framework of information theory,
there are several definitions of information (e.g. see the last
chapter of Renyi's book on Probability Theory). This situation
arises often in mathematics: e.g., there are several concepts of
"distance", and, despite the basic axioms they all satisfy, nobody
would say that they have the same meaning, even when they are defined
on a common space.

Then, mathematical tools are potential (and sometimes demonstrated)
simplified models for physical phenomenons. On the other hand,
scientists may publish various definitions of information for physical
situations. It does not mean that any of these definitions should be
confused between themselves and confused with the mathematical ones.
In many papers, the authors insist on the analogies between their
own concepts and those previously published by other authors: this
attitude may convince the reviewers of the manuscript that the work
has interest, but contribute to the general confusion, particularly
when the confusing terms are recorded in the bibliographic databases.
Searching in databases with the keyword "information" would lead to
a considerable number of hits: nobody would try it without constraining
the search with other terms (did some of you tried?).

We consider now the Entropy concepts. The two main ones are the
informational entropy and the thermodynamical entropy. The first
one has non ambiguous relations with information (in the sense of
information theory), since both are defined within the framework of
a common theory. Let us look now to the thermodynamical entropy,
which was defined by Rudolf Clausius in 1865.
It is a physical concept, usually introduced from the Carnot Cycle.
The existence of entropy is postulated, and it is a state function
of the system. Usual variables are temperature and pressure. Entropy
calculations are sometimes made discarding the implicit assumptions
done for an idealized Carnot Cycle. Here come difficulties. E.g., the
whole universe is sometimes considered as a system for which the
the entropy is assumed to have sense. Does the equilibrium of
such a system has sense? Does thermodynamical state functions
make sense here? And what about "the" temperature? These latter
variable, even when viewed as a function of coordinates and/or
time, has sense only for a restricted number of situations.
These difficulties appear for many other systems. At other scales,
they may appear for microscopic systems, and for macroscopic
systems unrelated to thermochemistry.
In fact, what is often implicitly postulated is that the
thermodynamical entropy theory could work outside thermodynamics.

Statistical mechanics creates a bridge between microscopic and
macroscopic models, as evidenced from the work of Boltzmann.
These two models are different. One is a mathematical model
for an idealized physical situation (punctual balls, elastic
collisions, distribution of states, etc..), and the other is
a simplified physical model, working upon a restricted number
of conditions. The expression of the entropy, calculated via
statistical mechanics methods, is formally similar to the
informational entropy. This latter has appeared many decades after
the former. Thus, the pioneers of information theory (Shannon,
von Neumann) who retain the term "entropy", are undoubtly responsible
of the historical link between <<Entropy>> and <<Information>>
(e.g. see http://www.bartleby.com/64/C004/024.html).

Although "entropy" is a well known term in information theory, and used
coherently with the term "information" in this area, the situation
is different in science. I do not know what is "information" in
theermodynamics (does anybody know?). However, "chemical information"
is a well known area of chemistry, which covers many topics, including
data mining in chemical data bases. In fact, chemical information
was reognized as a major field when the ACS decided in 1975 to rename
one of its journals "Journal of Chemical Information and Computer Sciences":
it was previously named the "Journal of Chemical Documentation".
There are little papers in this journal which are connected with
entropy (thermodunamical of informational). An example is the
1996 paper of Shu-Kun Lin, relating entropy with similarity and
symmetry. Similarity is itself a major area in chemical information,
but I consider that the main area of chemical information is related
to chemical databases, such that the chemical information is represented
by the nodes and edges graph associated to a structural formula.
Actually, mathematical tools able to work on this kind of chemical
information are lacking, particulary for statistics (did anyone
performed statistics on graphs?).

In 1999, the links between information sciences and entropy were again
recognized, when Shu-Kun lin created the open access journal "Entropy":
<<An International and Interdisciplinary Journal of Entropy and
Information Studies>>. Although most pluridisciplinary journals
are at the intersection of two areas, Shu-Kun Lin is a pionneer
in the field of transdisciplinarity, permitting the publication
in a single journal of works related to entropy and/or information
theory, originating from mathematics, physics, chemistry, biology,
economy, and philosophy.

The concept of information exists in other sciences for which the
term entropy is used. Bioinformation is a major concept in bioinformatics,
for which I am not specialist. Thus I hope that Pedro Marijuan would like
to help us to understand what are the links between bioinformation and
entropy. Entropy and information are known from economists and philosophers.
I also hope they add their voice to those of scientists and mathematicians,
to enlight our discussions during the session.

Now I would like to draw some provocative conclusions. Analogies between
concepts or between formal expressions of quantities are useful
for the spirit, for the quality of the papers, and sometimes they
are used by modellers to demonstrate why their work merit funds
(does anybody never do that?). The number of new concepts in
sciences (includes mathematics, economy, humanities, and so on)
is increasing, and new terms are picked in our natural language:
the task of the teachers becomes harder and harder. Entropy
and Information are like the "fourth dimension", one century ago:
they offer in common the ability to provide exciting topics
to discuss. Unfortunately, Entropy and Information are much more
difficult to handle.

Michel Petitjean Email: petitjean@itodys.jussieu.fr
Editor-in-Chief of Entropy entropy@mdpi.org
ITODYS (CNRS, UMR 7086) ptitjean@ccr.jussieu.fr
1 rue Guy de la Brosse Phone: +33 (0)1 44 27 48 57
75005 Paris, France. FAX : +33 (0)1 44 27 68 14
http://www.mdpi.net http://www.mdpi.org
http://petitjeanmichel.free.fr/itoweb.petitjean.html
http://petitjeanmichel.free.fr/itoweb.petitjean.freeware.html
_______________________________________________
fis mailing list
fis@listas.unizar.es
http://webmail.unizar.es/mailman/listinfo/fis
Received on Mon Apr 5 09:22:34 2004