Information Theory and Chaos
SHANNON AND WEAVER'S INFORMATION THEORY
Gregory
Information Theory, originally developed by Claude Shannon for Bell Telephone Laboratories, was originally published in 1948. It is a highly abstract, general mathematical theory of communication. Shannon hoped that it could be used as a model for all types of communication, whether human, animal, cellular, or mechanical. In human communication its application is also very broad. Warren Weaver, in his paper explaining the theory said, AThe word communication will be used here in a very broad sense to include all of the procedures by which one mind may affect another.@ (Shannon and Weaver, 3). This may include speech, vocalics, proxemics, music, dance, or any other form of human communication.
Information theory is difficult for many people two understand at first glance because of its mathematical nature. It is also difficult because the theory uses common words, but assigns to them new meanings. For example, the word information is used in a way that is actually at odds with the way we have been taught to understand it. We associate information with meaning. In this theory information is not associated with meaning at all. Rather, it has to do with uncertainty, unpredictability, entropy and freedom of choice. The lack of information is associated with concepts of redundancy, certainty predictability, and a lack of free choice.
The diagram below is Shannon and Weaver's Model of Communication. It is made up of six elements. I will identify these elements in the context of human speech communication sense that is what we are primarily concerned with.
The source of the communication signal is the speaker's brain, the transmitter is his vocal system, the channel is air, the receiver is the listener's ear, and the destination is the listener's brain. Noise refers to any interfering signals that are added to the signal before it reaches the listener's ear (Littlejohn, 54). The source, i.e. the speaker's brain, chooses a message from all the possible message choices. The message is then encoded and transmitted by the vocal system through the air to the listener's ear. It then reaches its destination, the listener's brain, where it is decoded.
It was stated above that information was related to uncertainty, entropy and free choice. How so? The speaker must choose which message he wants to send. The number of choices he has depends upon his shared code with the listener, the context of the situation and other factors. He may be more inclined to make some choices over the others, meaning some of the messages are more probable than others. Conversely, all of the messages may be equally probable.
If the first situation is considered, the speaker has very little freedom of choice. There are only a few messages that are appropriate to the context, therefore they are much more probable than the others. For the listener this means a high level of certainty as to what he should expect. A good example of this is the message A, B, (?). What is likely to be the next signal? Most people will automatically say C. Why? Because this is immediately recognized as a highly redundant message. AC@ almost always follows the pattern created by A, B. Theoretically anything could be inserted after A,B, but this is very unlikely. The familiarity of this message to English speakers is so great that all other possible choices are considered highly improbable.
If the second situation applies, then the speaker has complete freedom of choice. There are no constraints on his ability or inclination to choose any particular message over the others. This means uncertainty for the listener. He is unable to predict what the message is likely to be with any accuracy because there is too much information for him to consider. This communication situation is high in entropy. An example of this is the message A, F, 4, (?). What will be the next signal in this pattern? Most people would hesitate to guess. Anything could follow - any letter of the English alphabet, and any number between 0 and 9. There is no recognizable pattern. The listener has too much information - too many possibilities to consider, therefore he is very uncertain of what the message will ultimately be or what it means.
As the number of possible messages increases, so does information, uncertainty, entropy and free choice. They also increase when all of the messages are equally probable, or nearly so. Information and the associated concepts decrease when the number of message choices decrease, or when a few or one message is far more probable than the others. This can also be shown by the following mathematical formula:
H = -Spilog2pi
H is the average amount of information and is equal to the negative sum of each probability p times the log of the probability p. H increases when the choices available are equally probable, and when the number of choices is greater. H decreases when one or a few choices are more probable than the others.
We have one more issue to discuss, and that is the three levels of communication problems.
Level A is: How accurately can the symbols of communication be transmitted. This is a technical problem.
Level B is: How precisely do the transmitted symbols convey the intended meaning. This is a semantic problem.
Level C is: How effectively does the received meaning affect conduct in the desired way; an effectiveness problem.
Information theory deals primarily with the Level A technical problem. The model is of a signal being sent from a source, through a channel to a final destination. In order to be received and decoded accurately, the message must have a sufficient level of redundancy or certainty. However, there must also be freedom of choice, otherwise the message would be completely redundant and would contain no new information at all. It is the balance between free choice and constraint, certainty and uncertainty, information and redundancy that makes communication possible.
CYBERNETICS
Jeanne
According to Theories of Human Communication (Littlejohn: 1996 p.42), the study of Cybernetics is one of two related fields that offer broad perspectives on how to look at the world. Cybernetics deals with the control and regulation in the systems. approaches to communication in the system theory. Cybernetics, is one of the general system theories, has been useful as a basis for many communication theories. (Monge: 1977) p.42
Cybernetics is part of the "open systems," the important feature in this study has emphasis on "feedack". "Open systems" are regulated, they seek goals, and are purposeful. Cybernetics, then, deals with the ways a system gauges its effect and makes necessary adjustments. (Handy and Kurtz: 1964) p.47
Figure 3.2: is a Model of Cybernetic Complexity: The Model illustrates the most basic distinction between active behavior: which comes from the system itself, and passive behavior that results strictly from outside stimulation. Active behavior can be considered purposeful or random, depending on something that is done to express and idea, or emphasize a point, then the action is done on purpose. Active can be subdivided into purposeless, or random; purposeful is directed toward an objective or aim; (e.g: wave hand) if on purpose is random behavior; however, if this is done to express an idea, or emphasize a point, then the action is clearly achieving a purpose. All purposeful behaviors requires "feedback". In simple systems, an organism responds to "feedback" by turning on or off; (e.g. thermostat). [Littlejohn, 5th ed. 1996: p.47). Complex systems use positive/negative "feedback"-adjusts and adapts during action itself. Complex systems may be predictive/nonpredictive depending on anticipated position or response rather than actual position or response. (e.g: quarterback) (Rosenbleuth, Weiner, Bigelow: 1943) p. 48.
There are three parts to the simplest cybernetic device which consist of a sensor, comparator, and activator. The sensor provides feedback that is needed to the comparator which determines whether a machine is deviating from its established norm. The comparator then provides guidance to the which in turn produces an output that affects the environment in some way. Cybernetics, then is a fundamental process of regulation and control in systems, with the emphasis on feedback. This is part of the simplest device known that is operated by cybernetics and which is called the process of "output-feedback-adjustment". (Buckley: (1968) and Guilbaud: 1959) p. 49
The most basic distinction is between active and passive behaviors. Active behavior comes from the system itself, while passive behavior strictly results from outside stimulation. (e.g: scratch, wave hand, rub face) are all active behavior; however, if a person is waving his/her hand, the action(s) may be on purpose, but is not random behavior, there is a reason for the wave. Active behavior can be further subdivided into purposeless or random behavior(s); thus if the action is done to express an idea, or emphasize a point then that action is clearly achieving a purpose. (e.g: quarterback-football-receiver etc) (Rosenblueth, Wiener, and Bigelow: 1943) p.48
This is part of the cybernetic system, or open system that receives matter and energy from its environment and then passes that matter and energy back to its environment. (Littlejohn 1996) p.48. "Open Systems" are oriented toward life and growth, and is a simple way to explain the "complex systems" which may be predictive or nonpredictive behaviors. Predictive behavior is based on the anticipated position or response rather than the actual position or response. (Buckley: 1967) p.49
Figure 3.3: A Simple Feedback Model: has a simple explanation: In this Figure- B is an energy source directing "outputs" to C. A is the control mechanism responding to "feedback" from C. Therefore, depending on the complexity of the system, and the nature of the "output" the control mechanism itself is restricted in the kind of control it can exert. (Buckley: 1968) (Guilbaud: 1959) p.48
The first model in Figure 3.4 demonstrates a situation, the signal itself is modified, (e.g: loudspeaker) is amplified by A the control mechanism. The second model illustrates a simple switch (e.g. thermostat or circuit breaker) that turns something on or off, as in a All-or-Nothing effect-either the loudspeaker is too loud, or shrieks, or it is modified, and not heard as clearly as before. The third model illustrates a selection control in which A chooses a channel or position on the basis of criteria: Take for instance a guided missile, it goes up toward a target, and something in the environment happens, causing a change in direction of the missile-the channel-control switch, send the change to the missile to go in a different direction based on the "feedback received from the target.
(Buckley, 1967) p.49
Figure 3.5 illustrates three system states: (1) steady: involves use of negative "feedback-keeps system on track. Negative "feedback" signals a deviation from the standard, and the system adjusts in order to return to the line. The system is always moving, changing: (2) growth: here system deviates, a positive "feedback" maintains the deviation, resulting further and further from the original state-acceleration continues-then system will disintegrate (e.g.relationship), (3) is change state:the system moves from one state to another, requiring both positive and negative "feedback". Positive feedback gets the system moving in a new direction, negative "feedback" at some level returns the system to balance. (e.g: manager, employees)
The discussion of "feedback" has given the impression that a system responds as a "unit" to feedback from the outside. This is realistic only in simplest systems,(e.g: heater) A series of hierarchically order of "subsystems", advanced systems are more complex. At any moment, a subsystem may be part of a larger system or part of the environment. We know that subsystems respond to one another and as a result we must expand the concept of "feedback" in complex systems, which is a series of "feedback loops" forming networks. At some point the "feedback loops" are positive, at other it is negative. (Maruyama 1963) P. 50.
Figure 3.6 A Simplified Feedback Network: is an example of urbanization; Pluses (+) are positive, minuses (-) are negative: (P) population(+)'number of people in city; goes over to (M)-(+)-Modernization: (M) (+) descends to (C) migration (+) into city and also down to (S) Sanitation facilities. (C) (+) goes back to (P) and (S) (-) goes up to (B) and to (D) (-) (D) (-) then returns to (P) (+) to (M); (M) to (C), (C) over to (P): left side of drawing: (P) (+) - (G) & (B) - (D): minuses: (-) (S) to (B) and (D) then (D) up to (P) (-). (Maruyama: 1963) p.51
In this report. I will show how Mathematical Modeling and the Super Computer have changed our lives. What effect it has had on past, present and in the future of man kind.
Who is William Farish? In 1792 he was the first person to use Mathematical concept in grading students papers. He was a TA at Cambridge University. He was able to come up with a quantitative value on grading students papers.
It was believed that if a number was given a quality, than you would be able to use it in determine mercy, love, hate, beauty, creativity, intelligence and even sanity itself.
Even Galileo once said, "Language of nature is written in Mathematics". In this statement we find that language has change. Language develops new meaning for words. New words like VCR, software, front wheel drive and etc is the new language.
With this technology of Satellites, computers, high-speed cables and other devices, we could see TV and computers replace teachers in the class room. For that matter schools will be replace by a TV and computer.
In my own experience in working for the United States Department of Agriculture, in Cotton Classification. When I started in 1975 there were no computers and everything was done by hand. A card was sent into the office with; sample of cotton. It was graded and sent back to the farmer. In 1979 the office received it's first IBM punch card machine. Then came the computers in 1984, for each office. In 1990 the offices throughout the cotton belt were a' 1 brought on line with Memphis TN. Then on to Washington D.C. The cotton gin was able to have direct lines to the cotton of ice to receive information on samples that were sent in to graded.
When I left in 1995 they employed 400 people at peak of the season. Today they only employ 150. In some way the computer is a good thing but it has left people unemployed.
Freud once said, "if there had been one railway to conquer distances, my child would never have left his native town and I should not need one telephone to hear his voice". "If travelling across the ocean by ship had not been introduced, my friend would not have embarked on sea-voyage and I should not need a cable to relieve my anxiety about him". "What is the use of reducing infantile mortality when it is precisely that reduction which imp s s the greater restraint on us in the begetting of children, so that, taken all round, we nevertheless rear no more children reign of hygiene".
So without computers and math there my not be a future.
"Chaos theory" has, in the space of two decades, emerged from the scientific literature into the popular spotlight. Chaos theory is seen as a revolutionary new way of thinking about complex systems -- brains, immune systems, atmosphere, ecosystems, you time it. The excitement about chaos theory stems from the perception that it somehow captures the complex "disorganized order" of the real world. But in fact, chaos theory in the technical sense has fewer well-developed real world applications.
Chaos theory is only a small part of the emerging paradigm of complex systems science. In the popular literature the word "chaos" is often interpreted very loosely, sometimes as a synonym for complex systems science. But the distinction is very important. Chaos theory has to do with determinism underlying apparent randomness. Complex systems science is more concerned with the emergent, synergetic behaviors of systems composed of a large number of interacting parts.
The word chaos has several different meanings. Today it doesn't stand for its original theological meaning, "Chaos" simply referred to the void existing between Heaven and Earth. In other words, it has virtually nothing to do with any of its current meanings.
When attempting to find the relation of chaos and communication certain factors come into play. You may ask, "What is the purpose of language?" The straightforward answer to this question is "communication." But what exactly does this term mean? The so-called "mathematical theory of communication," defined by Claude Shannon, deals with the surprise value of a message relative to a given ensemble of messages. But although this is marvelous mathematics and engineering, it has little to do with meaning. The communication of patterns is different from the communication of statistical information.
Let us consider five "illocutionary categories" into which Searle (1983) claims all speech acts may be categorized:
Dimetives, which attempts to get the speaker to do something. This category is inclusive of both commands and questions.
Commissives, which commit the speaker to do something -- say to join the Navy, or tell the truth in a court proceeding.
Declaritives, which, by virtue of being uttered, bring about the content of the utterance. For instance, "I pronounce you man and wife."
All of these categories have one obvious thing in common. They say that the speaker, by using a speech act, is trying to cause some infrin to obtain. In case of expressives and assertives, one is mainly trying to cause an infon (the content of one's statement) to obtain in the mind of the listener. In particular, among other things, one is telling the listener the situation in question/speaker--this content. In the case of assertives, one may also be trying to cause the situation in question/listener--this content to appear --that is, one may be trying to convince the listener to agree with you (Goertzel, 82. But at any rate, the most basic thing you are doing is trying to cause a record of what you think or feel to occur in his/her mind.
In the case of directiveness, one is trying to cause the listener to respond either with an assertive statement of her own (in the case of a question) or with some other sort of action. One is trying to make a certain infon appear in one's present physical situation, or in some future situation.
Finally, in the case of commissives and declaratives, things me even more direct. One is swearing oneself into the Navy, or declaring two people married. Within the network of beliefs that makes up one's subjective world, one actually causing certain infons to obtain.
So what communication really comes down to, is molding the world in a certain way. It partakes of the deductive and analogical system associated with a given language. Rather that defining language as that which communicates, we can define it as communication as the process of doing something with language.
In conclusion, all of this is obviously only a beginning; despite the numerous examples, it is fairly abstract and general, and many details remain to be filled in.
