How Engineers Get Thermodynamics And Information Theory All Wrong

There is probably no other area of higher education where what is taught is so out of step with what is in fact valid. Engineering programs the world over, in the interest of simplicity and practicality, teach thermodynamics and information theory towards practicality and real-world solutions. What could be wrong with that? What is the negative side of practicality?

Well, usually, nothing. In most cases, cutting corners doesn't invert the causal bedrock upon which engineering is based. The field equations used to abstract relativity, do not usurp or demand a reformulation of E=mC^2. Neither do feynman diagrams mess with or disrupt an accurate understanding of quantum electro-dynamics. But in thermodynamics and information theory, the practical methods taught and used by engineers are based on assumptions that have resulted in an almost universal and wholesale misunderstanding of the base meaning and the causality that animates the bedrock of energy and information dynamics.

In thermodynamics, the problem is probably best described by the idea of "the perfect wall". To cut corners, engineers are taught arithmetic tricks that work in the usual atmospherically-dense and energy-conductive environments in which human's live. Unfortunately, these computational short-cuts do far more then introduce the usual errors of computational fidelity, they actually reverse the meaning of thermodynamics as a science. Thermodynamics as a science is about the way systems interact with the systems they are embedded within. But more than that, thermodynamics asserts the absolute necessity and inevitability of interaction and transference of energy that will result from ANY change within or without a system.

It should therefore be obvious that the teaching and use of practical methods that sidestep the central tenet of a field of science will have an unusually strong an adverse effect on the understanding of that science. Whole generations of engineers are being unleashed into the world with an absolutely backwards understanding of the very dynamic that universally informs all other dynamics. This is more than unfortunate. The growing population of scientists and engineers that march forward from universities with a backwards understanding of thermodynamics interferes with progress in all fields of science.

Same can be said of thermodynamics' sister, information theory. Because everything we do is increasingly keyed to progress in computation, the miss-map between the causal truths that inform information theory and the practical methods taught in their stead, may potentially have a much larger and deleterious impact on our potential as a species.

Where thermodynamics dictates the way energy leaks across the spacial dimensions, information theory dictates how information leaks across time. Purists will say that energy and information are equivalent. Ultimately, this is true. So when energy is measured in its more general form, as information, as bits, then information theory also dictates the lossy transfer of energy across time.

Because the two disciplines show how no system exists independent of other systems, we must concern ourselves with how systems are related through this leaking of energy and information. What can be said absolutely about the way information and energy set up directional relationships between systems with regard to space and time?

The Butterfly Effect; Isn't

In the none academic world, causality suffers a different abuse altogether. It is tempting for people to take notions of system interconnectedness to ridiculous and self-defeating extremes. We loose ground when the perfectly valid logic showing why a system can never act in isolation is illogically extrapolated to, "All systems effect all other systems equally". Making exceptions for speed of light (event cone) isolation, it can indeed be shown that all gravitational systems effect all other gravitational systems… the movement of a butterfly in South America will indeed effect (however infinitesimally) a dam in Montana. But if one were to rank, by degree of effect, all of the systems effecting the gravity fields surrounding a dam in Montana, a butterfly in Argentina would be very very low on the list. Even if one is butterfly obsessed, wants to ignore the one dog on the corner who has more mass than all of the butterfly's in the rocky mountains, there are tens of millions of butterflies closer, each of whom's infinitesimal gravitational pull would none the less have a larger causal effect on our poor dam's future.

This particularly populist breed of cause-and-effect miss-mappings is not the focus of my essay. As wacky as pedestrian notions become, they probably can't significantly derail scientific progress to any great degree. But when entire generations of science students are raised on incorrect understandings of basic science, we are all in trouble. This is especially devastating when the topic of delusion is as fundamental to the causal stack as is thermodynamics, energy and information.

"The law that entropy always increases, holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell's equations - then so much the worse for Maxwell's equations. If it is found to be contradicted by observation - well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation."

Sir Arthur Stanley Eddington, The Nature of the Physical World (1927)

What determines the causal morphology and behavior of the hierarchy of influence (dictated by thermodynamics and information theory)? If we define the shape of causality we define process itself, and by extension, the shape of reality.

Information Theory specifies ways to measure the capacity of a storage matrix and the reliability of a communication channel. But all of it's metrics are agnostic to the meaning encoded and transmitted. Each bit and each bit pattern are treated as equal. Only frequency and order, not meaning, not saliency, not fidelity of representation.

What would you have to fold into or add to information theory and thermodynamics in order to measure meaning and saliency? Is it there already? Are we missing something in our approach to and use of an already semantically robust set of laws and equations?

Several years ago, the mathematician Stephen Wolfram (founder of the maths software "Mathematica") wrote a book called "A New Kind Of Science". It is a dense and repetitive work over twelve hundred pages long. I tried to get through it and gave up. Feels like a giant fractal, built of some obscure philosophy based on fractals. Not feeling OK with my initial critique, I forced myself to come up with a theory, any theory, that said or not, I could attribute to his work. The best I could do was to suppose that Wolfram was trying to say that science had historically used equations to understand the components of nature that could be accurately described by equations, but the really interesting things about nature were iterative, and open ended, they required logical descriptions that required continuous computation. To bad he couldn't have just said that.

At about the same time, the social biologist Edward O. Wilson wrote a book called "Consilience". He argued for a cross-discipline coming-together of the various branches of scientific exploration, a holism, for the advantages of looking at nature (and those who study it) as the one large and interdependent super-system it is.

Of course dynamic, ever changing, "evolving" systems are systems simple equations (calculated once) will never accurately represent. Traditional thermodynamics and information theory engineering maths and methods work best on simple systems that are or can be thought of as repetitive and isolated. The conditions (input energy, output work) might change, but the conditions of the conditions never do. At any sufficiently salient level, real systems are never that well behaved or that removed from their environments or situations.

Real systems are direction of time dependent. It is more than ironic that the one scientific law that defines exactly why causal systems are non-reversible is used primarily by engineers who choose to use it in ways that ignore the direction of time it demands. I can forgive newtonian or relativistic or quantum physicists for ignoring the asymmetry of time… their maths don't require it. But thermodynamicists? Information theorists?

[more to come…]

Randall Reetz

Are We But Crows? (Where Pattern is Noise)

Lets look at yet another way that the human mind gets tripped up and falters. Our propensity to find pattern makes us vulnerable to an inane form of overindulgence and counterproductive obsession with attributes of systems that do not carry essence or salience. Seems that our evolutionarily-shaped affinity for pattern make us particularly vulnerable to a level of indulgence that in smaller quantities is reasonable and healthy, even necessary.

Here is the problem. Most pattern is indicative of a lack of information or salience. If a system can get away with simple duplication along a scheme, it means that that part of the system is not very important. It means that it is cutting resource corners and systems can only get away with this type of cost cutting in areas that are not salient to the main purpose of the system at large.

Imagine going to a bookstore and obsessing over the fact that all of the books are made of the same stuff, of paper sheets cut and stacked in approximately the same geometric ratio. Imagine, in fact that this pattern overwhelmed your attention to such a degree that you could not be bothered with the content encoded as printed words on the pages of those books.

That would render all books equivalent. A book printed with random strings of words or with no words at all would be informationally equivalent to Darwin's, Origin Of The Species. In the face of an overwhelming attraction to inessential pattern, essential pattern is dismissed as noise, is ignored.

I call this the "Crow" problem… an obsessive attraction to shiny objects. There are so many examples of this type of cognitive distraction; rainbows, crystals, mirages, amber waves of grain, camouflage, fractals, golden rectangles, sacred numbers, etc.  It is where our own cognitive process amplifies some pattern in the world wether or not it is important in any information rich manor.

We are for instance, especially susceptible to and sensitive to right/left symmetry. Probably because filtering for this pattern allows you to quickly pull animals out of a complex sensory field. Having such a filter gave us evolutionary advantage, so we now have brains with a tendency to favor right/left symmetry in visual fields.

The rainbow is an other great example of pattern that is attractive but virtually meaningless. A rainbow is not actually a thing of course, but a mix of the refractive nature of light and a behavioral anomaly of our visual apparatus. A rainbow is like a hologram. What we are seeing is light reflected off of raindrops or fog. Of course it is a poor indicator of the location of rain in our visual field… there is rain in other parts of the sky than where the colored bands appear. The phenomena that is visible as a rainbow is happening every time rain is present but we only see the rainbow when our orientation to the sun is within certain fairly restrictive parameters. Yet, so attractive is the rainbow stimuli, under the influence of this pattern detection stimulus, we might just miss more pertinent information (a lion approaching?).

Another example of particular importance is the attraction to crystals and the latest crystal fad, fractal geometry.

Most people are interested in fractals (or think they should be). But what exactly do you actually KNOW about fractals? Does your knowledge of fractals include an understanding of the WHY of n-dimensional radial n-scale self-similar patterns, why they appear in dissipative systems that develop over time? Do you care to know or understand? I get the feeling that people who are the most attracted to fractals are more interested in some sort of pseudo-spiritual grooviness, the stare at your navel aspect of fractals, than the simple truth of least-energy dictated growth patterns.

What is abstracted in mathematics as the "fractal" is in fact the only pattern that nature can exhibit in systems that are dominated by parts that are very much the same. Sand is a good example. Homogenous liquids and gasses are another. Fractals are a map or abstraction of dissipative systems, and all systems are dissipative. But the events and situations that matter in the evolution of complexity, the WHY that makes US possible, are not the even dissipation within a scheme, but the CREATIVE events that allow for new (faster and bigger) dissipative paradigms. An explosion is fractal in the same way that a capillary web is fractal in the same way that both a tributary and alluvial plane is fractal. It is just the end result of a history where least energy dictates the most efficient moment to moment dissipation of energy. The way you are saying that fractals are important feels religious and rapturous. As though fractal patterns in nature are a THING or a GOAL. Fractals don't know they are fractals any more than the ink making up the word "Fractal" on a printed page "knows" or "wants" to be that particular word or any word at all.

People who act this way towards pattern scare me. What matters in process is the places where pattern breaks. Else we wouldn't be here. Else we couldn't find salience.

I am afraid of and purposefully vigilant against the grand attractors of human thought, virgin births, miracles of any kind, shiny baubles, trickery, omnipotence, anecdote, life ever after, self importance, power over others, a desire to know or have access to the prediction of future events, predetermination, and pertinent to this discussion, crystals and fractals and the types of pseudo-cyclical pattern that makes us think nature is more simple than it is and that nature is of our own imagination or invention.

All systems seek equilibrium. In systems made up of subsystems, each of these subsystems seek their own equilibrium or resonance. Frequently when the resonances of several co-systems fall into an overlapping cycles, additive standing waves can and do overwhelm the integrity of the system as a whole. Feedback loops are anti-complexity mechanisms. Overarching patterns overwhelm systems and keep them from creative or information rich activities and interactions. Systems that seek to compete in the edge of evolution game are systems that spend an inordinate amount of energy keeping feedback loops in check. A creative system is a system working overtime AGAINST the information killer that is natural tendency towards simple pattern (fractals).

If you are trying to get rid of a planet or smooth out the heat cline in an ocean, fractal patterns are important. If you are a member of a species that is carrying complexity forward through the accumulation and control over abstraction, dissipative pattern and the systems dominated by them, are like Kryptonite to the program at hand… best to avoid them at all cost.

It is important to know when to appreciate pattern and when to run like hell when you see it developing. The forces that produce pattern are information destroyers.

We had better look long and hard at this issue. Choosing a false god at this level in the evolutionary game could end up causing the death of the whole complexity scheme just as it is becoming aware of its purpose and salient pattern.

As another example of pattern distraction, I give you DNA.

My favorite morphological description of DNA: "the non-periodic crystal". It harnesses the strong self-organizing (but anti-information) properties that give rise to a crystal (in this case, an almost endless wound chain of identical and stable "double-helix" spiral twists) as a stable superstructure for that will carry the information (anti-crystal) bearing base-pairs at its axillary center. The crystal spiral holds and protects the integrity of the always-vulnerable low-entropy sequence that holds our genetic recipe… DNA's real structure of importance. Even so, we humans seem to be more attracted (like crows) to the simplicity of the simple and un-remarkable spiral crystal armature instead of the non-crystaline information the crystal armature makes possible!

The larger implication of our self-destructive crow-like attraction to simplicity is that we are romantic about the very things that don't matter and ignore-ant of the anti-crystal configurations that are complex and information rich… that are salient to the information we are and the information our information may yet create.

I suspect that our attraction to simplicity in pattern is an idiosyncratic after-effect of a salience detector within our thinking apparatus that is constantly looking for self-similarity. When computer scientists and logicians attempt to design compression schemes, they are looking for ways to algorithmically discover those sections or sequences of a set of information that are un-important to the overall structure… much of data is repetitive, if you can find these repetitions, patterns, you can reduce them to simple equations or pointers to prototypical modules that can be stored once and forever duplicated as filler. Obviously, these patters are compressible because they cary so little salient information.

While it is somewhat surprising that so much of an image, musical recording, even string of text, can, using fractal math, be detected as filler and deleted, the subsequent and predictable human reaction… that of honoring the trash simply because it is pretty, is to my mind, one of the most scary attributes of human proclivity. Fractals illustrate how much of nature's structure is non-salient, is the noise, the tailings of dissipation. That we would be enamored by the pattern of noise to such an extent that we ignore the real meat of information (that which is not pattern-able) is our potential downfall as a species.

A snail isn't trying to build spiral. A snail grows a shell that happens to be a spiral because that is THE ONLY SHAPE allowed in three dimensions that is both a cone (expanding tube) and dimensionally minimal of solid permanent material. The snail has better things to waste its limited energy on than the shape of its protective home. So it chooses the one shape that takes the least energy and most minimal construction algorithm to pull off. We dishonor the snail as a biological scheme by paying attention to the aspect of its survival that is the least interesting. That simplicity of that spiral shell is what should tell us not to pay attention to that aspect of its being or strategy.

But in this regard and others, we act as crows despite our 100 billion brain cells and the potential therein.

Sad.

Randall Reetz