Search This Blog

How Engineers Get Thermodynamics And Information Theory All Wrong

There is probably no other area of higher education where what is taught is so out of step with what is in fact valid. Engineering programs the world over, in the interest of simplicity and practicality, teach thermodynamics and information theory towards practicality and real-world solutions. What could be wrong with that? What is the negative side of practicality?

Well, usually, nothing. In most cases, cutting corners doesn't invert the causal bedrock upon which engineering is based. The field equations used to abstract relativity, do not usurp or demand a reformulation of E=mC^2. Neither do feynman diagrams mess with or disrupt an accurate understanding of quantum electro-dynamics. But in thermodynamics and information theory, the practical methods taught and used by engineers are based on assumptions that have resulted in an almost universal and wholesale misunderstanding of the base meaning and the causality that animates the bedrock of energy and information dynamics.

In thermodynamics, the problem is probably best described by the idea of "the perfect wall". To cut corners, engineers are taught arithmetic tricks that work in the usual atmospherically-dense and energy-conductive environments in which human's live. Unfortunately, these computational short-cuts do far more then introduce the usual errors of computational fidelity, they actually reverse the meaning of thermodynamics as a science. Thermodynamics as a science is about the way systems interact with the systems they are embedded within. But more than that, thermodynamics asserts the absolute necessity and inevitability of interaction and transference of energy that will result from ANY change within or without a system.

It should therefore be obvious that the teaching and use of practical methods that sidestep the central tenet of a field of science will have an unusually strong an adverse effect on the understanding of that science. Whole generations of engineers are being unleashed into the world with an absolutely backwards understanding of the very dynamic that universally informs all other dynamics. This is more than unfortunate. The growing population of scientists and engineers that march forward from universities with a backwards understanding of thermodynamics interferes with progress in all fields of science.

Same can be said of thermodynamics' sister, information theory. Because everything we do is increasingly keyed to progress in computation, the miss-map between the causal truths that inform information theory and the practical methods taught in their stead, may potentially have a much larger and deleterious impact on our potential as a species.

Where thermodynamics dictates the way energy leaks across the spacial dimensions, information theory dictates how information leaks across time. Purists will say that energy and information are equivalent. Ultimately, this is true. So when energy is measured in its more general form, as information, as bits, then information theory also dictates the lossy transfer of energy across time.

Because the two disciplines show how no system exists independent of other systems, we must concern ourselves with how systems are related through this leaking of energy and information. What can be said absolutely about the way information and energy set up directional relationships between systems with regard to space and time?

The Butterfly Effect; Isn't
In the none academic world, causality suffers a different abuse altogether. It is tempting for people to take notions of system interconnectedness to ridiculous and self-defeating extremes. We loose ground when the perfectly valid logic showing why a system can never act in isolation is illogically extrapolated to, "All systems effect all other systems equally". Making exceptions for speed of light (event cone) isolation, it can indeed be shown that all gravitational systems effect all other gravitational systems… the movement of a butterfly in South America will indeed effect (however infinitesimally) a dam in Montana. But if one were to rank, by degree of effect, all of the systems effecting the gravity fields surrounding a dam in Montana, a butterfly in Argentina would be very very low on the list. Even if one is butterfly obsessed, wants to ignore the one dog on the corner who has more mass than all of the butterfly's in the rocky mountains, there are tens of millions of butterflies closer, each of whom's infinitesimal gravitational pull would none the less have a larger causal effect on our poor dam's future.

This particularly populist breed of cause-and-effect miss-mappings is not the focus of my essay. As wacky as pedestrian notions become, they probably can't significantly derail scientific progress to any great degree. But when entire generations of science students are raised on incorrect understandings of basic science, we are all in trouble. This is especially devastating when the topic of delusion is as fundamental to the causal stack as is thermodynamics, energy and information.

"The law that entropy always increases, holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell's equations - then so much the worse for Maxwell's equations. If it is found to be contradicted by observation - well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation."

Sir Arthur Stanley Eddington, The Nature of the Physical World (1927)

What determines the causal morphology and behavior of the hierarchy of influence (dictated by thermodynamics and information theory)? If we define the shape of causality we define process itself, and by extension, the shape of reality.

Information Theory specifies ways to measure the capacity of a storage matrix and the reliability of a communication channel. But all of it's metrics are agnostic to the meaning encoded and transmitted. Each bit and each bit pattern are treated as equal. Only frequency and order, not meaning, not saliency, not fidelity of representation.

What would you have to fold into or add to information theory and thermodynamics in order to measure meaning and saliency? Is it there already? Are we missing something in our approach to and use of an already semantically robust set of laws and equations?

Several years ago, the mathematician Stephen Wolfram (founder of the maths software "Mathematica") wrote a book called "A New Kind Of Science". It is a dense and repetitive work over twelve hundred pages long. I tried to get through it and gave up. Feels like a giant fractal, built of some obscure philosophy based on fractals. Not feeling OK with my initial critique, I forced myself to come up with a theory, any theory, that said or not, I could attribute to his work. The best I could do was to suppose that Wolfram was trying to say that science had historically used equations to understand the components of nature that could be accurately described by equations, but the really interesting things about nature were iterative, and open ended, they required logical descriptions that required continuous computation. To bad he couldn't have just said that.

At about the same time, the social biologist Edward O. Wilson wrote a book called "Consilience". He argued for a cross-discipline coming-together of the various branches of scientific exploration, a holism, for the advantages of looking at nature (and those who study it) as the one large and interdependent super-system it is.

Of course dynamic, ever changing, "evolving" systems are systems simple equations (calculated once) will never accurately represent. Traditional thermodynamics and information theory engineering maths and methods work best on simple systems that are or can be thought of as repetitive and isolated. The conditions (input energy, output work) might change, but the conditions of the conditions never do. At any sufficiently salient level, real systems are never that well behaved or that removed from their environments or situations.

Real systems are direction of time dependent. It is more than ironic that the one scientific law that defines exactly why causal systems are non-reversible is used primarily by engineers who choose to use it in ways that ignore the direction of time it demands. I can forgive newtonian or relativistic or quantum physicists for ignoring the asymmetry of time… their maths don't require it. But thermodynamicists? Information theorists?

[more to come…]

Randall Reetz