The 2nd Law: Is Increased Entropy Stochastic (incidental) or Causal (intrinsic)?

Recent science news is dominated by the multi-trillion dollar experimental search for the Higgs boson particle. A definitive observation of the theorized, but illusive, Higgs will finally complete the verification of the Standard Model – the most respected mathematical model of the evolution of our universe, explaining the emergence of each of the known forces and all of the matter we can observe. In the Standard Model, the Higgs is responsible for gravity – surrounding the more pedestrian particles – lending them the property we call "mass". If the Higgs exists, it is important as the causal bridge between the quantum world of the small and the relativistic world of the large. How could a particle that causes gravity be so hard to find? Because it doesn't actually have mass. It is as a result, known as "weakly interacting". It is only when a whole bunch of Higgs get together and surround other particles that mass is detected, and then, only in the surrounded particles. The Higgs binds so tightly to other particles, that it takes an extraordinary amount of energy, to break it free so that its presence can be detected. This is what the "Large Hadron Collider" does – it smashes heavy atomic nucleus (stripped of their electrons) at energies equivalent to those of the first moments after the Big Bang when all of the matter and energy in the entire universe was still smaller than a single star.

But there is a far more fundamental question. Gravity is a property. It is domain-dependent. It is specific to and belongs to a class of objects of a particular makeup and composition. The existence or nonexistence of the Higgs has no effect upon other properties of the universe like electromagnetism.

But there is a candidate for a domain-independent attribute of any and all causal systems. This attribute has been labeled the "Causal Entropic Principle" – it is generally discussed within the context of the transfer of heat (at astronomical scales) – within the study of thermodynamics. It is the logical extension of the concept of increased entropy, as first postulated, measured, and later described as the 2nd Law of Thermodynamics. But now, a hundred and fifty years after the formalization the laws of thermodynamics (of the phenomena and parameters of the transfer of heat, of the ratio of potential energy and work) correlative investigations in the fields of information, communication, computation, language, energy/mass, logic, and structure have uncovered parallel principles and constraints.  It is reasonable now to understand the 2nd Law as a description of a fundamental constraint on any change, in any system, no matter what forces and materials are at play. We now understand the 2nd Law to describe the reduction in the quality (density) of the energy and or structure of the universe (or any part therein) as results any change at all. We have come to understand the 2nd Law as a constraint on the outcome of change in structure, which is to say "information", on its construction, maintenance, and or transfer. This insight has rendered an equivalence between energy and structure in much the same way that Einsteinian Relativity exposed the equivalence between energy and mass.

There is however a daemon lurking within our understanding of the 2nd Law, a daemon that threatens to undermine our understanding of causality itself, a daemon that, once defined, may provide the basis for an understanding of any self-consistent causal system, including but not exclusive of our own universe and its particular set of properties and behaviors.

The daemon of the 2nd Law is the daemon of stochastic – is 2nd Law dictated dissipation (entropy) statistical, or is statistics simply a tool we use in the absence of microscopic knowledge? Asked another way, is the reduction in the quality of energy or information that the 2nd Law demands of every action, a property of the universe or is it a property of the measurement or observation of the universe? Is action equivalent to measurement? Is there a measurement or stochastic class of action free of the entropy-increase demanded by the 2nd Law?

This question is of far greater consequence to the universe and the understanding of the universe than the mechanics of mass as it would describe and thus parameterize ALL action and ALL configuration and the precipitation or evolution of all possible action and configuration. Where the existence of the Higgs Boson may explain the source of mass and gravity in this universe, an understanding of the causal attributes leading to the behavior described by the 2nd Law of Thermodynamics might just provide a foundation from which any and all causal systems must precipitate.

The implications and issues orbiting this problem are many and deep. At stake is an demonstrative understanding of change itself. We tend to think of change as exception. But, can a thing exist without change? If not, what is the difference between data and computation, between thing and abstraction of thing, and profoundly, an answer to the question, can data exist without computation? Can thing exist outside of abstraction of thing?

In thermodynamics and information theory, an effort is made to distinguish process and stochastic process. Heat is defined as an aggregate property describing the average or holistic state of systems composed so many interacting parts to keep track of all of them individually. Heat is a calculous of sorts, a system of shortcuts that allows mathematics to be employed successfully to determine the gross state of a huge collection of similar parts. There is a tendency then to assume that the laws that describe heat are laws that only apply to aggregate systems where knowledge is incomplete.

Are there non-stochastic systems? Are there discrete systems or dynamic changes within systems for which the laws of thermodynamics don't apply? Does the Causal Entropic Principle apply if you know and can observe every attribute of, and calculate the exact and complete state of a dynamic system?

Such questions are more involved than they may seem on first reading. Answering them will expose the very nature of change, independent of domain, illuminating the causal chain that has resulted from full evolutionary lineage of the universe.

Randall Lee Reetz

Note: The Causal Entropic Principle isn't a complex concept. It is the simple application of the 2nd Law's demand for increased universal entropy as a result of every change in any system. It says that every action in every system must be that action that causes the largest reduction in the quality of information or energy (the greatest dissipation). It says that a universe has only one possible end state – heat death – and that processes that maximize the rate towards this end state will be evolutionarily favored (selected), simply because entropy-maximizing processes and structures demand a higher throughput of energy and thus end up dominating their respective locality. Such entropy-maximizing schemes are thus more likely to determine the structure and behavior of the event cone stretching off into the future. An obvious extension of this principle is that complexity, or more precisely, the family of complexity that can find, record, and process abstractions that represent the salient aspects (physics) of the (an) universe, will help that complexity better predict the shape and behavior it must assume to maximize its competitive influence upon the future of entropy maximization. The "Causal Entropic Principle" thus represents a logically self-consistant (scientific) replacement for the awkwardly self-centered and causally impossible "anthropomorphic principle" (which lacks a physical or causal explanation and leans heavily on painfully erroneous macroscopic stretching of the quantum electro dynamics). Stretching circular logic to its most obvious and illogical end, the anthropomorphic principle borrows awkwardly and erroneously and ironically form the Heisenberg / Uncertainty Principle by asserting the necessity of "observers" as a precursor to the emergence of complexity. The Causal Entropic Principle explains the production of localized complexity without the need for prior-knowledge, and does so within the bounds of, as a result of, the 2nd Law of Thermodynamics, by showing that localized complexity can both come into existence as a result of the constant increase in universal entropy, and more specifically, that localized complexity has an evolutionary advantage, and will thus out-compete, less complex structures. In a Causal Entropic Principle universe, intelligence is the expected evolutionary result of competition to reach heat death faster. Falling down is enhanced by a particular class of complexity that can come into existence as a natural result of things falling down. Should one form of such complexity "understand" the universe better than another form, it will have an advantage and will be more likely to influence the shape of complexity in the future. The better a system gets at abstracting the dynamics of its environment the more likely it will be able to eat other systems than be eaten by them. Where the anthropomorphic principle requires an a-priori "observer", the causal entropic principle simply requires the 2nd Law's demand for increased entropy, for things falling down.

Compression as Intelligence

Let me take a stab at defending compression as equivalent to intelligence.

Standard string compression (LZW, etc.) works by understanding and then exploiting the sequencing rules that result in the redundancy built into most (all?) languages and communication protocols.

Compression is necessary in any storage/retrieval/manipulation system for the simple reason that all systems are finite.  Any library, any hard drive, any computer memory… all finite.  If working with primary in-situ environments was as efficient as working with maps or abstractions we would never have to go through the trouble of making maps or abstracting and filtering and representing.

It might seem sarcastic even to say it, but a universe is larger than a brain.

You have however stumbled upon an interesting insight.  Where exactly is intelligence?  In classic Shannon information theory, and the communication metrics (signal/noise ratio) upon which it is based, information is a duality where data and cypher are interlocked.  In this model, you can reduce the size of your content, but only if you increase the size (or capacity) of the cypher.  Want to reduce the complexity of the cypher, well you are forced to accept the fact that your content will grow in size or complexity.  No free lunch!

In order to build a more robust cypher, one has to generalize in order find salience (the difference that make a difference) in a greater and greater chunk of the universe.  It is one thing to build an data crawler for a single content protocol, quite another to build a domain and protocol independent data crawler.  It is one thing to build hash trees based on word or token frequency and quite another to build them based on causal semantics (not how the words are sequenced, but how the concepts they refer to are graphed.

I think the main trouble you are having with this compression = intelligence concept has to do with a limited mapping of the word "compression".

Lets say you are driving and need to know which way to turn as you approach a fork in the road.  If you are equipped with some sort of mental abstraction of the territory ahead, or on a map, you can choose based on the information encoded into these representations.  But what if you didn't?  What if you could not build a map, either on paper, or in your head.  Then you would be forced to drive up each fork in turn.  In fact, had you no abstraction device, you would have to do this continually as you would not be able to remember the first road by the time you took the second.

What if you had to traverse every road in every city you came to just to decide which road you were meant to take in the first place?  What if the universe it self was the best map you could ever build of the universe?  Surely you can see that a map is a form of compression.

But lets say that your brain can never be big enough to build a perfect map of every part of the universe important to you.  Lets imagine that the map-building map you build in order to create mental memories of roads and cities is ineffective at building maps of biological knowledge or physics or the names and faces of your friends.  You will have to go about building unique map builders for each domain of knowledge important to you.  Eventually, every cubic centimeter of your brain will be full of domain-specific map making algorithms.  No room for the maps!

What you need to build sited is a universal map builder.  A map builder that works just as well for topological territory as it does for concepts and lists and complex n-dimensional pattern-scapes.

Do so and you will end up with the ultimate compression algorithm!

But your point about where the intelligence lies is important.  I haven't read the rules for the contest you sight, but if I were to design such a contest, I would insist that the final byte count of each entrants' data also include the byte count of the code necessary to unpack it.

I realize that even this doesn't go far enough.  You are correctly asserting that most of the intelligence is in the human minds that build these compression algorithms in the first place.

How would you go about designing a contest that correctly or more accurately measures the full complexity of both cypher and the content it interprets?

But before you do, you should take the time to realize that a compression algorithm becomes a smaller and smaller component of the total complexity metric the more often it is used.  How many trillions of trillions of bytes have been trimmed from the global data tree over the lifespan of use of MPEG or JPEG on video and images?  Even if you factor in a robust calculation of the quantum wave space inhabited by the humans brains that created these protocols it is plain to see that use continues to diminish the complexity contribution of the cypher no matter how complex.

Now what do you think?

Randall Lee Reetz

How Engineers Get Thermodynamics And Information Theory All Wrong

There is probably no other area of higher education where what is taught is so out of step with what is in fact valid. Engineering programs the world over, in the interest of simplicity and practicality, teach thermodynamics and information theory towards practicality and real-world solutions. What could be wrong with that? What is the negative side of practicality?

Well, usually, nothing. In most cases, cutting corners doesn't invert the causal bedrock upon which engineering is based. The field equations used to abstract relativity, do not usurp or demand a reformulation of E=mC^2. Neither do feynman diagrams mess with or disrupt an accurate understanding of quantum electro-dynamics. But in thermodynamics and information theory, the practical methods taught and used by engineers are based on assumptions that have resulted in an almost universal and wholesale misunderstanding of the base meaning and the causality that animates the bedrock of energy and information dynamics.

In thermodynamics, the problem is probably best described by the idea of "the perfect wall". To cut corners, engineers are taught arithmetic tricks that work in the usual atmospherically-dense and energy-conductive environments in which human's live. Unfortunately, these computational short-cuts do far more then introduce the usual errors of computational fidelity, they actually reverse the meaning of thermodynamics as a science. Thermodynamics as a science is about the way systems interact with the systems they are embedded within. But more than that, thermodynamics asserts the absolute necessity and inevitability of interaction and transference of energy that will result from ANY change within or without a system.

It should therefore be obvious that the teaching and use of practical methods that sidestep the central tenet of a field of science will have an unusually strong an adverse effect on the understanding of that science. Whole generations of engineers are being unleashed into the world with an absolutely backwards understanding of the very dynamic that universally informs all other dynamics. This is more than unfortunate. The growing population of scientists and engineers that march forward from universities with a backwards understanding of thermodynamics interferes with progress in all fields of science.

Same can be said of thermodynamics' sister, information theory. Because everything we do is increasingly keyed to progress in computation, the miss-map between the causal truths that inform information theory and the practical methods taught in their stead, may potentially have a much larger and deleterious impact on our potential as a species.

Where thermodynamics dictates the way energy leaks across the spacial dimensions, information theory dictates how information leaks across time. Purists will say that energy and information are equivalent. Ultimately, this is true. So when energy is measured in its more general form, as information, as bits, then information theory also dictates the lossy transfer of energy across time.

Because the two disciplines show how no system exists independent of other systems, we must concern ourselves with how systems are related through this leaking of energy and information. What can be said absolutely about the way information and energy set up directional relationships between systems with regard to space and time?

The Butterfly Effect; Isn't

In the none academic world, causality suffers a different abuse altogether. It is tempting for people to take notions of system interconnectedness to ridiculous and self-defeating extremes. We loose ground when the perfectly valid logic showing why a system can never act in isolation is illogically extrapolated to, "All systems effect all other systems equally". Making exceptions for speed of light (event cone) isolation, it can indeed be shown that all gravitational systems effect all other gravitational systems… the movement of a butterfly in South America will indeed effect (however infinitesimally) a dam in Montana. But if one were to rank, by degree of effect, all of the systems effecting the gravity fields surrounding a dam in Montana, a butterfly in Argentina would be very very low on the list. Even if one is butterfly obsessed, wants to ignore the one dog on the corner who has more mass than all of the butterfly's in the rocky mountains, there are tens of millions of butterflies closer, each of whom's infinitesimal gravitational pull would none the less have a larger causal effect on our poor dam's future.

This particularly populist breed of cause-and-effect miss-mappings is not the focus of my essay. As wacky as pedestrian notions become, they probably can't significantly derail scientific progress to any great degree. But when entire generations of science students are raised on incorrect understandings of basic science, we are all in trouble. This is especially devastating when the topic of delusion is as fundamental to the causal stack as is thermodynamics, energy and information.

"The law that entropy always increases, holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell's equations - then so much the worse for Maxwell's equations. If it is found to be contradicted by observation - well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation."

Sir Arthur Stanley Eddington, The Nature of the Physical World (1927)

What determines the causal morphology and behavior of the hierarchy of influence (dictated by thermodynamics and information theory)? If we define the shape of causality we define process itself, and by extension, the shape of reality.

Information Theory specifies ways to measure the capacity of a storage matrix and the reliability of a communication channel. But all of it's metrics are agnostic to the meaning encoded and transmitted. Each bit and each bit pattern are treated as equal. Only frequency and order, not meaning, not saliency, not fidelity of representation.

What would you have to fold into or add to information theory and thermodynamics in order to measure meaning and saliency? Is it there already? Are we missing something in our approach to and use of an already semantically robust set of laws and equations?

Several years ago, the mathematician Stephen Wolfram (founder of the maths software "Mathematica") wrote a book called "A New Kind Of Science". It is a dense and repetitive work over twelve hundred pages long. I tried to get through it and gave up. Feels like a giant fractal, built of some obscure philosophy based on fractals. Not feeling OK with my initial critique, I forced myself to come up with a theory, any theory, that said or not, I could attribute to his work. The best I could do was to suppose that Wolfram was trying to say that science had historically used equations to understand the components of nature that could be accurately described by equations, but the really interesting things about nature were iterative, and open ended, they required logical descriptions that required continuous computation. To bad he couldn't have just said that.

At about the same time, the social biologist Edward O. Wilson wrote a book called "Consilience". He argued for a cross-discipline coming-together of the various branches of scientific exploration, a holism, for the advantages of looking at nature (and those who study it) as the one large and interdependent super-system it is.

Of course dynamic, ever changing, "evolving" systems are systems simple equations (calculated once) will never accurately represent. Traditional thermodynamics and information theory engineering maths and methods work best on simple systems that are or can be thought of as repetitive and isolated. The conditions (input energy, output work) might change, but the conditions of the conditions never do. At any sufficiently salient level, real systems are never that well behaved or that removed from their environments or situations.

Real systems are direction of time dependent. It is more than ironic that the one scientific law that defines exactly why causal systems are non-reversible is used primarily by engineers who choose to use it in ways that ignore the direction of time it demands. I can forgive newtonian or relativistic or quantum physicists for ignoring the asymmetry of time… their maths don't require it. But thermodynamicists? Information theorists?

[more to come…]

Randall Reetz