Prediction Schemes: Classicism vs. Non-linear vs. Thermodynamics

Thermodynamics and information theory are often grouped with classical dynamics. This is especially true where theory space is cleaved with quantum dynamics and other quote/unquote "non-deterministic" or "non-linear" theories on one side. But such classifications are problematic for several important reasons. Traditionally, the criteria of inclusion within the rubric "classical" has leaned heavily upon the concept of computation from knowledge of initial conditions. in Newtonian (and Relativistic) dynamics, knowing the initial state of a system allows one to calculate and thus predict the state of that system at any time in the future. Accuracy in prediction, from a classical perspective, is gated only by accuracy of knowledge of the original conditions of that system. Enter now, the strange world of quantum dynamics, where indeterminacy and sensitivity to observation turn classical calculations on their head. Non-clasical systems are systems in which determinism actually works against accuracy of prediction. The more you try to increase your knowledge of the initial conditions of a quantum situation, the less accurately you can predict that system's future.  Much is made of the philosophical implications of observer "relativity" in an Einsteinium space/time model, but vantage-sensitivity is absolutely classical – the more you know about the initial conditions, the more accurate will be your relativistic predictions. In the quantum world, knowledge is itself, a cost of business attribute. In the quantum world, knowledge perturbs. In the quantum world, a system that seeks to know itself, is a system that is changed. In the quantum world, there are two types of systems, systems that are statistically perturbed, and systems that are locally perturbed. Meaning, you can measure (observe) aspects of a whole system without messing with that system, but should you want discrete knowledge of individual particles within that system, you must pay the price of a system that is forever thereafter disturbed. It is interesting how closely the empirically observed quantum world mimics the limits Kurt Godel placed on absolute knowledge.  OK, let us now contrast thermodynamics, specifically the second law of thermodynamics, against both classical or deterministic dynamics and quantum indeterminacy. If one accepts that purpose of knowledge is prediction, is fidelity of calculation to actual future states, than both classical and non-linear theory are self-limitiing. Classical prediction is hampered by limits to the accuracy of observation of the initial state. Quantum prediction is limited by the way systems are perturbed by measurement, the more you know, the more you must include yourself into to prediction calculations, and the more said act is limited by Godel's caps on self-knowledge. One could say that classical prediction is dependent at base upon naiveté, and that quantum prediction is limited by knowledge itself. But what of the second law? The second law allows for absolute knowledge of the end state, of "heat death" or complete dissipation. Unlike all other forms of theoretical abstraction, the second law is absolutely agnostic to initial condition(s). You can use Newton's laws to look into the immediate future of a gravitationally bound system, but the same laws are meaningless in a system perturbed by other forces. Thermodynamic theory doesn't care what forces or materials are at play, it only cares about difference. In fact, thermodynamics doesn't know for the difference between material and force. The second law says that difference will always be less after any change in any system. The second law says that a change in any system will always result in the greatest possible reduction in difference. And importantly, the second law flips determinism on its head by providing perfect knowledge of the final state and doing so absolutely independent of any knowledge of initial conditions. Well that is certainly interesting, a theory that can predict the ultimate future independent of any past or present configuration, or, for that matter, any knowledge what so ever. What can be said of the quality or quantity of action that can be taken as result of this strange sort of knowledge? If success in competition can be linked to accuracy and capacity to predict, than what can be said of competitive success as a function of range of prediction? Imagine one could make and than order all possible predictions from most immediate to most long term. Comparing short-term against long-term predictions, which have the greatest impact on competitive advantage? If someone came into your office today and said, "I can say with absolute confidence that you will die as an artist in Copenhagen", how would such knowledge effect your future decisions and actions? How would absolute knowledge of your ultimate future effect your behavior?  What if we were to compare the influence of such knowledge to short term knowledge of the same certainty? What if that same person came into your office and instead declared, "I have no knowledge of your ultimate fate, but I do know that you will not be able to fall asleep tonight". Would you be more (or less) likely to change or conform your plans or to take action based on short term predictions? There might be a tendency to ignore predictions that are far removed in time. One might reasonably think, "Even if I know that I will become an artist and eventually die in Copenhagen, I have a life to live until then, concentrating on long term eventualities interferes with my ability to successfully negotiate success in the short term, in the here and now. But it might also be reasonable to try to conform local goals to long term eventualities. One might eliminate actions that one feels will make it harder to plot a path towards know eventualities. Or, one might take risks they would not otherwise have taken. If I know I will die in Copenhagen, I might as well go base jumping in the Andes or climb Everest sans bottled oxygen. Surely, the heat death of the universe is an eventuality of much greater philosophical remove. What's more, evolution, as a process, seems to work just fine in the absence of any knowledge of eventualities. Can one make an argument that knowledge of universal eventuality gains its owner any special form of evolutionary advantage? Lets pit two entities against each other, one knows of heat death, the other doesn't. Which has the evolutionary advantage?

Randall Lee Reetz, January 26, 2012

The 2nd Law: Is Increased Entropy Stochastic (incidental) or Causal (intrinsic)?

Recent science news is dominated by the multi-trillion dollar experimental search for the Higgs boson particle. A definitive observation of the theorized, but illusive, Higgs will finally complete the verification of the Standard Model – the most respected mathematical model of the evolution of our universe, explaining the emergence of each of the known forces and all of the matter we can observe. In the Standard Model, the Higgs is responsible for gravity – surrounding the more pedestrian particles – lending them the property we call "mass". If the Higgs exists, it is important as the causal bridge between the quantum world of the small and the relativistic world of the large. How could a particle that causes gravity be so hard to find? Because it doesn't actually have mass. It is as a result, known as "weakly interacting". It is only when a whole bunch of Higgs get together and surround other particles that mass is detected, and then, only in the surrounded particles. The Higgs binds so tightly to other particles, that it takes an extraordinary amount of energy, to break it free so that its presence can be detected. This is what the "Large Hadron Collider" does – it smashes heavy atomic nucleus (stripped of their electrons) at energies equivalent to those of the first moments after the Big Bang when all of the matter and energy in the entire universe was still smaller than a single star.

But there is a far more fundamental question. Gravity is a property. It is domain-dependent. It is specific to and belongs to a class of objects of a particular makeup and composition. The existence or nonexistence of the Higgs has no effect upon other properties of the universe like electromagnetism.

But there is a candidate for a domain-independent attribute of any and all causal systems. This attribute has been labeled the "Causal Entropic Principle" – it is generally discussed within the context of the transfer of heat (at astronomical scales) – within the study of thermodynamics. It is the logical extension of the concept of increased entropy, as first postulated, measured, and later described as the 2nd Law of Thermodynamics. But now, a hundred and fifty years after the formalization the laws of thermodynamics (of the phenomena and parameters of the transfer of heat, of the ratio of potential energy and work) correlative investigations in the fields of information, communication, computation, language, energy/mass, logic, and structure have uncovered parallel principles and constraints.  It is reasonable now to understand the 2nd Law as a description of a fundamental constraint on any change, in any system, no matter what forces and materials are at play. We now understand the 2nd Law to describe the reduction in the quality (density) of the energy and or structure of the universe (or any part therein) as results any change at all. We have come to understand the 2nd Law as a constraint on the outcome of change in structure, which is to say "information", on its construction, maintenance, and or transfer. This insight has rendered an equivalence between energy and structure in much the same way that Einsteinian Relativity exposed the equivalence between energy and mass.

There is however a daemon lurking within our understanding of the 2nd Law, a daemon that threatens to undermine our understanding of causality itself, a daemon that, once defined, may provide the basis for an understanding of any self-consistent causal system, including but not exclusive of our own universe and its particular set of properties and behaviors.

The daemon of the 2nd Law is the daemon of stochastic – is 2nd Law dictated dissipation (entropy) statistical, or is statistics simply a tool we use in the absence of microscopic knowledge? Asked another way, is the reduction in the quality of energy or information that the 2nd Law demands of every action, a property of the universe or is it a property of the measurement or observation of the universe? Is action equivalent to measurement? Is there a measurement or stochastic class of action free of the entropy-increase demanded by the 2nd Law?

This question is of far greater consequence to the universe and the understanding of the universe than the mechanics of mass as it would describe and thus parameterize ALL action and ALL configuration and the precipitation or evolution of all possible action and configuration. Where the existence of the Higgs Boson may explain the source of mass and gravity in this universe, an understanding of the causal attributes leading to the behavior described by the 2nd Law of Thermodynamics might just provide a foundation from which any and all causal systems must precipitate.

The implications and issues orbiting this problem are many and deep. At stake is an demonstrative understanding of change itself. We tend to think of change as exception. But, can a thing exist without change? If not, what is the difference between data and computation, between thing and abstraction of thing, and profoundly, an answer to the question, can data exist without computation? Can thing exist outside of abstraction of thing?

In thermodynamics and information theory, an effort is made to distinguish process and stochastic process. Heat is defined as an aggregate property describing the average or holistic state of systems composed so many interacting parts to keep track of all of them individually. Heat is a calculous of sorts, a system of shortcuts that allows mathematics to be employed successfully to determine the gross state of a huge collection of similar parts. There is a tendency then to assume that the laws that describe heat are laws that only apply to aggregate systems where knowledge is incomplete.

Are there non-stochastic systems? Are there discrete systems or dynamic changes within systems for which the laws of thermodynamics don't apply? Does the Causal Entropic Principle apply if you know and can observe every attribute of, and calculate the exact and complete state of a dynamic system?

Such questions are more involved than they may seem on first reading. Answering them will expose the very nature of change, independent of domain, illuminating the causal chain that has resulted from full evolutionary lineage of the universe.

Randall Lee Reetz

Note: The Causal Entropic Principle isn't a complex concept. It is the simple application of the 2nd Law's demand for increased universal entropy as a result of every change in any system. It says that every action in every system must be that action that causes the largest reduction in the quality of information or energy (the greatest dissipation). It says that a universe has only one possible end state – heat death – and that processes that maximize the rate towards this end state will be evolutionarily favored (selected), simply because entropy-maximizing processes and structures demand a higher throughput of energy and thus end up dominating their respective locality. Such entropy-maximizing schemes are thus more likely to determine the structure and behavior of the event cone stretching off into the future. An obvious extension of this principle is that complexity, or more precisely, the family of complexity that can find, record, and process abstractions that represent the salient aspects (physics) of the (an) universe, will help that complexity better predict the shape and behavior it must assume to maximize its competitive influence upon the future of entropy maximization. The "Causal Entropic Principle" thus represents a logically self-consistant (scientific) replacement for the awkwardly self-centered and causally impossible "anthropomorphic principle" (which lacks a physical or causal explanation and leans heavily on painfully erroneous macroscopic stretching of the quantum electro dynamics). Stretching circular logic to its most obvious and illogical end, the anthropomorphic principle borrows awkwardly and erroneously and ironically form the Heisenberg / Uncertainty Principle by asserting the necessity of "observers" as a precursor to the emergence of complexity. The Causal Entropic Principle explains the production of localized complexity without the need for prior-knowledge, and does so within the bounds of, as a result of, the 2nd Law of Thermodynamics, by showing that localized complexity can both come into existence as a result of the constant increase in universal entropy, and more specifically, that localized complexity has an evolutionary advantage, and will thus out-compete, less complex structures. In a Causal Entropic Principle universe, intelligence is the expected evolutionary result of competition to reach heat death faster. Falling down is enhanced by a particular class of complexity that can come into existence as a natural result of things falling down. Should one form of such complexity "understand" the universe better than another form, it will have an advantage and will be more likely to influence the shape of complexity in the future. The better a system gets at abstracting the dynamics of its environment the more likely it will be able to eat other systems than be eaten by them. Where the anthropomorphic principle requires an a-priori "observer", the causal entropic principle simply requires the 2nd Law's demand for increased entropy, for things falling down.

The Problem with Darwin…

Ya… how would you look as Darwin?

Darwin Darwin Darwin. Darwin is a problem. It isn't that he was wrong. In fact, it is very very hard to find any kind of mistake in his theory or his supporting data and arguments. What makes Darwin problematic is his myopic assignment of the process of evolution to the domain of biology. In doing so, Darwin has (inadvertently) misled generations of readers, who now confuse biology's "how" in evolution with big "E" Evolution in all domains. Big "E" Evolution is informative because it describes the more general "why" driving the direction of change in ALL domains.

When understood as a "how", the process of evolution is reduced to orrery – like the awkward clockworks that spin planets and moons around concentric bearings – substituting method where there should be cause. How is always specific to domain, but why, the ultimate why, is general enough to explain all of the how's. Armed with a robust understanding of the big WHY of evolution, one should be able to walk into any domain and predict and then map it's how. Again, it isn't that Darwin's evolution orrery doesn't accurately predict biological patterns of change, or even that Darwin's evolution orrery doesn't accurately abstract the salient causal aspects of biological change, it is that Darwin's how of evolution in biology leads people to the idea that evolution is specific and exclusive to biology, or that one can understand evolution in other domains by overlaying biology's how.

Darwin never generalized the process of evolution. Imagine had Newton and Einstein had not generalized dynamics and motion and that we had, as a result, built all of our machines on the principle that motion was caused by legs and feet.

The people who have come the closest to the generalization of evolution, the thermodynamisists, have never been able to or interested in the development of a generalization of the direction of change and the cause of that direction. I will get back to this absence of generalization in the understanding of evolution but right now will only hint at an explanation… in the aftermath of the all too human race and cultural superiority wars and atrocities, it has been socially dangerous to think of evolution as having a direction as such thoughts can be read as rhetorical arguments for superiority and pre-judgement, the likes of which were used by Hitler, Stalin, Pol Pot, Mao, and others as justification for mass exterminations and other exclusionary policies. That humans have the proclivity to exploit incomplete knowledge in the pursuit of ridiculous selfishness at absurd scales should be nothing new or noteworthy. But no one would advocate the cessation of the study of chemistry simply because arsenic is a chemical, or the study of high energy physics simply because the atom bomb can be built from such knowledge.

Or would we? Cautionary reactions to the self-superior pogroms that so blighted the 20th century have driven several generations of researchers towards the relativist rhetoric we see most prominently in the post-modernist movement, but which is evident in the works of less irrational and otherwise, empirical scientists like Stephen J. Gould and Richard Dawkins. Both represent an interesting study in overcompensation. In their quest to irradiate the all-to-natural self-superiority that seems to cause humans to erect unfounded tautologies that place humans on top of pre-destined hierarchies, both argue and argue brilliantly, for a flat evolutionary environment in which change happens but without any directionality at all. Again, this is like saying that because metal can be shaped into swards and knives and guns it shouldn't be produced even should we need plows and trains and dynamos and bridges and buildings and printing presses and lab equipment and computers.

Of course, caution is its own form of rhetoric, as potentially dangerous as its more obviously tyrannous cousins.

And, yes! Evolution has a direction. There I said it! Say it with me. You won't be struck down by post-modernist lightning. Trust me. Trust your self. It is more than a little absurd that one would have to argue for direction in a process that explains directionality. They are of course correct in their assertion that evolution isn't pre-determined. Nothing is. Of course. But the "brilliance" of evolution is that it results in a direction without need for prior knowledge, plan, or determination of any kind. To toss this most salient aspect of the evolutionary process simply to make a sociological point seems reckless in the maximum.

Randall Lee Reetz

Compression as Intelligence

Let me take a stab at defending compression as equivalent to intelligence.

Standard string compression (LZW, etc.) works by understanding and then exploiting the sequencing rules that result in the redundancy built into most (all?) languages and communication protocols.

Compression is necessary in any storage/retrieval/manipulation system for the simple reason that all systems are finite.  Any library, any hard drive, any computer memory… all finite.  If working with primary in-situ environments was as efficient as working with maps or abstractions we would never have to go through the trouble of making maps or abstracting and filtering and representing.

It might seem sarcastic even to say it, but a universe is larger than a brain.

You have however stumbled upon an interesting insight.  Where exactly is intelligence?  In classic Shannon information theory, and the communication metrics (signal/noise ratio) upon which it is based, information is a duality where data and cypher are interlocked.  In this model, you can reduce the size of your content, but only if you increase the size (or capacity) of the cypher.  Want to reduce the complexity of the cypher, well you are forced to accept the fact that your content will grow in size or complexity.  No free lunch!

In order to build a more robust cypher, one has to generalize in order find salience (the difference that make a difference) in a greater and greater chunk of the universe.  It is one thing to build an data crawler for a single content protocol, quite another to build a domain and protocol independent data crawler.  It is one thing to build hash trees based on word or token frequency and quite another to build them based on causal semantics (not how the words are sequenced, but how the concepts they refer to are graphed.

I think the main trouble you are having with this compression = intelligence concept has to do with a limited mapping of the word "compression".

Lets say you are driving and need to know which way to turn as you approach a fork in the road.  If you are equipped with some sort of mental abstraction of the territory ahead, or on a map, you can choose based on the information encoded into these representations.  But what if you didn't?  What if you could not build a map, either on paper, or in your head.  Then you would be forced to drive up each fork in turn.  In fact, had you no abstraction device, you would have to do this continually as you would not be able to remember the first road by the time you took the second.

What if you had to traverse every road in every city you came to just to decide which road you were meant to take in the first place?  What if the universe it self was the best map you could ever build of the universe?  Surely you can see that a map is a form of compression.

But lets say that your brain can never be big enough to build a perfect map of every part of the universe important to you.  Lets imagine that the map-building map you build in order to create mental memories of roads and cities is ineffective at building maps of biological knowledge or physics or the names and faces of your friends.  You will have to go about building unique map builders for each domain of knowledge important to you.  Eventually, every cubic centimeter of your brain will be full of domain-specific map making algorithms.  No room for the maps!

What you need to build sited is a universal map builder.  A map builder that works just as well for topological territory as it does for concepts and lists and complex n-dimensional pattern-scapes.

Do so and you will end up with the ultimate compression algorithm!

But your point about where the intelligence lies is important.  I haven't read the rules for the contest you sight, but if I were to design such a contest, I would insist that the final byte count of each entrants' data also include the byte count of the code necessary to unpack it.

I realize that even this doesn't go far enough.  You are correctly asserting that most of the intelligence is in the human minds that build these compression algorithms in the first place.

How would you go about designing a contest that correctly or more accurately measures the full complexity of both cypher and the content it interprets?

But before you do, you should take the time to realize that a compression algorithm becomes a smaller and smaller component of the total complexity metric the more often it is used.  How many trillions of trillions of bytes have been trimmed from the global data tree over the lifespan of use of MPEG or JPEG on video and images?  Even if you factor in a robust calculation of the quantum wave space inhabited by the humans brains that created these protocols it is plain to see that use continues to diminish the complexity contribution of the cypher no matter how complex.

Now what do you think?

Randall Lee Reetz