Evolution: The changes that will have the greatest effect on the longest future... and what it takes for those changes to survive the present long enough to make it there.
That sentence describes evolution better than any attributed to Darwin. The refinement-on-a-scheme process Darwin described is only the metabolism, the power plant, that fuels evolution. "Fitness" in the present is the necessary evil, not the goal of evolution. Evolution is not the struggle for the right beak shape. That kind of refinement-on-a-scheme only gets you the biggest slice of the local pie. Refinement makes for a powerful now, a perquisite for a powerful then, but it is self limiting. Resources spent adapting to the now are resources not available for adaptation to the future. It is ironic to have to utter these words, but evolution isn't about the here and now. No, evolution is about the biggest there and the longest then. If your purpose is to facilitate the future, you are in the prediction business. So let's re-write our definition.
Evolution: The development and selection of better and better prediction schemes.
However, a prediction scheme is useless unless it can get you from what works now to what will work then. What ensues, is a tug-of-war, a tight-wire-walk, in which an evolution scheme must support both refinement in the now and prediction of the future. The causal implications are complex. Refinement is antithetical to prediction. While the finch is involved in the struggle for a better beak, the larger sphere of resources, the whole of the universe over the longest spans of the future, go ignored and un-tapped. You might argue that a bird doesn't have the capacity to understand the universe or the concept of resources or the maximization of exploitation to which all evolving systems must ultimately compete, and you would be, after a fashion, correct. That is the magic of evolution, the evolving thing doesn't need to understand the process it is engaged in. So long as there are enough individuals and these individuals are each even just slightly different from each other, selection will direct change towards better and better survival and better and better prediction schemes. Changes that promote survival are the easy part. And yet, even though it often works against survival in the present, the capacity to predict will win in the long run.
Darwin can't be faulted for not seeing the big picture. He had to lay down the ground work and the groundwork in evolution is survival. Beak shape is an easily to observe population variance in finch morphology. But beak shape variation is no different than leg length variation, or protean variation, or cognitive and behavioral variation. Critics of evolution in general or Darwin argue that he never actually explained "…the origin of species". Implied in that challenge is a description of a process that results in qualitative changes; animals vs. plants for instance. And this is because he chose to explain his theory through the more overt and obvious adaptations that fall into the category of refinement or fitness in the present. The larger picture of evolution must be told as a conflict between refinement and prediction and that would have been an impossible sell to an already suspicious victorian audience.
Randall Lee Reetz
The 2nd Law: Is Increased Entropy Stochastic (incidental) or Causal (intrinsic)?
Recent science news is dominated by the multi-trillion dollar experimental search for the Higgs boson particle. A definitive observation of the theorized, but illusive, Higgs will finally complete the verification of the Standard Model – the most respected mathematical model of the evolution of our universe, explaining the emergence of each of the known forces and all of the matter we can observe. In the Standard Model, the Higgs is responsible for gravity – surrounding the more pedestrian particles – lending them the property we call "mass". If the Higgs exists, it is important as the causal bridge between the quantum world of the small and the relativistic world of the large. How could a particle that causes gravity be so hard to find? Because it doesn't actually have mass. It is as a result, known as "weakly interacting". It is only when a whole bunch of Higgs get together and surround other particles that mass is detected, and then, only in the surrounded particles. The Higgs binds so tightly to other particles, that it takes an extraordinary amount of energy, to break it free so that its presence can be detected. This is what the "Large Hadron Collider" does – it smashes heavy atomic nucleus (stripped of their electrons) at energies equivalent to those of the first moments after the Big Bang when all of the matter and energy in the entire universe was still smaller than a single star.
But there is a far more fundamental question. Gravity is a property. It is domain-dependent. It is specific to and belongs to a class of objects of a particular makeup and composition. The existence or nonexistence of the Higgs has no effect upon other properties of the universe like electromagnetism.
But there is a candidate for a domain-independent attribute of any and all causal systems. This attribute has been labeled the "Causal Entropic Principle" – it is generally discussed within the context of the transfer of heat (at astronomical scales) – within the study of thermodynamics. It is the logical extension of the concept of increased entropy, as first postulated, measured, and later described as the 2nd Law of Thermodynamics. But now, a hundred and fifty years after the formalization the laws of thermodynamics (of the phenomena and parameters of the transfer of heat, of the ratio of potential energy and work) correlative investigations in the fields of information, communication, computation, language, energy/mass, logic, and structure have uncovered parallel principles and constraints. It is reasonable now to understand the 2nd Law as a description of a fundamental constraint on any change, in any system, no matter what forces and materials are at play. We now understand the 2nd Law to describe the reduction in the quality (density) of the energy and or structure of the universe (or any part therein) as results any change at all. We have come to understand the 2nd Law as a constraint on the outcome of change in structure, which is to say "information", on its construction, maintenance, and or transfer. This insight has rendered an equivalence between energy and structure in much the same way that Einsteinian Relativity exposed the equivalence between energy and mass.
There is however a daemon lurking within our understanding of the 2nd Law, a daemon that threatens to undermine our understanding of causality itself, a daemon that, once defined, may provide the basis for an understanding of any self-consistent causal system, including but not exclusive of our own universe and its particular set of properties and behaviors.
The daemon of the 2nd Law is the daemon of stochastic – is 2nd Law dictated dissipation (entropy) statistical, or is statistics simply a tool we use in the absence of microscopic knowledge? Asked another way, is the reduction in the quality of energy or information that the 2nd Law demands of every action, a property of the universe or is it a property of the measurement or observation of the universe? Is action equivalent to measurement? Is there a measurement or stochastic class of action free of the entropy-increase demanded by the 2nd Law?
This question is of far greater consequence to the universe and the understanding of the universe than the mechanics of mass as it would describe and thus parameterize ALL action and ALL configuration and the precipitation or evolution of all possible action and configuration. Where the existence of the Higgs Boson may explain the source of mass and gravity in this universe, an understanding of the causal attributes leading to the behavior described by the 2nd Law of Thermodynamics might just provide a foundation from which any and all causal systems must precipitate.
The implications and issues orbiting this problem are many and deep. At stake is an demonstrative understanding of change itself. We tend to think of change as exception. But, can a thing exist without change? If not, what is the difference between data and computation, between thing and abstraction of thing, and profoundly, an answer to the question, can data exist without computation? Can thing exist outside of abstraction of thing?
In thermodynamics and information theory, an effort is made to distinguish process and stochastic process. Heat is defined as an aggregate property describing the average or holistic state of systems composed so many interacting parts to keep track of all of them individually. Heat is a calculous of sorts, a system of shortcuts that allows mathematics to be employed successfully to determine the gross state of a huge collection of similar parts. There is a tendency then to assume that the laws that describe heat are laws that only apply to aggregate systems where knowledge is incomplete.
Are there non-stochastic systems? Are there discrete systems or dynamic changes within systems for which the laws of thermodynamics don't apply? Does the Causal Entropic Principle apply if you know and can observe every attribute of, and calculate the exact and complete state of a dynamic system?
Such questions are more involved than they may seem on first reading. Answering them will expose the very nature of change, independent of domain, illuminating the causal chain that has resulted from full evolutionary lineage of the universe.
Randall Lee Reetz
Note: The Causal Entropic Principle isn't a complex concept. It is the simple application of the 2nd Law's demand for increased universal entropy as a result of every change in any system. It says that every action in every system must be that action that causes the largest reduction in the quality of information or energy (the greatest dissipation). It says that a universe has only one possible end state – heat death – and that processes that maximize the rate towards this end state will be evolutionarily favored (selected), simply because entropy-maximizing processes and structures demand a higher throughput of energy and thus end up dominating their respective locality. Such entropy-maximizing schemes are thus more likely to determine the structure and behavior of the event cone stretching off into the future. An obvious extension of this principle is that complexity, or more precisely, the family of complexity that can find, record, and process abstractions that represent the salient aspects (physics) of the (an) universe, will help that complexity better predict the shape and behavior it must assume to maximize its competitive influence upon the future of entropy maximization. The "Causal Entropic Principle" thus represents a logically self-consistant (scientific) replacement for the awkwardly self-centered and causally impossible "anthropomorphic principle" (which lacks a physical or causal explanation and leans heavily on painfully erroneous macroscopic stretching of the quantum electro dynamics). Stretching circular logic to its most obvious and illogical end, the anthropomorphic principle borrows awkwardly and erroneously and ironically form the Heisenberg / Uncertainty Principle by asserting the necessity of "observers" as a precursor to the emergence of complexity. The Causal Entropic Principle explains the production of localized complexity without the need for prior-knowledge, and does so within the bounds of, as a result of, the 2nd Law of Thermodynamics, by showing that localized complexity can both come into existence as a result of the constant increase in universal entropy, and more specifically, that localized complexity has an evolutionary advantage, and will thus out-compete, less complex structures. In a Causal Entropic Principle universe, intelligence is the expected evolutionary result of competition to reach heat death faster. Falling down is enhanced by a particular class of complexity that can come into existence as a natural result of things falling down. Should one form of such complexity "understand" the universe better than another form, it will have an advantage and will be more likely to influence the shape of complexity in the future. The better a system gets at abstracting the dynamics of its environment the more likely it will be able to eat other systems than be eaten by them. Where the anthropomorphic principle requires an a-priori "observer", the causal entropic principle simply requires the 2nd Law's demand for increased entropy, for things falling down.
The Problem with Darwin…
 |
| Ya… how would you look as Darwin? |
When understood as a "how", the process of evolution is reduced to orrery – like the awkward clockworks that spin planets and moons around concentric bearings – substituting method where there should be cause. How is always specific to domain, but why, the ultimate why, is general enough to explain all of the how's. Armed with a robust understanding of the big WHY of evolution, one should be able to walk into any domain and predict and then map it's how. Again, it isn't that Darwin's evolution orrery doesn't accurately predict biological patterns of change, or even that Darwin's evolution orrery doesn't accurately abstract the salient causal aspects of biological change, it is that Darwin's how of evolution in biology leads people to the idea that evolution is specific and exclusive to biology, or that one can understand evolution in other domains by overlaying biology's how.
Darwin never generalized the process of evolution. Imagine had Newton and Einstein had not generalized dynamics and motion and that we had, as a result, built all of our machines on the principle that motion was caused by legs and feet.
The people who have come the closest to the generalization of evolution, the thermodynamisists, have never been able to or interested in the development of a generalization of the direction of change and the cause of that direction. I will get back to this absence of generalization in the understanding of evolution but right now will only hint at an explanation… in the aftermath of the all too human race and cultural superiority wars and atrocities, it has been socially dangerous to think of evolution as having a direction as such thoughts can be read as rhetorical arguments for superiority and pre-judgement, the likes of which were used by Hitler, Stalin, Pol Pot, Mao, and others as justification for mass exterminations and other exclusionary policies. That humans have the proclivity to exploit incomplete knowledge in the pursuit of ridiculous selfishness at absurd scales should be nothing new or noteworthy. But no one would advocate the cessation of the study of chemistry simply because arsenic is a chemical, or the study of high energy physics simply because the atom bomb can be built from such knowledge.
Or would we? Cautionary reactions to the self-superior pogroms that so blighted the 20th century have driven several generations of researchers towards the relativist rhetoric we see most prominently in the post-modernist movement, but which is evident in the works of less irrational and otherwise, empirical scientists like Stephen J. Gould and Richard Dawkins. Both represent an interesting study in overcompensation. In their quest to irradiate the all-to-natural self-superiority that seems to cause humans to erect unfounded tautologies that place humans on top of pre-destined hierarchies, both argue and argue brilliantly, for a flat evolutionary environment in which change happens but without any directionality at all. Again, this is like saying that because metal can be shaped into swards and knives and guns it shouldn't be produced even should we need plows and trains and dynamos and bridges and buildings and printing presses and lab equipment and computers.
Of course, caution is its own form of rhetoric, as potentially dangerous as its more obviously tyrannous cousins.
And, yes! Evolution has a direction. There I said it! Say it with me. You won't be struck down by post-modernist lightning. Trust me. Trust your self. It is more than a little absurd that one would have to argue for direction in a process that explains directionality. They are of course correct in their assertion that evolution isn't pre-determined. Nothing is. Of course. But the "brilliance" of evolution is that it results in a direction without need for prior knowledge, plan, or determination of any kind. To toss this most salient aspect of the evolutionary process simply to make a sociological point seems reckless in the maximum.
Randall Lee Reetz
Causalism
Though I have been writing about "causality" for years, I continued to label myself an atheist. I really thought the label fit, but that was before I attended to a few atheism events and talks and found that atheists are not, as I had understood, interested in rational thought. The atheists I have met are simply and aggressively apposed to religion. They have an axe to grind, are pathologically obsessed with, focused upon, and have a general need to work bad juju against – religion. The single minded obsessive combativeness exhibited by the atheists that I have met, seems, well, in a word – religious.
Of course I will take responsibility for my mistake, my assumption, it is right there in the word: "atheist", a-theist, anti-theist, against theism. And though I am not a believer, my disinterest in belief does not define my self or my thoughts or more specifically, the way I choose to think things through. Nobody who likes the color red describes this preference as anti-green.
Of course I will take responsibility for my mistake, my assumption, it is right there in the word: "atheist", a-theist, anti-theist, against theism. And though I am not a believer, my disinterest in belief does not define my self or my thoughts or more specifically, the way I choose to think things through. Nobody who likes the color red describes this preference as anti-green.
The word "causality" describes a world view in which every system exists as a hierarchy of cause and effect. Even though dictionaries don't yet list the word – a "causalist" is a person who "believes" that all structure and action is the result of physical causality. That describes my way of thinking perfectly.
I am a causalist!
There, I said it. It is now a word. That is what makes the english language so dynamic and alive. If you need a word. Go out and build one. In this case, anyone who understands the word causal will immediately understand the word causalist. 'Wish I could say the same of my spell checker.
I believe that the universe is what it is and acts like it acts solely as a result of physical properties and the way these properties are effected by and effect each-other. No back doors. No end-arounds. No special cases. No miracles. No exceptions. No preferential treatment. Simple. Expectable. Demonstrable. Regular. I believe that thoughts are built of atoms, not the other way around. That atoms and their properties effect thought but that thoughts can't effect the basic fundamental properties of atoms.
Causality is a description of a system as a hierarchy of the things that influence it. Causality strongly implies an asymmetry between cause and effect. A causal system is a system in which some parts and attributes have more influence over the shape and behavior of that system than do others (which are more effect).
A causalist assumes that all systems exist as the result of a history and that this history defines a linearity of construction – first this happened, then this, and finally this. A causalist assumes that this history of a system, exposes a hierarchy of the energies that were required to built it. The most energetic processes shaped the earliest subsystems, and the most recent additions were laid down in much less energetic environments. This makes for stable structures – strongly associated foundations upon which more delicate subsystems are layered.
And, low and behold, when we pick systems apart, we always find this hierarchical strata of causally deposited layers. Always! No exceptions have ever been found.
The other key aspect to the causalist's cosmology is the fact that earlier systems are more similar and systems added later are more diverse. This means that everything shares and is built upon the same past. So no matter if you have gills or feathers, hands or tentacles, eyes or echo-location – you will all be built of cells and these cells, muscle or neuron, blood cell or phagocyte will all be made of atoms and these atoms helium or hydrogen, led or plutonium will always be made of protons, neutrons and electrons, etc. etc.
A world or universe built up from ever more shared and similar parts is a universe that can be known. It is a stable universe that has expectable patterns that exist within the boundaries of limits defined by their own causally stacked history. Everything shares the same history. Everything is built of the same history. Though the present may look diverse and confusing, further inquiry will always reveal patterns and similarities at base, shared by everything. Exceptions and difference is always an aspect of the surface or most recent manifestations of evolution. The base gets more and more similar the deeper or further back you look.
This regularity is exactly why spiritualists and the religious have always been apposed to science and the causal view of reality. It doesn't allow exceptions. You can't win the race cause you prayed longer or to the right god. You can only win the race if you have the physicality and the emotional drive to run harder than everyone else. There are no exceptions. There are no special cases. There is no favoritism. There is no OZ pulling levers. Nobody and no thing to assuage towards your interests.
We know that adding heat to a gas causes it to expand. We know that compressing a gas causes it to get hotter. We know the exact ratio between heat and pressure or volume. With this ratio we can predict the exact changes that will result when we add energy or change the volume of a given amount of gas. It always works the way the math says it should. But of course the same can not be said for the spiritual arts. In fact. There has never been an event that has been shown to have been effected by prayer or thinking (no matter how concentrated, repeated, or "conscious"). Never once has the temperature of a gas or anything else been effected by anything spiritual or religious. This is not just true of gasses and heat. It is true of everything we can measure. We know the entire electromagnetic spectrum. We understand and have defined the parameters that show why this spectrum is bounded on both the cold and the energetic end. We know how gravity is effected by energy, time and distance. We know how quanta effect other quanta. And in no system, caused by no dynamics, have we ever encountered any behavior that can not be explained by simple causality of the things effecting those systems.
It is easy to see why causality would disturb the anxious. There is no way to play a causal system. No way to gain advantage. No free lunch. No favoritism. No exceptions. No para-normal. No meta-physics. No magic. No luck. Just the awesome beauty and incomprehensible complexity that can't help but happen in a universe this big and this full of energy and stuff with this much time summing into an almost infinite number interactions and the constructs can survive. In a causal universe, you get fit by working out – not by chanting mantras. You live longer by taking care of your body and your mind, not by praying for longevity. You gain advantage in sports, business, romance, academia, and culture, by understanding the causal influences effecting these systems, not by sleeping under a pyramid or praying to a 10 armed blue goddess or a sandal wearing guy nailed to a couple of wood beams. You gain knowledge through measurement and building theories that abstract the greatest domain of measurements – not by listening reverently to some guy in a robe who slickly explains away your loneliness and fear of death with a 24 voice choir and 60 foot pipe organ as backup.
It may in fact be true that the human brain has evolved into a configuration that demands religious and spiritual thoughts and that these thoughts can result in a better sense of well-being and that the resulting sense of inner peace gives rise to physical benefits, but this itself is a causal (if Byzantine) system. But any causal assessment of the human penchant for religious and spiritual thought would certainly give it a low score for effectiveness and a high score for self-delusion. Thoughts that make you feel better aren't necessarily the thoughts that increase your knowledge of yourself or the universe. This rift between reality and what our minds would like reality to be is the single most dangerous side effect of evolving a great big layer of grey matter over the top of our lizard brain within. It is important to remember that the lizard brain, an emotion-to-action processing center, is still very much in charge of everything we do. This is non-negotiable. It is true because of physical wiring, not philosophy. Adding to a sober assessment of our scary neural architecture is the fact that the inner brain, the old brain, the brain in charge, isn't sophisticated enough to understand any of the wonderfully sophisticated products (reason, logic, abstract mapping, systems modeling) our more recently acquired layers are able to compute. Be you Einstein or Leonardo, Shakespeare or Confucius, Newton or Feynman, your inner brain is still as dumb as a lizard, and it, unfortunately, is both blind to your brilliance, and directing the actions you take.
A causalist isn't as depressed by this seemingly hopeless understanding as one might expect. A causalist just accepts reality and looks for solutions that work with and are informed by it. If humans have great big thinking machines that are hard to hear above the ruckus and clatter of emotions, the challenge is to build cultural solutions that protect and automate higher order knowledge from the vagrancies of low level emotions. The playing field shifts with this shift in perspectives. Instead of looking for solutions within the arena of emotional thought (meditation, mood music, a harmonic convergence, global conciseness, etc.) the cuasalist, looks for solutions outside of the lizard brain, looks for solutions that accept the lizard brain but work to lessen its control over policy and society. A causalist looks to the infrastructure. A causalist has noticed that people act more constructive when their environment takes care of the simple needs of the lizard brain. A causalist has noticed that clean running water at every faucet, enough food, reasonable shelter, and reliable stability make people with the same lizard brain end up acting more like their grey matter ideas and less like a lizard. No amount of prayer or philosophy or yoga will yield the same productivity improvements and cultural growth as a reliable and supportive infrastructure.
Take the time to look at the statistics that compare a culture's confidence in its infrastructure and its per labor hour productivity or its crime rate. Then try to find a better predictor of a culture's ability to build towards greater and greater progress of knowledge and opportunity. Confidence doesn't build an infrastructure, infrastructure builds confidence. Those of us confident enough to see this hierarchy for what it is must work constantly and vigorously against the grain. We must build the infrastructure that will produce in tomorrow's masses the confidence we are lucky enough to feel today.
It is way way way cheaper to run clean water and nutritious food into every home than it is to deal with the chaos that ALWAYS results when people must live with the fear of thirst and hunger. Going to church for an hour a week isn't ever going to have the same effect on productivity and stability. Even the religious would surely rather pray for more interesting and complex goals than a bowl of rice and clear water.
Accepting the weird irrationality of the human thinking and emotional system doesn't mean that you have to be irrational about your acceptance of it or how you choose to go about building towards a better future.
A causalist isn't as depressed by this seemingly hopeless understanding as one might expect. A causalist just accepts reality and looks for solutions that work with and are informed by it. If humans have great big thinking machines that are hard to hear above the ruckus and clatter of emotions, the challenge is to build cultural solutions that protect and automate higher order knowledge from the vagrancies of low level emotions. The playing field shifts with this shift in perspectives. Instead of looking for solutions within the arena of emotional thought (meditation, mood music, a harmonic convergence, global conciseness, etc.) the cuasalist, looks for solutions outside of the lizard brain, looks for solutions that accept the lizard brain but work to lessen its control over policy and society. A causalist looks to the infrastructure. A causalist has noticed that people act more constructive when their environment takes care of the simple needs of the lizard brain. A causalist has noticed that clean running water at every faucet, enough food, reasonable shelter, and reliable stability make people with the same lizard brain end up acting more like their grey matter ideas and less like a lizard. No amount of prayer or philosophy or yoga will yield the same productivity improvements and cultural growth as a reliable and supportive infrastructure.
Take the time to look at the statistics that compare a culture's confidence in its infrastructure and its per labor hour productivity or its crime rate. Then try to find a better predictor of a culture's ability to build towards greater and greater progress of knowledge and opportunity. Confidence doesn't build an infrastructure, infrastructure builds confidence. Those of us confident enough to see this hierarchy for what it is must work constantly and vigorously against the grain. We must build the infrastructure that will produce in tomorrow's masses the confidence we are lucky enough to feel today.
It is way way way cheaper to run clean water and nutritious food into every home than it is to deal with the chaos that ALWAYS results when people must live with the fear of thirst and hunger. Going to church for an hour a week isn't ever going to have the same effect on productivity and stability. Even the religious would surely rather pray for more interesting and complex goals than a bowl of rice and clear water.
Accepting the weird irrationality of the human thinking and emotional system doesn't mean that you have to be irrational about your acceptance of it or how you choose to go about building towards a better future.
Randall Lee Reetz (a causalist!)
Compression as Intelligence
Let me take a stab at defending compression as equivalent to intelligence.
Standard string compression (LZW, etc.) works by understanding and then exploiting the sequencing rules that result in the redundancy built into most (all?) languages and communication protocols.
Compression is necessary in any storage/retrieval/manipulation system for the simple reason that all systems are finite. Any library, any hard drive, any computer memory… all finite. If working with primary in-situ environments was as efficient as working with maps or abstractions we would never have to go through the trouble of making maps or abstracting and filtering and representing.
It might seem sarcastic even to say it, but a universe is larger than a brain.
You have however stumbled upon an interesting insight. Where exactly is intelligence? In classic Shannon information theory, and the communication metrics (signal/noise ratio) upon which it is based, information is a duality where data and cypher are interlocked. In this model, you can reduce the size of your content, but only if you increase the size (or capacity) of the cypher. Want to reduce the complexity of the cypher, well you are forced to accept the fact that your content will grow in size or complexity. No free lunch!
In order to build a more robust cypher, one has to generalize in order find salience (the difference that make a difference) in a greater and greater chunk of the universe. It is one thing to build an data crawler for a single content protocol, quite another to build a domain and protocol independent data crawler. It is one thing to build hash trees based on word or token frequency and quite another to build them based on causal semantics (not how the words are sequenced, but how the concepts they refer to are graphed.
I think the main trouble you are having with this compression = intelligence concept has to do with a limited mapping of the word "compression".
Lets say you are driving and need to know which way to turn as you approach a fork in the road. If you are equipped with some sort of mental abstraction of the territory ahead, or on a map, you can choose based on the information encoded into these representations. But what if you didn't? What if you could not build a map, either on paper, or in your head. Then you would be forced to drive up each fork in turn. In fact, had you no abstraction device, you would have to do this continually as you would not be able to remember the first road by the time you took the second.
What if you had to traverse every road in every city you came to just to decide which road you were meant to take in the first place? What if the universe it self was the best map you could ever build of the universe? Surely you can see that a map is a form of compression.
But lets say that your brain can never be big enough to build a perfect map of every part of the universe important to you. Lets imagine that the map-building map you build in order to create mental memories of roads and cities is ineffective at building maps of biological knowledge or physics or the names and faces of your friends. You will have to go about building unique map builders for each domain of knowledge important to you. Eventually, every cubic centimeter of your brain will be full of domain-specific map making algorithms. No room for the maps!
What you need to build sited is a universal map builder. A map builder that works just as well for topological territory as it does for concepts and lists and complex n-dimensional pattern-scapes.
Do so and you will end up with the ultimate compression algorithm!
But your point about where the intelligence lies is important. I haven't read the rules for the contest you sight, but if I were to design such a contest, I would insist that the final byte count of each entrants' data also include the byte count of the code necessary to unpack it.
I realize that even this doesn't go far enough. You are correctly asserting that most of the intelligence is in the human minds that build these compression algorithms in the first place.
How would you go about designing a contest that correctly or more accurately measures the full complexity of both cypher and the content it interprets?
But before you do, you should take the time to realize that a compression algorithm becomes a smaller and smaller component of the total complexity metric the more often it is used. How many trillions of trillions of bytes have been trimmed from the global data tree over the lifespan of use of MPEG or JPEG on video and images? Even if you factor in a robust calculation of the quantum wave space inhabited by the humans brains that created these protocols it is plain to see that use continues to diminish the complexity contribution of the cypher no matter how complex.
Now what do you think?
Randall Lee Reetz
Standard string compression (LZW, etc.) works by understanding and then exploiting the sequencing rules that result in the redundancy built into most (all?) languages and communication protocols.
Compression is necessary in any storage/retrieval/manipulation system for the simple reason that all systems are finite. Any library, any hard drive, any computer memory… all finite. If working with primary in-situ environments was as efficient as working with maps or abstractions we would never have to go through the trouble of making maps or abstracting and filtering and representing.
It might seem sarcastic even to say it, but a universe is larger than a brain.
You have however stumbled upon an interesting insight. Where exactly is intelligence? In classic Shannon information theory, and the communication metrics (signal/noise ratio) upon which it is based, information is a duality where data and cypher are interlocked. In this model, you can reduce the size of your content, but only if you increase the size (or capacity) of the cypher. Want to reduce the complexity of the cypher, well you are forced to accept the fact that your content will grow in size or complexity. No free lunch!
In order to build a more robust cypher, one has to generalize in order find salience (the difference that make a difference) in a greater and greater chunk of the universe. It is one thing to build an data crawler for a single content protocol, quite another to build a domain and protocol independent data crawler. It is one thing to build hash trees based on word or token frequency and quite another to build them based on causal semantics (not how the words are sequenced, but how the concepts they refer to are graphed.
I think the main trouble you are having with this compression = intelligence concept has to do with a limited mapping of the word "compression".
Lets say you are driving and need to know which way to turn as you approach a fork in the road. If you are equipped with some sort of mental abstraction of the territory ahead, or on a map, you can choose based on the information encoded into these representations. But what if you didn't? What if you could not build a map, either on paper, or in your head. Then you would be forced to drive up each fork in turn. In fact, had you no abstraction device, you would have to do this continually as you would not be able to remember the first road by the time you took the second.
What if you had to traverse every road in every city you came to just to decide which road you were meant to take in the first place? What if the universe it self was the best map you could ever build of the universe? Surely you can see that a map is a form of compression.
But lets say that your brain can never be big enough to build a perfect map of every part of the universe important to you. Lets imagine that the map-building map you build in order to create mental memories of roads and cities is ineffective at building maps of biological knowledge or physics or the names and faces of your friends. You will have to go about building unique map builders for each domain of knowledge important to you. Eventually, every cubic centimeter of your brain will be full of domain-specific map making algorithms. No room for the maps!
What you need to build sited is a universal map builder. A map builder that works just as well for topological territory as it does for concepts and lists and complex n-dimensional pattern-scapes.
Do so and you will end up with the ultimate compression algorithm!
But your point about where the intelligence lies is important. I haven't read the rules for the contest you sight, but if I were to design such a contest, I would insist that the final byte count of each entrants' data also include the byte count of the code necessary to unpack it.
I realize that even this doesn't go far enough. You are correctly asserting that most of the intelligence is in the human minds that build these compression algorithms in the first place.
How would you go about designing a contest that correctly or more accurately measures the full complexity of both cypher and the content it interprets?
But before you do, you should take the time to realize that a compression algorithm becomes a smaller and smaller component of the total complexity metric the more often it is used. How many trillions of trillions of bytes have been trimmed from the global data tree over the lifespan of use of MPEG or JPEG on video and images? Even if you factor in a robust calculation of the quantum wave space inhabited by the humans brains that created these protocols it is plain to see that use continues to diminish the complexity contribution of the cypher no matter how complex.
Now what do you think?
Randall Lee Reetz
Subscribe to:
Posts (Atom)