Search This Blog

Building Pattern Matching Graphs

I talk a lot about the integral relationship between compression and intelligence.  Here are some simple methods.  We will talk of images but images are not special in any way (just easier to visualize).  Recognizing pattern in an image is easier if you can't see very well.

What?

Blur your eyes and you vastly reduce the information that has to be processed.  Garbage in, brilliance out!



Do this with every image you want to compare.  Make copies and blur them heavily.  Now compress their size down to a very small bitmap (say 10 by 10 pixels) using a pixel averaging algorithm.  Now convert each to grey scale.  Now increase the contrast (about, 150 percent).  Store them thus compressed.  Now compare each image to all of the rest: subtract the target image from the compared image. The result will be the delta between the two. Reduce this combined image to one pixel.  It will have a value somewhere between pure white (0) and pure black (256), representing the gross difference between the two images. Perform this comparison between your target image and all of the images in your data base. Rank and group them from most similar to least.

Now perform image averages of the top 10 percent matches. Build a graph that has all of the source images at the bottom, the next layer is the image averages you just made. Now perform the same comparison to the 10 percent that make up this new layer of averages, that will be your next layer. Repeat until your top layer contains two images. 

Once you have a graph like this, you can quickly find matching images by moving down the graph and making simple binary choices for the next best match. Very fast. If you also take the trouble to optimize your whole salience graph each time you add a new image, your filter should get smarter and smarter.

To increase the fidelity of your intelligence, simply compare individual regions of your image that were most salient in the hierarchical filtering that cascaded down to cause the match. This process can back-propagate up the match hierarchy to help refine salience in the filter graph. Same process works for text or sound or video or topology of any kind. If you have information, this process will find pattern in it. Lots of parameters to tweak. Work the parameters into your fitness or salience breading algorithm and you have a living breathing learning intelligence. Do it right and you shouldn't have to know which category your information originated from (video, sound, text, numbers, binary, etc.). Your system should find those categories automatically.

Remember that intelligence is a lossy compression problem. What to pay attention to, what to ignore. What to save, what to throw away. And finally, how to store your compressed patterns such that the graph that results says something real about the meta-paterns that exist natively in your source set. 

This whole approach has a history of course. Over the history of human scientific and practical thought many people have settled in on the idea that fast filtering is most efficient when it is initiated on a highly compressed pattern range. It is more efficient for instance to go right to the "J's" than to compare the word "joy" to every word in a dictionary or database. This efficiency is only available if your match set is highly structured (in this example, alphabetically ordered). One can do way way way better than alphabetically ordered lists of 3 million words. Lets say there are a million words in a dictionary. If one sets up a graph, an inverted pyramid, where each level where the level one has 2 "folders" and each folder is named for the last word in the subset of all words at that level divided into two groups. The first folder would reference all words from "A" to something like "Monolith" (and is named "Monolith") The second folder at that level contains all words alphabetically larger than "Monolith" (maybe starting with "Monolithic") and is named "Zyzer" (or what ever the last word is in the dictionary). Now, put two folders in each of these folders to make up the second tier of your sorting graph. At the second level you will have 4 folders. Do this again at the third level and you will have 8 folders each named for the last word in the graph referenced in the tiers of the graph above them. It will only take 20 levels to reference a million words, 24 levels for 15 million words. That represents a 6 order of magnitude savings over an unstructured sort. 

A cleaver administrative assistant working for Edward Hubble (or was it Wilson, I can't find the reference?) made punch cards of star positions from observational photo plates of the heavens and was able to perform fast searches for quickly moving stars by running knitting needles into the punch holes in a stack of cards.



Pens A and B found their way through all cards. Pen C hits the second card.

What matters, what is salient, is always that which is proximal in the correct context. What matters is what is near the object of focus at some specific point in time.

Lets go back to the image search I introduced earlier. As in the alphabetical word search just mentioned, what should matter isn't the search method (that is just a perk), but rather the association graph that is produced over the course of many searches. This structured graph represents a meta-pattern inherent in the source data set. If the source data is structurally non-random, its structure will encode part of its semantic content.  If this is the case, the data can be assumed to have been encoded according to a set of structural rules themselves encoding a grammar.

For each of these grammatical rule sets (chunking/combinatorial schemes) one should be able to represent content as a meta-pattern graph. One of the graphs representing a set of words might be pointers to the full lexicon graph. A second graph of the same source text might represent the ordered proximity of each word to its neighbors (remember the alphabetical meta-pattern graph simply represents the neighbors at the character chunk level).

What gets interesting of course are the meta-graphs that can be produced when these structured graphs are cross compressed. In human cognition these meta-graphs are called associative memory (experience) and are why we can quickly reference a memory when we see a color or our nose picks up a scent.

At base, all of these storage and processing tricks depend on two things, storing data structures that allow fast matching, and getting rid of details that don't matter. In concert these two goals result in a self optimization towards maximal compression.

The map MUST be smaller than the territory or it isn't of any value.

It MUST hold ONLY those aspects of the territory that matter to the entity referencing them. The difference between photos and text: A photo-sensor in a digital camera doesn't know for human salience. It sees all points of the visual plane as equal. The memory chips upon which these color points are stored see all pixels as equal. So far, no compression, and no salience. Salience only appears at the level of where digital photos originate (who took them, where, and when). On the other hand, text is usually highly compressed from the very beginning. What a person writes about and how they write it always represents a very very very small subset of 

Compression as Intelligence (Garbage Out, Brilliance In)

I am convinced that the secret to developing intelligence (in any substrate, including your brain) lies in the percentage of the data coming in that you are willing (or forced) to toss. Lossy compression is the key to intelligence. Of course there is a caveat… you can't just trash anything and everything.

The first line of the book I am writing about evolution: "What matters is what matters, knowing what matters and how to know it matters the most."

I am convinced that evolving systems can only work towards mechanisms that process salience if they are forced to maximize the amount of stuff they can trash.

If you are forced to get rid of 99.999 percent of everything that comes in, well you will have to get good at knowing the difference between needles and hay and you will have to get good at knowing the difference in a hurry. The "needles and hay" metaphor doesn't map well to what I am talking towards. If the system you are dealing with is so unstructured as to fit the haystack metaphor, you really aren't doing anything I would classify as intelligence. If there is nothing of structure in the haystack you are storing than your compression system should already have tossed the whole thing out.

Many techniques for the filtering of essence, for finding pattern, for storing pattern and for storing pattern of pattern have been developed. The most impressive reduce raw input streams and store pattern from the most general to the most specific as hierarchically stratified graphs.

Being forced to reduce data to storage formats that maximize lossy-ness minimizes necessary storage. But that is just a perk. What really gates intelligence is the amount of a complex system (or map thereof) that can be made proximal to immediate processing. Our brains might be big and mighty, but what really matters is how much of the right parts of what is stored can be brought together in one small space for semi-real-time simulations processing. Information, when organized optimally for maximal storage density, will also be information that is ideally organized for localized serialization and simultaneity of processing.

To think, a system has to be able to grab highly compressed pattern hierarchies and move them into superposition on top of each other for near instantaneous comparison. You can't do this with a whole brain's worth of data, no matter how well organized it is.

Lets say you have to store everything you know about every sport you have ever heard of, and you have to do it in a very limited space. You will be forced to build a hierarchy of grammars in which general concepts shared in every sport (opponents, the goal to win, a set of rules and consequences, physical playing geometries, equipment, etc.), with layers of groupings that allow for the similarities between some sports and so on up to the specifics that are are only present in each individual sport. Keep compressing this set. Always compress. Try all day (or all night) for even more compression. Compress until you can't even get to lots of the specifics any more. Keep compressing. Dump the sports you don't care about. Keep on throwing stuff out.

Now lets say I have some sort of morbid sense of humor and I tell you that you are going to have to store everything you encounter and everything you think about, your entire life, in that same database that you have optimized for sports.

You will have to learn to look for the meta-patterns that will allow you to store your first romance in a structure that also allows you to store everything you know about kitchen utensils and geo-politics and the way the Beatles White Album makes you feel when it is windy outside.

The necessity to toss, enforced by limited storage and an obsession to compress will result in domain-blending salience hierarchies. It is why we can find deep similarities between music and geological topologies. It is why we can "think".

For years people have tried to come up with the algorithms of thought. What we need instead is to build into our artificial systems, a very mean and ornery compression task master that forces over time, all of our disparate sensation streams into the same shared graph.

Once you have all of your memories stored within the same graph, by necessity sharing the same meta-pattern, the job of evolving processing algorithms is made that much easier.

An intelligent system will spend most if not all of its time compressing data. We have a tendency to bifurcate the behavior of a mind into storage on the one hand, and processing on the other. I am beginning to think that the thing we call "thinking" and "thought" is exclusively and only a side-effect of constant attempts at compression – that there really isn't anything separate that happens outside of compression. Is this possible?

Randall Reetz

Computing: What Went Wrong

The year, 2010.  The state of computing?  Applications that build obfuscating document types that act as black boxes, hiding and separating information and intent.  File systems that store these documents largely blind of their content, of the context of their origination, and of the associations hidden in the meaning that binds them to the flow of the author's life and work.

There are but a finite set of ways that information needs to be associated.  Yet almost none of these association types are supported by our computers.  If you don't know before hand that everything you want to do will fit into the format of a linear text document, or the grid of a spread sheet, or the fields and records of a data base, well you might as well not even start.  If you want some information in one of your documents to reference information in another, well you had better be content with copying and pasting (live links are forbidden in all but the most expensive (and self-restrictive) application "suites".

Why is this true?  Why don't we yet have computers that can compute?  Mainly because the application layer is the WRONG place for the association of information.  The file system is the RIGHT place.  Information must be related and associated in a strata far below the application layer.  The only authority that should be granted to applications is user affordance – how humans are helped through the assignment, understanding, and management of the associations within their life's data.

To do this, the data (file) management layer needs to be beefed up (and the application layer slimmed down).  Where today's file systems only know a document by its wrapper (name, enclosing folder, parent application, document type, size, date, and on-disc storage address), a true data model layer would "understand" and dictate the structure of all of the ways information is related both within and between "documents".  In fact, in a data-model driven architecture, documents become arbitrary "collections", "instances", and "presentations" specific to the context of presentation or use.  The underlying data and associations between data from which documents are derived remain intact, separate, and agnostic of the documents that serve to reference, blend, display, and associate.

In the proposed data-model driven architecture, applications don't define information association, they must call on the data model to ask which types of associations are allowed and how these associations dictate data type, grammatical hierarchies, and chunking.  Applications build documents not as strings of binary, but from pointers into the information content stored according to the meta-archetectural rules of the master data model.  A document becomes instead an instance of assembled data and data associations (either frozen or live)… more like the "edit decision lists" that video and music editing professionals use to assemble linear streams of media from multiple sources.

Today's computational model's awkward emphasis on application authority and autonomy promotes an informational ecosystem that promotes informational islands dictated by the whims of application developers.

[to be continued]

Randall Reetz

The Incomputable Heaviness of Knowledge

Is the universe conceivable?  Does scientific knowledge improve our ability to think about the universe?

What happens when our knowledge reaches a level of sophistication such that the  human brain can no longer comfortably hold it, or compute on it?  For thousands of years, scholars have optimistically preached the benefits of knowledge.  Our world is rich and safe as a result.  People live longer, people live in greater personal control over the options they face.  All of this is an obvious result of our hard won understanding of how the universe and its parts actually work.  We arm our engineers with these knowledges and send them out to solve the problems that lead to a more and more desire-mitigated environment.  Wish you weren't hungry, go to the fridge or McDonnalds.  Wish you were somewhere else, get in your car and go there.  Wish you could be social, but your friends are in Prague, call them.  Wish you knew something, look it up on the internet.  Lonely, log in to a dating service and set up a rendezvous. Wish your leg wasn't fractured, go to a doc-in-the-box and get it set and cast.

But what if you want to put it all together?  What if your interests run to integration and consolidation.  What if you want to understand your feelings about parking meters as an ontological stack of hierarchical knowledge built all the way up from the big bang?

Evolution: Optimizing a Definition of Fitness

We think of evolution as a process that optimizes organisms (things) through the filter of fitness. Fitness as means - the species - as end. I have long suspected that this interpretation is wrong-headed, and results in conceptual mistakes that ripple though all of science, blinding us to much that could be understood about the Universe, process, and the basic shape and behavior of reality.

So let's flip it. We'll instead, re-frame evolution as a process that uses things (organisms, species, systems, ideas, etc.) as a means (channel, resource, armature, vehicle) for the optimization of fitness. From this inverted vantage, optimizing the criteria of fitness is the goal – species, nothing more than a convenient means.

It always feels wrong to talk of evolution's "goals".  Certainly a universe doesn't start out with a plan or agenda.  Things like plans and agendas are only possible within advanced abstraction apparatus like a brain or computer.  Universe's start out simple and chaotic.  Only chance causal interactions played out amongst a universe sized accumulation of matter and force over ridiculous amounts of time will lead to the types of rare and energy demanding structures that can "think" up things like plans and agendas.  So when I talk here of "a process that optimizes",  I make use concepts and terms that are more generally associated with self, ideation, and will – with the products of advanced abstraction machinery found in humans and maybe eventually in thinking machines.  But what I mean to convey is the direction of a process.  That processes have direction and that direction is (or can be) independent of the types of advanced computation necessary for things like planing and intent is in fact, the exact conceptual jump that the idea or discovery of "evolution" demands.  Evolution = direction without intent.

The directionality we see in evolving systems (all systems) is blatantly and obviously non-random.  Our job then is to understand, explain, and ultimately, exploit this understanding. Because we humans have trouble imagining non-random direction coming from systems without a brain, a soul, an agenda, we are left with a slim set of emotionally acceptable options; anthropomorphizing the universe and evolution, inserting a deity, or simply rejecting evolution (or reality) out of hand.  The non-emotional option, the science option, evolution, recovers from this dissonance through the application of inductive logic, physical evidence, and frankly, by simply offering a emotionally dissonant option.

The thesis of this essay is the suggestion that evolution might be agnostic to optimization of species and is instead simply using species as a conduit for the optimization of this thing called fitness.  That fitness might in fact be more real, and species, ethereal.

This entire domain is so fraught with potential miss-interpretation.  I feel a constant urge to over-explain, to be extra careful, to make sure the reader isn't thinking one thing when I mean something else.  For instance I feel a need to define the term "species", especially because I am using it in a more general way than is usually required within the boundaries of its original domain, biology.  This is because I am convinced that evolution is a universal process, that it has nothing in particular to do with biology or life, that it happens in all systems, all of the time, an unavoidable aspect of any reality.

So when I write "species"  I mean the thing or system that is "evolving" – the animal, the planet, the culture, the idea, the group attitude in line at the post office this morning.  And in the context of this essay, I use "species" to mean the thing upon which "fitness" acts (as judge, jury, pimp, or executioner).  Species is the thing, fitness the criteria that molds the thing.

But by this definition, species is corporal and measurable, suggesting that fitness is… is what?  If we are talking about something, shouldn't we have some way of examining it, measuring it, comparing it, holding it in our hands, flipping it over, squeezing it, spitting it open and looking at its parts?  That seems a more reasonable proposition for species than for fitness.

We like to think we can man-handle a thing like species, take it to the lab and do lab things.  But maybe that is more illusion than truth.  We can dissect a frog, but that particular frog isn't really the species "frog".  The species "frog" is an average, a canonical concept, a Platonic solid, a moving target, an arbitrarily bounded collection, a gelatinous arrow through foggy potentialities.

I was in route to show that "fitness" is a real thing, but all I accomplished was a picking away at the real-ness of "species".  Maybe that will end up being more helpful anyway.  The colloquial image of species, even amongst evolution theorists has always seemed more visceral, more thing-like than fitness.  We point to a single nervous animal on the savanna and declare, "that is gazelle".  Worse, we often fail to make a semantics distinction between that declaration and the categorical; "gazelle is that".  That fitness is a much harder thing to point to, really doesn't mean it is less real, or as I have shown, that real-ness applies to either.

Now that I've reduced both species and fitness to the realm of concept, it should be easier to argue my thesis.

Even at the concept level, "species" is a thorny concept fraught with pedagogy and hubris.  It is hard to look at a penguin, a porpoise, or a planet and imagine something more amazing, more evolved than its current form.  Which probably goes a long way to explain why we have a natural tendency to overlay onto the concept "species" notions of perfect form, of an apex, a pre-determined goal.  But this certainly has less to do with species and more to do with the limits of our cognitive facility.  It would be absurd to assume that this particular now is in some way special, that forms are complete and that we just happen to inhabit the planet just at the point when evolution has finally and completely finished its big 14 billion year project.

OK, the apologies have been met out, the slippery territory marked, the standard arguments abutted, the inconsistencies delineated, the usual misinterpretations admitted. These are standard precursors to any serious discussion in the study of evolution and bare witness to both the complexity of the subject and the apparent inability of the brain to readily make sense of its many dimensions.

So why should I want to reorder the relative hierarchy of fitness and species?  For one, I have always felt the standard Darwinian definition of evolution to be a bit circular.  Wow, before that comment ruins my standing, I had better get to work defending Darwin.  I am a "standard model" realist.  Darwin got most or all of evolution correct.  Especially if you restrict your focus to biology.  Darwin is the dude!  The positions I detail here are meant as additions, as icing on the cake Darwin baked.  But Darwin built his theory around life and his bio-centrist focus on evolution restricts and warps the applicable idea-space it scopes.  I always say that Darwin explained the how of evolution with regard to biology, and that I am interested in the why of evolution with regard to all systems.

To restrict the scope of evolution to biology, is to somehow draw a line in the sand between life and not-life, a special sauce within life that categorically separates it from all other systems.  I can't find that line.  So I am left with the responsibility of understanding and defining evolution as a domain independent attribute of any system or system of systems.

Structurally, all systems are ordered as hierarchical stacks.  Each level receives aggregate structures from lower (previously constructed) levels and produces from these, new super-aggragates, that it in turn passes to the next higher level.  That this process of aggregate layering is historically dependent is obvious.  The non-obvious mapping is to energy.  The lowest levels of the hierarchy, the earliest levels, represent high energy processes, energy levels that would rip apart aggregates at higher levels.  In this universe, all systems are built upon the aggregation processes laid down in the earliest moments, aggregations that occur at the upper limits of heat and pressure –  strings, quarks, sub-atomic particles, atoms, molecules.  Each corresponding to a matching environmental energy level, an energy level that is cooler and less pressurized than the ones that came before it.  The universe gets cooler and more dispersed.  Always.  The growth of complexity, evolution, is dependent upon this predictable and unavoidable dissipation of energy over time.

Those who would argue that life is special, that evolution is exclusive to it, well they are obligated to draw a definitive boundary between life and everything else, and because life, like everything else, is dependent upon the historical layering of aggregate systems, will have to draw that line historically.  They will have to show a moment in time before which there was not life or evolution and after which there was life and evolution.

There are many ways to define life in order that such a line could be drawn.  If you say, as most do, that life is that set of systems that incorporate and utilize both R and D Nucleic acids, well there is surely some moment in the past which would accurately delineate those earlier systems which didn't have both RNA and  DNA, from the later systems that did.  Such a definition is some what arbitrary, but all categorical definitions are.  But if you seek instead to hinge your definition of life to the process of evolution, then you are faced with a tautologically intractable problem.  Either you must accept the nonsensical proposition that the universe started with RNA/DNA preformed, or the more rational causal proposition that evolution is independent of and proceeded biology, preparing over vast periods of time, the aggregate ingredients necessary for the super-aggregate we call life.  If you insist despite this logic, that evolution is a property exclusive to biology, then you are left with the thorny problem of defining aggregation processes happening simultaneous to and independent of biology.  Processes that continue to produce atoms, molecules, stars, planets, galaxies, cultures, ideas, sand dunes, ocean currents, etc. And, you must also show how these continuous and omnipresent processes are qualitatively different when they happen outside of systems that use RNA and DNA from those that do.  But that isn't enough, you must also show either that no system after biology will ever evolve, of that the entire future of evolution will happen within the confines of biological systems.

The evidence and logic weighs overwhelmingly on the side of life being an arbitrarily bounded category, and evolution defining a process unbounded by domain, history, or complexity. Both of which are difficult concepts for humans to accept.  We like to think we belong to a category made exclusive by some secret sauce, some magic that applies in some measure only to life, and which has reached its zenith in the human form or spirit.  We like to imagine evolution to be that process that shaped the shapeless gasses of primal soup into the perfect form that we now enjoy.  Wow.  The ego and hubris drips and pools.

If I may, back to fitness.  The above arguments are crafted to shake we humans free of our innate bio/human/self centrism and to show how such hubris works to emphasize contemporary corporal form over timeless ephemeral process, placing a sort of artificial spotlight on species and downgrading the in contrast, fitness.  Its only natural.  And it is wrong.

The tendency to focus on species is easy to understand.  If you are looking at an animal and asking questions about evolution and process it is only natural that the scope of your thinking would be restricted to that animal, that species, that family of life and its struggle to survive.  Even when you back your self out to a vantage wide enough to include all of life, the full fan of Linnaean Taxonomy over the full 4.5 billion year crawl, the focus is still thing, still survival, still some sort of cosmic engineering project.  It is only when you back all the way out, when you look at all that is, the entire Universe, every moment since the big bang, life and the stuff between, in, and of it, that you might be forced to ask questions big enough to frame the why of evolution.

The why of evolution has to be big enough to comfortably hold all change, all systems, any aggregate and any aggregate chain, not just those that succeed, not just those that are fit, not just things that can be called things… everything!  Any process that explains the existence of one system should also be able to explain every other system.  Universality, at this depth of scope demands a bigger reason than can be explained by the concept "species".  Darwin's big how in biology then becomes a local mapping to a specific domain.  It isn't wrong, it just isn't universal.  You can know everything about pianos, but won't really understand music until you know enough about enough instruments that you begin to see the formative patterns that unite, from which all instruments are informed.

Species, be it a valid concept at all, must be but a subset, an example, a non-special representative, a member of a perfectly inclusive, and domain independent set.  Sets that include everything are not informative as a set.  So we look elsewhere.  That species, as a label, pointing to the subject of evolution, can equally be applied to any thing, forces us to look elsewhere for that which explains the big why of change. Change must not reside in thing, product, tailings, result, or even detritus.  If the big why isn't thing, but has to explain thing, any thing, all things, than the big why must be a process or action or modifier or pressure.  Some common attribute of any change regardless of domain.  What process is agnostic to domain?

In a word, entropy. In an attempt to determine the maximum work that could be extracted from any source of energy, steam era engineers teased apart the relationship between source and output and found an intriguing and strangely universal leakage.  Energy, when used, degrades, diffuses, is no longer as useful or available to the original process.  When scientists discovered the same leak, this time with structure, a strange universality began to appear.  Energy and information, force and structure, an unexpected symmetry.  Then Einstein revealed the exact relationship between energy, time, space, and mass, allowing thermodynamic transforms on all physical terms.  Despite initial objections by Stephen Hawking (and others attracted to the notion that nooks and crannies of the universe might provide respite from the second law's rigid causal prescriptions), Leonard Susskind and others have brought both the quantum world of the impossibly small and the black hole world of the impossibly big, together under a shared entropic umbrella.  What we are left with, like it or not, is a universal.  A universal that is universal to all physical domains and dimensions, regardless of scale.  Wow. That doesn't happen very often in nature.  That hasn't happened in science.  Ever.  Significant?

In his 1927 book, The Nature of the Physical World, Sir Arthur Eddington, put it this way:

"The law that entropy always increases, holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell's equations - then so much the worse for Maxwell's equations. If it is found to be contradicted by observation - well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation."

Any theory or assessment of evolution that is not written in response to thermodynamics, information theory, and entropy would seem to be a theory not particularly interested in validity.  That the laws of thermodynamics and evolution both direct their unblinking stares upon the domain of change would seem to me an invitation to at least begin to consider the possibility of a concerted union between the two.

[more to come…]

Randall Reetz