The Metaweb: The Global Mind Just Got Smarter

One of the many cool things about the Metaweb is that it functions as a vast bottom-up collaborative filtering system. RSS feeds represent perspectives of publishers. Because feed publishers can automatically or manually include content from other feeds they can "republish," annotate and filter content. Every feed is effectively a switch, routing content to and from other feeds. You are my filter. I am your filter.

Entire communities can collaboratively filter information, in a totally bottom-up way. The community as a whole acts to filter and route content in an emergent fashion, without any central coordination. On top of this sites can then provide value-added aggregation and information-refinery services by tracking memes across any number of feeds and then repackaging and redistributing them in virtual feeds for particular topics or interests. And these new feeds are fed right back into the collective mind, becoming raw materials for still other feeds that pick them up.

What we have here is the actual collective consciousness of humanity thinking collective thoughts in real-time, and we get to watch and participate! We are the "neurons" in the collective minds of our organizations, communities, marketplaces. Our postings comprise the memes, the thoughts, in these collective thought processes. Already the Metaweb is thinking thoughts that no individual can comprehend -- they are too big, too distributed, too complex. As the interactions of millions of people, groups and memes evolve we will see increasing layers of intelligence taking place in the Metaweb.

I am speaking here of the evolution of cybernetic feedback loops that act to filter, amplify, dampen and trigger global-scale virtual collective thought processes with unprecedented levels of intelligence. Perhaps, from a distance, it might even look as though the system as a whole is goal-directed or "intelligent" and maybe even "self-aware." I have always thought that we can learn more about the human brain-mind from the Internet than anyone realizes.

Nature has a way of evolving the best solutions, and as I watch the Metaweb evolve I wonder whether it will take a similar path to the evolution of the human nervous system. By studying the way the Metaweb "thinks" will we be able to derive new understands of how the human mind thinks? I think this is likely. I believe that there are abstract functional principles that apply to all intelligent systems of sufficient scale and complexity.

The Metaweb is already approaching the scale of complex nervous systems and in the future it will definitely surpass even the complexity of the human brain. While complexity itself does not create intelligence, it is a necessary condition for high-level intelligence at least. It would appear that the Metaweb is rapidly evolving the basic building blocks of higher-order intelligence: Neurons (people), axons and dendrites (social relationships), synapses (links, addresses, and applications that "fire" on events).

Like the human brain, the Metaweb is a distributed, decentralized, emergent computational system. Like the human brain, the Metaweb processes information flows. Like the human brain, the Metaweb has sensory inputs (human beings, software applications, sensors) and can sense and even reflect upon its own state and content (for example, when someone reads an article and then comments on it). And like humans the Metaweb is truly self-aware in the strong sense of the word -- there is actual consciousness in it -- whatever that is. Where is this consciousness? It is in us, the humans. As an astronaut once said, "We are the sensing elements of humanity." I would add, "We are the self-awareness of the Metaweb."

I don't believe that consciousness can be synthesized by any formal information processor, but I do think that consciousness that is already there can be brought into another system. In this case we may view the Metaweb as a vast collective extension of our own nervous systems and bodies -- it enables our minds (and perhaps eventually our bodies) to reach out around the globe, to sense ideas and images, sounds, and processes that take place beyond the limits of the individual human mind and body. It is also breaking down the boundaries between individual minds (and bodies someday?). And it is starting to cause us to engage in collective behaviors (consider Flashmobs for example).

The Metaweb is a reflection of ourselves and our communities -- and like any mirror, it at first merely reflects behavior, but soon begins to condition and guide behavior. Just as we may look in the mirror to decide what to wear, we may look in the Metaweb -- a vast societal meta-mirror -- to decide what to think and do next. As individuals begin to increasingly engage in collective thought processes on the Metaweb they will also begin to be conditioned by those thought processes which in turn will influence their individual behavior. Because their individual behavior will be conditioned by collective thoughts, they will be engaging in collectively intelligent behaviors, without even necessarily knowing that they are doing so. These collective behaviors are intelligent yet transcend the individuals that comprise them. This is analagous to the human brain. Individual neurons could barely be considered to be "intelligent" yet the brain as a whole conducts intelligent processes across them. The neurons cannot "understand" or even perceive the thoughts they are part of. In the Metaweb, the intelligence of the whole is greater than the sum of the intelligences of the parts.

For example, when you decide to buy Product X, was it because that idea originated in your mind on its own or because it came from somewhere else? What if millions of people read a product review on the Metaweb and link to it? That conditions the buying behavior of millions of people, causing many to go out and buy (or not buy) that product. This is an example of a collective thought conditioning a collective behavior. Another collective behavior is electronic democracy -- for example the kinds of large-scale actions being coordinated by MoveOn.org or the Howard Dean campaign. As we become increasingly involved in collective feedback loops the boundary between the individual and the group begins to blur. The boundary between our own minds and the collective mind begins to blur as well.

The first Web (what I call "Web1") made it easy for any one to publish a thought (a meme) into the collective mind, but it lacked an efficient "thought processing" capability. New "personal publishing" technologies such as Weblogs and RSS feeds shorten the feedback loop between collective thinking and behavior by providing the necessary enabling infrastructure for memes to propagate, interact, compete, collaboration, reproduce, mututate more quickly across the Net. On Web1 memes were relatively stationary and slow moving. For a meme to propagate, it took real work to publish it, real work for others to find it, real work for them to redistribute it and they propagated from site to site relatively slowly.

On the Metaweb ("Web2") however, it is trivially easy for even non-technical consumers to create and post memes to weblogs, find or get memes from others they subscribe to, and link and redistribute memes to others. In other words, the Metaweb makes memes "smarter" and "faster." It vectors them more efficiently, in less time, over broader areas, with better targeting, resulting in better transmission. By reducing the barriers to effective collective cognition, the Metaweb will enable individuals, groups, organizations, communities and socieities to think and act more intelligently, both individually and collectively. As increasing semantic richness is added to the Metaweb the memes will become even more self-describing and machine-understandable, which will in turn enable software applications to become smarter too.

Think of the Metaweb as a vast distributed superorganism that is evolving around us. We humans are the brain cells of this system -- we constitute it's consiousness, intelligence and intention. But already its memory is spreading beyond our own brains -- the millions of pieces of data on the Web are all parts of its collective memory and they link to memories stored in our own minds. I can retreive a memory from the Metaweb without caring whether that memory comes from a database or a human. I can also access intelligence in the Metaweb without caring where it comes from. An example of this: Once I was challenged to solve a very difficult math problem faster than the best programmer in my company. The programmer, call him Dr. Foo Bar went to work diligently writing a program to solve the problem. I, on the other hand, emailed my 10 smartest math friends and asked them to solve the problem. Who solved it first? I did, even though I could not have solved it myself given my limited theoretical math skills. Now both myself and Dr. Foo Bar were black boxes, which of us would you judge as more intelligent? In the end it is the black box that provides the best answer fastest.

Like humans, as the Metaweb evolves, software applications -- such as future intelligent agents for example -- will also begin to evolve high levels of intelligence (although not self-aware!) in this system as well. This will be enabled by increasing amounts of "smart data" -- metadata-enabled data -- that machines are able to better understand. I see RSS as the first step in this process -- the first consumer-level "killer app" of XML.

As humans and software become increasingly locked into close symbiosis we will see a high degree of co-evolution taking place that will likely influence both the direction of software evolution and human evolution. A case in point: Humans have opposable thumbs as a direct consequence of tool use. If the Metaweb is just another "tool" then how might it change our long-term evolution? Will our bodies and minds change as a result of this technology? I think the answer is definitely "yes." In fact, I think the long-term trend here is that the very concept of the individual is evolving -- we are becoming less distinct (as the Metaweb dissolves the boundaries between your mind and mind) but more unique (as the Metaweb gives us each more expressive power and reach).

Where else has this happened? It happened billions of years ago as simple single-cellular organisms evolved. At first, in order to survive, single-celled organisms had to be fairly distinct and generalized -- they needed to be able to ingest nutrients, digest them, excrete waste, defend against attack, reproduce, etc. But over as different types of single-celled organisms interacted it turned out that in certain combinations such organisms could be more successful than on their own. For example if organism A has a lot of surface area and is good at sticking to nutrients and organism B is really good at breaking down nutrients into digestible components and organism C is good at repelling attack and processing waste, then as a community they can be more successful.

Some evolutionary theorists have posited that because communities provided selective advantages to their members, evolution favored "social" organisms. In this case even single-celled organisms could be considered social in a primitive sense. But here's the catch: by joining a community an organisms tend to become increasingly specialized; they lose generality. This occurs because if organism A is great at doing X but not Y, while organism B is great at Y but not X, then together A can focus on X and B can focus on Y.

In other words, by joining forces A overcomes the selective disadvantage of being bad at Y and B overcomes the selective disadvantage of being bad at X -- they protect each other from the negative effects of natural selection by compensating for one another's weaknesses and enhancing one another's strengths. Thus A and B are more likely to survive and reproduce successfully together than apart. What I am getting to here is that symbiosis leads to specialization.

Today we can see the same thing happening on the human-level. For example consider the field of medicine. It used to be that all doctors were general practitioners, familiar with the whole field of medicine. But today that is no longer the case. Doctors have become so specialized that they rarely know or understand one another's disciplines. To conduct even a basic surgery usually requires the involvement of several specialists. No one doctor can function on their own anymore. A modern hospital is in fact a superorganism, much like an ant colony or a bee hive although organized differently, that enables extremely specialized organisms to collaborate as if they were a single entity on collective behaviors. The same is true of the modern military, or even any large corporation. They are all superorganisms. And this is of course also the case with the Metanet, the largest superorganism we know of.

Now the important question here is: at what point can we say that a bunch of parts becomes a new whole in its own right? At what point can we speak of "the superorganism" as an entity? The crucial boundary is called a "metasystem transition" which is a transition from one order of organization to a higher order of organization.

Metasystem transitions occur throughout nature and may be key to understanding evolution on every level. In the case of single-celled organisms, a metasystem transition occurs when, over many generations, organisms in communities co-evolve to the point where they are so specialized that they lose their generality and can no longer function individually. The parts -- the individual organisms -- become a new whole -- the superorganism -- at such time as they can no longer survive and reproduce effectively apart from the whole: they are just too specialized.

On the human level I believe that we are in the early stages of a 1000 year metasystem transition driven by the symbiosis of humans, organizations and machines (in particular, software, networks and network computing devices). Human organizations combined with network computing systems enable the ultimate level of human specialization. Not only is it no longer necessary for a human individual to know everything in order to quickly access information about it, but they can easily leverage or participate in the combined expertise of groups of other specialists located anywhere on the planet, in real time or asynchronously.

In other words the final boundaries of human specialization are being transcended: the boundaries of space, time, body and mind. The human superorganism is even evolving the ability to change its own code -- as genetic engineering advances we may soon see feedback loops from the Metaweb to the genome and back again. In fact, this may already be happening as teams of biologists interact via the Metaweb, in turn changing their behavior and resulting in new genetic experiments on a daily basis.

We are in the process of "going virtual." And this is really what the Metaweb is doing -- it is virtualizing the human species, starting with our minds and eventually even encompassing our organizations, bodies and environments. Does this mean that someday in the not-too-distant future we will all live in the Matrix? Does this mean that someday we will all be avatars in a virtual reality? Maybe we already are. It's hard to know for sure, but recent progress in information physics (such as the field of cellular automata, especially the works of Ed Fredkin and more recently, Stephen Wolfram) seem to indicate that it may be possible that the universe is a simulation running on a giant computer of sorts.

However, I think there is infinitely more to the story -- if the universe is running on some kind of computer, it is not equivalent to a Turing Machine -- it is something that totally transcends the limitations of what we think of as a "computer" (evidence for this: consciousness is not an information process and does not require any information and the fact that the quantum world does not appear to be computable in the classical sense) and may be a lot more like what we think of as a "sentient being." But that is the subject of another essay!

For now what is important is the present. We are in the process of evolving the Metaweb -- the beginnings of our species' superorganism. Perhaps the stage of evolving into a superorganism represents the highest degree of evolution of a species. It's an interesting thought.

Perhaps ants, bees, and termites really are more advanced as a species on the evolutionary scale (not necessarily on the intelligence dimension however) than the human species. We are remarkably primitive and self-destructive as a species. Is our lack of self-awareness and self-organization on a species-level a positive or a negative? On a million-year scale some would suggest that it is this very chaos -- bloody as it may be -- that has enable the great leaps in human innovation and evolution that set us apart from other animals. I believe that there may be something to that hypothesis. However I would suggest that perhaps in the end the highest achievement for a species is to evolve a sustainable balance between chaos and order, part and whole. This is a very Taoist way of thinking, and for that reason it is probably right.

Comments

[kvantomme]

I believe that it won't take 1000s of years, in less than 1 generation we are already loosing our generality. I am a 26 year old consultant and I can't imagine how I could do any type of "knowledge work" without the Internet. Should the Internet stop to exist, I would have to fall back to a much lower income generating profession.

[Otov]

Damn, it looks like skynet... or matrix :-/

[Otov]

Damn, it looks like skynet :-/

[Lucas]

This is all well and true, but doesn't take into account the critical bottoms-up dichotimizing factor within these organizational principles. That is, the push and pull between producer and consumer. It is the producer's drive to self aggrandize, it is the consumer's drive to find useful stuff. How to align these drives lies at the heart of the shape of the beast we are forming.