This Blog has Moved to NovaSpivack.com

Radar Networks Coming Out of Stealth - Friday, October 19

News Flash!

My company, Radar Networks, is coming out of stealth this Friday, October 19, 2007 at the Web 2.0 Summit, in San Francisco. I'll be speaking on "The Semantic Edge Panel" at 4:10 PM, and publicly showing our Semantic Web online service for the first time. If you are planning to come to Web 2.0, I hope to see you at my panel.

Here's the official Media Alert below:

               

(PRWEB) October 15, 2007 -- At the Web2.0 Summit on October 19th, Radar Networks will announce a revolutionary new service that uses the power of the emerging Semantic Web to enable a smarter way of sharing, organizing and finding information. Founder and CEO Nova Spivack will also give the first public preview of Radar’s application, which is one of the first examples of “Web 3.0” – the next-generation of the Web, in which the Web begins to function more like a database, and software grows more intelligent and helpful.

Join Nova as he participates in “The Semantic Edge” panel discussion with esteemed colleagues including Powerset’s Barney Pell and Metaweb’s Daniel Hillis, moderated by Tim O’Reilly.

Who:   
Radar Networks Founder and CEO Nova Spivack

When:   
Friday, October 19, 2007
4:10 – 4:55 p.m.
   
Where: 
Web2.0 Summit
Palace Hotel
Grand Ballroom
2 New Montgomery Street
San Francisco,  California  94105
   

Understanding The Semantic Web: A Response to Tim O'Reilly's Recent Defense of Web 2.0

Tim O'Reilly, recently blogged another article about Web 2.0 Versus Web 3.0 in which he responded to some of my points about what Web 3.0 is and is not. There are several points in his post that I need to respond to. Here is what I am going to cover in this article:

• Correcting some factual errors in Tim's post about Web 3.0
• Web 2.0 = Industry Renaissance + Marketing Hype
• Web 2.0 was NOT mainly about back-end innovation
• Web 2.0 = The Social Web
• What's After Web 2.0?
• The Semantic Web = The Data Web
• The Value of Open Data
• Key Points of Differentiation

FACTUAL ERRORS THAT NEED TO BE CLARIFIED

Before I address where I agree/disagree with his article, there are some factual errors that should be corrected. Contrary to what Tim states, the term "Web 3.0" was NOT originated by me, and I've never claimed it was. Nor have I designed a definition for it that was tailor-made for what my startup, Radar Networks, is up to.

In fact, the term, Web 3.0, was  independently originated by Jeffrey Zeldman, Tim Berners-Lee, Reed Hastings, John Markoff, and Dan Gillmor. They all had one thing in common however -- a feeling that "Web 2.0" wasn't the end of the story for the Web -- and that something new was brewing.

Personally speaking, the first time I ever heard the term  was actually in John Markoff's New York Times Article on the Intelligent Web, in which he mentioned several companies including my own. That article made the first connection between "Web 3.0" and The Semantic Web -- and since that time many people have come to think of these terms as synonymous.

My only contribution to the whole Web 3.0 debate has been to try to define the term as something that makes more sense, namely, just a decade characterized by a range of technologies that are coming to the fore, which will include The Semantic Web for sure, but will not be limited to it. I've probably put more effort into trying to clarify this term than most, and for that I apologize!

If you are interested in the history, I would encourage you to read the Wikipedia page on the subject for a more detailed account.

WEB 2.0 WAS A RENAISSANCE

I agree with Tim that the Web 2.0 era was a renaissance -- and that there were certain trends and patterns that I think Tim recognized first, and that he has explained better, than just about anyone else. Tim helped the world to see what Web 2.0 was really about -- collective intelligence.

In short Tim deserves a lot of credit for defining Web 2.0 and plugging for that meme incessantly for years. Had he not done that, the industry might not have come back the way it did. I am very thankful for those efforts -- after all they probably got my company funded.

WEB 2.0 = INDUSTRY RENAISSANCE + MARKETING HYPE

Tim is annoyed because he thinks that Web 3.0 is "marketing hype." But let's face it, Web 2.0 is essentially just a buzzword that is nothing other than a marketing term that was designed to promote a conference. Tim even admits this in his article:

Web 2.0 started out as the name of a conference!  And that name had a very specific purpose:  to signify that the web was roaring back after the dot com bust! The 2.0 bit wasn't about the technology, but about the resurgence of interest in the web. When we came up with the idea back in 2003, a lot of programmers were out of work, and there was a general lack of interest in web applications. But we saw a resurgence coming, and designed a conference to tell the story of what was going to be different this time.

Tim is tacitly agreeing with my view on how these terms should be used in his passage above. Web 2.0 is a period of time that began "back in 2003" when there was a resurgence. People who speak of Web 3.0 are also referring to a period of time -- THE NEXT period of time AFTER Web 2.0, in which some kind of discernible new pattern is starting to emerge.

WEB 2.0 WAS NOT MAINLY ABOUT BACK-END INNOVATION

Tim does, I think make a good point about Google being a backend play and being a great Web 2.0 company. But then he goes on to say that "Every major web 2.0 play is a back-end story." I would have to disagree with that statement in just about every case but Google. In fact, every major Web 2.0 play has been built on the LAMP stack, or on Google, or the equivalent, for the most part, not on some fancy new backend technology.

What other Web 2.0 companies can you think of, besides Google, that are truly backend-innovations? Technorati maybe? I'm trying to think of some but the fact is, most Web 2.0 era companies were about getting apps built fast, cheap and simple, by re-using open-source backend components (which were mostly just non-commercial remakes of technologies which had previously existed commercially).

When we look at Web 2.0, the major contributions have been focused towards user-experience. Del.icio.us: A better way to share bookmarks. Flickr: A better way to share photos. YouTube: A better way to share videos. Blogs: A better way to publish user-generated content. Wikis: A better way to share documents. Myspace: A better way to make hideously ugly homepages. LinkedIn: A better way to do social networking. Facebook: a better way to keep in touch with friends. Etc.

The backends of all of these kinds of apps (other than perhaps Facebook, lately) have been nothing to write home about. It's what they did on the front-end that really sang. And not just the front-end in terms of user-interface, but also in terms of design patterns and approach.

WEB 2.0 = THE SOCIAL WEB

Web 2.0 has been about "the social Web" in my opinion.That's the big distinction. Google is really a social network measurement algorithm based on similar ideas that had been brewing in the field of bibliometrics for a decade prior, at least.

Most of the big Web 2.0 successes have also been social in one form or another. They have been about harnessing social networks, user-generated content, folksonomies, the wisdom of crowds, and collective intelligence. That's the real contribution of this era to the Web. Web 2.0 is the Social Web.

WEB 3.0 IS WHAT'S AFTER THE SOCIAL WEB

If Web 2.0 has largely been about new social applications of existing back-end technologies, the question is what comes next? Clearly there is still a lot of room for improving on the ideas of Web 2.0. But that's not NEW.

When we think about what would actually be new, it would have to be a characteristic shift that would enable a lot of innovation, new capabilities, new kinds of applications, new design patterns, new technologies. The Semantic Web certainly fits the bill. But it's not the only technology that will matter.

Tim seems to think that Web 3.0 should be a completely new take on the Web. He says:

So for starters, I'd say that for "Web 3.0" to be meaningful we'll need to see a serious discontinuity from the previous generation of technology. That might be another bust and resurgence, or more likely, it will be something qualitatively different. I like Stowe Boyd's musings on the subject:

Personally, I feel the vague lineaments of something beyond Web 2.0, and they involve some fairly radical steps. Imagine a Web without browsers. Imagine breaking completely away from the document metaphor, or a true blurring of application and information. That's what Web 3.0 will be, but I bet we will call it something else.

I'm with Stowe. There's definitely something new brewing, but I bet we will call it something other than Web 3.0. And it's increasingly likely that it will be far broader and more pervasive than the web, as mobile technology, sensors, speech recognition, and many other new technologies make computing far more ambient than it is today.

Ironically, Stowe suggests that Web 3.0 will be a "blurring of application and information" -- that is exactly what the Semantic Web is in fact. Tim then goes on to agree, stating that he thinks there is going to be something far more pervasive than the web, etc. Again that's another point in support of The Semantic Web -- which in fact is not just about the Web, but all information and all applications. It's a better way to handle the DATA that they use.

The Semantic Web "blurs applications and information" because it starts to move the semantics out of applications and into the information itself. Applications can therefore be smarter while also being thinner -- more of what used to be application logic moves into the data itself; the data becomes "smarter."

THE SEMANTIC WEB IS THE DATA WEB

The Semantic Web is not about AI or anything fancy like that, it is really just about data. Another and perhaps better name for it would be "The Data Web."

RDF enables something as potentially important as HTML. Just as HTML enabled a universally reusable Web of content, RDF enables the Data Web, a universally reusable Web of data. The Web browser is a universal client for content, but not really for data. Web browsers can render any content written in HTML in a standard way. That was a big leap back in the early 1990's. Previously each type of content required a different application to view it. The browser unified them all -- this separation of rendering from data made life easier for programmers, and for end-users. A single tool could render any data because the data carried metadata (HTML) that described how to render it.

But currently although browsers can render the formatting and layout of data, they don't know anything about the  meaning of the data, unless they are explicitly programmed to do so. The same is true for all applications today -- they have to be explicitly programmed in advance to interpret each kind of data they need to use.

The Semantic Web provides a solution for this problem that is analogous to what HTML did for content -- RDF and OWL provide a standard way to describe the meaning of any data structure, such that any application that speaks these languages can correctly interpret the meaning without having to have been explicitly programmed to do so in advance.

In other words, the Semantic Web offers the promise of a "universal client for data." That would be a big improvement over how applications are written and how data is managed and stored today. It's a significant back-end level upgrade, and it requires not only that data is represented differently, but new tools for managing it (new kinds of databases, new API's, new forms of search, etc.).

There's also an added benefit to the Semantic Web -- one which is usually OVER-emphasized -- and that is reasoning. The rich semantics of the RDF and OWL languages enable metadata that not only describes the meaning of data, but also the logical relationships between data and various concepts.

This richer metadata can be used to support machine reasoning, such as simple inferencing, across data on the Web. That's powerful and will enable a whole new generation of smarter applications and services -- the so-called "Intelligent Web" -- but it's not the main point! I think that is rather far off in the future still. Today, just making the "Data Web" would be a huge innovation. Transforming the Web from a distributed file-server to a distributed database is a huge enough step on its own.

THE REAL POINT OF THE SEMANTIC WEB = OPEN DATA

The fact is, while I have great respect for Tim as a thinker, I don't think he truly "gets" the Semantic Web yet. In fact, he consistently misses the real point of where these technologies add value, and instead gets stuck on edge-cases (like artificial intelligence) that all of us who are really working on these technologies actually don't think about at all. We don't care about reasoning or artificial intelligence, we care about OPEN DATA.

From what I can see, Tim thinks the Semantic Web is some kind of artificial intelligence system. If that is the case, he's completely missing the point. Yes, of course it enables better, smarter applications. But it's fundamentally NOT about AI and it never was. It's about OPEN DATA. The Semantic Web should be renamed to simply The Data Web.

Watch how Tim Berners-Lee talks about it these days as a "universal data bus", for example in this video. That would be a much more accurate description of where the real thrust of these technologies is headed.

The real benefit of RDF and OWL is that they disrupt the idea of what a database is -- making it something that is much more open, more richly described, more decentralized, more extensible, more maintainable, more portable, more precisely definable, and more useful.

If you really look at RDF, OWL, and in particular GRDDL and SPARQL, it becomes crystal clear that this is a set of technologies about freeing data from platform and application lock-in. That is really what the Semantic Web is for.

The benefit of Open Data is that it enables databases and the data they contain to be designed, shared, and mashed-up in a totally bottom-up, user-driven, Web 2.0 manner. This is in fact collective-intelligence applied to data.

I'm really looking forward to the day when Tim O'Reilly sees that the true value of the RDF and OWL, not to mention SPARQL and GRDDL (aka "The Semantic Web") is Open Data. I think when he finally "gets it" he will actually be quite excited about it. When he sees how these technologies enable a bottom-up, distributed, user-generated open Web of Data, I think he will have an epiphany.

KEY POINTS OF DIFFERENTIATION

There are few important distinctions that Tim is starting to agree with however:

1. Web 3.0 is not equivalent the Semantic Web. Whatever Web 3.0 is, it is not only going to be limited to the Semantic Web. Many other technologies are emerging that are also going to matter. Collectively they will bring about a shift that is fundamentally disruptive and qualitatively different from the Web 2.0 content and applications we know today.
2. The Semantic Web is completely orthogonal to the issue collective intelligence. The Semantic Web is no more or less about collective intelligence than Web 2.0. In fact, it has the potential to provide a better medium for collective intelligence. Services like Metaweb's Freebase, and the new service that Radar Networks will be launching soon, both help to illustrate this point.
3. The Semantic Web is precisely the kind shift that can define a new era of the Web.  Transforming the Web from a fileserver model to a database model is a big, fundamental shift. The kind of shift that qualifies for a new term like "Web 3.0" if that is necessary.
4. This is only the very beginning. Let's not Over-Hype The Semantic Web. The Semantic Web has the potential to really upgrade the back-end of the Web and all the applications and content on it. But it has not done that YET. It is going to be an incremental process -- just like Web 2.0 was. Web 3.0 (2010 - 2020) has not really started yet. We are just in 2007 and there is much to be done to get there. We should be careful not to over-hype what we are doing.  The real value of the Semantic Web will take many years to truly emerge. The "proof is in the pudding" -- but the pudding is going to take some time before it is fully ready for mass-consumption.

In closing I want to also point out that while I am enthusiastic about The Semantic Web, I am not a purist -- I actually believe in using what works best, rather than being stuck on some ideology. To that extent, Radar Networks is making use of the Semantic Web, where appropriate, but we also use other techniques and technologies. We're pragmatists at heart.

Web 3.0 -- The Best Official Definition Imaginable

Jason just blogged his take on an official definition of "Web 3.0" -- in his case he defines it as better content, built using Web 2.0 technologies. There have been numerous responses already, but since I am one of the primary co-authors of the Wikipedia page on the term Web 3.0, I thought I should throw my hat in the ring here.

Web 3.0, in my opinion is best defined as the third-decade of the Web (2009 - 2019), during which time several key technologies will become widely used. Chief among them will be RDF and the technologies of the emerging Semantic Web. While Web 3.0 is not synonymous with the Semantic Web (there will be several other important technology shifts in that period), it will be largely characterized by semantics in general.

Web 3.0 is an era in which we will upgrade the back-end of the Web, after a decade of focus on the front-end (Web 2.0 has mainly been about AJAX, tagging, and other front-end user-experience innovations.) Web 3.0 is already starting to emerge in startups such as my own Radar Networks (our product is Twine) but will really become mainstream around 2009.

Why is defining Web 3.0 as a decade of time better than just about any other possible definition of the term? Well for one thing, it's a definition that can't easily be co-opted by any company or individual around some technology or product. It's also a completely unambiguous definition -- it refers to a particular time period and everything that happens in Web technology and business during that period. This would end the debate about what the term means and move it to something more useful to discuss: What technologies and trends will actually become important in the coming decade of the Web?

It's time to once again pull out my well-known graph of Web 3.0 to illustrate what I mean...

(Click the thumbnail for a larger, reusable version)

I've written fairly extensively on the subjects of defining Web 3.0 and the Semantic Web. Here are some links to get you started if you want to dig deeper:

The Semantic Web: From Hypertext to Hyperdata
The Meaning and Future of the Semantic Web
How the WebOS Evolves
Web 3.0 Roundup
Gartner is Wrong About Web 3.0
Beyond Keyword (And Natural Language) Search
Enriching the Connections of the Web: Making the Web Smarter
Next Step for the Web
Doing for Data What HTML Did for Documents

My Burma Meme Spreads to 17,000 Web Pages in just one week!

I've been tracking the progress of my Burma protest meme. In just under one week it has spread to almost 17,000 web pages and it continues to grow. (For the latest number, click here). It's great to see the blogosphere pick this up, and I'm glad to be able to do something to help raise awareness of this important human rights issue.

This meme is also an example of an interesting new way to spread content on the Web -- whether for a protest or an ad or any other kind of announcement. It's kind of like a chain letter, but via weblogs. There are many different ways to structure these memes with varying levels of virality and benefit to participants. For some earlier work I've done on meme propagation on the Web see my GoMeme experiments from a few years ago. In those experiments I created a series of memes that spread widely through the blogosphere, based on different viral messages, surveys, and benefits to participants. Other people then tracked the statistics of the memes as they spread. It turned out to be a very interesting study of superdistribution of content along social networks.

Gartner is Wrong about Web 3.0

I have a lot of respect for the folks at Gartner, but their recent report in which they support the term "Web 2.0" yet claim that the term "Web 3.0" is just a marketing ploy, is a bit misguided.

In fact, quite the opposite is true.

The term Web 2.0 is in fact just a marketing ploy. It has only come to have something resembling a definition over time. Because it is in fact so ill-defined, I've suggested in the past that we just use it to refer to a decade: the second decade of the Web (2000 - 2010). After all there is no actual technology that is called "Web 2.0" -- at best there are a whole slew of things which this term seems to label, and many of them are design patterns, not technologies. For example "tagging" is not a technology, it is a design pattern. A tag is a keyword, a string of text -- there is not really any new technology there. AJAX is also not a technology in its own right, but rather a combination of technologies and design patterns, most of which existed individually before the onset of what is called Web 2.0.

In contrast, the term Web 3.0 actually does refer to a set of new technologies, and changes they will usher in during the third decade of the Web (2010 - 2020). Chief among these is the Semantic Web. The Semantic Web is actually not one technology, but many. Some of them such as RDF and OWL have been under development for years, even during the Web 2.0 era, and others such as SPARQL and GRDDL are recent emerging standards. But that is just the beginning. As the Semantic Web develops there will be several new technology pieces added to the puzzle for reasoning, developing and sharing open rule definitions, handling issues around trust, agents, machine learning, ontology development and integration, semantic data storage, retrieval and search, and many other subjects.

Essentially, the Semantic Web enables the gradual transformation of the Web into a database. This is a profound structural change that will touch every layer of Web technology eventually. It will transform database technology, CMS, CRM, enterprise middleware, systems integration, development tools, search engines, groupware, supply-chain integration, and all the other topics that Gartner covers.

The Semantic Web will manifest in several ways. In many cases it will improve applications and services we already use. So for example, we will see semantic social networks, semantic search, semantic groupware, semantic CMS, semantic CRM, semantic email, and many other semantic versions of apps we use today. For a specific example, take social networking. We are seeing much talk about "opening  up the social graph" so that social networks are more connected and portable. Ultimately to do this right, the social graph should be represented using Semantic Web standards, so that it truly is not only open but also easily extensible and mashable with other data. 

Web 3.0 is not ONLY the Semantic Web however. Other emerging technologies may play a big role as well. Gartner seems to think Virtual Reality will be one of them. Perhaps, but to be fair, VR is actually a Web 1.0 phenomenon. It's been around for a long time, and it hasn't really changed that much. In fact the folks at the MIT Media Lab were working on things that are still far ahead of Second Life, even back in the early 1990's.

So what other technologies can we expect in Web 3.0 that are actually new? I expect that we will have a big rise in "cloud computing" such as open peer-to-peer grid storage and computing capabilities on the Web -- giving any application essentially as much storage and computational power as needed for free or a very low cost. In the mobile arena we will see higher bandwidth, more storage and more powerful processors in mobile devices, as well as powerful built-in speech recognition, GPS and motion sensors enabling new uses to emerge. I think we will also see an increase in the power of personalization tools and personal assistant tools that try to help users manage the complexity of their digital lives. In the search arena, we will see search engines get smarter -- among other things they will start to not only answer questions, but they will accept commands such as "find me a cheap flight to NYC" and they will learn and improve as they are used. We will also see big improvements in integration and data and account portability between different Web applications. We will also see a fundamental change in the database world as databases move away from the relational model and object model, towards the associative model of data (graph databases and triplestores).

In short, Web 3.0 is about hard-core new technologies and is going to have a much greater impact on enterprise IT managers and IT systems than Web 2.0. But ironically, it may not be until Web 4.0 (2020 - 2030) that Gartner comes to this conclusion!

Cool Women in Tech

Hey, I'm posting this to support my friend Kaliya's  (un)conference for women in  tech called She's Geeky, A Women's Tech (un)conference (October 22-23 in Mountain View, CA).

Here at  Radar Networks we're proud that we have a bunch of really smart women across all areas of the company including Sonja Erickson our awesome VP of systems, Jennifer Agostinelli who runs operations and makes our culture great, Karen Marcelo who hacks code and in her free time builds fire-breathing robots, Lara Fields who hacks RDF in a land called Cucamonga, Susan Mayo who is not only a woman in tech but also a woman in soccer, Tricia Royal who is a designer of many things, and just this week we added Kim Laama on UI, and soon two more women are joining our team. More than 1/3 of our staff are women -- Not bad for a company working on bleeding-edge tech!!!

The Semantic Web, Collective Intelligence and Hyperdata

I'm posting this in response to a recent post by Tim O'Reilly which focused on disambiguating what the Semantic Web is and is not, as well as the subject of Collective Intelligence. I generally agree with Tim's post, but I do have some points I would add by way of clarification. In particular, in my opinion,  the Semantic Web is all about collective intelligence, on several levels. I would also suggest that the term "hyperdata" is a possibly useful way to express what the Semantic Web is really all about.

What Makes Something a Semantic Web Application?

I agree with Tim that the term "Semantic Web" refers to the use of a particular set of emerging W3C open standards. These standards include RDF, OWL, SPARQL, and GRDDL. A key requirement for an application to have "Semantic Web inside" so to speak, is that it makes use of or is compatible with, at the very least, basic RDF. Another alternative definition is that for an application to be "Semantic Web" it must make at least some use of an ontology, using a W3C standard for doing so.

Semantic Versus Semantic Web

Many applications and services claim to be "semantic" in one manner or another, but that does not mean they are "Semantic Web." Semantic applications include any applications that can make sense of meaning, particularly in language such as unstructured text, or structured data in some cases. By this definition, all search engines today are somewhat "semantic" but few would qualify as "Semantic Web" apps.

The Difference Between "Data On the Web" and a "Web of Data"

The Semantic Web is principally about working with  data in a new and hopefully better way, and making that data available on the Web if desired in an open fashion such that other applications can understand and reuse it more easily. We call this idea "The Data Web" -- the notion is that we are transforming the Web from a distributed file server into something that is more like a distributed database.

Instead of the basic objects being web pages, they are actually pieces of data (triples) and records formed from them (sets, trees, graphs or objects comprised of triples). There can be any number of triples within a Web page, and there can also be triples on the Web that do not exist within Web pages at all -- they can come directly from databases for example.

One might respond to this by noting that there is already a lot of data on the Web, in XML and other formats -- how is the Semantic Web different from that? What is the difference between "Data on the Web" and the idea of "The Data Web?"

The best answer to this question that I have heard was something that Dean Allemang said at a recent Semantic Web SIG in Palo Alto. Dean said, "Sure there is data on the Web, but it's not actually a web of data."  The difference is that in the Semantic Web paradigm, the data can be linked to other data in other places, it's a web of data, not just data on the Web.

I call this concept of interconnected data, "Hyperdata." It does for data what hypertext did for text. I'm probably not the originator of this term, but I think it is a very useful term and analogy for explaining the value of the Semantic Web.

Another way to think of it is that the current Web is a big graph of interconnected nodes, where the nodes are usually HTML documents, but in the Semantic Web we are talking about a graph of interconnected data statements that can be as general or specific as you want. A data record is a set of data statements about the same subject, and they don't have to live in one place on the network -- they could be spread over many locations around the Web.

A statement to the effect of "Sue lives in Palo Alto" could exist on site A, refer to a URI for a statement defining Sue on site B, a URI for a statement that defines "lives in" on site C, and a URI for a statement defining "Palo Alto" on site D. That's a web of data. What's cool is that anyone can potentially add statements to this web of data, it can be completely emergent.

The Semantic Web is Built by and for Collective Intelligence

This is where I think Tim and others who think about the Semantic Web may be missing an essential point. The Semantic Web is in fact highly conducive to "collective intelligence." It doesn't require that machines add all the statements using fancy AI. In fact, in a next-generation folksonomy, when tags are created by human users, manually, they can easily be encoded as RDF statements. And by doing this you get lots of new capabilities, like being able to link tags to concepts that define their meaning, and to other related tags.

Humans can add tags that become semantic web content. They can do this manually or software can help them. Humans can also fill out forms that generate RDF behind the scenes, just as filling out a blog posting form generates HTML, XML, ATOM etc. Humans don't actually write all that code, software does it for them, yet blogging and wikis for example are considered to be collective intelligence tools.

So the concept of folksonomy and tagging is truly orthogonal to the Semantic Web. They are not mutually exclusive at all. In fact the Semantic Web -- or at least "Semantic Web Lite" (RDF + only basic use of OWL + basic SPARQL) is capable of modelling and publishing any data in the world in a more open way.

Any application that uses data could do everything it does using these technologies. Every single form of social, user-generated content and community could, and probably will, be implemented using RDF in one manner or another within the next decade or so. And in particular, RDF and OWL + SPARQL are ideal for social networking services -- the data model is a much better match for the structure of the data and the network of users and the kinds of queries that need to be done.

Folktologies

This notion that somehow the Semantic Web is not about folksonomy needs to be corrected. For example, take Metaweb's Freebase. Freebase is what I call a "folktology" -- it's an emergent, community generated ontology. Users collaborate to add to the ontology and the knowledge base that is populated within it. That's a wonderful example of collective intelligence, user generated content, and semantics (although technically to my knowledge they are not using RDF for this, their data model is from what I can see functionally equivalent and I would expect at least a SPARQL interface from them eventually).

But that's not all -- check out TagCommons and this Tag Ontology discussion, and also the SKOS ontology -- all of which are working on semantic ways of characterizing simple tags in order to enrich folksonomies and enable better collective intelligence.

There are at least two other places where the Semantic Web naturally leverages and supports collective intelligence. The first is the fact that people and software can generate triples (people could do it by hand, but generally they will do it by filling out Web forms or answering questions or dialog boxes etc.) and these triples can live all over the Web, yet interconnect or intersect (when they are about the same subjects or objects).

I can create data about a piece of data you created, for example to state that I agree with it, or that I know something else about it. You can create data about my data. Thus a data-set can be generated in a distributed way -- it's not unlike a wiki for example. It doesn't have to work this way, but at least it can if people do this.

The second point is that OWL, the ontology language, is designed to support an infinite number of ontologies -- there doesn't have to be just one big ontology to "rule them all." Anyone can make a simple or complex ontology and start to then make data statements that refer to it. Ontologies can link to or include other ontologies, or pieces of them, to create bigger distributed ontologies that cover more things.

This is kind of like not only mashing up the data, but also mashing up the schemas too. Both of these are examples of collective intelligence. In the case of ontologies, this is already happening, for example many ontologies already make use of other ontologies like the Dublin Core and Foaf.

The point here is that there is in fact a natural and very beneficial fit between the technologies of the Semantic Web and what Tim O'Reilly defines Web 2.0 to be about (essentially collective intelligence). In fact the designers of the underlying standards of the Semantic Web specifically had "collective intelligence" in mind when they came up with these ideas. They were specifically trying to rectify several problems in the closed, data-silo world of old fashioned databases. The big motivation was to make data more integrated, to enable applications to share data more easily, and to be able to build data with other data, and to build schemas with other schemas. It's all about enabling connections and network effects.

Now, whether people end up using these technologies to do interesting things that enable human-level collective intelligence (as opposed to just software level collective intelligence) is an open question. At least some companies such as my own Radar Networks and Metaweb, and Talis (thanks, Danny), are directly focused on this, and I think it is safe to say this will be a big emerging trend. RDF is a great fit for social and folksonomy-based applications.

Web 3.0 and the concept of "Hyperdata"

Where Tim defines Web 2.0 as being about collective intelligence generally, I would define Web 3.0 as being about "connective intelligence." It's about connecting data, concepts, applications and ultimately people. The real essence of what makes the Web great is that it enables a global hypertext medium in which collective intelligence can emerge. In the case of Web 3.0, which begins with the Data Web and will evolve into the full-blown Semantic Web over a decade or more, the key is that it enables a global hyperdata medium (not just hypertext).

As I mentioned above, hyperdata is to data what hypertext is to text. Hyperdata is a great word -- it is so simple and yet makes a big point. It's about data that links to other data. It does for data what hypertext does for text. That's what RDF and the Semantic Web are really all about. Reasoning is NOT the main point (but is a nice future side-effect...). The main point is about growing a web of data.

Just as the Web enabled a huge outpouring of collective intelligence via an open global hypertext medium, the Semantic Web is going to enable a similarly huge outpouring of collective knowledge and cognition via a global hyperdata medium. It's the Web, only better.

Proposal for GRDDLR: The Open Source GRDDL Repository

Now that GRDDL is officially on its way to becoming a W3C open standard, I'm even more interested in its potential. For those who don't know, GRDDL provides a way to map from non-RDF data into RDF, on-the-fly. So for example, if a semantic web app needs data from some Web page, and that page does not contain RDF, a GRDDL transform can be provided that maps that Web page's content to appropriate RDF. The semantic web app can simply look for the presence of a GRDDL transform for that page and then generate RDF on the fly for it. Ok well that's probably an oversimplification but it gets the point across. Here are some nicely illustrative use-case examples that shed more light on the subject.

So this got me thinking. Where can semantic web app x find good GRDDL transforms to use? The current idea is that content creators will create and publish GRDDL for their own content. So the publisher of Web page x would have to create a GRDDL transform for it and host it on their site. When apps come along that need RDF they can check to see if this GRDDL information exists there, and if so they can use it.

But that approach puts all the burden on content publishers -- many of whom may be slow to adopt GRDDL until they see some real demand for it. So how could we get this started faster?

This is pure speculation at this point, but what if there was a central open site -- a sourceforge of sorts -- where anyone could create and publish a GRDDL transform for anything that has a URI. Let's call it GRDDLR (GRDDL + R for "Repository").

Open Source Projects for Extracting Data and Metadata from Files & the Web

I've been looking around for open-source libraries (preferably in Java, but not required) for extracting data and metadata from common file formats and Web formats. One project that looks very promising is Aperture. Do you know of any others that are ready or almost ready for prime-time use? Please let me know in the comments! Thanks.

Microsoft Astoria Not Supporting RDF

Microsoft's Astoria project has decided to make RDF a lower priority and is not supporting it for now. So much for Microsoft participating in the Semantic Web.

The Astoria project page describes the project thusly:

The goal of Microsoft Codename Astoria is to enable applications to expose data as a data service that can be consumed by web clients within a corporate network and across the internet. The data service is reachable over HTTP, and URIs are used to identify the various pieces of information available through the service. Interactions with the data service happens in terms of HTTP verbs such as GET, POST, PUT and DELETE, and the data exchanged in those interactions is represented in simple formats such as XML and JSON.

That sounds like a perfect description of a Web 3.0 service-- and a perfect use-case for RDF and SPARQL!  But the team at Astoria doesn't seem to feel anyone is actually using RDF enough to warrant supporting that open standard. Well, perhaps among Microsoft's customers that is true enough. But on the other hand, this is a missed opportunity to exercise forward-thinking leadership that will pay dividends to Microsoft in the future when millions of developers and applications will be using RDF as their common language. When is that future? Perhaps as soon as 2 - 3 years. Certainly not more than 5 years.

Welcome to the age of openness: The more open your platform and data, the more developers will want to use it. RDF is one of the ways to make data more open, and supporting it in a data-web platform makes good sense. But it's hard to teach a an old god new tricks!

DBpedia.org is Among the Coolest Semantic Web Datasets I've Seen

I've been poking around in the DBpedia, and I'm amazed at the progress. It is definitely one of the coolest (launched) example of the Semantic Web I've seen. It's going to be a truly useful resource to everyone. If you haven't heard of it yet, check it out!

Knowledge Networking

I've been thinking for several years about Knowledge Networking. It's not a term I invented, it's been floating around as a meme for at least a decade or two. But recently it has started to resurface in my own work.

So what is a knowledge network? I define a knowledge network as a form of collective intelligence in which a network of people (two or more people connected by social-communication relationships) creates, organizes, and uses a collective body of knowledge. The key here is that a knowledge network is not merely a site where a group of people work on a body of information together (such as the wikipedia), it's also a social network -- there is an explicit representation of a social relationship within it. So it's more like a social network than for example a discussion forum or a wiki.

I would go so far as to say that knowledge networks are the third-generation of social software. (Note this is based in-part on ideas that emerged in conversations I have had with Peter Rip, so this also his idea):

• First-generation social apps were about communication (eg. messaging such as Email, discussion boards, chat rooms, and IM)
• Second-generation social apps were about people and content (eg. Social networks, social media sharing, user-generated content)
• Third-generation social apps are about relationships and knowledge  (eg. Wikis, referral networks, question and answer systems, social recommendation systems, vertical knowledge and expertise portals, social mashup apps, and coming soon, what we're building at Radar Networks)

Just some thoughts on a Saturday morning...

What's cooking on GRDDL?

Peter Royal got me interested in GRDDL recently. It looks like something we may want to use here at Radar Networks for scraping data from the Web and turning it into RDF triples. After doing a little digging around lately, I'm wondering if there is a list of sites that support GRDDL yet? Or any apps that use GRDDL yet?

The Rise of the Social Operating System

In recent months we have witnessed a number of social networking sites begin to open up their platforms to outside developers. While this trend has been exhibited most prominently by Facebook, it is being embraced by all the leading social networking services, such as Plaxo, LinkedIn, Myspace and others. Along separate dimensions we also see a similar trend towards "platformization" in IM platforms such as Skype as well as B2B tools such as Salesforce.com.

If we zoom out and look at all this activity from a distance it appears that there is a race taking place to become "the social operating" system of the Web. A social operating system might be defined as a system that provides for systematic management and facilitation of human social relationships and interactions.

We might list some of the key capabilities of an ideal "social operating system" as:

• Identity management
  • Open portable identity
  • Personal profiles ("personas")
  • Privacy control
• Relationship management
  • Directory and lookup services (location of people to communicate with)
  • Social networking (opt-in relationship formation, indirect social connectivity via social networks)
  • Spam control
• Communication
  • Person to person communication
    • Synchronous (IM, VOIP)
    • Asynchronous (email, SMS)
  • Group communication
    • Synchronous (conferencing)
    • Asynchronous (group discussions)
• Social Content distribution
  
  • Personal publishing (blogging, home pages)
  • Public content distribution
• Social Coordination
  • Event management (scheduling, invitations, RSVP's)
  • Calendaring
• Social Collaboration
  • File sharing
  • Document collaboration (communal authoring/editing)
  • Collaborative filtering
  • Recommendation systems
  • Knowledge management
  • Human powered search
  • Project management
  • Workflow
• Commerce
  • Classified advertising
  • Auctions
  • Shopping

Today I have not seen any single player that provides a coherent solution to this entire "social stack" however Microsoft, Yahoo, and AOL are probably the strongest contenders. Can Facebook and other social networks truly compete or will they ultimately be absorbed into one of these larger players?

Radar Networks Progress Update

I'm sitting the Dynasty Lounge in Taipei, enroute to Singpore where I will be addressing ministers of the government there on the potential of the Semantic Web. Singapore is a very forward-looking country and they have some very exciting new initiatives in the works there. After that I hope to have a little time for a vacation and then I'm heading back to San Francisco, returning on August 1.

I should have email for all or most of the time here, so that is the best way to reach me directly. And of course you can comment on this blog too.

As for the company -- lots of good news here at Radar Networks.

First of all the team has gotten the next version of our alpha up (our hosted Web service for the Semantic Web) and it's getting awesome! We're on track for a invite only launch in the fall timeframe as planned.

We also chose a brand for our product, with help from the mad geniuses at Igor International. The new brand is secret until launch but we love it. We'll be announcing the brand close to launch.

If you want to be invited to our launch and be one of the first to see how useful the Semantic Web really can be -- sign up for our mailing list at http://www.radarnetworks.com/ -- and feel free to invite your friends to sign up too. Only people who sign up will get on our waiting list. We already have around 2000 bloggers and other influencers pre-registered, and more are coming every day, so don't wait -- it will be on a first-come, first-serve basis. We'll be letting people into the service in waves.

Another exciting development: Several of the world's big media empires have started approaching me to see how they can get involved in the network we are building here at Radar Networks. They are interested in the potential of the Semantic Web for adding new capabilities to their content and new services for their audiences. That's an exciting direction to explore for us. If you have large collections of interesting, useful, content of value to particular audiences, or if you have large audiences that need a better way to do stuff on the Web, feel free to drop me a line and we can discuss how you might be able to get involved with the Semantic Web in partnership with us.

In other news, I am still inundated with hundreds of emails from interesting people who read the articles about us in this month's Business 2.0 and BusinessWeek. It's been very interesting to connect with so many other thinkers and businesses. Forgive me in advance if takes me a while to write back -- I promise I will.

I can't wait to come back to San Francisco and start playing with our alpha -- it's really getting there. All the credit should go to our awesome development team. They've been writing tons of code and it's starting to really pay off.

Enriching the Connections of the Web -- Making the Web Smarter

Web 3.0 -- aka The Semantic Web -- is about enriching the connections of the Web. By enriching the connections within the Web, the entire Web may become smarter.

I  believe that collective intelligence primarily comes from connections -- this is certainly the case in the brain where the number of connections between neurons far outnumbers the number of neurons; certainly there is more "intelligence" encoded in the brain's connections than in the neurons alone. There are several kinds of connections on the Web:

1. Connections between information (such as links)
2. Connections between people (such as opt-in social relationships, buddy lists, etc.)
3. Connections between applications (web services, mashups, client server sessions, etc.)
4. Connections between information and people (personal data collections, blogs, social bookmarking, search results, etc.)
5. Connections between information and applications (databases and data sets stored or accessible by particular apps)
6. Connections between people and applications (user accounts, preferences, cookies, etc.)

Are there other kinds of connections that I haven't listed -- please let me know!

I believe that the Semantic Web can actually enrich all of these types of connections, adding more semantics not only to the things being connected (such as representations of information or people or apps) but also to the connections themselves.

In the Semantic Web approach, connections are represented with statements of the form (subject, predicate, object) where the elements have URIs that connect them to various ontologies where their precise intended meaning can be defined. These simple statements are sometimes called "triples" because they have three elements. In fact, many of us are working with statements that have more than three elements ("tuples"), so that we can represent not only subject, predicate, object of statements, but also things like provenance (where did the data for the statement come from?), timestamp (when was the statement made), and other attributes. There really is no limit to what kind of metadata can be stored in these statements. It's a very simple, yet very flexible and extensible data model that can represent any kind of data structure.

The important point for this article however is that in this data model rather than there being just a single type of connection (as is the case on the present Web which basically just provides the HREF hotlink, which simply means "A and B are linked" and may carry minimal metadata in some cases), the Semantic Web enables an infinite range of arbitrarily defined connections to be used.  The meaning of these connections can be very specific or very general.

For example one might define a type of connection called "friend of" or a type of connection called "employee of" -- these have very different meanings (different semantics) which can be made explicit and also machine-readable using OWL. By linking a page about a person with the "employee of" link to another page about a different person, we can express that one of them employs the other. That is a statement that any application which can read OWL is able to see and correctly interpret, by referencing the underlying definition of "employee of" which is defined in some ontology and might for example specify that an "employee of" relation connects a person to a person or organization who is their employer. In other words, rather than just linking things with the generic "hotlink" we are all used to, they can now be linked with specific kinds of links that have very particular and unambiguous meaning and logical implications.

This has the potential at least to dramatically enrich the information-carrying capacity of connections (links) on the Web. It means that connections can carry more meaning, on their own. It's a new place to put meaning in fact -- you can put meaning between things to express their relationships. And since connections (links) far outnumber objects (information, people or applications) on the Web, this means we can radically improve the semantics of the structure of the Web as a whole -- the Web can become more meaningful, literally. This makes a difference, even if all we do is just enrich connections between gross-level objects (in other words, connections between Web pages or data records, as opposed to connections between concepts expressed within them, such as for example, people and companies mentioned within a single document).

Even if the granularity of this improvement in connection technology is relatively gross level it could still be a major improvement to the Web. The long-term implications of this have hardly been imagined let alone understood -- it is analogous to upgrading the dendrites in the human brain; it could be a catalyst for new levels of computation and intelligence to emerge.

It is important to note that, as illustrated above, there are many types of connections that involve people. In other words the Semantic Web, and Web 3.0, are just as much about people as they are about other things. Rather than excluding people, they actually enrich their relationships to other things. The Semantic Web, should, among other things, enable dramatically better social networking and collaboration to take place on the Web. It is not only about enriching content.

Now where will all these rich semantic connections come from? That's the billion dollar question. Personally I think they will come from many places: from end-users as they find things, author content, bookmark content, share content and comment on content (just as hotlinks come from people today), as well as from applications which mine the Web and automatically create them. Note that even when Mining the Web a lot of the data actually still comes from people -- for example, mining the Wikipedia, or a social network yields lots of great data that was ultimately extracted from user-contributions. So mining and artificial intelligence does not always imply "replacing people" -- far from it! In fact, mining is often best applied as a means to effectively leverage the collective intelligence of millions of people.

These are subtle points that are very hard for non-specialists to see -- without actually working with the underlying technologies such as RDF and OWL they are basically impossible to see right now. But soon there will be a range of Semantically-powered end-user-facing apps that will demonstrate this quite obviously. Stay tuned!

Of course these are just my opinions from years of hands-on experience with this stuff, but you are free to disagree or add to what I'm saying. I think there is something big happening though. Upgrading the connections of the Web is bound to have a significant effect on how the Web functions. It may take a while for all this to unfold however. I think we need to think in decades about big changes of this nature.

Web 3.0 -- Next-Step for Web?

The Business 2.0 Article on Radar Networks and the Semantic Web just came online. It's a huge article. In many ways it's one of the best popular articles written about the Semantic Web in the mainstream press. It also goes into a lot of detail about what Radar Networks is working on.

One point of clarification, just in case anyone is wondering...

Web 3.0 is not just about machines -- it's actually all about humans -- it leverages social networks, folksonomies, communities and social filtering AS WELL AS the Semantic Web, data mining, and artificial intelligence. The combination of the two is more powerful than either one on it's own. Web 3.0 is Web 2.0 + 1. It's NOT Web 2.0 - people. The "+ 1" is the addition of software and metadata that help people and other applications organize and make better sense of the Web. That new layer of semantics -- often called "The Semantic Web" -- will add to and build on the existing value provided by social networks, folksonomies, and collaborative filtering that are already on the Web.

So at least here at Radar Networks, we are focusing much of our effort on facilitating people to help them help themselves, and to help each other, make sense of the Web. We leverage the amazing intelligence of the human brain, and we augment that using the Semantic Web, data mining, and artificial intelligence. We really believe that the next generation of collective intelligence is about creating systems of experts not expert systems.

Business 2.0 and BusinessWeek Articles About Radar Networks

It's been an interesting month for news about Radar Networks. Two significant articles came out recently:

Business 2.0 Magazine published a feature article about Radar Networks in their July 2007 issue. This article is perhaps the most comprehensive article to-date about what we are working on at Radar Networks, it's also one of the better articulations of the value proposition of the Semantic Web in general. It's a fun read, with gorgeous illustrations, and I highly recommend reading it.

BusinessWeek  posted an article about Radar Networks on the Web. The article covers some of the background that led to my interests in collective intelligence and the creation of the company. It's a good article and covers some of the bigger issues related to the Semantic Web as a paradigm shift. I would add one or two points of clarification in addition to what was stated in the article: Radar Networks is not relying solely on software to organize the Internet -- in fact, the service we will be launching combines human intelligence and machine intelligence to start making sense of information, and helping people search and collaborate around interests more productively. One other minor point related to the article -- it mentions the story of EarthWeb, the Internet company that I co-founded in the early 1990's: EarthWeb's content business actually was sold after the bubble burst, and the remaining lines of business were taken private under the name Dice.com. Dice is the leading job board for techies and was one of our properties. Dice has been highly profitable all along and recently filed for a $100M IPO.

Jim Wissner's Latest Semantic Web Creation Rocks

I met today with Jim Wissner, Chief Architect and co-founder of my company, Radar Networks. Jim started the company along with me and Kris Thorisson several years ago when we were just a few guys doing futuristic R&D. Jim has been working for a few months on a "secret project" in his spare time, and today I finally got to see it.

I have been hearing rumors of Jim's secret project from some of the other guys on the team for weeks -- but whenever I asked Jim about it he would just get a gleam in his eye and say something like, "Wellllll yeah, I've been thinking about something interesting... but there's nothing to show yet..." Anyway, I've known Jim long enough to know that whenever he's coy about some "something interesting" it is probably going to be something pretty damn impressive. So I didn't bug him too much -- I just figured when it was time he would show me.

Today Jim started out by saying he hoped he hadn't set my expectations too high. So I automatically assumed it was just going to be some little widget. But as soon as I saw it I realized that he has built something really MAJOR. In fact, I'm kind of in awe that one developer -- even a developer as good as Jim -- could have done it all himself in just a few months of hacking. He really outdid himself this time.

Jim has built something that I think could become a cornerstone of the coming Semantic Web infrastructure. It's something that he has been dreaming about for a couple of years -- but I never imagined that he would actually find time to build it. As Jim put it today, "Well I finally just decided, since I've been talking about this for years, I should just go ahead and build it. It's a lot easier to understand when you see it working."

So what has Jim built? I can't say what it is yet, but suffice to say it's something that I think every developer involved in the semantic web space is going to benefit from. It's definitely more of a developer oriented thing -- but something that could catalyze a lot of new innovation, growth and collaboration.

After Jim's demo I met with Chris and Lew (who were equally impressed and had seen it earlier during technical discussions with Jim) and we began to discuss the timeline for releasing what Jim has made. We all think it will be a big piece of what we roll out when we go beta. So those of you who are curious -- you won't have to wait that long (but be sure to sign up for our mailing list on the Radar Networks site so you get notified of the closed beta).

I haven't blogged much about Jim's contribution to the company yet, but for those who don't know, he singlehandedly built several  iterations of our platform. He is a huge part of the DNA of our company but I haven't acknowledged his contributions in public enough. Partly that's because he's our secret weapon, and partly that's because he tends to prefer to avoid any sort of hype -- so he's been patiently waiting until we launch before speaking publicly about what he's built.

Jim is a true semantic web "guru" -- as well as inventing our platform, he was also the chief architect for our work on the DARPA CALO program. Jim is a big piece of what Radar Networks is, and in the future, as we begin to roll out our platform I think people in the Semantic Web development community are going to be extremely interested in what he has built.

As Jim would be the first to point out: we've got a lot of other great talent in our company and they have all made enormous contributions to various aspects of what we're doing. It's not just him. To do something as broad as what we're building requires a great engineering team, and several great architects, not just one person. That's true. But it's also fair to say that Jim started our engineering team and has been quietly laying the foundations for several years and without his leadership it would never have happened.

So this post is a special thank you to Jim. We couldn't have done what we're doing without him and I feel extremely grateful that he joined me in co-founding this back in our New York days before we moved to San Francisco. This post is in recognition of the truly impressive work Jim's been doing.

As we begin to roll out our platform, Jim (and our other developers) will begin to blog and speak more publicly about our platform. I do think other folks working in the semantic web area, or writing about it, will find what they have built to be quite interesting, comprehensive, and useful.

OK well I've said enough. Jim is probably going to be somewhat horrified by such high praise. But it's well deserved and long overdue.

Danny's Tips for Web 3.0 Builders

Danny Ayers has posted some good general guidelines for Web 3.0 system builders. I've commented on the thread on his post, so I won't add too much more here. But it's a good set of guidelines for the best-practices of building sites and services for Web 3.0 -- the Data-Web.

Good Post: Semantic Web & World's Collective Knowledge

I really liked this post. A brief explanation of the value of the Semantic Web for organizing the world's collective knowledge.

Next SF Web Innovators Event - This Thursday 4/26 in SF

Please join me at the next SF Web Innovators monthly meetup, this Thursday evening in San Francisco.

CLICK HERE TO RSVP

April Event 2007

                                                                                   

THURSDAY April 26th AT 6 PM – ORRICK

405 Howard Street - the Orrick Building

Here's some photos from the last event.

For a year and a half, SFWIN has been the event for professionals within the Web 2.0 space to meet and greet each other in a friendly and relaxing atmosphere. We've had companies founded over drinks and deals brokered over the finest mini tacos/pigs-in-the-blanket on the event scene. Unlike some of the more "party" events in the area, SFWIN is a great place to relax and discover new business opportunities and emerging technologies. And did I mention all the free food and booze?

Scoble Gets the Semantic Web

Robert Scoble spent 2 hours with us looking at our app yesterday. We had a great conversation and he had many terrific ideas and suggestions for us. We are still in stealth, so we asked him to agree not say much about what we showed him yet. He blogged a very nice post about us today, providing a few hints.

Listen to this Discussion on the Future of the Web

If you are interested in the future of the Web, you might enjoy listening to this interview with me, moderated by Dr. Paul Miller of Talis. We discuss, in-depth: the Semantic Web, Web 3.0, SPARQL, collective intelligence, knowledge management, the future of search, triplestores, and Radar Networks.

A Bunch of New Press About Radar Networks

We had a bunch of press hits today for my startup, Radar Networks...

PC World  Article on  Web 3.0 and Radar Networks

Entrepreneur Magazine interview

We're also proud to announce that Jim Hendler, one of the founding gurus of the Semantic Web, has joined our technical advisory board.

Come to SFWIN Thursday 7 PM in SF!!!

Hey everyone, it's that time of the month again -- yes, you guessed it, the monthly SFWIN meetup (San Francisco Web Innovators Network). Come join our high-quality network of entrepreneurs, venture capitalists, techies, and vendors for an evening of conversation, food and drinks in San Francisco (the Orrick Building). It's close to transit and there's plenty of parking nearby. I hope to see you there. Please RSVP at the link. Also -- invite your friends in the industry! This event is different -- no hype, no fluff -- real networking.

(PLEASE Note: we are going to start the event at 7 pm instead of 6 pm this month due to some building maintenance).

Excellent Overview of Benefits of RDF and SPARQL

This article on XML.com is a very good summary of the benefits of RDF and SPARQL -- two of the key technologies of the emerging Semantic Web.

Radar Networks in the Chronicle

The SF Chronicle just did an interview with Chris Jones, of my startup, Radar Networks. Here's the link.

Radar Networks Profiled in Technology Review

The MIT Technology Review just published a large article on the Semantic Web and Web 3.0, in which Radar Networks, Metaweb, Joost, RealTravel and other ventures are profiled.

Metaweb and Radar Networks

This is just a brief post because I am actually slammed with VC meetings right now. But I wanted to congratulate our friends at Metaweb for their pre-launch announcement. My company, Radar Networks, is the only other major venture-funded play working on the Semantic Web for consumers so we are thrilled to see more action in this sector.

Metaweb and Radar Networks are working on two very different applications (fortunately!). Metaweb is essentially making the Wikipedia of the Semantic Web. Here at Radar Networks we are making something else -- but equally big -- and in a different category. Just as Metaweb is making a semantic analogue to something that exists and is big, so are we: but we're more focused on the social web -- we're building something that everyone will use. But we are still in stealth so that's all I can say for now.

This is now an exciting two-horse space. We look forward to others joining the excitement too. Web 3.0 is really taking off this year.

An interesting side note: Danny Hillis (founder of Metaweb), myself (founder of Radar Networks) and Lew Tucker (CTO of Radar Networks) all worked together at Thinking Machines (an early AI massively parallel computer company). It's fascinating that we've all somehow come to think that the only practical way to move machine intelligence forward is by having us humans and applications start to employ real semantics in what we record in the digital world.

Is it Only Wednesday?

Is it only Wednesday? It feels like a whole week already! I've been in back-to-back VC meetings, board discussions and strategy meetings since last week. I think this must be related to the heating-up of the "Web 3.0" meme and the semantic sector in general. Perhaps it is also due to the coverage we got in the Guidewire Report and newsletter which went out to everyone who went to DEMO, and also perhaps because of some influential people in the biz have been talking about us. We've been very careful not to show our app to anyone because it does some things that are really new. We don't want to spread that around (yet). Anyway it's been pretty busy -- not just for me, but for the whole team. Everyone is on full afterburners right now.

By the way -- I'm really proud or product team (hope you guys are reading this)-- the team has made an alpha that is not only a breakthrough on the technical level, but it also looks incredibly good too. Some of the select few who have seen our app so far have said, "the app looks beautiful" and "wow, that's amazing" etc. We've done some cool things with NLP, graph analysis, and statistics under the hood. And the GUI is also very slick. Probably the best team I've worked with.

If you are interested in helping to beta-test the consumer Semantic Web, We're planning on doing invite-only beta trials this summer -- sign up at our website to be on our beta invite list.

Breaking the Collective IQ Barrier -- Making Groups Smarter

I've been thinking since 1994 about how to get past a fundamental barrier to human social progress, which I call "The Collective IQ Barrier." Most recently I have been approaching this challenge in the products we are developing at my stealth venture, Radar Networks.

In a nutshell, here is how I define this barrier:

The Collective IQ Barrier: The potential collective intelligence of a human group is exponentially proportional to group size, however in practice the actual collective intelligence that is achieved by a group is inversely proportional to group size. There is a huge delta between potential collective intelligence and actual collective intelligence in practice. In other words, when it comes to collective intelligence, the whole has the potential to be smarter than the sum of its parts, but in practice it is usually dumber.

Why does this barrier exist? Why are groups generally so bad at tapping the full potential of their collective intelligence? Why is it that smaller groups are so much better than large groups at innovation, decision-making, learning, problem solving, implementing solutions, and harnessing collective knowledge and intelligence? 

I think the problem is technological, not social, at its core. In this article I will discuss the problem in more depth and then I will discuss why I think the Semantic Web may be the critical enabling technology for breaking through the Collective IQ Barrier.

Doing for Data What HTML Did for Documents

I've been saying for quite some time that the first thing the Semantic Web will accomplish is "doing for data what HTML did for documents." HTML made it possible for everyone to publish, access and connect documents on the Internet. RDF and OWL -- the languages of the Semantic Web -- make it possible to do the same thing for data records -- structured fragments or collections of data. Today data records exist in a variety of non-integrated formats and standards. RDF and OWL provide a common language for publishing, accessing and linking data around the Web.

Tim Berners-Lee recently gave Congressional testimony on the future of the Web where he explained this as well:

“Digital information about nearly every aspect of our lives is being created at an astonishing rate. Locked within all of this data is the key to knowledge about how to cure diseases, create business value, and govern our world more effectively. The good news is that a number of technical innovations (RDF which is to data what HTML is to documents, and the Web Ontology Language (OWL) which allows us to express how data sources connect together) along with more openness in information sharing practices are moving the World Wide Web toward what we call the Semantic Web. … The Semantic Web will enable better data integration by allowing everyone who puts individual items of data on the Web to link them with other pieces of data using standard formats.”

This first stage of evolution of the Semantic Web can be called the "data web" - a Web of connected data. Once this is in place and lots of RDF and OWL data records exist, and can be queried via SPARQL interfaces by any application, the Web will start to actually function like a giant globally distributed database.  This in turn will enable a new era of Web applications that understand the structure and interconnections of data around the Web. For example a search engine that could search for different kinds of product listings across many different sites that publish RDF/OWL data for their product records.

Today such a data-aware search engine is hard to make -- each new site that is searched has to be integrated with in a completely custom way. This is sometimes referred to as the challenge of doing "deep Web search." To solve this problem, if product listings of various types were at least mapped to a common ontology (or even a set of ontologies) this would make integration of disparate data types about products easier. By making it easier to operate across data around the Web, data-aware applications can  be made to do more sophisticated things with less effort, and they in turn will produce more data for the data-web in a virtuous-cycle. That is the first step. After this we can then begin to make smart applications that help to filter, make sense of, track, suggest, organize, data and assist users. First we have to have rich data -- the data-web -- then we can move to smart applications from there.

Diagram: Beyond Keyword (and Natural Language) Search

Here at Radar Networks we are working on practical ways to bring the Semantic Web to end-users. One of the interesting themes that has come up a lot, both internally, as well as in discussions with VC's, is the coming plateau in the productivity of keyword search. As the Web gets increasingly large and complex, keyword search becomes less effective as a means for making sense of it. In fact, it will even decline in productivity in the future. Natural language search will be a bit better than keyword search, but ultimately won't solve the problem either -- because like keyword search it cannot really see or make use of the structure of information.

I've put together a new diagram showing how the Semantic Web will enable the next step-function in productivity on the Web. It's still a work in progress and may change frequently for a bit, so if you want to blog it, please link to this post, or at least the .JPG image behind the thumbnail below so that people get the latest image. As always your comments are appreciated. (Click the thumbnail below for a larger version).

Today a typical Google search returns up to hundreds of thousands or even millions of results -- but we only really look at the first page or two of results. What about the other results we don't look at? There is a lot of room to improve the productivity of search, and the help people deal with increasingly large collections of information.

Keyword search doesn't understand the meaning of information, let alone its structure. Natural language search is a little better at understanding the meaning of information -- but it still won't help with the structure of information. To really improve productivity significantly as the Web scales, we will need forms of search that are data-structure-aware -- that are able to search within and across data structures, not just unstructured text or semistructured HTML. This is one of the key benefits of the coming Semantic Web: it will enable the Web to be navigated and searched just like a database.

Starting with the "data web" enabled by RDF, OWL, ontologies and SPARQL, structured data is becoming increasingly accessible, searchable and mashable. This in turn sets the stage for a better form of search: semantic search. Semantic search combines the best of keyword, natural language, database and associative search capabilities together.

Without the Semantic Web, productivity will plateau and then gradually decline as the Web, desktop and enterprise continue to grow in size and complexity. I believe that with the appropriate combination of technology and user-experience we can flip this around so that productivity actually increases as the size and complexity of the Web increase.

See Also: A Visual Timeline of the Past, Present and Future of the Web

Get the Wikipedia to Make a Web 3.0 Page

Editor's NOTE, February 27, 2007: I guess this post must have helped. The Wikipedia has reinstated a page about Web 3.0 and hopefully it will improve over time. Thanks if you helped argue the case.

Someone at Wikipedia has repeatedly deleted the Web 3.0 page and locked it -- they've done this despite the fact that there are 905,000 results in Google for "Web 3.0" and dozens of major newspaper articles on the subject all around the world, not to mention (hundreds or thousands?) of blog posts listed in Technorati on the term. Whoever is doing this claims that Web 3.0 is not defined and/or is synonymous with "Semantic Web" for which there is already a node in the Wikipedia. I disagree on several counts:

1. Web 3.0 is defined in many places. My definition may be the most practical and objective proposal: define "Web 3.0" is as the third-decade (ie. third-generation) of the Web (2010 - 2020) -- and likewise Web 2.0 would be the second decade, etc. This is more rational and objective even than the current way of defining Web 2.0. I've defined this more extensively here.
2. Web 2.0 is not a more "defined term" in fact, there are many more differing definitions of that term than Web 3.0 in fact, yet there IS a node in Wikipedia for that term!.
3. Web 3.0 is broader than just the Semantic Web (although the Semantic Web is one key enabler of it) -- thus it DOES deserve it's own Wikipedia node apart from the Semantic Web node; It is not equivalent to Semantic Web. I think Web 3.0 is an upgrade of the backend of the Web and several technologies will play a part in that.

If you are working on the Semantic Web or other emerging Web content and data technologies that go beyond Web 2.0 AJAX front-end innovation, then please post into this discussion to express why this term should exist in Wikipedia.

We are on the cusp of Web 3.0 -- the third-generation Web -- and many companies such as my own Radar Networks, are working on key pieces of the puzzle. If you or your organization are working in this area, please help by chiming in on this wikipedia node so we can overcome the very unfortunate bias that is obviously being imposed there presently.

Envisioning the Whole Digital Person

Another article of note on the subject of our evolving digital lives and what user-experience designers should be thinking about:

Our lives are becoming increasingly digitized—from the ways we communicate, to our entertainment media, to our e-commerce transactions, to our online research. As storage becomes cheaper and data pipes become faster, we are doing more and more online—and in the process, saving a record of our digital lives, whether we like it or not.

(snip...)

In the coming years, our ability to interact with the information we’re so rapidly generating will determine how successfully we can manage our digital lives. There is a great challenge at our doorsteps—a shift in the way we live with each other.

As designers of user experiences for digital products and services, we can make people’s digital lives more meaningful and less confusing. It is our responsibility to envision not only techniques for sorting, ordering, and navigating these digital information spaces, but also to devise methods of helping people feel comfortable with such interactions. To better understand and ultimately solve this information management problem, we should take a holistic view of the digital person. While our data might be scattered, people need to feel whole.

Capturing Your Digital Life

Nice article in Scientific American about Gordon Bell's work at Microsoft Research on the MyLifeBits project. MyLifeBits provides one perspective on the not-too-far-off future in which all our information, and even some of our memories and experiences, are recorded and made available to us (and possibly to others) for posterity. This is a good application of the Semantic Web -- additional semantics within the dataset would provide many more dimensions to visualize, explore and search within, which would help to make the content more accessible and grokkable.

Intelligence is in the Connections

Google's Larry Page recently gave a talk to the AAAS about how Google is looking towards a future in which they hope to implement AI on a massive scale. Larry's idea is that intelligence is a function of massive computation, not of "fancy whiteboard algorithms." In other words, in his conception the brain doesn't do anything very sophisticated, it just does a lot of massively parallel number crunching. Each processor and its program is relatively "dumb" but from the combined power of all of them working together "intelligent" behaviors emerge.

Larry's view is, in my opinion, an oversimplification that will not lead to actual AI. It's certainly correct that some activities that we call "intelligent" can be reduced to massively parallel simple array operations. Neural networks have shown that this is possible -- they excel at low level tasks like pattern learning and pattern recognition for example. But neural networks have not proved capable of higher level cognitive tasks like mathematical logic, planning, or reasoning. Neural nets are theoretically computationally equivalent to Turing Machines, but nobody (to my knowledge) has ever succeeded in building a neural net that can in practice even do what a typical PC can do today -- which is still a long way short of true AI!

Somehow our brains are capable of basic computation, pattern detection and learning, simple reasoning, and advanced cognitive processes like innovation and creativity, and more. I don't think that this richness is reducible to massively parallel supercomputing, or even a vast neural net architecture. The software -- the higher level cognitive algorithms and heuristics that the brain "runs" -- also matter. Some of these may be hard-coded into the brain itself, while others may evolve by trial-and-error, or be programmed or taught to it socially through the process of education (which takes many years at the least).

Larry's view is attractive but decades of neuroscience and cognitive science have shown conclusively that the brain is not nearly as simple as we would like it to be. In fact the human brain is far more sophisticated than any computer we know of today, even though we can think of it in simple terms. It's a highly sophisticated system comprised of simple parts -- and actually, the jury is still out on exactly how simple the parts really are -- much of the computation in the brain may be sub-neuronal, meaning that the brain may actually a much much more complex system than we think.

Perhaps the Web as a whole is the closest analogue we have today for the brain -- with millions of nodes and connections. But today the Web is still quite a bit smaller and simpler than a human brain. The brain is also highly decentralized and it is doubtful than any centralized service could truly match its capabilities. We're not talking about a few hundred thousand linux boxes -- we're talking about hundreds of billions of parallel distributed computing elements to model all the neurons in a brain, and this number gets into the trillions if we want to model all the connections. The Web is not this big, and neither is Google.

AOL Adds 63 Million Users to OpenID ... Digg Joins Too

The folks at Read/Write Web have a great analysis of AOL's recent move to support the OpenID initiative. AOL has integrated with OpenID, adding 63 million users in a bid to make AIM handles sticky.

Digg also just announced they are joining OpenID.

OpenID is one of the technologies that I am tracking as a key enabler of Web 3.0.

'Bemes' are Defining the Blogosphere

Tom Hayes has an interesting post in which he coins the word 'beme" to mean a meme that spreads in the blogosphere.

Michael Malone's ABC News column on Thursday mentioning "bemes" has certainly produced a lot of interest.  Originally, I coined the word beme to describe a meme propagated by blogs and bloggers.  Now I can see that the turn of phrase has a much bigger potential to capture the rapidly-moving cultural touchstones of the Bubble Generation.

As you may know, "meme" was first defined by Richard Dawkins in 1976 as "a unit of cultural information" spread from one mind to another. In other words, a viral idea that eventually becomes common knowledge.

Fast forward three decades, and it seems to me that technology has turbo-charged the meme process.  Looking for the juste mot to describe a "purposeful" meme fed into the vast human network of the Internet, either by blog, email, video, phonecast, social media or other viral means, beme seems to fit the bill. 

A beme is a turbo-charged meme made possible entirely by the existence of the network affect.  A beme can be impactful because it is lurid--a photo of a panty-less Britney Spears, or humorous--a whimisical video of the band OKGO on treadmills, or gut-wrenching--the sad tirade by comedian Michael Richards.  A beme can cement an idea with the public in a way that cannot be legislated or regulated.  No legal effort by Cisco to enforce a trademark, for example, will make the public unlearn that Apple produces the iPhone.

• A meme is old media, a beme is new media.
• A meme takes off by accident, a beme by design.
• A meme can take years to surface, a beme hours.

Web 3.0 Roundup: Radar Networks, Powerset, Metaweb and Others...

It's been a while since I posted about what my stealth venture, Radar Networks, is working on. Lately I've been seeing growing buzz in the industry around the "semantics" meme -- for example at the recent DEMO conference, several companies used the word "semantics" in their pitches. And of course there have been some fundings in this area in the last year, including Radar Networks and other companies.

Clearly the "semantic" sector is starting to heat up. As a result, I've been getting a lot of questions from reporters and VC's about how what we are doing compares to other companies such as for example, Powerset, Textdigger, and Metaweb. There was even a rumor that we had already closed our series B round! (That rumor is not true; in fact the round hasn't started yet, although I am getting very strong VC interest and we will start the round pretty soon).

In light of all this I thought it might be helpful to clarify what we are doing, how we understand what other leading players in this space are doing, and how we look at this sector.

Indexing the Decades of the Web

First of all, before we get started, there is one thing to clear up. The Semantic Web is part of what is being called "Web 3.0" by some, but it is in my opinion really just one of several converging technologies and trends that will define this coming era of the Web. I've written here about a proposed definition of Web 3.0, in more detail.

For those of you who don't like terms like Web 2.0, and Web 3.0, I also want to mention that  I agree --- we all want to avoid a rapid series of such labels or an arms-race of companies claiming to be > x.0. So I have a practical proposal: Let's use these terms to index decades since the Web began. This is objective -- we can all agree on when decades begin and end, and if we look at history each decade is characterized by various trends. 

I think this is reasonable proposal and actually useful (and also avoids endless new x.0's being announced every year). Web 1.0 was therefore the first decade of the Web: 1990 - 2000. Web 2.0 is the second decade, 2000 - 2010. Web 3.0 is the coming third decade, 2010 - 2020 and so on. Each of these decades is (or will be) characterized by particular technology movements, themes and trends, and these indices, 1.0, 2.0, etc. are just a convenient way of referencing them. This is a useful way to discuss history, and it's not without precedent. For example, various dynasties and historical periods are also given names and this provides shorthand way of referring to those periods and their unique flavors. To see my timeline of these decades, click here.

So with that said, what is Radar Networks actually working on? First of all, Radar Networks is still in stealth, although we are planning to go beta in 2007. Until we get closer to launch what I can say without an NDA is still limited. But at least I can give some helpful hints for those who are interested. This article provides some hints, as well as what I hope is a helpful tutorial about natural language search and the Semantic Web, and how they differ. I'll also discuss how Radar Networks compares some of the key startup ventures working with semantics in various ways today (there are many other companies in this sector -- if you know of any interesting ones, please let me know in the comments; I'm starting to compile a list).

 

(click the link below to keep reading the rest of this article...)

Wow! Watch this Multi-Touch UI Demo Video

If you are interested on what computer user-interfaces are going to feel like in the future -- you must see this video of a demo of a new multi-touch computer monitor. This is amazing technology -- and the various demos themselves are interactive artworks in their own right. For more information about the researchers and projects behind this, click here. I want one of these NOW!

How the WebOS Evolves?

Here is my timeline of the past, present and future of the Web. Feel free to put this meme on your own site, but please link back to the master image at this site (the URL that the thumbnail below points to) because I'll be updating the image from time to time.

This slide illustrates my current thinking here at Radar Networks about where the Web (and we) are heading. It shows a timeline of technology leading from the prehistoric desktop era to the possible future of the WebOS...

Note that as well as mapping a possible future of the Web, here I am also proposing that the Web x.0 terminology be used to index the decades of the Web since 1990. Thus we are now in the tail end of Web 2.0 and are starting to lay the groundwork for Web 3.0, which fully arrives in 2010.

This makes sense to me. Web 2.0 was really about upgrading the "front-end" and user-experience of the Web. Much of the innovation taking place today is about starting to upgrade the "backend" of the Web and I think that will be the focus of Web 3.0 (the front-end will probably not be that different from Web 2.0, but the underlying technologies will advance significantly enabling new capabilities and features).

See also: This article I wrote redefining what the term "Web 3.0" means.

See also: A Visual Graph of the Future of Productivity

Please note: This is a work in progress and is not perfect yet. I've been tweaking the positions to get the technologies and dates right. Part of the challenge is fitting the text into the available spaces. If anyone out there has suggestions regarding where I've placed things on the timeline, or if I've left anything out that should be there, please let me know in the comments on this post and I'll try to readjust and update the image from time to time. If you would like to produce a better version of this image, please do so and send it to me for inclusion here, with the same Creative Commons license, ideally.

Congratulations to Adam Cheyer and Change.org

My friend and colleague, Adam Cheyer, from SRI, recently helped to advise and launch Change.org -- a social network for nonprofit activism. It's a great site for a great cause. It's also a great example of a special-purpose social network -- a useful social network. Congratulations to Adam and the team over there! Read the TechCrunch review here.

I'm at DEMO

Hey I'm at DEMO this week. If you're here too and you want to chat, drop me a line.

Radar Networks News -- We've Moved into a Bigger Office!

My startup company, Radar Networks, has been busy working in stealth on the next-generation of the Web (sometimes called "Web 3.0"). We've been quietly growing and putting the pieces in place all year. Today we are celebrating the opening of our new, much larger office at 410 Townsend in the SOMA neighborhood of San Francisco. It's a gorgeous (and huge) space with room to grow the company for several years. We're excited to be here. This marks a key milestone in the growth of the venture. 2007 is going to be a big year for us.

Very Cool Desktop Interface Prototype Video -- Bumptop

Check out this very impressive user-interface prototype for a desktop that works more like a real desk -- a messy desk in fact. Very delightful design work that makes want to use it now!

SFWIN This Thursday -- See you There!

Hey everyone, please RSVP if you are planning to come to the first SFWIN event of the year, in San Francisco -- this Thursday, January 18. SFWIN is a gathering of technologists. There are about 200 people in the network and about 50 to 80 usually show up at each event. Please help get the word out by inviting your friends in the industry as well.

Web 3.0 Redefined

KurzweilAI.net has published an article I wrote redefining the meaning of Web 3.0. Basically, I am proposing that Web 3.0 include a set of emerging technologies that are all reaching new levels of maturity at the same time.

Minding the Planet: From Semantic Web to Global Mind

Draft 1.1 for Review (integrates some fixes from readers)
Nova Spivack (www.mindingtheplanet.net)

INTRODUCTION

This article presents some thoughts about the future of intelligence on Earth. In particular, I discuss the similarities between the Internet and the brain, and how I believe the emerging Semantic Web will make this similarity even greater.

DISTRIBUTED INTELLIGENCE

The Semantic Web enables the formal communication of a higher level of language -- metalanguage. Metalanguage is language about language -- language that encodes knowledge about how to interpret and use information. Metalanguages – particularly semantic metalanguages for encoding relationships between information and systems of concepts – enable a new layer of communication and processing. The combination of computing networks with semantic metalanguages represents a major leap in the history of communication and intelligence.

The invention of written language long ago changed the economics of communication by making it possible for information to be represented and shared independently of human minds. This made it less costly to develop and spread ideas widely across populations in space and time. Similarly, the emergence of software based on semantic metalanguages will dramatically change the economics not only of information distribution, but of intelligence -- the act of processing and using information.

Semantic metalanguages provide a way to formally express, distribute and share the knowledge necessary to interpret and use information, independently of the human mind. In other words, they make it possible not just to write down and share information, but also to encode and share the background necessary for intelligently making use of that information. Prior to the invention of such a means to share this background knowledge about information, although information could be written and shared, the recipients of such information had to be intelligent and appropriately knowledgeable in advance in order to understand it. Semantic metalanguages remove this restriction by making it possible to distill the knowledge necessary to understand information into a form that can be shared just as easily as the information itself.

The recipients of information – whether humans or software – no longer have to know in advance (or attempt to deduce) how to interpret and use the information; this knowledge is explicitly coded in the metalanguage about the information. This is important for artificial intelligence because it means that expertise for specific domains does not have to be hard-coded into programs anymore -- instead programs simply need to know how to interpret the metalanguage. By adding semantic metalanguage statements to information data becomes “smarter,” and programs can therefore become “thinner.” Once programs can speak this metalanguage they can easily import and use knowledge about any particular domain, if and when needed, so long as that knowledge is expressed in the metalanguage.

In other words, whereas basic written languages simply make raw information portable, semantic metalanguages make knowledge (conceptual systems) and even intelligence (procedures for processing knowledge) about information portable. They make it possible for knowledge and intelligence to be formally expressed, stored digitally, and shared independently of any particular minds or programs. This radically changes the economics of communicating knowledge and of accessing and training intelligence. It makes it possible for intelligence to be more quickly, easily and broadly distributed across time, space and populations of not only humans but also of software programs.

The emergence of standards for sharing semantic metalanguage statements that encode the meaning of information will catalyze a new era of distributed knowledge and intelligence on the Internet. This will effectively “make the Internet smarter.” Not just monolithic expert systems and complex neural networks, but even simple desktop programs and online software agents will begin to have access to a vast decentralized reserve of knowledge and intelligence.

The externalization, standardization and sharing of knowledge and intelligence in this manner, will make it possible for communities of humans and software agents to collaborate on cognition, not just on information. As this happens and becomes increasingly linked into our daily lives and tools, the "network effect" will deliver increasing returns. While today most of the intelligence on Earth still resides within human brains, In the near future, perhaps even within our lifetimes, the vast majority of intelligence will exist outside of human brains on the Semantic Web.

THE INTERNET IS A BRAIN AND THE WEB IS ITS MIND

Anyone familiar with the architecture and dynamics of the human nervous system cannot help but notice the striking similarity between the brain and the Internet. But is this similarity more than a coincidence - is the Internet really a brain in its own right - the brain of our planet? And is its collective behavior intelligent - does it constitute a global mind? How might this collective form of intelligence compare to that of an individual human mind, or a group of human minds?

I believe that the Internet (the hardware) is already evolving into a distributed global brain, and its ongoing activity (the software, humans and data) represents the cognitive process of an increasingly intelligent global mind. This global mind is not centrally organized or controlled, rather it is a bottom-up, emergent, self-organizing phenomenon formed from flows of trillions of information-processing events comprised of billions of independent information processors.

As with other types of emergent computing systems, for example John Conway’s familiar cellular automaton “The Game of Life,” on the Internet large scale homeostatic systems and seemingly intentional or guided information processes naturally emerge and interact within it. The emergence of sophisticated information systems does not require top-down design or control, it can happen in an evolutionary bottom-up manner as well.

Like a human brain, the Internet is a vast distributed computing network comprised of billions of interacting parallel processors. These processors include individual human beings as well as software programs, and systems of them such as organizations, which can all be referred to as "agents" in this system. Just as the computational power of the human brain as a whole is vastly greater than that of any of the individual neurons or systems within it, the computational power of the Internet is vastly beyond any of the individual agents it contains. Just as the human brain is not merely the sum of its parts, the Internet is more than the sum of its parts - like other types of distributed emergent computing systems, it benefits from the network effect. The power of the system grows exponentially as agents and connections between them are added.

The human brain is enabled by an infrastructure comprised of networks of organic neurons, dendrites, synapses and protocols for processing chemical and electrical messages. The Internet is enabled by an infrastructure of synthetic computers, communications networks, interfaces, and protocols for processing digital information structures. The Internet also interfaces with organic components however – the human beings who are connected to it. In that sense the Internet is not merely an inorganic system – it could not function without help from humans, for the moment at least. The Internet may not be organized in exactly the same form as the human brain, but it is at least safe to say it is an extension of it.

The brain provides a memory system for storing, locating and recalling information. The Internet also provides shared address spaces and protocols for using them. This enables agents to participate in collaborative cognition in a completely decentralized manner. It also provides a standardized shared environment in which information may be stored, addressed and retrieved by any agent of the system. This shared information space functions as the collective memory of the global mind.

Just as no individual neuron in the human brain could be said to have the same form or degree of intelligence as the brain as-a-whole - we individual humans cannot possibly comprehend the distributed intelligence that is evolving on the Internet. But we are part of it nonetheless, whether we know it or not.  The global mind is emerging all around us, and via us, is our creation but it is already becoming independent of us - truly it represents the evolution of a new form of meta-level intelligence that has never before existed on our planet.

Although we created it, the Internet is already far beyond our control or comprehension - it surrounds us and penetrates our world - it is inside our buildings, our tools, our vehicles, and it connects us together and modulates our interactions. As this process continues and the human body and biology begins to be networked into this system we will literally become part of this network - it will become an extension of our nervous systems and eventually, via brain-computer interfaces, it will be an extension of our senses and our minds. Eventually the distinction between humans and machines, and the individual and the collective, will gradually start to dissolve, along with the distinction between human and artificial forms of intelligence.