I've posted a link to a video of my best talk -- given at the GRID '08 Conference in Stockholm this summer. It's about the growth of collective intelligence and the Semantic Web, and the future and role the media. Read more and get the video here. Enjoy!
I love how my friend, Jason Calacanis, announces the death of blogging and his retirement from blogging...on a blog. He's a genius. You can read more here and here.
But as anyone who reads this blog knows, I try to be ahead of the curve. In keeping with that, I announced something similar last week. Sorry Jason, I beat you to it.
Unlike Jason, however, I haven't left blogging. Instead, my blogging has evolved and is moving into a better medium. Jason wants to go back to old fashioned email mailing lists, but I'm moving forward into something even better than blogs.
Blogs will continue of course, but for those of us who do a lot of blogging and online bookmarking, social networking, online discussions, and social media sharing -- blogs just aren't productive enough anymore. Neither are social networks or the other tools available today. Because of that, I believe that an increasing amount of activity is going to move off of today's blogs and into a new kind of social media environment. I call these next generation services, interest networks, and my company is building one of the first: Twine.
Interest networking is, I believe, the next evolution of social media. The next step after blogs, aggregators, personal home pages, and social networks. It brings them all together into a new synthesis that is finally what we all really were trying to achieve with all those separate tools in the first place. Interest networks are a big step towards a more unified, productive, and manageable social media environment.
I still blog here from time to time, but it's just a fraction of the online authoring and conversations I participate in now -- instead, the vast majority of my social media activity is taking place in Twine. I'm Twining a lot more than I'm blogging or participating in social networking -- probably by a factor of 100X.
Why have I been doing so much more Twining than blogging and social networking? First of all, I'm not interested in having a conversation with the entire general public, or ever being an A-List blogger, or interacting with networks of random strangers. What I want is to efficiently participate in many different specific groups and communities around particular interests and relationships I have.
Some of my interest communities are relevant to each other but many are not, and it would not make sense to combine them all into one blog or network (which is what present-day blogs and social networks require). In Twine it's super easy to create the equivalent of microblogs, microcommunities, and micronetworks, or to join and participate in, existing ones started by others, around any person, group or interest.
Some of my interest communities are public and many are private (for teams, for private discussions, etc). But I don't want to have to run many blogs or networks for all these different communities I want to converse with -- that's just too much work. Instead, in Twine it is extremely easy to do everything I need to do in one place.
In Twine I can create, join and converse with a large number of different individuals, groups and communities around particular interests, goals, activities and relationships I have. Many of my Twine interactions are public but some are private. They are not all structured or governed in the same way. Twine helps me manage them all, and participate efficiently to author content, track what others are saying, as well as to share and distribute my ideas, participate in discussions, and discover new things around my interests. That's what Twine was built to for.
Twine enables me to connect with others around my interests, the way I really want to. It's easier and smarter than blogging or other forms of social media. Of course, it's still a work in progress; we're still in beta and it's not perfect or even fully built yet. But it's getting there.
Twine has been making great headway in inventing and defining what interest networks should be like. Soon we are going to start opening Twine up to the wider Web. Very soon in fact. And then, as we move into the fall and winter release timeframes, we are going to be rolling out some important new capablities that may be very disruptive. Stay tuned. There are going to be a series of cool things happening in Twine during the remainder of this summer, and this year.
I agree with Jason -- blogging isn't good enough. The first wave of social media is ending -- for some because they couldn't cope with the overload, for others because they want a better medium. I believe interest networks are what's next and I hope to make Twine into the best place to network around interests on the Web. So don't give up blogging or move to email lists, Jason. Just move to a better platform. See you in Twine!
By the way, you can respond to this post in Twine, here. (I'm turning off comments on all new articles in this blog. If you want to have a real discussion with me and others, it's easier in Twine, so my discussions will be there from now on.)
And please join the FriendFeed room about Twine if you are there too.
Now that I have been asked by several dozen people for the slides from my talk on "Making Sense of the Semantic Web," I guess it's time to put them online. So here they are, under the Creative Commons Attribution License (you can share it with attribution this site).
You can download the Powerpoint file at the link below:
Or you can view it right here:
Enjoy! And I look forward to your thoughts and comments.
Last night I saw that the video of my presentation of Twine at the Web 2.0 Summit is online. My session, "The Semantic Edge," featured Danny Hillis of Metaweb demoing Freebase, Barney Pell demoing Powerset, and myself Demoing Twine, followed by a brief panel discussion with Tim O'Reilly (in that order). It's a good panel and I recommend the video, however, the folks at Web 2.0 only filmed the presenters; they didn't capture what we were showing on our screens, so you have to use your imagination as we describe our demos.
An audio cast of one of my presentations about Twine to a reporter was also put online recently, for a more in-depth description.
My company, Radar Networks, has just come out of stealth. We've announced what we've been working on all these years: It's called Twine.com. We're going to be showing Twine publicly for the first time at the Web 2.0 Summit tomorrow. There's lot's of press coming out where you can read about what we're doing in more detail. The team is extremely psyched and we're all working really hard right now so I'll be brief for now. I'll write a lot more about this later.
I've been thinking for several years about Knowledge Networking. It's not a term I invented, it's been floating around as a meme for at least a decade or two. But recently it has started to resurface in my own work.
So what is a knowledge network? I define a knowledge network as a form of collective intelligence in which a network of people (two or more people connected by social-communication relationships) creates, organizes, and uses a collective body of knowledge. The key here is that a knowledge network is not merely a site where a group of people work on a body of information together (such as the wikipedia), it's also a social network -- there is an explicit representation of a social relationship within it. So it's more like a social network than for example a discussion forum or a wiki.
I would go so far as to say that knowledge networks are the third-generation of social software. (Note this is based in-part on ideas that emerged in conversations I have had with Peter Rip, so this also his idea):
Just some thoughts on a Saturday morning...
The Business 2.0 Article on Radar Networks and the Semantic Web just came online. It's a huge article. In many ways it's one of the best popular articles written about the Semantic Web in the mainstream press. It also goes into a lot of detail about what Radar Networks is working on.
One point of clarification, just in case anyone is wondering...
Web 3.0 is not just about machines -- it's actually all about humans -- it leverages social networks, folksonomies, communities and social filtering AS WELL AS the Semantic Web, data mining, and artificial intelligence. The combination of the two is more powerful than either one on it's own. Web 3.0 is Web 2.0 + 1. It's NOT Web 2.0 - people. The "+ 1" is the addition of software and metadata that help people and other applications organize and make better sense of the Web. That new layer of semantics -- often called "The Semantic Web" -- will add to and build on the existing value provided by social networks, folksonomies, and collaborative filtering that are already on the Web.
So at least here at Radar Networks, we are focusing much of our effort on facilitating people to help them help themselves, and to help each other, make sense of the Web. We leverage the amazing intelligence of the human brain, and we augment that using the Semantic Web, data mining, and artificial intelligence. We really believe that the next generation of collective intelligence is about creating systems of experts not expert systems.
It's been an interesting month for news about Radar Networks. Two significant articles came out recently:
Business 2.0 Magazine published a feature article about Radar Networks in their July 2007 issue. This article is perhaps the most comprehensive article to-date about what we are working on at Radar Networks, it's also one of the better articulations of the value proposition of the Semantic Web in general. It's a fun read, with gorgeous illustrations, and I highly recommend reading it.
BusinessWeek posted an article about Radar Networks on the Web. The article covers some of the background that led to my interests in collective intelligence and the creation of the company. It's a good article and covers some of the bigger issues related to the Semantic Web as a paradigm shift. I would add one or two points of clarification in addition to what was stated in the article: Radar Networks is not relying solely on software to organize the Internet -- in fact, the service we will be launching combines human intelligence and machine intelligence to start making sense of information, and helping people search and collaborate around interests more productively. One other minor point related to the article -- it mentions the story of EarthWeb, the Internet company that I co-founded in the early 1990's: EarthWeb's content business actually was sold after the bubble burst, and the remaining lines of business were taken private under the name Dice.com. Dice is the leading job board for techies and was one of our properties. Dice has been highly profitable all along and recently filed for a $100M IPO.
If you are interested in the future of the Web, you might enjoy listening to this interview with me, moderated by Dr. Paul Miller of Talis. We discuss, in-depth: the Semantic Web, Web 3.0, SPARQL, collective intelligence, knowledge management, the future of search, triplestores, and Radar Networks.
This is just a brief post because I am actually slammed with VC meetings right now. But I wanted to congratulate our friends at Metaweb for their pre-launch announcement. My company, Radar Networks, is the only other major venture-funded play working on the Semantic Web for consumers so we are thrilled to see more action in this sector.
Metaweb and Radar Networks are working on two very different applications (fortunately!). Metaweb is essentially making the Wikipedia of the Semantic Web. Here at Radar Networks we are making something else -- but equally big -- and in a different category. Just as Metaweb is making a semantic analogue to something that exists and is big, so are we: but we're more focused on the social web -- we're building something that everyone will use. But we are still in stealth so that's all I can say for now.
This is now an exciting two-horse space. We look forward to others joining the excitement too. Web 3.0 is really taking off this year.
An interesting side note: Danny Hillis (founder of Metaweb), myself (founder of Radar Networks) and Lew Tucker (CTO of Radar Networks) all worked together at Thinking Machines (an early AI massively parallel computer company). It's fascinating that we've all somehow come to think that the only practical way to move machine intelligence forward is by having us humans and applications start to employ real semantics in what we record in the digital world.
Is it only Wednesday? It feels like a whole week already! I've been in back-to-back VC meetings, board discussions and strategy meetings since last week. I think this must be related to the heating-up of the "Web 3.0" meme and the semantic sector in general. Perhaps it is also due to the coverage we got in the Guidewire Report and newsletter which went out to everyone who went to DEMO, and also perhaps because of some influential people in the biz have been talking about us. We've been very careful not to show our app to anyone because it does some things that are really new. We don't want to spread that around (yet). Anyway it's been pretty busy -- not just for me, but for the whole team. Everyone is on full afterburners right now.
By the way -- I'm really proud or product team (hope you guys are reading this)-- the team has made an alpha that is not only a breakthrough on the technical level, but it also looks incredibly good too. Some of the select few who have seen our app so far have said, "the app looks beautiful" and "wow, that's amazing" etc. We've done some cool things with NLP, graph analysis, and statistics under the hood. And the GUI is also very slick. Probably the best team I've worked with.
If you are interested in helping to beta-test the consumer Semantic Web, We're planning on doing invite-only beta trials this summer -- sign up at our website to be on our beta invite list.
I've been thinking since 1994 about how to get past a fundamental barrier to human social progress, which I call "The Collective IQ Barrier." Most recently I have been approaching this challenge in the products we are developing at my stealth venture, Radar Networks.
In a nutshell, here is how I define this barrier:
The Collective IQ Barrier: The potential collective intelligence of a human group is exponentially proportional to group size, however in practice the actual collective intelligence that is achieved by a group is inversely proportional to group size. There is a huge delta between potential collective intelligence and actual collective intelligence in practice. In other words, when it comes to collective intelligence, the whole has the potential to be smarter than the sum of its parts, but in practice it is usually dumber.
Why does this barrier exist? Why are groups generally so bad at tapping the full potential of their collective intelligence? Why is it that smaller groups are so much better than large groups at innovation, decision-making, learning, problem solving, implementing solutions, and harnessing collective knowledge and intelligence?
I think the problem is technological, not social, at its core. In this article I will discuss the problem in more depth and then I will discuss why I think the Semantic Web may be the critical enabling technology for breaking through the Collective IQ Barrier.
Continue reading "Breaking the Collective IQ Barrier -- Making Groups Smarter" »
I've been reading some of the further posts on various blogs in reaction to the Markoff article in the New York Times last Sunday. There is a tremendous amount of misconception about the Semantic Web-- as evidenced for example by Ross Mayfield's post recently. Ross implied that the Semantic Web is about automating the Web, rather than facilitating people. This is a misconception that others have taken to even further extremes -- some people have characterized it as an effort to replace humans, replace social networks and social software, etc. etc. That is simply NOT at all correct! Quite the opposite in fact.
The Semantic Web is just a way to augment and improve the EXISTING Web and all the existing relationships, groups, communities, social networks, user-experiences, apps, content, and online services on it. It doesn't replace the Web we have, it just makes it smarter. It doesn't replace human intelligence and decision-making, it just augments human thinking, so that individuals and groups can overcome the growing complexity of information overload on the Web.
Continue reading "The Semantic Web is About Helping People Use the Web More Productively" »
NOTES
Prelude
Many years ago, in the late 1980s, while I was still a college student, I visited my late grandfather, Peter F. Drucker, at his home in Claremont, California. He lived near the campus of Claremont College where he was a professor emeritus. On that particular day, I handed him a manuscript of a book I was trying to write, entitled, "Minding the Planet" about how the Internet would enable the evolution of higher forms of collective intelligence.
My grandfather read my manuscript and later that afternoon we sat together on the outside back porch and he said to me, "One thing is certain: Someday, you will write this book." We both knew that the manuscript I had handed him was not that book, a fact that was later verified when I tried to get it published. I gave up for a while and focused on college, where I was studying philosophy with a focus on artificial intelligence. And soon I started working in the fields of artificial intelligence and supercomputing at companies like Kurzweil, Thinking Machines, and Individual.
A few years later, I co-founded one of the early Web companies, EarthWeb, where among other things we built many of the first large commercial Websites and later helped to pioneer Java by creating several large knowledge-sharing communities for software developers. Along the way I continued to think about collective intelligence. EarthWeb and the first wave of the Web came and went. But this interest and vision continued to grow. In 2000 I started researching the necessary technologies to begin building a more intelligent Web. And eventually that led me to start my present company, Radar Networks, where we are now focused on enabling the next-generation of collective intelligence on the Web, using the new technologies of the Semantic Web.
But ever since that day on the porch with my grandfather, I remembered what he said: "Someday, you will write this book." I've tried many times since then to write it. But it never came out the way I had hoped. So I tried again. Eventually I let go of the book form and created this weblog instead. And as many of my readers know, I've continued to write here about my observations and evolving understanding of this idea over the years. This article is my latest installment, and I think it's the first one that meets my own standards for what I really wanted to communicate. And so I dedicate this article to my grandfather, who inspired me to keep writing this, and who gave me his prediction that I would one day complete it.
This is an article about a new generation of technology that is sometimes called the Semantic Web, and which could also be called the Intelligent Web, or the global mind. But what is the Semantic Web, and why does it matter, and how does it enable collective intelligence? And where is this all headed? And what is the long-term far future going to be like? Is the global mind just science-fiction? Will a world that has a global mind be good place to live in, or will it be some kind of technological nightmare?
Continue reading "Minding The Planet -- The Meaning and Future of the Semantic Web" »
I was reading this article in Wired magazine about wikis, where the article itself is a wiki that the readers can contribute to -- and an idea occurred to me. What if you could make an entire magazine that was in a fact a wiki? This magazine would be published online via a Website running a wiki engine. Every issue would be by and for the community of readers. There would be an editorial group among the readers that would decide what to write articles about for the next issue of the magazine, and then the community would work to write the articles. To get into the editorial group, remain there, and have a vote as an editor, a community member would have to make a certain number of (non-spurrious) contributions to articles on an ongoing basis (and/or maintain a certain reputation in the community as measured in some other manner).
I can imagine this idea taking off and a lot of these "wikazines" forming around various subject areas. It makes sense that communities of people who are interested in subjects could help to research and write about them. Of course in such communities there would be some people who put more effort in than others, and some who were more like readers or lurkers. But it would still be much more involving than old "one-way media."
In some ways communities like Digg simulate this -- people essentially vote on what is interesting and this filters up to become the featured content on the site. But that is still one step removed from the creative process itself -- only the readers participate, not the content authors. What's interesting about this proposal is that it blurs the distinction between an author and a reader, and provides a way for a magazine to be truly emergent and community-driven. OK, I'm too busy to start this, but I hope someone out there on the lazyweb takes this idea and runs with it. Please let me know if you find examples of this.
Shel Israel and I just finished up working together for 10 days. I needed Shel's perspective on what we are working on at Radar Networks. Shel lived up to his reviews as a brilliant thinker on strategic messaging, branding and positioning. So what are the 15 people at Radar Networks working on? It's still a secret, but yes, it's related to the Semantic Web, and yes, Shel has hinted on his blog at some of it. But it's probably not what you think. And, no, it's not semantic video blogging either. More hints later on. For now, if you are a blogger and you have a wish-list for what wikis or blogs could do next, feel free to submit your list in the comments on this post: I'm doing some informal market research...
[Corrected due to typo.]
Check out this video demo of Microsoft Photosynth -- an experimental technology that combines multiple photos of the same thing into a 3-D model that can then be navigated and explored -- it's beautiful, visionary and well... just awesome.
Grupthink is a service where anyone can create and participate in polls on various subjects. It's similar to an idea I once had about polling the global mind in real-time (although their system does not show votes happening in real-time, presently). It reminds me of several other Web 2.0 sites, but it's nicely done. Worth a look.
Check out The Broth -- it's a "global mosaic" in which you can move tiles around in real time with other people to create emergent artworks. It's really cool to watch images grow and morph from the combined imagination of people around the Net. Beautiful.
My father, Mayer Spivack, has written an interesting piece on managing thinking styles in organizations. He points out the difference between the thinking styles in early and later stage companies, and the challenge of managing and integrating these two aspects of the organization's cognitive process. I think that the syncretic-associative mode (curious, inventive, exploratory, enthusiastic, adventurous) tends to be more externally focused, whereas the linear-logical mode tends to be more internally focused (careful, reductivist, analyticial, skeptical, habitual). You could say this is the difference between an organization being extraverted or introverted, and the challenge is for organizations to evolve balanced personalities. This could be an interesting way to approach management consulting -- and I wouldn't be surprised if there are others thinking about organizations in these terms.
The Syncretic Management Process
Understanding The Value Of Associative Thinking In A World Of Linear Decision-MakingDecisions and communications among individuals in organizations are frequently initiated, managed and concluded almost entirely from within a framework of linear-logical thinking. Paradoxically, the products and services comprising the intellectual property and capital creative upon which most business are founded owe their existence to a non-linear, but nonetheless logical, syncretic process of associative thinking. Syncretic thinking is a mental process that makes non-linear, and therefore often unexpected, connections among seemingly divergent phenomena or data on the basis of common qualities.
By understanding this paradox between the creative syncretic process characterizing the founding stage of an organization, and the conservative linear processes that characterize later stages we can generate a new mix of creative thinking that effectively includes both elements. These two modes of thought highlight several differences between the mind-sets that typify the young innovative start-up phase of a business compared with that same business at a later more mature phase, settled into it’s niche. The associative and inventive thinking that generated a novel product or service and founded an organization or industry may, at maturity, have yielded to a more rigorous calculus of risk, competition and strategic analysis. In this later phase of organization, linear frameworks of thinking that tend to conserve capital and to advance the organizations goals incrementally within an established niche are strongly reinforced and rewarded. Thus linearity, alone, is widely believed to be essential for survival. However, neither framework in isolation is likely to encourage the growth of new ideas that may form the future of the organization. Nor could an organization develop far beyond the initial concept stage without the benefit of both modalities operating together.
Today I read an interesting article in the New York Times about a company called Rite-Solutions which is using a home-grown stock market for ideas to catalyze bottom-up innovation across all levels of personnel in their organization. This is a way to very effectively harness and focus the collective creativity and energy in an organization around the best ideas that the organization generates.
Using virtual stock market systems to measure community sentiment is not a new concept but it is a new frontier. I don't think we've even scratched the surface of what this paradigm can accomplish. For lots of detailed links to resources on this topic see the wikipedia entry on prediction markets. This prediction markets portal also has collected interesting links on the topic. Here is an informative blog post about recent prediction market attempts. Here is a scathing critique of some prediction markets.
There are many interesting examples of prediction markets on the Web:
Here are some interesting, more detailed discussions of prediction market ideas and potential features.
Another area that is related, but highly underleveraged today, are ways to enable communities to help establish whether various ideas are correct using argumentation. By enabling masses of people to provide reasons to agree or disagree with ideas, and with those reasons as well, we can automatically rate what ideas are most agreed with or disagreed with. One very interesting example of this is TruthMapping.com. Some further concepts related to this approach are discussed in this thread.
Introduction:
This article proposes the creation of a new open, nonprofit service on the Web
that will provide something akin to “collective self-awareness” back to the
Web. This service is like a "Google Zeitgeist" on steroids, but with a
lot more real-time, interactive, participatory data, technology and features in
it. The goal is to measure and visualize the state of the collective mind of
humanity, and provide this back to humanity in as close to real-time as is possible,
from as many data sources as we can handle -- as a web service.
By providing this service, we will enable higher levels of collective intelligence to emerge and self-organize on the Web. The key to collective intelligence (or any intelligence in fact) is self-awareness. Self-awareness is, in essence, a feedback loop in which a system measures its own internal state and the state of its environment, then builds a representation of that state, and then reasons about and reacts to that representation in order to generate future behavior. This feedback loop can be provided to any intelligent system -- even the Web, even humanity as-a-whole. If we can provide the Web with such a service, then the Web can begin to “see itself” and react to its own state for the first time. And this is the first step to enabling the Web, and humanity as-a-whole, to become more collectively intelligent.
It should be noted that by "self-awareness" I don’t mean consciousness or sentience – I think that the consciousness comes from humans at this point and we are not trying to synthesize it (we don't need to; it's already there). Instead, by "self-awareness" I mean a specific type of feedback loop -- a specific Web service -- that provides a mirror of the state of the whole back to its parts. The parts are the conscious elements of the system – whether humans and/or machines – and can then look at this meta-mirror to understand the whole as well as their place in it. By simply providing this meta-level mirror, along with ways that the individual parts of the system can report their state to it, and get the state of the whole back from it, we can enable a richer feedback loop between the parts and the whole. And as soon as this loop exists the entire system suddenly can and will become much more collectively intelligent.
What I am proposing is something quite common in artificial intelligence. For example, in the field of robotics, such as when building an autonomous robot. Until a robot is provided with a means by which it can sense its own internal state and the state of its nearby environment, it cannot behave intelligently or very autonomously. But once this self-representation and feedback loop is provided, it can then react to it’s own state and environment and suddenly can behave far more intelligently. All cybernetic systems rely on this basic design pattern. I’m simply proposing we imp
