The BBC World Service's Business Daily show interviewed the CTO of Xerox and me, about the future of the Web, printing, newspapers, search, personalization, the real-time Web. Listen to the audio stream here. I hear this will only be online at this location for 6 more days. If anyone finds it again after that let me know and I'll update the link here.
The next generation of Web search is coming sooner than expected. And with it we will see several shifts in the way people search, and the way major search engines provide search functionality to consumers.
Web 1.0, the first decade of the Web (1989 - 1999), was characterized by a distinctly desktop-like search paradigm. The overriding idea was that the Web is a collection of documents, not unlike the folder tree on the desktop, that must be searched and ranked hierarchically. Relevancy was considered to be how closely a document matched a given query string.
Web 2.0, the second decade of the Web (1999 - 2009), ushered in the beginnings of a shift towards social search. In particular blogging tools, social bookmarking tools, social networks, social media sites, and microblogging services began to organize the Web around people and their relationships. This added the beginnings of a primitive "web of trust" to the search repertoire, enabling search engines to begin to take the social value of content (as evidences by discussions, ratings, sharing, linking, referrals, etc.) as an additional measurment in the relevancy equation. Those items which were both most relevant on a keyword level, and most relevant in the social graph (closer and/or more popular in the graph), were considered to be more relevant. Thus results could be ranked according to their social value -- how many people in the community liked them and current activity level -- as well as by semantic relevancy measures.
In the coming third decade of the Web, Web 3.0 (2009 - 2019), there will be another shift in the search paradigm. This is a shift to from the past to the present, and from the social to the personal.
A very good article about Twine.com has come out in Fast Company, June issue, in print and online. It calls Twine "red hot" and even mentions some hints about Twine 2.0 that I gave the reporter.
I've written a new article about how content distribution has evolved, and where it is heading. It's published here: http://www.siliconangle.com/social-media/content-distribution-is-changing-again/.
If you are interested in semantics, taxonomies, education, information overload and how libraries are evolving, you may enjoy this video of my talk on the Semantic Web and the Future of Libraries at the OCLC Symposium at the American Library Association Midwinter 2009 Conference. This event focused around a dialogue between David Weinberger and myself, moderated by Roy Tennant. We were forutnate to have an audience of about 500 very vocal library directors in the audience and it was an intensive day of thinking together. Thanks to the folks at OCLC for a terrific and really engaging event!
If you are interested in collective intelligence, consciousness, the global brain and the evolution of artificial intelligence and superhuman intelligence, you may want to see my talk at the 2008 Singularity Summit. The videos from the Summit have just come online.
(Many thanks to Hrafn Thorisson who worked with me as my research assistant for this talk).
Twine has been growing at 50% per month since launch in October. We've been keeping that quiet while we wait to see if it holds. VentureBeat just noticed and did an article about it. It turns out our January numbers are higher than Compete.com estimates and February is looking strong too. We have a slew of cool viral features coming out in the next few months too as we start to integrate with other social networks. Should be an interesting season.
UPDATE: There's already a lot of good discussion going on around this post in my public twine.
I’ve been writing about a new trend that I call “interest networking” for a while now. But I wanted to take the opportunity before the public launch of Twine on Tuesday (tomorrow) to reflect on the state of this new category of applications, which I think is quickly reaching its tipping point. The concept is starting to catch on as people reach for more depth around their online interactions.
In fact – that’s the ultimate value proposition of interest networks – they move us beyond the super poke and towards something more meaningful. In the long-term view, interest networks are about building a global knowledge commons. But in the short term, the difference between social networks and interest networks is a lot like the difference between fast food and a home-cooked meal – interest networks are all about substance.
At a time when social media fatigue is setting in, the news cycle is growing shorter and shorter, and the world is delivered to us in soundbytes and catchphrases, we crave substance. We go to great lengths in pursuit of substance. Interest networks solve this problem – they deliver substance.
So, what is an interest network?
In short, if a social network is about who you are interested in, an interest network is about what you are interested in. It’s the logical next step.
Twine for example, is an interest network that helps you share information with friends, family, colleagues and groups, based on mutual interests. Individual “twines” are created for content around specific subjects. This content might include bookmarks, videos, photos, articles, e-mails, notes or even documents. Twines may be public or private and can serve individuals, small groups or even very large groups of members.
I have also written quite a bit about the Semantic Web and the Semantic Graph, and Tim Berners-Lee has recently started talking about what he calls the GGG (Giant Global Graph). Tim and I are in agreement that social networks merely articulate the relationships between people. Social networks do not surface the equally, if not more important, relationships between people and places, places and organizations, places and other places, organization and other organizations, organization and events, documents and documents, and so on.
This is where interest networks come in. It’s still early days to be clear, but interest networks are operating on the premise of tapping into a multi--dimensional graph that manifests the complexity and substance of our world, and delivers the best of that world to you, every day.
We’re seeing more and more companies think about how to capitalize on this trend. There are suddenly (it seems, but this category has been building for many months) lots of different services that can be viewed as interest networks in one way or another, and here are some examples:
What all of these interest networks have in common is some sort of a bottom-up, user-driven crawl of the Web, which is the way that I’ve described Twine when we get the question about how we propose to index the entire Web (the answer: we don’t. We let our users tell us what they’re most interested in, and we follow their lead).
Most interest networks exhibit the following characteristics as well:
This last bullet point is where I see next-generation interest networks really providing the most benefit over social bookmarking tools, wikis, collaboration suites and pure social networks of one kind or another.
To that end, we think that Twine is the first of a new breed of intelligent applications that really get to know you better and better over time – and that the more you use Twine, the more useful it will become. Adding your content to Twine is an investment in the future of your data, and in the future of your interests.
At first Twine begins to enrich your data with semantic tags and links to related content via our recommendations engine that learns over time. Twine also crawls any links it sees in your content and gathers related content for you automatically – adding it to your personal or group search engine for you, and further fleshing out the semantic graph of your interests which in turn results in even more relevant recommendations.
The point here is that adding content to Twine, or other next-generation interest networks, should result in increasing returns. That’s a key characteristic, in fact, of the interest networks of the future – the idea that the ratio of work (input) to utility (output) has no established ceiling.
Another key characteristic of interest networks may be in how they monetize. Instead of being advertising-driven, I think they will focus more on a marketing paradigm. They will be to marketing what search engines were to advertising. For example, Twine will be monetizing our rich model of individual and group interests, using our recommendation engine. When we roll this capability out in 2009, we will deliver extremely relevant, useful content, products and offers directly to users who have demonstrated they are really interested in such information, according to their established and ongoing preferences.
6 months ago, you could not really prove that “interest networking” was a trend, and certainly it wasn’t a clearly defined space. It was just an idea, and a goal. But like I said, I think that we’re at a tipping point, where the technology is getting to a point at which we can deliver greater substance to the user, and where the culture is starting to crave exactly this kind of service as a way of making the Web meaningful again.
I think that interest networks are a huge market opportunity for many startups thinking about what the future of the Web will be like, and I think that we’ll start to see the term used more and more widely. We may even start to see some attention from analysts -- Carla, Jeremiah, and others, are you listening?
Now, I obviously think that Twine is THE interest network of choice. After all we helped to define the category, and we’re using the Semantic Web to do it. There’s a lot of potential in our engine and our application, and the growing community of passionate users we’ve attracted.
Our 1.0 release really focuses on UE/usability, which was a huge goal for us based on user feedback from our private beta, which began in March of this year. I’ll do another post soon talking about what’s new in Twine. But our TOS (time on site) at 6 minutes/user (all time) and 12 minutes/user (over the last month) is something that the team here is most proud of – it tells us that Twine is sticky, and that “the dogs are eating the dog food.”
Now that anyone can join, it will be fun and gratifying to watch Twine grow.
Still, there is a lot more to come, and in 2009 our focus is going to shift back to extending our Semantic Web platform and turning on more of the next-generation intelligence that we’ve been building along the way. We’re going to take interest networking to a whole new level.
Stay tuned!
I've posted a link to a video of my best talk -- given at the GRID '08 Conference in Stockholm this summer. It's about the growth of collective intelligence and the Semantic Web, and the future and role the media. Read more and get the video here. Enjoy!
I've posted a new article in my public twine about how we are moving from the World Wide Web to the Web Wide World. It's about how the Web is spreading into the physical world, and what this means.
Video from my panel at DEMO Fall '08 on the Future of the Web is now available.
I moderated the panel, and our panelists were:
Howard Bloom, Author, The Evolution of Mass Mind from the Big Bang to the 21st Century
Peter Norvig, Director of Research, Google Inc.
Jon Udell, Evangelist, Microsoft Corporation
Prabhakar Raghavan, PhD, Head of Research and Search Strategy, Yahoo! Inc.
The panel was excellent, with many DEMO attendees saying it was the best panel they had ever seen at DEMO.
Many new and revealing insights were provided by our excellent panelists. I was particularly interested in the different ways that Google and Yahoo describe what they are working on. They covered lots of new and interesting information about their thinking. Howard Bloom added fascinating comments about the big picture and John Udell helped to speak about Microsoft's longer-term views as well.
Enjoy!!!
(Brief excerpt from a new post on my Public Twine -- Go there to read the whole thing and comment on it with me and others...).
I have spent the last year really thinking about the future of the Web. But lately I have been thinking more about the future of the desktop. In particular, here are some questions I am thinking about and some answers I've come up so far.
This is a raw, first-draft of what I think it will be like.
Is the desktop of the future going to just be a web-hosted version of the same old-fashioned desktop metaphors we have today?
No. We've already seen several attempts at doing that -- and they never catch on. People don't want to manage all their information on the Web in the same interface they use to manage data and apps on their local PC.
Partly this is due to the difference in user experience between using real live folders, windows and menus on a local machine and doing that in "simulated" fashion via some Flash-based or HTML-based imitation of a desktop.
Web desktops to-date have simply have been clunky and slow imitations of the real-thing at best. Others have been overly slick. But one thing they all have in common: None of them have nailed it.
Whoever does succeed in nailing this opportunity will have a real shot at becoming a very important player in the next-generation of the Web, Web 3.0.
From the points above it should be clear that I think the future of the desktop is going to be significantly different from what our desktops are like today.
It's going to be a hosted web service
Is the desktop even going to exist anymore as the Web becomes increasingly important? Yes, there is going to be some kind of interface that we consider to be our personal "home" and "workspace" -- but it will become unified across devices.
Currently we have different spaces on different devices (laptop, mobile device, PC). These will merge. In order for that to happen they will ultimately have to be provided as a service via the Web. Local clients may be created for various devices, but ultimately the most logical choice is to just use the browser as the client.
Our desktop will not come from any local device and will always be available to us on all our devices.
The skin of your desktop will probably appear within your local device's browser as a completely dynamically hosted web application coming from a remote server. It will load like a Web page, on-demand from a URL.
This new desktop will provide an interface both to your local device, applications and information, as well as to your online life and information.
Instead of the browser running inside, or being launched from, some kind of next-generation desktop web interface technology, it's will be the other way around: The browser will be the shell and the desktop application will run within it either as a browser add-in, or as a web-based application.
The Web 3.0 desktop is going to be completely merged with the Web -- it is going to be part of the Web. There will be no distinction between the desktop and the Web anymore.
Today we think of our Web browser running inside our desktop as an applicaiton. But actually it will be the other way around in the future: Our desktop will run inside our browser as an application.
The focus shifts from information to attention
As our digital lives shift from being focused on the old fashioned desktop (space-based metaphor) to the Web environment we will see a shift from organizing information spatially (directories, folders, desktops, etc.) to organizing information temporally (river of news, feeds, blogs, lifestreaming, microblogging).
Instead of being a big directory, the desktop of the future is going to be more like a Feed reader or social news site. The focus will be on keep up with all the stuff flowing through and what the trends are, rather than on all the stuff that is stored there already.
The focus will be on helping the user to manage their attention rather than just their information.
This is a leap to the meta-level. A second-order desktop. Instead of just being about the information (the first-order), it is going to be about what is happening with the information (the second-order).
It's going to shift us from acting as librarians to acting as daytraders.
Our digital roles are already shifting from effectively acting as "librarians" to becoming more like "daytraders." We are all focusing more on keep up with change than on organizing information today. This will continue to eat up more of our attention...
Read the rest of this on my public Twine! http://www.twine.com/item/11bshgkbr-1k5/the-future-of-the-desktop
I love how my friend, Jason Calacanis, announces the death of blogging and his retirement from blogging...on a blog. He's a genius. You can read more here and here.
But as anyone who reads this blog knows, I try to be ahead of the curve. In keeping with that, I announced something similar last week. Sorry Jason, I beat you to it.
Unlike Jason, however, I haven't left blogging. Instead, my blogging has evolved and is moving into a better medium. Jason wants to go back to old fashioned email mailing lists, but I'm moving forward into something even better than blogs.
Blogs will continue of course, but for those of us who do a lot of blogging and online bookmarking, social networking, online discussions, and social media sharing -- blogs just aren't productive enough anymore. Neither are social networks or the other tools available today. Because of that, I believe that an increasing amount of activity is going to move off of today's blogs and into a new kind of social media environment. I call these next generation services, interest networks, and my company is building one of the first: Twine.
Interest networking is, I believe, the next evolution of social media. The next step after blogs, aggregators, personal home pages, and social networks. It brings them all together into a new synthesis that is finally what we all really were trying to achieve with all those separate tools in the first place. Interest networks are a big step towards a more unified, productive, and manageable social media environment.
I still blog here from time to time, but it's just a fraction of the online authoring and conversations I participate in now -- instead, the vast majority of my social media activity is taking place in Twine. I'm Twining a lot more than I'm blogging or participating in social networking -- probably by a factor of 100X.
Why have I been doing so much more Twining than blogging and social networking? First of all, I'm not interested in having a conversation with the entire general public, or ever being an A-List blogger, or interacting with networks of random strangers. What I want is to efficiently participate in many different specific groups and communities around particular interests and relationships I have.
Some of my interest communities are relevant to each other but many are not, and it would not make sense to combine them all into one blog or network (which is what present-day blogs and social networks require). In Twine it's super easy to create the equivalent of microblogs, microcommunities, and micronetworks, or to join and participate in, existing ones started by others, around any person, group or interest.
Some of my interest communities are public and many are private (for teams, for private discussions, etc). But I don't want to have to run many blogs or networks for all these different communities I want to converse with -- that's just too much work. Instead, in Twine it is extremely easy to do everything I need to do in one place.
In Twine I can create, join and converse with a large number of different individuals, groups and communities around particular interests, goals, activities and relationships I have. Many of my Twine interactions are public but some are private. They are not all structured or governed in the same way. Twine helps me manage them all, and participate efficiently to author content, track what others are saying, as well as to share and distribute my ideas, participate in discussions, and discover new things around my interests. That's what Twine was built to for.
Twine enables me to connect with others around my interests, the way I really want to. It's easier and smarter than blogging or other forms of social media. Of course, it's still a work in progress; we're still in beta and it's not perfect or even fully built yet. But it's getting there.
Twine has been making great headway in inventing and defining what interest networks should be like. Soon we are going to start opening Twine up to the wider Web. Very soon in fact. And then, as we move into the fall and winter release timeframes, we are going to be rolling out some important new capablities that may be very disruptive. Stay tuned. There are going to be a series of cool things happening in Twine during the remainder of this summer, and this year.
I agree with Jason -- blogging isn't good enough. The first wave of social media is ending -- for some because they couldn't cope with the overload, for others because they want a better medium. I believe interest networks are what's next and I hope to make Twine into the best place to network around interests on the Web. So don't give up blogging or move to email lists, Jason. Just move to a better platform. See you in Twine!
By the way, you can respond to this post in Twine, here. (I'm turning off comments on all new articles in this blog. If you want to have a real discussion with me and others, it's easier in Twine, so my discussions will be there from now on.)
And please join the FriendFeed room about Twine if you are there too.
This is a note to readers of this blog. As many of you know, I'm the CEO of Radar Networks, the makers of a new service called Twine.
Twine is a service for "interest networking," which I believe is the next evolution of social media.
How are social networks and interest networks different?
I now use Twine as my main place for authoring and sharing content on the Web. (I also use Twine as my main place for keeping up with my many interests. The Twine community does a great job of scouring the Web to find the content that I want to know about. Generally if there is an article that matters to me, it shows up in Twine very quickly. I no longer have to read as many RSS feeds. This is the power of collective intelligence at its best.)
However, although Twine can be used both to author and discover content around interest, in this article I will focus on the authoring side of the story.
Of course I am biased, but speaking from the perspective of a blogger, I can say that Twine is rapidly becoming the personal publishing environment I always dreamed of having. It's an ideal environment to author content and distribute it to highly relevant audiences.
In Twine, I have many different public and private microblogs on various topics that matter to me, and I also participate in microblogs that others have created. It's super easy to post to one or many of them at once.
Twine also has good support for discussions. It's very easy to have discussions around any piece of content -- and the discussions simply work better than they do in my Typepad blog. And of course, Twine has cool features such as automatic semantic tagging of all my posts, great content management features for finding all the content I have added, and powerful contextual recommendations to other interesting content that are added to my content.
As a result of these benefits, in the last month, I have found that my blogging activity in Twine has become about 100X my blogging activity here in Typepad (no offense to Typepad, by the way -- I really like Typepad too, but as a means of distributing content, it just isn't as useful to me as Twine).
Posting in a traditional blog is a labor intensive process and in the end my post only appears to the readers of one blog. But in Twine it is as easy as bookmarking something, or authoring a note, and then sharing it across a bunch of different communities. And Twine helps me keep track of the discussion around each of my posts as it evolves.
So if you are interested in what I'm reading, what I'm thinking about, and what matters to me, you'll find a lot more of that in Twine.
If you are not yet a Twine member already, register and you will be let in very quickly.
Here is where I hang out in Twine:
And if you want to track all my public posts in Twine, go to my profile and subscribe to my RSS feed in Twine.
Twine is still in invite only beta -- but in the second half of July we will be opening up all the public content in Twine to the open Web. Anyone will be able to read it and we will be letting people in faster as well.
I will still blog here when I have larger articles to share. But on a day-to-day basis, I will be posting a lot more in Twine. Hope to see you there!
(By the way, if you are a member of Twine and you are also finding that Twine is becoming the center of your social media life, feel free to copy and paste this post and adapt it into your own blog)
Tim Berners-Lee is giving a talk, and then we're on a panel, live, today, discussing the Semantic Web, Net Neturality and Web Science. Watch the live Webcast and submit your questions to the panel interactively. Details and times are here.
Melissa Pierce is a filmmaker who is making a film about "Life in Perpetual Beta." It's about how people who are adapting and reinventing themselves in the moment, and a new philosophy or approach to life. She's interviewed a number of interesting people, and while I was in Chicago recently, she spoke with me as well. Here is a clip about how I view the philosophy of living in Beta. Her film is also in perpetual beta, and you can see the clips from her interviews on her blog as the film evolves. Eventually it will be released through the indie film circuit, and it looks like it will be a cool film. By the way, she is open to getting sponsors so if you like this idea and want your brand on the opening credits, drop her a line!
Here is the full video of my talk on the Semantic Web at The Next Web 2008 Conference. Thanks to Boris and the NextWeb gang!
John Mills, one of the engineers behind Twine, recently wrote up an interesting article discussing our approach to semantic tags. It's a good read for folks who think about the Semantic Web and tags.
Continue reading "Tagging and the Semantic Web: Tags as Objects" »
Our present day search engines are a poor match for the way that our brains actually think and search for answers. Our brains search associatively along networks of relationships. We search for things that are related to things we know, and things that are related to those things. Our brains not only search along these networks, they sense when networks intersect, and that is how we find things. I call this associative search, because we search along networks of associations between things.
Human memory -- in other words, human search -- is associative. It works by "homing in" on what we are looking for, rather than finding exact matches. Compare this to the the keyword search that is so popular on the Web today and there are obvious differences. Keyword searching provides a very weak form of "homing in" -- by choosing our keywords carefully we can limit the set of things which match. But the problem is we can only find things which contain those literal keywords.
There is no actual use of associations in keyword search, it is just literal matching to keywords. Our brains on the other hand use a much more sophisticated form of "homing in" on answers. Instead of literal matches, our brains look for things things which are associatively connected to things we remember, in order to find what we are ultimately looking for.
For example, consider the case where you cannot remember someone's name. How do you remember it? Usually we start by trying to remember various facts about that person. By doing this our brains then start networking from those facts to other facts and finally to other memories that they intersect. Ultimately through this process of "free association" or "associative memory" we home in on things which eventually trigger a memory of the person's name.
Both forms of search make use of the intersections of sets, but the
associative search model is exponentially more powerful because for
every additional search term in your query, an entire network of
concepts, and relationships between them, is implied. One additional
term can result in an entire network of related queries, and when you
begin to intersect the different networks that result from multiple
terms in the query, you quickly home in on only those results that make
sense. In keyword search on the other hand, each additional search term
only provides a linear benefit -- there is no exponential amplification
using networks.
Keyword search is a very weak approximation of associative search because there really is no concept of a relationship at all. By entering keywords into a search engine like Google we are simulating an associative search, but without the real power of actual relationships between things to help us. Google does not know how various concepts are related and it doesn't take that into account when helping us find things. Instead, Google just looks for documents that contain exact matches to the terms we are looking for and weights them statistically. It makes some use of relationships between Web pages to rank the results, but it does not actually search along relationships to find new results.
Basically the problem today is that Google does not work the way our brains think. This difference creates an inefficiency for searchers: We have to do the work of translating our associative way of thinking into "keywordese" that is likely to return results we want. Often this requires a bit of trial and error and reiteration of our searches before we get result sets that match our needs.
A recently proposed solution to the problem of "keywordese" is natural language search (or NLP search), such as what is being proposed by companies like Powerset and Hakia. Natural language search engines are slightly closer to the way we actually think because they at least attempt to understand ordinary language instead of requiring keywords. You can ask a question and get answers to that question that make sense.
Natural language search engines are able to understand the language of a query and the language in the result documents in order to make a better match between the question and potential answers. But this is still not true associative search. Although these systems bear a closer resemblance to the way we think, they still do not actually leverage the power of networks -- they are still not as powerful as associative search.
I highly recommend this new book on Collective Intelligence. It features chapters by a Who's Who of thinkers on Collective Intelligence, including a chapter by me about "Harnessing the Collective Intelligence of the World Wide Web."
Here is the full-text of my chapter, minus illustrations (the rest of the book is great and I suggest you buy it to have on your shelf. It's a big volume and worth the read):
Continue reading "Great Collective Intelligence Book; Includes a Chapter I Wrote" »
Earlier this month I had the opportunity to visit, and speak at, the Digital Enterprise Research Institute (DERI), located in Galway, Ireland. My hosts were Stefan Decker, the director of the lab, and John Breslin who is heading the SIOC project.
DERI has become the world's premier research institute for the Semantic Web. Everyone working in the field should know about them, and if you can, you should visit the lab to see what's happening there.
Part of the National University of Ireland, Galway. With over 100 researchers focused solely on the Semantic Web, and very significant financial backing, DERI has, to my knowledge, the highest concentration of Semantic Web expertise on the planet today. Needless to say, I was very impressed with what I saw there. Here is a brief synopsis of some of the projects that I was introduced to:
In summary, my visit to DERI was really eye-opening and impressive. I recommend that major organizations that want to really see the potential of the Semantic Web, and get involved on a research and development level, should consider a relationship with DERI -- they are clearly the leader in the space.
This week we began letting the second wave of beta users into the Twine invite-only beta. It's been a very busy and exciting time for the Twine team. I'll be providing more detailed stats on an ongoing basis in a few weeks once we have more data to analyze. For now, I will just provide some qualitative observations.
Twine is still in the early beta process, but already we are seeing a rapid increase in adoption and scale. We have only let in a few hundred more users to get the process started, but we will be letting more and more in every week as we go forward.
It has been really exciting to watch Twine grow. I find that I am increasingly glued to my Interest Feed watching the fascinating information that is flowing through from all the new members. There have been many new twines created around a wide and growing range of interests and large amount of content added. The recommendations are also quite interesting -- I have already discovered a wide range of new people, twines and content that I didn't know about.
As of this writing, I now have 157 social connections in Twine. My social network in Twine has doubled in size in a week and is rapidly approaching the size of my Facebook network. That's pretty impressive considering this happened in a week (it took about half a year for my Facebook network to grow to that size).
We also had our first outside Twine client app, called "Entwine," written spontaneously by a beta user -- it browses through the RDF data from various items in Twine. That was very cool and unexpected! It really got the team jazzed to see this happen.
Twine is now full of active discussions around interests, questions, ideas, suggestions, current events, technologies and products. I have been pleasantly surprised to see so much interaction among users develop so quickly. As we had hypothesized, discussions are turning out to be a very key feature.
We have received a lot of great feedback from beta users within Twine, as well as many suggestions for how to improve Twine, streamline the user experience, and integrate Twine with other applications and services. This is exactly what we had hoped for from our beta. The team is hard at work analyzing this and prioritizing our next development sprints in light of what we are learning from our users (we do minor releases every week and major ones every 3 weeks).
Most of the press reviews and user stories point to Twine being very exciting, useful and full of potential, which has been great to hear after so much work --- they also universally agree that we still have room to improve the user experience and we need to work on making Twine easier to learn and use. That's not unexpected -- we opened the beta well before the app is finished in order to understand user priorities better. We are really focusing on usability and bug fixes for the next several sprints. All this feedback has been incredibly valuable to the team. Keep it coming!
Another interesting observation. The quality of the users in Twine is distinctly impressive. It's a very smart community of leading-edge thinkers, builders, and technology adopters. Kind of like having your own TED Conference, 24/7 around the world. We will be inviting in a wider range of users in later phases, once the app is further along. In the meantime it is really great to see so many of my colleagues in Twine, and to be making so many new contacts and friends here. For this initial phase this is exactly the audience we need -- people who will really roll up their sleeves and help us make Twine into a great application.
Twine is also rapidly aggregating most of the leading minds in the worldwide Semantic Web development and research community into a social and collaborative interest network. It is great to have this global community of people interested in building and using the Semantic Web come together in Twine, an application that is built using Semantic Web technologies on the Radar Networks Semantic Web Applications platform. I look forward to beginning to share Twine with this worldwide community, and to collaborate with others to extend it and integrate it with other semantic apps and data sets. This is definitely our goal.
It's been a great week. I haven't slept much. I'm having too much fun in Twine!
Special offer to readers of my blog...
There are now well over 30,000 users in the queue to get into the Twine beta. We're going to start letting people in from the waiting list in waves and it should take about a month or two to let everyone in.
But what good is a waiting list if there's no way to cut to the front, right? Fortunately, there is a way to skip ahead to the front of the line...
Write a blog post about Twine on your blog and why you want early access, and send me the link to nova (at) radarnetworks (dot) com. along with your first name, last name, and email address. If I like your post, I'll get you an early access VIP pass to front of the line.
See you in Twine!
I'm here at the BlogTalk conference in Cork, Ireland with a range of bloggers and technologists discussing the emerging social Web. Including myself, Ian Davis and Paul Miller from Talis, there are also a bunch of other Semantic Web folks including Dan Brickley, and a group from DERI Galway.
Over dinner a few of us were discussing the terms "Semantic Web" versus "Web 3.0" and we all felt a better term was needed. After some thinking, Ian Davis suggested "Web 3G." I like this term better than Web 3.0 because it loses the "version number" aspect that so many objected to. It has a familiar ring to it as well, reminding me of the 3G wireless phone initiative. It also suggests Tim Berners-Lee's "Giant Global Graph" or GGG -- a synonym for the Semantic Web. Ian stayed up late and put together a nice blog post about the term, echoing many of my own sentiments about how this term should apply to a decade (the third decade of the Web), rather than to a particular technology.
I am pleased to announce that my company Radar Networks, has raised a $13M Series B investment round to grow our product, Twine. The investment comes from Velocity Interactive Group, DFJ, and Vulcan. Ross Levinsohn -- the man who acquired and ran MySpace for Fox Interactive -- will be joining our board. I'm very excited to be working with Ross and to have his help guiding Twine as it grows.
We are planning to use these funds to begin rolling Twine out to broader consumer markets as part of our multi-year plan to build Twine into the leading service for organizing, sharing and discovering information around interests. One of the key themes of Web 3.0 is to be help people make sense of the overwhelming amount of information and change in the online world, and at Twine, we think interests are going to play a key organizing role in that process.
Your interests comprise the portion of your information and relationships that are actually important enough that you want to keep track of them and share them with others. The question that Twine addresses is how to help individuals and groups more efficiently locate, manage and communicate around their interests in the onslaught of online information they have to cope with. The solution to information overload is not to organize all the information in the world (an impossible task), it is to help individuals and groups organize THEIR information (a much more feasible goal).
In March we are going to expand the Twine beta to begin letting more people in. Currently we have around 30,000 people on the wait-list and more coming in steadily. In March we will start letting all of these people in, gradually in waves of a few thousand at a time, and letting them invite their friends in. So to get into Twine you need to sign up on the list on the Twine site, or have a friend who is already in the service invite you in. I look forward to seeing you in Twine!
The last few months of closed beta have been very helpful in getting a lot of useful feedback and testing that has helped us improve the product in many ways. This next wave will be an exciting phase for Twine as we begin to really grow the service with more users. I am sure there will be a lot of great feedback and improvements that result from this.
However, even though we will be letting more people in soon, we are still very much in beta and will be for quite some time to come -- There will still be things that aren't finished, aren't perfect, or aren't there yet -- so your patience will be appreciated as we continue to work on Twine over the coming year. We are letting people in to help us guide the service in the right direction, and to learn from our users. Today Twine is about 10% of what we have planned for it. First we have to get the basics right -- then, in the coming year, we will really start to surface more of the power of the underlying semantic platform. We're psyched to get all this built -- what we have planned is truly exciting!
This is a video of me giving commentary on my "Understanding the Semantic Web" talk and how it relates to Twine, to a group of French business school students who made a visit to our office last month.
Nova Spivack - Semantic Web Talk from Nicolas Cynober on Vimeo.
The Crunchies are done. At Radar Networks we are really honored to have our product, Twine.com, nominated as a finalist for Best Technology Innovation of 2007. It was very cool to see our Twine logo up there on stage next to Facebook, Digg, LinkedIn and so many other incredible companies -- especially considering we were the only company that was still in closed Beta in the awards (and yes, we are coming out of closed beta in March, so get ready!).
Meanwhile, one of things that made the Crunchies fun was that every company was asked to submit a video. Not all companies did, and not all of them were that creative. Some however were really funny, including ours. Here is a link to the "director's cut" of the Twine Crunchies video for 2007. Enjoy!!!
ps. For those who don't live in the USA... CoolWhip is a synthetic dessert topping we have here in the States. Imagine whipped cream, made out of some kind of industrial byproduct. It actually tastes pretty good, whatever it is. And it has almost no calories -- possibly because there is nothing in that is actually digestible by humans. It's really a wonderful technological innovation. Thus our choice.
Question: What do you do if you're not a computer scientist but you are interested in understanding what all this Semantic Web stuff is about?
Answer: Watch this video!
Scoble came over and filmed a full conversation and video demo of Twine. You can watch the long version (1 hour) or the short version (10 mins) on his site. Here's the link.
This is written in response to a post by Anne Zelenka.
I've been talking about the coming "semantic graph" for quite some time now, and it seems the meme has suddenly caught on thanks to a recent article by Tim Berners-Lee in which he speaks of an emerging "Giant Global Graph" or "GGG." But if the GGG emerges it may or may not be semantic. For example social networks are NOT semantic today, even though they contain various kinds of links between people and other things.
So what makes a graph "semantic?" How is the semantic graph different from social networks like Facebook for example?
Many people think that the difference between a social graph and a semantic graph is that a semantic graph contains more types of nodes and links. That's potentially true, but not always the case. In fact, you can make a semantic social graph or a non-semantic social graph. The concept of whether a graph is semantic is orthogonal to whether it is social.
A graph is "semantic" if the meaning of the graph is defined and exposed in an open and machine-understandable fashion. In other words, a graph is semantic if the semantics of the graph are part of the graph or at least connected from the graph. This can be accomplished by representing a social graph using RDF and OWL, the languages of the Semantic Web.
Continue reading "Defining the Semantic Graph -- What is it Really?" »
Slideshare is a site where people post and share their Powerpoints. You can watch the powerpoints quickly with a little viewer widget that let's you click through them in your browser. There are some really interesting, creative, and informative presentations there. It's addictive, I've been looking at presentations all morning. Can't stop. (Thanks to Peter Royal for telling me about this).
Now that I have been asked by several dozen people for the slides from my talk on "Making Sense of the Semantic Web," I guess it's time to put them online. So here they are, under the Creative Commons Attribution License (you can share it with attribution this site).
You can download the Powerpoint file at the link below:
Or you can view it right here:
Enjoy! And I look forward to your thoughts and comments.
The most interesting and exciting new app I've seen this month (other than Twine of course!) is a new semantic search engine called True Knowledge. Go to their site and watch their screencast to see what the next generation of search is really going to look like.
True Knowledge is doing something very different from Twine -- whereas Twine is about helping individuals, groups and teams manage their private and shared knowledge, True Knowledge is about making a better public knowledgebase on the Web -- in a sense they are a better search engine combined with a better Wikipedia. They seem to overlap more with what is being done by natural language search companies like Powerset and companies working on public databases, such as Metaweb and Wikia.
I don't yet know whether True Knowledge is supporting W3C open-standards for the Semantic Web, but if they do, they will be well-positioned to become a very central service in the next phase of the Web. If they don't they will just be yet another silo of data -- but a very useful one at least. I personally hope they provide SPARQL API access at the very least. Congratulations to the team at True Knowledge! This is a very impressive piece of work.
Last night I saw that the video of my presentation of Twine at the Web 2.0 Summit is online. My session, "The Semantic Edge," featured Danny Hillis of Metaweb demoing Freebase, Barney Pell demoing Powerset, and myself Demoing Twine, followed by a brief panel discussion with Tim O'Reilly (in that order). It's a good panel and I recommend the video, however, the folks at Web 2.0 only filmed the presenters; they didn't capture what we were showing on our screens, so you have to use your imagination as we describe our demos.
An audio cast of one of my presentations about Twine to a reporter was also put online recently, for a more in-depth description.
What a week it has been for Radar Networks. We have worked so hard these last few days to get ready to unveil Twine, and it has been a real thrill to show our work and get such positive feedback and support from the industry, bloggers, the media and potential users.
We really didn't expect so much excitement and interest. In fact we've been totally overwhelmed by the response as thousands upon thousands of people have contacted us in the last 24 hours asking to join our beta, telling us how they would use Twine for their personal information management, their collaboration, their organizations, and their communities. Clearly there is such a strong and growing need out there for the kind of Knowledge Networking capabilities that Twine provides, and it's been great to hear the stories and make new connections with so many people who want our product. We love hearing about your interest in Twine, what you would use it for, what you want it to do, and why you need it! Keep those stories coming. We read them all and we really listen to them.
Today, in unveiling Twine, over five years of R&D, and contributions from dozens of core contributors, a dedicated group of founders and investors, and hundreds of supporters, advisors, friends and family, all came to fruition. As a company, and a team, we achieved an important milestone and we should all take some time to really appreciate what we have accomplished so far. Twine is a truly ambitious and pardigm-shifting product, that is not only technically profound but visually stunning -- There has been so much love and attention to detail in this product.
In the last 6 months, Twine has really matured into a product, a product that solves real and growing needs (for a detailed use-case see this post). And just as our product has matured, so has our organization: As we doubled in size, our corporate culture has become tremendously more interesting, innovative and fun. I could go on and on about the cool things we do as a company and the interesting people who work here. But it's the passion, dedication and talent of this team that is most inspiring. We are creating a team and a culture that truly has the potential to become a great Silicon Valley company: The kind of company that I've always wanted to build.
Although we launched today, this is really just the beginning of the real adventure. There is still much for us to build, learn about, and improve before Twine will really accomplish all the goals we have set out for it. We have a five-year roadmap. We know this is a marathon, not a sprint and that "slow and steady wins the race." As an organization we also have much learning and growing to do. But this really doesn't feel like work -- it feels like fun -- because we all love this product and this company. We all wake up every day totally psyched to work on this.
It's been an intense, challenging, and rewarding week. Everyone on my team has impressed me and really been at the top of their game. Very few of us got any real sleep, and most of us went far beyond the call of duty. But we did it, and we did it well. As a company we have never cut corners, and we have always preferred to do things the right way, even if the right way is the hard way. But that pays off in the end. That is how great products are built. I really want to thank my co-founders, my team, my investors, advisors, friends, and family, for all their dedication and support.
Today, we showed our smiling new baby to the world, and the world smiled back.
And tonight , we partied!!!
My company, Radar Networks, has just come out of stealth. We've announced what we've been working on all these years: It's called Twine.com. We're going to be showing Twine publicly for the first time at the Web 2.0 Summit tomorrow. There's lot's of press coming out where you can read about what we're doing in more detail. The team is extremely psyched and we're all working really hard right now so I'll be brief for now. I'll write a lot more about this later.
News Flash!
My company, Radar Networks, is coming out of stealth this Friday, October 19, 2007 at the Web 2.0 Summit, in San Francisco. I'll be speaking on "The Semantic Edge Panel" at 4:10 PM, and publicly showing our Semantic Web online service for the first time. If you are planning to come to Web 2.0, I hope to see you at my panel.
Here's the official Media Alert below:
(PRWEB) October 15, 2007 -- At the Web2.0 Summit on October 19th, Radar Networks will announce a revolutionary new service that uses the power of the emerging Semantic Web to enable a smarter way of sharing, organizing and finding information. Founder and CEO Nova Spivack will also give the first public preview of Radar’s application, which is one of the first examples of “Web 3.0” – the next-generation of the Web, in which the Web begins to function more like a database, and software grows more intelligent and helpful.
Join Nova as he participates in “The Semantic Edge” panel discussion with esteemed colleagues including Powerset’s Barney Pell and Metaweb’s Daniel Hillis, moderated by Tim O’Reilly.
Who:
Radar Networks Founder and CEO Nova Spivack
When:
Friday, October 19, 2007
4:10 – 4:55 p.m.
Where:
Web2.0 Summit
Palace Hotel
Grand Ballroom
2 New Montgomery Street
San Francisco, California 94105
Jason just blogged his take on an official definition of "Web 3.0" -- in his case he defines it as better content, built using Web 2.0 technologies. There have been numerous responses already, but since I am one of the primary co-authors of the Wikipedia page on the term Web 3.0, I thought I should throw my hat in the ring here.
Web 3.0, in my opinion is best defined as the third-decade of the Web (2009 - 2019), during which time several key technologies will become widely used. Chief among them will be RDF and the technologies of the emerging Semantic Web. While Web 3.0 is not synonymous with the Semantic Web (there will be several other important technology shifts in that period), it will be largely characterized by semantics in general.
Web 3.0 is an era in which we will upgrade the back-end of the Web, after a decade of focus on the front-end (Web 2.0 has mainly been about AJAX, tagging, and other front-end user-experience innovations.) Web 3.0 is already starting to emerge in startups such as my own Radar Networks (our product is Twine) but will really become mainstream around 2009.
Why is defining Web 3.0 as a decade of time better than just about any other possible definition of the term? Well for one thing, it's a definition that can't easily be co-opted by any company or individual around some technology or product. It's also a completely unambiguous definition -- it refers to a particular time period and everything that happens in Web technology and business during that period. This would end the debate about what the term means and move it to something more useful to discuss: What technologies and trends will actually become important in the coming decade of the Web?
It's time to once again pull out my well-known graph of Web 3.0 to illustrate what I mean...
(Click the thumbnail for a larger, reusable version)
I've written fairly extensively on the subjects of defining Web 3.0 and the Semantic Web. Here are some links to get you started if you want to dig deeper:
The Semantic Web: From Hypertext to Hyperdata
The Meaning and Future of the Semantic Web
How the WebOS Evolves
Web 3.0 Roundup
Gartner is Wrong About Web 3.0
Beyond Keyword (And Natural Language) Search
Enriching the Connections of the Web: Making the Web Smarter
Next Step for the Web
Doing for Data What HTML Did for Documents
I've been tracking the progress of my Burma protest meme. In just under one week it has spread to almost 17,000 web pages and it continues to grow. (For the latest number, click here). It's great to see the blogosphere pick this up, and I'm glad to be able to do something to help raise awareness of this important human rights issue.
This meme is also an example of an interesting new way to spread content on the Web -- whether for a protest or an ad or any other kind of announcement. It's kind of like a chain letter, but via weblogs. There are many different ways to structure these memes with varying levels of virality and benefit to participants. For some earlier work I've done on meme propagation on the Web see my GoMeme experiments from a few years ago. In those experiments I created a series of memes that spread widely through the blogosphere, based on different viral messages, surveys, and benefits to participants. Other people then tracked the statistics of the memes as they spread. It turned out to be a very interesting study of superdistribution of content along social networks.
I have a lot of respect for the folks at Gartner, but their recent report in which they support the term "Web 2.0" yet claim that the term "Web 3.0" is just a marketing ploy, is a bit misguided.
In fact, quite the opposite is true.
The term Web 2.0 is in fact just a marketing ploy. It has only come to have something resembling a definition over time. Because it is in fact so ill-defined, I've suggested in the past that we just use it to refer to a decade: the second decade of the Web (2000 - 2010). After all there is no actual technology that is called "Web 2.0" -- at best there are a whole slew of things which this term seems to label, and many of them are design patterns, not technologies. For example "tagging" is not a technology, it is a design pattern. A tag is a keyword, a string of text -- there is not really any new technology there. AJAX is also not a technology in its own right, but rather a combination of technologies and design patterns, most of which existed individually before the onset of what is called Web 2.0.
In contrast, the term Web 3.0 actually does refer to a set of new technologies, and changes they will usher in during the third decade of the Web (2010 - 2020). Chief among these is the Semantic Web. The Semantic Web is actually not one technology, but many. Some of them such as RDF and OWL have been under development for years, even during the Web 2.0 era, and others such as SPARQL and GRDDL are recent emerging standards. But that is just the beginning. As the Semantic Web develops there will be several new technology pieces added to the puzzle for reasoning, developing and sharing open rule definitions, handling issues around trust, agents, machine learning, ontology development and integration, semantic data storage, retrieval and search, and many other subjects.
Essentially, the Semantic Web enables the gradual transformation of the Web into a database. This is a profound structural change that will touch every layer of Web technology eventually. It will transform database technology, CMS, CRM, enterprise middleware, systems integration, development tools, search engines, groupware, supply-chain integration, and all the other topics that Gartner covers.
The Semantic Web will manifest in several ways. In many cases it will improve applications and services we already use. So for example, we will see semantic social networks, semantic search, semantic groupware, semantic CMS, semantic CRM, semantic email, and many other semantic versions of apps we use today. For a specific example, take social networking. We are seeing much talk about "opening up the social graph" so that social networks are more connected and portable. Ultimately to do this right, the social graph should be represented using Semantic Web standards, so that it truly is not only open but also easily extensible and mashable with other data.
Web 3.0 is not ONLY the Semantic Web however. Other emerging technologies may play a big role as well. Gartner seems to think Virtual Reality will be one of them. Perhaps, but to be fair, VR is actually a Web 1.0 phenomenon. It's been around for a long time, and it hasn't really changed that much. In fact the folks at the MIT Media Lab were working on things that are still far ahead of Second Life, even back in the early 1990's.
So what other technologies can we expect in Web 3.0 that are actually new? I expect that we will have a big rise in "cloud computing" such as open peer-to-peer grid storage and computing capabilities on the Web -- giving any application essentially as much storage and computational power as needed for free or a very low cost. In the mobile arena we will see higher bandwidth, more storage and more powerful processors in mobile devices, as well as powerful built-in speech recognition, GPS and motion sensors enabling new uses to emerge. I think we will also see an increase in the power of personalization tools and personal assistant tools that try to help users manage the complexity of their digital lives. In the search arena, we will see search engines get smarter -- among other things they will start to not only answer questions, but they will accept commands such as "find me a cheap flight to NYC" and they will learn and improve as they are used. We will also see big improvements in integration and data and account portability between different Web applications. We will also see a fundamental change in the database world as databases move away from the relational model and object model, towards the associative model of data (graph databases and triplestores).
In short, Web 3.0 is about hard-core new technologies and is going to have a much greater impact on enterprise IT managers and IT systems than Web 2.0. But ironically, it may not be until Web 4.0 (2020 - 2030) that Gartner comes to this conclusion!
Hey, I'm posting this to support my friend Kaliya's (un)conference for women in tech called She's Geeky, A Women's Tech (un)conference (October 22-23 in Mountain View, CA).
Here at Radar Networks we're proud that we have a bunch of really smart women across all areas of the company including Sonja Erickson our awesome VP of systems, Jennifer Agostinelli who runs operations and makes our culture great, Karen Marcelo who hacks code and in her free time builds fire-breathing robots, Lara Fields who hacks RDF in a land called Cucamonga, Susan Mayo who is not only a woman in tech but also a woman in soccer, Tricia Royal who is a designer of many things, and just this week we added Kim Laama on UI, and soon two more women are joining our team. More than 1/3 of our staff are women -- Not bad for a company working on bleeding-edge tech!!!
I'm posting this in response to a recent post by Tim O'Reilly which focused on disambiguating what the Semantic Web is and is not, as well as the subject of Collective Intelligence. I generally agree with Tim's post, but I do have some points I would add by way of clarification. In particular, in my opinion, the Semantic Web is all about collective intelligence, on several levels. I would also suggest that the term "hyperdata" is a possibly useful way to express what the Semantic Web is really all about.
What Makes Something a Semantic Web Application?
I agree with Tim that the term "Semantic Web" refers to the use of a particular set of emerging W3C open standards. These standards include RDF, OWL, SPARQL, and GRDDL. A key requirement for an application to have "Semantic Web inside" so to speak, is that it makes use of or is compatible with, at the very least, basic RDF. Another alternative definition is that for an application to be "Semantic Web" it must make at least some use of an ontology, using a W3C standard for doing so.
Semantic Versus Semantic Web
Many applications and services claim to be "semantic" in one manner or another, but that does not mean they are "Semantic Web." Semantic applications include any applications that can make sense of meaning, particularly in language such as unstructured text, or structured data in some cases. By this definition, all search engines today are somewhat "semantic" but few would qualify as "Semantic Web" apps.
The Difference Between "Data On the Web" and a "Web of Data"
The Semantic Web is principally about working with data in a new and hopefully better way, and making that data available on the Web if desired in an open fashion such that other applications can understand and reuse it more easily. We call this idea "The Data Web" -- the notion is that we are transforming the Web from a distributed file server into something that is more like a distributed database.
Instead of the basic objects being web pages, they are actually pieces of data (triples) and records formed from them (sets, trees, graphs or objects comprised of triples). There can be any number of triples within a Web page, and there can also be triples on the Web that do not exist within Web pages at all -- they can come directly from databases for example.
One might respond to this by noting that there is already a lot of data on the Web, in XML and other formats -- how is the Semantic Web different from that? What is the difference between "Data on the Web" and the idea of "The Data Web?"
The best answer to this question that I have heard was something that Dean Allemang said at a recent Semantic Web SIG in Palo Alto. Dean said, "Sure there is data on the Web, but it's not actually a web of data." The difference is that in the Semantic Web paradigm, the data can be linked to other data in other places, it's a web of data, not just data on the Web.
I call this concept of interconnected data, "Hyperdata." It does for data what hypertext did for text. I'm probably not the originator of this term, but I think it is a very useful term and analogy for explaining the value of the Semantic Web.
Another way to think of it is that the current Web is a big graph of interconnected nodes, where the nodes are usually HTML documents, but in the Semantic Web we are talking about a graph of interconnected data statements that can be as general or specific as you want. A data record is a set of data statements about the same subject, and they don't have to live in one place on the network -- they could be spread over many locations around the Web.
A statement to the effect of "Sue lives in Palo Alto" could exist on site A, refer to a URI for a statement defining Sue on site B, a URI for a statement that defines "lives in" on site C, and a URI for a statement defining "Palo Alto" on site D. That's a web of data. What's cool is that anyone can potentially add statements to this web of data, it can be completely emergent.
The Semantic Web is Built by and for Collective Intelligence
This is where I think Tim and others who think about the Semantic Web may be missing an essential point. The Semantic Web is in fact highly conducive to "collective intelligence." It doesn't require that machines add all the statements using fancy AI. In fact, in a next-generation folksonomy, when tags are created by human users, manually, they can easily be encoded as RDF statements. And by doing this you get lots of new capabilities, like being able to link tags to concepts that define their meaning, and to other related tags.
Humans can add tags that become semantic web content. They can do this manually or software can help them. Humans can also fill out forms that generate RDF behind the scenes, just as filling out a blog posting form generates HTML, XML, ATOM etc. Humans don't actually write all that code, software does it for them, yet blogging and wikis for example are considered to be collective intelligence tools.
So the concept of folksonomy and tagging is truly orthogonal to the Semantic Web. They are not mutually exclusive at all. In fact the Semantic Web -- or at least "Semantic Web Lite" (RDF + only basic use of OWL + basic SPARQL) is capable of modelling and publishing any data in the world in a more open way.
Any application that uses data could do everything it does using these technologies. Every single form of social, user-generated content and community could, and probably will, be implemented using RDF in one manner or another within the next decade or so. And in particular, RDF and OWL + SPARQL are ideal for social networking services -- the data model is a much better match for the structure of the data and the network of users and the kinds of queries that need to be done.
Folktologies
This notion that somehow the Semantic Web is not about folksonomy needs to be corrected. For example, take Metaweb's Freebase. Freebase is what I call a "folktology" -- it's an emergent, community generated ontology. Users collaborate to add to the ontology and the knowledge base that is populated within it. That's a wonderful example of collective intelligence, user generated content, and semantics (although technically to my knowledge they are not using RDF for this, their data model is from what I can see functionally equivalent and I would expect at least a SPARQL interface from them eventually).
But that's not all -- check out TagCommons and this Tag Ontology discussion, and also the SKOS ontology -- all of which are working on semantic ways of characterizing simple tags in order to enrich folksonomies and enable better collective intelligence.
There are at least two other places where the Semantic Web naturally leverages and supports collective intelligence. The first is the fact that people and software can generate triples (people could do it by hand, but generally they will do it by filling out Web forms or answering questions or dialog boxes etc.) and these triples can live all over the Web, yet interconnect or intersect (when they are about the same subjects or objects).
I can create data about a piece of data you created, for example to state that I agree with it, or that I know something else about it. You can create data about my data. Thus a data-set can be generated in a distributed way -- it's not unlike a wiki for example. It doesn't have to work this way, but at least it can if people do this.
The second point is that OWL, the ontology language, is designed to support an infinite number of ontologies -- there doesn't have to be just one big ontology to "rule them all." Anyone can make a simple or complex ontology and start to then make data statements that refer to it. Ontologies can link to or include other ontologies, or pieces of them, to create bigger distributed ontologies that cover more things.
This is kind of like not only mashing up the data, but also mashing up the schemas too. Both of these are examples of collective intelligence. In the case of ontologies, this is already happening, for example many ontologies already make use of other ontologies like the Dublin Core and Foaf.
The point here is that there is in fact a natural and very beneficial fit between the technologies of the Semantic Web and what Tim O'Reilly defines Web 2.0 to be about (essentially collective intelligence). In fact the designers of the underlying standards of the Semantic Web specifically had "collective intelligence" in mind when they came up with these ideas. They were specifically trying to rectify several problems in the closed, data-silo world of old fashioned databases. The big motivation was to make data more integrated, to enable applications to share data more easily, and to be able to build data with other data, and to build schemas with other schemas. It's all about enabling connections and network effects.
Now, whether people end up using these technologies to do interesting things that enable human-level collective intelligence (as opposed to just software level collective intelligence) is an open question. At least some companies such as my own Radar Networks and Metaweb, and Talis (thanks, Danny), are directly focused on this, and I think it is safe to say this will be a big emerging trend. RDF is a great fit for social and folksonomy-based applications.
Web 3.0 and the concept of "Hyperdata"
Where Tim defines Web 2.0 as being about collective intelligence generally, I would define Web 3.0 as being about "connective intelligence." It's about connecting data, concepts, applications and ultimately people. The real essence of what makes the Web great is that it enables a global hypertext medium in which collective intelligence can emerge. In the case of Web 3.0, which begins with the Data Web and will evolve into the full-blown Semantic Web over a decade or more, the key is that it enables a global hyperdata medium (not just hypertext).
As I mentioned above, hyperdata is to data what hypertext is to text. Hyperdata is a great word -- it is so simple and yet makes a big point. It's about data that links to other data. It does for data what hypertext does for text. That's what RDF and the Semantic Web are really all about. Reasoning is NOT the main point (but is a nice future side-effect...). The main point is about growing a web of data.
Just as the Web enabled a huge outpouring of collective intelligence via an open global hypertext medium, the Semantic Web is going to enable a similarly huge outpouring of collective knowledge and cognition via a global hyperdata medium. It's the Web, only better.
I've been looking around for open-source libraries (preferably in Java, but not required) for extracting data and metadata from common file formats and Web formats. One project that looks very promising is Aperture. Do you know of any others that are ready or almost ready for prime-time use? Please let me know in the comments! Thanks.
Microsoft's Astoria project has decided to make RDF a lower priority and is not supporting it for now. So much for Microsoft participating in the Semantic Web.
The Astoria project page describes the project thusly:
The goal of Microsoft Codename Astoria is to enable applications to expose data as a data service that can be consumed by web clients within a corporate network and across the internet. The data service is reachable over HTTP, and URIs are used to identify the various pieces of information available through the service. Interactions with the data service happens in terms of HTTP verbs such as GET, POST, PUT and DELETE, and the data exchanged in those interactions is represented in simple formats such as XML and JSON.
That sounds like a perfect description of a Web 3.0 service-- and a perfect use-case for RDF and SPARQL! But the team at Astoria doesn't seem to feel anyone is actually using RDF enough to warrant supporting that open standard. Well, perhaps among Microsoft's customers that is true enough. But on the other hand, this is a missed opportunity to exercise forward-thinking leadership that will pay dividends to Microsoft in the future when millions of developers and applications will be using RDF as their common language. When is that future? Perhaps as soon as 2 - 3 years. Certainly not more than 5 years.
Welcome to the age of openness: The more open your platform and data, the more developers will want to use it. RDF is one of the ways to make data more open, and supporting it in a data-web platform makes good sense. But it's hard to teach a an old god new tricks!
I've been thinking for several years about Knowledge Networking. It's not a term I invented, it's been floating around as a meme for at least a decade or two. But recently it has started to resurface in my own work.
So what is a knowledge network? I define a knowledge network as a form of collective intelligence in which a network of people (two or more people connected by social-communication relationships) creates, organizes, and uses a collective body of knowledge. The key here is that a knowledge network is not merely a site where a group of people work on a body of information together (such as the wikipedia), it's also a social network -- there is an explicit representation of a social relationship within it. So it's more like a social network than for example a discussion forum or a wiki.
I would go so far as to say that knowledge networks are the third-generation of social software. (Note this is based in-part on ideas that emerged in conversations I have had with Peter Rip, so this also his idea):
Just some thoughts on a Saturday morning...
Peter Royal got me interested in GRDDL recently. It looks like something we may want to use here at Radar Networks for scraping data from the Web and turning it into RDF triples. After doing a little digging around lately, I'm wondering if there is a list of sites that support GRDDL yet? Or any apps that use GRDDL yet?
In recent months we have witnessed a number of social networking sites begin to open up their platforms to outside developers. While this trend has been exhibited most prominently by Facebook, it is being embraced by all the leading social networking services, such as Plaxo, LinkedIn, Myspace and others. Along separate dimensions we also see a similar trend towards "platformization" in IM platforms such as Skype as well as B2B tools such as Salesforce.com.
If we zoom out and look at all this activity from a distance it appears that there is a race taking place to become "the social operating" system of the Web. A social operating system might be defined as a system that provides for systematic management and facilitation of human social relationships and interactions.
We might list some of the key capabilities of an ideal "social operating system" as:
Today I have not seen any single player that provides a coherent solution to this entire "social stack" however Microsoft, Yahoo, and AOL are probably the strongest contenders. Can Facebook and other social networks truly compete or will they ultimately be absorbed into one of these larger players?
I'm sitting the Dynasty Lounge in Taipei, enroute to Singpore where I will be addressing ministers of the government there on the potential of the Semantic Web. Singapore is a very forward-looking country and they have some very exciting new initiatives in the works there. After that I hope to have a little time for a vacation and then I'm heading back to San Francisco, returning on August 1.
I should have email for all or most of the time here, so that is the best way to reach me directly. And of course you can comment on this blog too.
As for the company -- lots of good news here at Radar Networks.
First of all the team has gotten the next version of our alpha up (our hosted Web service for the Semantic Web) and it's getting awesome! We're on track for a invite only launch in the fall timeframe as planned.
We also chose a brand for our product, with help from the mad geniuses at Igor International. The new brand is secret until launch but we love it. We'll be announcing the brand close to launch.
If you want to be invited to our launch and be one of the first to see how useful the Semantic Web really can be -- sign up for our mailing list at http://www.radarnetworks.com/ -- and feel free to invite your friends to sign up too. Only people who sign up will get on our waiting list. We already have around 2000 bloggers and other influencers pre-registered, and more are coming every day, so don't wait -- it will be on a first-come, first-serve basis. We'll be letting people into the service in waves.
Another exciting development: Several of the world's big media empires have started approaching me to see how they can get involved in the network we are building here at Radar Networks. They are interested in the potential of the Semantic Web for adding new capabilities to their content and new services for their audiences. That's an exciting direction to explore for us. If you have large collections of interesting, useful, content of value to particular audiences, or if you have large audiences that need a better way to do stuff on the Web, feel free to drop me a line and we can discuss how you might be able to get involved with the Semantic Web in partnership with us.
In other news, I am still inundated with hundreds of emails from interesting people who read the articles about us in this month's Business 2.0 and BusinessWeek. It's been very interesting to connect with so many other thinkers and businesses. Forgive me in advance if takes me a while to write back -- I promise I will.
I can't wait to come back to San Francisco and start playing with our alpha -- it's really getting there. All the credit should go to our awesome development team. They've been writing tons of code and it's starting to really pay off.
Web 3.0 -- aka The Semantic Web -- is about enriching the connections of the Web. By enriching the connections within the Web, the entire Web may become smarter.
I believe that collective intelligence primarily comes from connections -- this is certainly the case in the brain where the number of connections between neurons far outnumbers the number of neurons; certainly there is more "intelligence" encoded in the brain's connections than in the neurons alone. There are several kinds of connections on the Web:
Are there other kinds of connections that I haven't listed -- please let me know!
I believe that the Semantic Web can actually enrich all of these types of connections, adding more semantics not only to the things being connected (such as representations of information or people or apps) but also to the connections themselves.
In the Semantic Web approach, connections are represented with statements of the form (subject, predicate, object) where the elements have URIs that connect them to various ontologies where their precise intended meaning can be defined. These simple statements are sometimes called "triples" because they have three elements. In fact, many of us are working with statements that have more than three elements ("tuples"), so that we can represent not only subject, predicate, object of statements, but also things like provenance (where did the data for the statement come from?), timestamp (when was the statement made), and other attributes. There really is no limit to what kind of metadata can be stored in these statements. It's a very simple, yet very flexible and extensible data model that can represent any kind of data structure.
The important point for this article however is that in this data model rather than there being just a single type of connection (as is the case on the present Web which basically just provides the HREF hotlink, which simply means "A and B are linked" and may carry minimal metadata in some cases), the Semantic Web enables an infinite range of arbitrarily defined connections to be used. The meaning of these connections can be very specific or very general.
For example one might define a type of connection called "friend of" or a type of connection called "employee of" -- these have very different meanings (different semantics) which can be made explicit and also machine-readable using OWL. By linking a page about a person with the "employee of" link to another page about a different person, we can express that one of them employs the other. That is a statement that any application which can read OWL is able to see and correctly interpret, by referencing the underlying definition of "employee of" which is defined in some ontology and might for example specify that an "employee of" relation connects a person to a person or organization who is their employer. In other words, rather than just linking things with the generic "hotlink" we are all used to, they can now be linked with specific kinds of links that have very particular and unambiguous meaning and logical implications.
This has the potential at least to dramatically enrich the information-carrying capacity of connections (links) on the Web. It means that connections can carry more meaning, on their own. It's a new place to put meaning in fact -- you can put meaning between things to express their relationships. And since connections (links) far outnumber objects (information, people or applications) on the Web, this means we can radically improve the semantics of the structure of the Web as a whole -- the Web can become more meaningful, literally. This makes a difference, even if all we do is just enrich connections between gross-level objects (in other words, connections between Web pages or data records, as opposed to connections between concepts expressed within them, such as for example, people and companies mentioned within a single document).
Even if the granularity of this improvement in connection technology is relatively gross level it could still be a major improvement to the Web. The long-term implications of this have hardly been imagined let alone understood -- it is analogous to upgrading the dendrites in the human brain; it could be a catalyst for new levels of computation and intelligence to emerge.
It is important to note that, as illustrated above, there are many types of connections that involve people. In other words the Semantic Web, and Web 3.0, are just as much about people as they are about other things. Rather than excluding people, they actually enrich their relationships to other things. The Semantic Web, should, among other things, enable dramatically better social networking and collaboration to take place on the Web. It is not only about enriching content.
Now where will all these rich semantic connections come from? That's the billion dollar question. Personally I think they will come from many places: from end-users as they find things, author content, bookmark content, share content and comment on content (just as hotlinks come from people today), as well as from applications which mine the Web and automatically create them. Note that even when Mining the Web a lot of the data actually still comes from people -- for example, mining the Wikipedia, or a social network yields lots of great data that was ultimately extracted from user-contributions. So mining and artificial intelligence does not always imply "replacing people" -- far from it! In fact, mining is often best applied as a means to effectively leverage the collective intelligence of millions of people.
These are subtle points that are very hard for non-specialists to see -- without actually working with the underlying technologies such as RDF and OWL they are basically impossible to see right now. But soon there will be a range of Semantically-powered end-user-facing apps that will demonstrate this quite obviously. Stay tuned!
Of course these are just my opinions from years of hands-on experience with this stuff, but you are free to disagree or add to what I'm saying. I think there is something big happening though. Upgrading the connections of the Web is bound to have a significant effect on how the Web functions. It may take a while for all this to unfold however. I think we need to think in decades about big changes of this nature.
The Business 2.0 Article on Radar Networks and the Semantic Web just came online. It's a huge article. In many ways it's one of the best popular articles written about the Semantic Web in the mainstream press. It also goes into a lot of detail about what Radar Networks is working on.
One point of clarification, just in case anyone is wondering...
Web 3.0 is not just about machines -- it's actually all about humans -- it leverages social networks, folksonomies, communities and social filtering AS WELL AS the Semantic Web, data mining, and artificial intelligence. The combination of the two is more powerful than either one on it's own. Web 3.0 is Web 2.0 + 1. It's NOT Web 2.0 - people. The "+ 1" is the addition of software and metadata that help people and other applications organize and make better sense of the Web. That new layer of semantics -- often called "The Semantic Web" -- will add to and build on the existing value provided by social networks, folksonomies, and collaborative filtering that are already on the Web.
So at least here at Radar Networks, we are focusing much of our effort on facilitating people to help them help themselves, and to help each other, make sense of the Web. We leverage the amazing intelligence of the human brain, and we augment that using the Semantic Web, data mining, and artificial intelligence. We really believe that the next generation of collective intelligence is about creating systems of experts not expert systems.
Give me a tip!
Recent Comments