Jul 22 2008
Collective Intelligence and Web 2.0
I have recently been exploring social networking sites and collective intelligence algorithms. I suppose I am of the generation that just missed out on the Facebook/MySpace phenomenon. But I have always found recommendation engines from places like Amazon and Netflix quite useful. They provide an indirect type of socially augmented intelligence by using algorithms to find people with similar tastes, and then rate or recommend items indirectly based on their recommendations and their similarity to your own tastes. Newer sites like del.icio.us and last.fm take this to an explicit next step. These types of social networking sites allow you to actively find not only items that interest you (web sites and music respectively), but also people with similar interests and tastes to form explicit and ad-hoc social networks. See the new items on my side bar which should follow my del.icio.us bookmarks and last.fm music interests and activity from these sites.
One of the themes of this blog is Metacortices, the idea that the collective cognition of many people can be harnessed to perform more intelligent behaviors than would be possible from any of the individuals. Social networks, as I am describing them, do fit this definition to a degree, though the intelligent behavior being generated is really one that benefits the individuals by helping them make better choices, and find more interesting stuff. Though, look at collective intelligence and social games research for examples where people are designing systems based on playing games that do appear to create behavior that is more intelligent than any of the individual participants might do on their own.
A related concept that comes to mind looking at my del.icio.us and last.fm logs is the lifelog (also known as lifestream or total history). This is more in the realm of building an Exocortex, the idea of a constantly created log or memory of activities, thoughts, findings, etc. The basic idea of a lifelog is a technologically enhanced long-term episodic memory. Episodic memory is a type of long-term memory recognized by psychologists and cognitive scientists. It is the type of memory that allows us to store and recall episodes in our life, like our last birthday party, or what we ate for lunch yesterday. A lifelog would consist of a technologically enhanced person with devices that capture all video, images, audio and other type of sensory inputs, and stores and processes these logs in ways that make them searchable. Imagine a camera in your glasses taking video at 10 frames per second, being automatically meta-tagged with gps location information of where the photos occurred, and associated through temporal tags with other streams, such as audio or other, possibly uploading them to a Flickr stream, with auto-generated tags, etc.. The link given previously of Stross’s ideas on this subject propose a rapidly approaching capability to log and process this level of data for individual people because of increasing computing storage and processing power and mobility.
A lot of this interest was because of a new book I have been reading Collective Intelligence by Tobey Segaran. I think this is a wonderful book, almost on the level of Norvig’s Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp, in terms of giving concrete and hands-on views of the important algorithms and methods behind these intelligent systems. My graduate AI course this fall of 2008 will be using this book as our text and will concentrate on machine intelligence and collective intelligence methods.