Harnessing the Wisdom of Crowds

We’ve heard a lot about collective intelligence, Web 2.0.  Internet-based examples abound: Wikipedia, Google, Threadless.  Thomas Malone and colleagues have written a new article proposing an analytic framework to help us think about these networks, what they call the “building blocks” or “genes” of collective intelligence.  Of course, I wanted to see how their framework compares to my own “collaborative web” model from Group Genius.  Based on a study of 250 examples of web communities, here’s what they propose.

The four building blocks are answers to these questions: Who is doing the task? Why? What is being done? How?  How these questions are answered determines which “gene” it is.

The Who question has two genes: (1) Hierarchical organization determines who.  (2) Crowd gene: Anyone can participate.

The Why question has three genes: (1) Money (2) Intrinsic motivation of the task (3) fame or reputation.

The What question has two genes: (1) Create something new; (2) Select among alternatives.

The How question has two genes for each of the what genes, depending on whether it is independent or collective.  For “Create” the two genes are Collect independent contributions and Collaborate together.  For “Select” the two genes are Group decisions (Aggregate individual group decisions by voting or consensus etc.) and Individual decisions, through markets or social networks.

Then continuing the biological analogy, they analyze specific examples of web-based collaboration and call them “genomes.”

This is a useful way of breaking down different web-based communities, although I found it not very surprising or new.  The key thing that’s missing from this model is what I think is the biggest challenge facing such communities: What is the right degree of central control and structure?  Communities with no central structure are usually a huge mess.  Linux succeeds only because of a strong central guiding body, led by Linus Torvalds.  The model presented in this article seems most appropriate for tasks where no central control is necessary, where small items are created that don’t need to coordinate with each other in complex systems (in Wikipedia each entry stands alone; with Threadless, each t-shirt design is independent).  But with Linux, everything has to work together.  That’s a key variable missing from this model, but I have no doubt Malone and colleagues are aware of this and are thinking about it. (The 7/19/2009 NYTimes article by Steve Lohr, where Prof. Malone was interviewed, explicitly addresses this topic.)  Perhaps another paper will emerge from the same research study.

*Thomas W. Malone, Robert Laubacher, and Chrysanthos Dellarocas. February 2009.  Harnessing Crowds: Mapping the Genome of Collective Intelligence.

Innovation in the 19th Century

I’ve just been reading a fascinating new book, Structures of Change in the Mechanical Age, by Ross Thompson (published 2009).  It’s a fairly detailed study of technological innovation from 1790 through 1865, in a range of industries.  Several histories have been written about how innovation systems emerged in the late 19th century: systems that included universities and technical institutes, research laboratories, and government agencies.  Thomas Edison’s research laboratory, with its structured approach to invention, flourished in the late 19th century.  But Thompson’s book is the first one to examine the innovation systems of the antebellum U.S.

The conclusion that I find the most intriguing is that, even in this early period, innovation occurred more rapidly in collaborative webs–networked groups of creators. As Thompson writes, “early in the development of any innovation, inventors and users formed networks that communicated technological knowledge and addressed problems….Networks sped diffusion by building on already high mobility among firms…For the economy as a whole, innovation consisted of a number of paths, each resting on distinct knowledge transmitted in different networks” (p. 315).

Thompson studied thirteen technologies that experienced significant patenting during this period: 1660 inventors from 1836 through 1865 that received over 6,900 patents.  The inventors who were networked with others were about two-fifths of all inventors.  The networked inventors averaged 2.8 patents whereas the non-networked inventors averaged only 1.9.  Many innovations occurred when inventors moved from one field to another. Machinists spread machine-tool techniques as they moved among industries (locomotives, sewing machines, and shoe machines). Engineers used canal methods to build railroads (p. 316).  I leave you with this important statement:

“The mobility of workers supported development in a wide range of sectors.” (p. 316)

Innovation at Google

Google has a famous strategy for innovation: Give each engineer one day every week to work on blue-sky, big potential ideas of their own choosing, and only require them to work on their “official” assigned project four days each week.  When I describe this to executives, I often get this question: “Why wouldn’t the engineer leave and start up his own company with that great idea? How does Google make sure they benefit from this investment in their staff?”

One way they might do it is by making their engineers sign some sort of agreement stating that “any ideas developed while employed at Google are Google property” etc. etc.   So far, I’ve seen no evidence that Google has done this.  (If they do, it’s never appeared in print.)  And in fact, Google does have a problem with employees leaving the company and taking their ideas with them; this was the focus of a recent Wall Street Journal article titled “Google searches for ways to keep big ideas at home”*.  Three start-ups that emerged from Google include Ooyala, Aardvark, and FriendFeed.

Google has never had a formal process for senior executives to review the blue-sky ideas that engineers work on one day a week.  So a lot of those ideas fail to move forward, sometimes because they require resources that only senior management can allocate.  So now, as the WSJ article reports, Google has initiated “innovation reviews” where each executive presents the most promising ideas from within their own division.  CEO Eric Schmidt is there to listen, and so are the founders Page and Brin.

The problem is in getting a promising idea from the early stage, to the next stage where it’s developed enough to see how a prototype might work.  And typically, that doesn’t happen without some management focus and some allocation of resources.  So there’s a new consensus at Google that too few of their one-day-a-week ideas are turning into blockbusters.

Another way Google is addressing the problem is to give a few engineers, with extremely promising ideas, even more than one day a week–in some cases, giving them full-time to work on their idea.  The new collaboration tool, Google Wave, resulted when two brothers in Australia were told to go all-out on their idea, a new communication system to replace email.  Top leaders also assigned the two brothers dozens of employees.

Now that’s something you wouldn’t get if you left the company with your idea–unless you got some very deep-pocketed venture capitalists to fund you.  That’s a good argument to stay at Google; as CEO Eric Schmidt puts it, to be “part of a start-up within Google.”  But it can’t happen without adding some structure, criteria, and resource allocation mechanisms to their innovation process.

*Jessica E. Vascellaro, WSJ, June 18, 2009, p. B1, B5: “Google searches for ways to keep big ideas at home”.

Teaching Creativity

I just returned from a visit to The Teaching Company.  You may have seen their ads in the Wall Street Journal or any number of other publications; they develop and sell videotape and audiotape courses by the U.S.’s best professors.  I was honored to be invited there to do what they call an “audition lecture” of 30 minutes.  We’re talking about developing a 24-lecture course in “cultivating creativity” that is research based.  What would you like to see in such a course?

My first thoughts are to use the outline of my book Explaining Creativity, an overview of the research about creativity.  I’ve just signed with Oxford University Press to write a second edition of this book.  Although it was just published in 2006, the research has progressed rather rapidly since then, and there are enough new findings that a second edition is necessary.

And on a related note: I’ll be on sabbatical from Washington University for the next 12 months, and will be in residence at three different locations:

1. Four weeks this summer at the San Francisco Exploratorium;

2. Three months, the fall term, at the University of Cambridge, England;

3. January through May 2010 at the Savannah College of Art & Design.

I plan to put these three experiences together and write a book about how to teach creativity.