Collaborative Technology Leads to Collaborative Leadership

In my 2007 book Group Genius, I predicted that the organization of the future would drive innovation with collaboration.

In the ten years since, this prediction has largely come true. Yesterday the Wall Street Journal described how several big companies have shifted to a more collaborative, more innovative organizational structure–enabled by collaborative software that didn’t exist back in 2007, like Slack or Microsoft Teams. This is a big reason why I’ve written a second edition of Group Genius (to be published later this year).

New data-driven capabilities are breaking down barriers between formerly siloed business units, flattening out management structures and streamlining production processes, prompting many firms to redraw leadership roles and responsibilities.

Companies moving toward innovative structures include Equifax, Liberty Mutual, and Procter & Gamble. For example, Equifax is moving to “small, cross-functional teams”. And the role of leaders changes, too: “rather than issue top-down directives, these managers instead strive to help self-directed teams leverage collaboration and sharing tools.” Managers are changing from “dictating how things should be done” to acting more like coaches who guide collaborative teams.

My own research on collaboration and creativity explains why and how this works: Innovation emerges, bottom up, from improvisational, nonlinear, and unpredictable processes. The organizations that can channel and foster this bottom-up, emergent process, will be the winners in the innovation competition of the future.

The organizational structures and cultures that lead to innovation have always been collaborative, distributed, and improvisational. Even before the Internet, a few rare organizations were able to design for innovation and collaboration. But today, Internet-based collaboration software is making it a lot easier for companies to shift to innovative organization designs.

The Emergence of Creativity: Matt Ridley’s New Book

You’ve got to read the excerpt from Matt Ridley’s new book in today’s Wall Street Journal. Just released this week, his book is called The Evolution of Everything: How New Ideas Emerge. I have a lot of respect for his previous books, so I’m delighted to learn that his new book makes the same points as my 2007 book Group Genius.

Here are the key features of innovation, described in both of our books:

  • The stories we hear about genius inventors, like Thomas Edison inventing the light bulb, are always myths. Ridley and I both describe the real history of the light bulb, which involves lots of people way before Edison. (Group Genius, pages 110, 196)
  • “Innovation emerges from the bottom up,” I write in Group Genius  (page 16). I show that innovation emerges from self-organizing systems, and this is Ridley’s main point, too.
  • Ridley writes that innovation is “incremental” rather than “revolutionary.” That’s why I called one of my chapters “Small Sparks”: to emphasize that innovation doesn’t come from a big flash of insight. “Successful creators know how to keep their sparks coming in a process that unfolds over time” (Group Genius, page 97).
  • Ridley describes the historical research on multiple discovery, as I do on pages 192-193, with this example: “In the 1920s, numerous teams invented television in parallel.”
  • Ridley argues that patent protection is too broad and is based on the mythical view of the lone inventor. I make the same point on pages 176-224, especially pages 221-225: “Current policy favors linear, centralized innovation and blocks the natural rhythm of innovation”.
  • Ridley demolishes the idea that innovation comes from a linear process; this is the most important point of Group Genius  (for example, pages 158-159, “Beyond Linear Innovation”)

Ridley’s WSJ  excerpt is filled with great stories of real innovations. I come to the same conclusions, with some of the same historical examples, and also by drawing on the science of creativity. Inspired by my studies of jazz and improv theater, I think of creativity as improvisation. Group Genius argues that the most creative improvisations are non-linear, emergent, unpredictable, and inefficient. Ridley has a bit more to say about the political and economic implications of this new, more realistic, understanding of innovation (for example, he concludes that government doesn’t need to fund scientific research). I have a bit more to say about how you can use this research to become more successfully creative, both on your own and in teams. It’s cool that Ridley and I come to the same conclusions from really different directions. If you like Group Genius, you really should check out The Evolution of Everything. (I’ll post a review after I’ve read the whole book.)



Big Company Innovation Labs Won’t Work

Creativity research* has shown that all companies benefit from very similar innovation strategies, whether they’re technology companies or not. We mostly hear about software and Internet-based startups these days; and most incubator spaces (sometimes called “innovation labs”) are filled with smartphone apps and web developers.

Other industries are setting up innovation labs, and they almost always get built in San Francisco’s Bay Area. There’s nothing new here: back in the 1970s, Xerox, the copier company based in Rochester New York, decided to open its innovation lab in Palo Alto. Today’s Wall Street Journal lists a few of the companies who’ve created spaces in the Bay Area: Lowe’s, Home Depot, Target, Walgreens, Sears, Visa.

I predict these efforts won’t work very well. We already know why, from the legendary failure of Xerox PARC. Back in the 1970s, developers at Xerox PARC invented the first windows-and-mouse computer, the first laser printer, and the first network to link computers (the Ethernet). It was called Smalltalk, and it was at decades ahead of its time. Both Microsoft and Apple based their windows operating systems on what came out of Xerox PARC. But the executives back in Rochester thought they were a bunch of crazy hippies and they said “Hey, we’re a copier company, why are you guys wasting your time on this stuff?” The book Fumbling the Future pretty much says the same thing that I say in my book Group Genius, and today’s Wall Street Journal identifies the key problem: if you create a separate R&D group, to keep the innovative people from being constrained by the traditional company culture, you also isolate the rest of the company from innovation. The labs are just too far removed; different organizational cultures develop; the innovation group just can’t communicate with the rest of the company.

Nordstrom, one of the earliest companies to build an innovation lab (in 2010) found this out. They’ve now shrunk their lab dramatically, and instead have spread innovators throughout the company. Another example: Amazon’s Silicon Valley innovation center failed to meet expectations.

I explain why in Group Genius: For successful innovation, you have to spread a culture of creativity throughout the organization. Creating a separate innovation lab doesn’t work.  It’s just a trendy name for what used to be called the R&D group. We learned that didn’t work back in the 1970s and 1980s. Calling it an “innovation lab” doesn’t make any difference in the underlying dynamics of innovation.


*Sawyer, Group Genius: The Creative Power of Collaboration.

The Pixar Film “Inside Out”: The Zigzag Creative Process Succeeds Again!

Pixar’s creative process has always followed a zig-zagging, improvisational process–one that’s perfectly aligned with the lessons emerging from creativity research. All of the research is showing that effective creativity, and exceptional creators, all follow an improvisational process: you don’t know where you’re going to end up, or when you’ll get there. It takes a while (and a few successes) to learn to trust in a process It’s because at first, it can feel like aimless failure.

Pixar, the animated movie powerhouse, has stayed true to the zigzag process, because they know their successes emerged from it. From their first hit, Toy Story (with its many twists and turns documented in the book The Pixar Touch), all the way through Frozen, creative success has emerged from this unpredictable, wandering process.

Pixar’s latest movie, arriving in theaters June 19, is Inside Out, and it’s been in process since 2010. It started with the thinnest of premises: we’ll go inside of a pre-adolescent girl’s head, and we’ll personify each of the emotions she feels every day, showing Sadness and Joy (for example) as cartoon characters. Think about that very simple idea, and ask yourself: How would you make a movie out of that? What happens at Pixar is that they start without knowing how it’s going to end. They start working it out, and then expect frequent changes to happen along the way.

It looks like the zigzag process has worked yet again: The bittersweet movie got a huge positive response from critics at the Cannes Film Festival, it’s expected to make $250 million just in the USA, and it’s already being discussed as an Oscar contender.

John Lasseter, in an article in today’s New York Times, says “We’re always tearing up work and starting over. At Pixar, we trust our process.” The article mentions several zigzags: for example, they were going to have the girl’s character go into a deep depression, but as they worked this idea out, they realized “that was not appropriate” says Pete Doctor, the director of the new movie (and also of the Oscar-winning “Up”). Another zigzag: One version of the script had Joy and Fear getting together. They worked for months, but couldn’t quite make this plot work. Eventually, they decided to turn to Sadness and give her a key role, when everyone had previously been leaving Sadness to be a peripheral character. This unexpected zigzag turned out to work surprisingly well.

Not many movie studios can afford, or can trust, a director to take four or five years to go through the zigzags that the creative process requires. But there aren’t any shortcuts; this is how you get surprisingly original creativity.

Marc Andreessen on Group Genius

Marc Andreesen, the Silicon Valley investor, has just published a free e-book containing his blog posts from 2007 to 2009. This excerpt from the book was just published in the Wall Street Journal–titled “retaining great people”. It’s really advice about how to build an innovative organization:

Don’t create a new group or organization within your company whose job is “innovation”. This takes various forms, but it happens reasonably often, and it’s hugely damaging. It sends the terrible message to the rest of the organization that you think they’re the B team. Instead, focus on boosting the innovation culture of the entire company.*

How do you get the whole company to innovate? I give the answer in my 2007 book, Group Genius:

Many companies say that they believe in empowering their employees through participation. But too often, participation is little more than a strategy to increase employee job satisfaction or to get their buy-in for senior management decisions. Real participatory companies are collaborative, improvisational, emerging from the bottom up. It’s a radical rethinking of the organization, and most companies aren’t willing to go there just yet. But as innovation becomes ever more important, there won’t be any other choice. (p. 155)

The reason you have to spread innovation throughout the organization is because innovation today isn’t linear. That’s why you can’t separate out the “idea stage” from the “execution stage”:

The skunk works model places all its hopes on one big flash of inspiration that must come from a select group of special people. In fact, successful innovative companies keep these small sparks coming from individuals throughout the organization, each spark inspiring the next one. (p. 159)

In Group Genius, I tell you the ten features of the most innovative organizations, grounded in this emergent, up bottom approach to innovation. That’s why Andreesen is right; not (only) because the skunk works approach damages morale, but because it never actually generates innovation.

*Andreeasen, The Pmarca Blog Archives.

Group Genius: Radical in 2007, Conventional Wisdom Today

In 2007, my book Group Genius made a radical claim: The discipline of psychology could never explain creativity, because creativity emerges from collaborative groups and networks. In 2007, this put me at odds with most of my creativity research colleagues; they studied solitary individuals. And it was a bit cutting edge for the business world, too; most business books were still focused on enhancing the creative potential of each employee:

We’re drawn to the image of the lone genius whose mystical moment of insight changes the world. But the lone genius is a myth; instead, it’s group genius that generates breakthrough innovation. Collaboration drives creativity because innovation always emerges from a series of sparks–never a single flash of insight. (p. 7)

My timing turned out to be perfect for the business world. In 2007, top executives were beginning to realize that collaboration was the key to innovation. They were eager to learn about my seven key characteristics of effective creative teams and companies:

  1. Innovation emerges over time
  2. Successful collaborative teams practice deep listening
  3. Team members build on their collaborator’s ideas
  4. Only afterwards does the meaning of each idea become clear
  5. Surprising questions emerge
  6. Innovation is inefficient
  7. Innovation emerges from the bottom up

In recent years, several new books have appeared that reinforce my argument: Great creativity always emerges over time, from collaborative pairs, teams, and distributed networks. I’ve just read two wonderful books that make a particularly strong case for collaborative creativity:

Both books are wonderfully written. They are true to the science and the historical record. Each of them have turned up surprising and little-known details about creativity. If you read these books, along with Group Genius, you’ll have a really good understanding of what science has discovered about innovation.

Steven Johnson, Where Good Ideas Come From

Johnson’s central claim is that good ideas don’t come from inside some genius’s brain:

If we want to understand where good ideas come from, we have to put them in context. The argument of this book is that a series of shared properties and patterns recur again and again in unusually fertile environments. (p. 17)

In the last three chapters of Group Genius, I describe the “collaborative webs” that foster innovation, and the characteristics of environments that make them grow. Johnson’s book builds on my work, and adds in some really fascinating stories. (He comes to the same conclusion that I do about what sort of intellectual property law regime results in the greatest innovation.) Consistent with my seven points above, he argues that innovation emerges from tinkering and bricolage. The most innovative environments are like my collaborative webs:

Good ideas are not conjured out of thin air; they are built out of a collection of existing parts. (p. 35) [These environments have] a capacity to make new connections with as many other elements as possible. And a “randomizing” environment that encourages collisions between all the elements in the system. (p. 51) The most productive tool for generating good ideas remains a circle of humans at a table. (p. 61)

Johnson cites a lot of the same research that I do, and tells many of the same stories (Kevin Dunbar’s research; Gruber’s book about Darwin’s notebooks; brainstorming research; Burt’s research on structural holes; MIT’s Building 20). He echoes my concept of group flow with his term collective flow (both of us building on Dr. Csikszentmihalyi).

Johnson’s book is a fascinating read; he’s a great storyteller. In the last chapter, he comes to the same conclusion that I did in 2007:

A majority of breakthrough ideas emerge in collaborative environments. (p. 228)

Walter Isaacson, The Innovators

Isaacson’s book focuses more narrowly–on the technology innovations that resulted in today’s tablet, smartphone, networked world. We sometimes take this world for granted, but it didn’t exist just a few years ago. I’m surprised to see how well Isaacson’s book is selling, because it’s highly detailed and very focused. Maybe there are more nerds out there that I realized! Personally, I loved it, because I participated in this history. I arrived at MIT in 1978, and received my computer science degree in 1982. I did my undergraduate thesis on MIT’s version of the Xerox PARC Smalltalk computer, the LISP Machine, so I was using a windows and mouse interface as early as 1980. I played the original video game, Space War, in the MIT student center. I remember how cool it was to use the Arpanet and log in to computers all over the world (one country I remember logging into was Norway). There were no passwords and no security; when I wanted to read a draft of Professor Marvin Minsky’s new book, I just went into his personal file folders and read his drafts. I met Richard Stallman, who tried to get me to participate in his “Free Unix!” project. Isaacson’s book was perfect for me.

Chapter after chapter, he takes up the core innovations: Computer hardware. Software and programming. Microchips. Video games. The Internet. The personal computer. And every single one emerged from collaboration:

The main lesson to draw from the birth of computers is that innovation is usually a group effort, involving collaboration between visionaries and engineers, and that creativity comes from drawing on many sources. Only in storybooks do inventions come like a thunderbolt, or a lightbulb popping out of the head of a lone individual in a basement or a garret or a garage. (p. 85)

The formation of ideas was shaped more by the iterative interplay within the group than by an individual tossing in a wholly original concept. The sparks come from ideas rubbing against each other rather than as bolts out of the blue. (p. 110)

As with Johnson’s book, Isaacson tells several of the same stories I tell in Group Genius: Xerox PARC, Richard Stallman and GNU/Linux, how the windows-and-mouse interface emerged from successive incremental ideas. He comes to the same conclusion I did in 2007:

First and foremost is that creativity is a collaborative process. Innovation comes from teams more often than from the lightbulb moments of lone geniuses. (p. 479)

Like any author, I hope that my book stands the test of time. Group Genius contains many stories that aren’t in these books: The creation of the airplane, the mountain bike, the Monopoly boardgame, emergency and disaster response teams, Honda’s motorcycles, basketball teams, the ATM cash machine, Tolkien’s Lord of the Rings, and more. And, I tend to provide a bit more practical advice for how to use this research to be more creative. So if you like these two books, I hope you’ll read mine too!

It’s About the Group, Genius

But what about those moments when you have a sudden realization, you get an idea while taking a walk, you experience a flash of insight? Isn’t that still really about solitary processes within your own private brain? No:

Researchers have discovered that the mind itself is filled with a kind of internal collaboration, that even the insights that emerge when you’re completely alone can be traced back to previous collaborations. (Group Genius, p. xii)

Forget the myths about historical inventors; the truth is always a story of group genius. (Group Genius, p. xiii)

Standing against this new consensus about how creativity and innovation work, many of my creativity research colleagues remain focused on individual creativity. If you skim the pages of the Creativity Research Journal, you’ll see almost exclusively psychological research that focuses on mental processes inside the minds of solitary people. But this narrow focus is holding us back, as I write at the end of Group Genius:

If you believe that creativity is reserved for special geniuses, you’re more likely to think that you can’t be creative. If you believe that creativity is an unexplainable gift that happens in a magical flash of insight, you won’t invest in the hard and sustained work that it takes to generate a long string of small sparks. If you believe that creativity happens to nonconforming, solo operators, you won’t work together with others to build group genius. (p. 225-226)

We need an interdisciplinary science of creativity, one that brings together psychologists with scholars who study groups, teams, and collaborative webs in organizations. Here’s what I hoped for in my 2012 overview of creativity research, Explaining Creativity:

Creativity research in the future will be increasingly interdisciplinary, bringing together scientists who are experts in multiple levels of analysis–neurons, mental states, groups, and organizations. An interdisciplinary science of creativity has the potential to provide a more complete scientific explanation of how new things emerge from human activity. (pp. 432-433)

Other books about collaborative creativity

My 2007 book wasn’t the first to emphasis the power of collaboration. I built on prior work by adding insights from my own scientific research, on jazz ensembles and improv theater groups, using interaction analysis methodology, and I wove it together with some cool case studies. Prior books that I loved include:

Some books after 2008 that jive with Group Genius include:

Collective Genius: The Art and Practice of Leading Innovation

Collective Genius is a wonderful new book by a team led by Professor Linda A. Hill of Harvard Business School. Of course, I had to read it, because my own business book is called Group Genius. With such similar titles, it’s not surprising there’s a lot of overlap with my book: we cite a lot of the same research, we choose many of the same companies for our case studies, and we provide very similar advice for leaders. Here are some quotations that resonated with my research:

People apparently prefer to believe in the rugged individualism of discovery, perhaps because they rarely get to see the sausage-making process behind every breakthrough innovation. Three decades of research has clearly revealed that innovation is most often a group effort. [Here, they cite Group Genius and many other books with similar findings.] The process of innovation needs to be collaborative because innovations most often arise from the interplay of ideas that occur during the interactions of people with diverse expertise, experience, or points of view. Flashes of insight may play a role, but most often they simply build on and contribute to the collaborative work of others. (pp. 16-17)

What makes this new book somewhat different from mine is that this book is squarely targeted at managers–it provides very practical advice, in the three-bullet-point style common in executive education programs. For example, the three key things that effective leaders of innovation do are (1) create collaborative organizations, (2) foster discovery-driven learning, (3) support and encourage integrative decision making. Their main foil is the stereotype that the most effective leader is someone who has a strong vision, and then persuades everyone else to execute that vision. Instead, they argue,

Great leaders of innovation see their role not as take-charge direction setters but primarily as creators of a context in which others are willing and able to make innovation happen. (p. 225)

In Group Genius, I call this type of context the “collaborative web” and I provide advice on how to foster their emergence. Since Group Genius was published in 2007, I’ve done countless executive education workshops for corporate leaders, and as I’ve translated the research from my book into workable advice for executives, my own message has become much more focused on leadership practice. This book is very similar in style and spirit to my one-day workshops. Practicing executives will definitely benefit from this book, but if you have the money, you should fly in Linda Hill, or myself, to do a one-day workshop with your leadership team. I suspect this book started out as a one-day workshop and then, in conversation with a literary agent and an acquisition editor, it gradually grew into this book. As a fellow business book author, I’m particularly impressed by their long list of CEO endorsements, by folks like Tim Brown (IDEO), Reid Hoffman (LinkedIn), Terri Kelly (Gore), and Tony Hsieh (Zappos). If you’re looking for a good read for your next business flight, or you can’t afford our speaking fees, I definitely recommend this book.