Skip to content

Posts tagged ‘Innocentive’

Is Too Much Familiarity Bad For Creativity?

Originally posted on

Several years ago University of California at Davis professor Dean Simonton conducted a study that examined more than three hundred creative geniuses born between 1450 and 1850. The list included thinkers Liebniz and Descartes, scientists Newton and Copernicus and artists Vinci and Rembrandt. He compared the relationship between their education and eminence, a metric he determined by an array of criteria. He plotted the graph and found an inverted U sparking the following conclusion: “The most eminent creators were those who had received a moderate amount of education, equal to about the middle of college.”

Simonton’s research highlights a commonly held notion: too much familiarity can be detrimental to creativity. The problem, Simonton hypothesizes, is that creativity benefits from an outsider’s mindset. “Too much experience…” on the other hand, “may restrict creativity because you know so well how things should be done that you are unable to escape to come up with new ideas.” It seems reasonable, then, to suggest that, “if you want a creative solution to a problem, you’d better find someone who knows a little about the situation but not too much.”

Consider the clever website The premise is simple: ‘Seekers’ go to post problems for ‘Solvers.’ The problems range from “Recovery of Bacillus Spore from Swabs,” to “Blueprints for a Medical Transportation Device for Combat Rescue.” They are usually posted by large corporations, so the rewards can be lucrative – sometimes millions of dollars.

There are two things remarkable about InnoCentive, each brought to light by astudy conducted by researchers at Harvard Business School. The first is that it works; about 33 percent of the problems are solved on time. The second is that solvers tend to solve problems that are at the fringe of their expertise. If a biochemistry problem only attracted biochemists it tended to remain unsolved. But if the same problem was tackled by, say, a molecular biologist or an organic chemist the chances were greater that the problem would be solved. Outside thinking was vital.

Think about the failures of expertise, as the author of Talent is Overrated Geoff Colvin does: “Why didn’t Western Union invent the telephone? Why didn’t U.S. Steel invent the minimill. Why didn’t IBM invent the personal computer? Over and over, the organizations that knew all there was to know about a technology or an industry failed to make the creative breakthrough that would transform the business.”

Is too much expertise killing creativity?

Well, not exactly. Colvin goes on to remind readers that the greatest innovators of any field share a few characteristics in common: years of intensive preparation and technical competence. Great innovations, he says, are roses that bloom after long and careful cultivation.

He considers James Watson’s and Francis Crick’s discovery of the structure of DNA. Colvin cites the research of Robert Weisberg, who showed that several other distinguished scientists were trying to solve the same problem at the same time. Colvin argues that, “if we presume that too much familiarity with a problem is a disadvantage, then we would expect to find that Watson and Crick came at this one unburdened by the excessive data that clouded the thinking of the other researchers. But in reality, the story was just the opposite.”

The larger point is that creative breakthroughs require about 10,000 hours or ten years of deliberate practice within a given field:

The most eminent creators are consistently those who have immersed themselves utterly in their chosen field, have devoted their lives to it, amassed tremendous knowledge of it, and continually pushed themselves to the front of it. Zero evidence supports the conclusion that too much knowledge might be a hindrance in creative achievement.

And what about the success of InnoCentive? What’s important is not to be an outsider, but to have an outsider’s mindset. People at the fringe of their expertise solved problems on InnoCentive, but they were still solving problems within their general field of expertise. Indeed, innovation occurs at the boundary of disciplines, but you’ll never hear about a novelist winning a Nobel Prize in physics.

As for Simonton’s study, it’s important to remember that during the period that his subjects existed – 1450-1850 – many fundamental principles of the scientific method were still unknown. It was still possible – especially in the first half of that 400 year stretch – for someone to be an expert in multiple disciplines. Moreover, a high-level degree in, say, 1650, didn’t confer much.

Today’s landscape is much different – all the low hanging fruit is good. A breakthrough in any field requires exclusive preparation in that field; even experts don’t know everything about their field. So it’s important to maintain a skeptical point of view and think like an outsider. But when it comes to creative breakthroughs, the more familiarity the better.

Jonah Lehrer and the New Science of Creativity

The following is a repost of my latest article originally posted on It is a review of Jonah Lehrer’s latest book, Imagine: How Creativity Works, which was released March 19th.

Bob Dylan was stuck. At the tail end of a grueling tour that took him across the United States and through England he told his manager that he was quitting music. He was physically drained – insomnia and drugs had taken their toll – and unsatisfied with his career. He was sick of performing “Blowin’ in the Wind” and answering the same questions from reporters. After finishing a series of shows at a sold out Royal Albert Hall in London, he escaped to a cabin in Woodstock, New York to rethink his creative direction.

What came next would change rock ‘n’ roll forever. As soon as Dylan settled into his new home he grabbed a pencil and started writing whatever came to his mind. Most of it was a mindless stream of conscious. “I found myself writing this song, this story, this long piece of vomit, twenty pages long,” he once told interviewers. The song he was writing started like any children’s book – “Once Upon a Time” – but what emerged was a tour de force that left people like Bruce Springsteen and John Lennon in awe. A few months later, “Like A Rolling Stone” was released to critical acclaim.

Creativity in the 21st Century

Every creative journey begins with a problem. For Dylan it was the predictability and shallowness of his previous songs. He wasn’t challenging his listeners enough; they were too comfortable. What Dylan really wanted to do to was replace the expected with the unexpected. He wanted to push boundaries and avoid appealing to the norm; he wanted to reinvent himself.

For most of human history, the creative process has been associated with higher powers; it was about channeling the muses or harnessing one’s inner Apollonian and Dionysian; it was otherwordly. Science has barely touched creativity. In fact, in the second half of the 20th century less than 1 percent of psychology papers investigated aspects of the creative process. This changed in the last decade. Creativity is now one of the most popular topics in cognitive science.

The latest installment is Jonah Lehrer’s Imagine: How Creativity Works – released today. With grandiose style á la Proust Was Neuroscientist and How We Decide, Lehrer tells stories of scientific invention and tales of artistic breakthroughs – including Dylan’s – while weaving in findings from psychology and neuroscience. What emerges from his chronicles is a clearer picture of what happens in the brain when we are exercising – either successfully or unsuccessfully – our creative juices. The question is: what are the secrets to creativity? 

How To Think

There’s nothing fun about creativity. Breakthroughs are usually the tale end of frustration, sweat and repeated failure. Consider the story of Swiffer. Back in the 1980s Procter and Gamble hired the design firm Continuum to study how people cleaned their floors. The team “visited people’s homes and watched dozens of them engage in the tedious ritual of floor cleaning. [They] took detailed notes on the vacuuming of carpets and the sweeping of kitchens. When the notes weren’t enough, they set up video cameras in living rooms.” The leader of the team, Harry West, described the footage as the most boring stuff imaginable. After months of poring through the tapes he and his team knew as much about how people cleaned their floors as anybody else – very little.

But they stuck with it. And eventually landed on a key insight: people spend more time cleaning their mops than they did cleaning the floor. That’s when they realized that a paper towel could be used as a disposable cleaning surface. Swiffer launched in the spring of 1999 and by the end of the year it generated more than $500 million in sales.

Discovery and invention require relentless work and focus. But when we’re searching for an insight stepping back from a problem and relaxing is also vital; the unconscious mind needs time to mull it over before the insight happens – what Steve Berlin Johnson calls the “incubation period.” This is the story of Arthur Fry, which Lehrer charmingly brings to life.

In 1974 Fry attended a seminar given by his 3M colleague Spencer Silver about a new adhesive. It was a weak paste, not even strong enough to hold two pieces of paper together. Fry tried to think of an application but eventually gave up.

Later in the year he found himself singing in his Church’s choir. He was frustrated with the makeshift bookmarkers he fashioned to mark the pages in his hymnal book; they either fell out or got caught in the seams. What he really needed was glue strong enough so his bookmarkers would stick to the page but weak enough so they wouldn’t rip the paper when he removed them. That’s when he had his moment of insight: why not use Silver’s adhesive for the bookmark? He called it the Post-it Note.

Fry’s story bodes well with tales of insight throughout history. Henrí Poincaré is famous for thinking up Non-Euclidean geometry while boarding a bus, and then there’s Newton’s apple-induced revelation about the law of gravity. Lehrer delves into the relevant research to make sense of these stories from the neurological level. Fascinating studies from Mark Jung-Beeman, John Kounios and Joy Bhattacharya give us good reason to take Lehrer’s advice: “Rather than relentlessly focusing, take a warm shower, or play some Ping-Pong, or walk on the beach.”

When it comes to the creative process, then, it’s important to balance repose with red bull. As Lehrer explains: “the insight process… is a delicate mental balancing act. At first, the brain lavishes the scarce resource of attention on a single problem. But, once the brain is sufficiently focused, the cortex needs to relax in order to seek out the more remote association in the right hemisphere, which will provide the insight.”

Other People

The flip side of the creative process is other people. The world’s great ideas are as much about our peers as they are about the individual who makes it into the textbook. To explore how the people around us influence our ideas Lehrer explains the research of Brian Uzzi who, a few years ago, set out to answer this question: what determines the success of a Broadway musical?

With his colleague Jarrett Spiro, Uzzi thoroughly examined a data set that included 2,092 people who worked on 474 musicals from 1945 to 1989. They considered metrics such as reviews and financial success and controlled for talent and any economic or geographic advantages – big New York City musicals would likely flub the data. They found that productions failed for two reasons. The first was too much like-mindedness: “When the artists were so close that they all thought in similar ways… theatrical innovation [was crushed].” On the other hand, when “the artists didn’t know one another, they struggled to work together and exchange ideas.” Successful productions, in contrast, found an even distribution between novelty and familiarity within its members. This is why West Side Story was such a hit: it balanced new blood with industry veterans.

This is what the website teaches us. InnoCentive is a website where, as Matt Ridley would suggest, ideas go to have sex. The framework is simple: “seekers” go to the website to post their problems for “solvers.” The problems aren’t trivial but the rewards are lucrative. For example, the Sandler-Kenner Foundation is currently offering a $10,000 reward for anybody who can create “diagnostic tools for identification of adenocarcinoma and neuroendocrine pancreatic cancer at early stages of development.” Another company is offering $8,000 to anyone who can prevent ice formation inside packages of frozen foods.

What’s remarkable about InnoCentive is that it works. Karim Lakhani, a professor at Harvard Business School, conducted a study that found that about 40 percent of the difficult problems posted on InnoCentive were solved within 6 months. A handful of the problems were even solved within days. “Think, for a moment,” Lehrer says, “about how strange this is: a disparate network of strangers managed to solve challenges that Fortune 500 companies like Eli Lilly, Kraft Foods, SAP, Dow Chemical, and General Electric—companies with research budgets in the billions of dollars—had been unable to solve.”

The secret was outside thinking:

The problem solvers on InnoCentive were most effective when working at the margins of their fields. In other words, chemists didn’t solve chemistry problems, they solved molecular biology problems, just as molecular biologists solved chemistry problems. While these people were close enough to understand the challenges, they weren’t so close that their knowledge held them back and caused them to run into the same stumbling blocks as the corporate scientists.

This is the lesson from West Side Story: great ideas flourish under the right balance of minds. John Donne was right: no man is an island.


There are so many wonderful nuggets to take away from Imagine, and Lehrer does an excellent job of gathering stories from history to bring the relevant psychological research to life. The few stories and studies I’ve mentioned here are just the tip of the iceberg.

When I asked him what the takeaway of his book is (if there could be just one) he said:

The larger lesson is that creativity is a catchall term for a bundle of distinct processes. If you really want to solve the hardest problems you will need all these little hacks in order to solve these problems. This is why what the cognitive sciences are saying about creativity is so important.

He’s right. We think about creativity as being a distinct thing and as people being either creative or not, but the empirical research Lehrer discusses tells a different story. Creativity engages multiple cognitive processes that anybody can access.

This is why Dylan’s story is so important: It’s the story of a musician being discontent with his creative direction, having a moment of insight, working tirelessly to bring this insight and new sounds to life to ultimately change the norm. Dylan’s genius isn’t about a specific skill that nobody else possessed. It’s about his ability to wage through the creative process by using the right parts of the brain at the right times.

Not everybody can be Dylan, but Imagine reminds us that as mysterious and magical as creativity seems, “for the first time in human history, it’s possible to learn how the imagination actually works.”

Why Intellectual Diversity Is Important

Below is my latest column at The Creativity Post in its entirety. I argue that good ideas benefit from intellectual diversity. Incidentally, I came across this wonderful NYTimes article on the same subject at Farnam Street blog this morning. It discusses Scott Page’s The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools and Societies.

A few years ago Brian Uzzi of Northwestern University and Jarrett Spiro of Stanford University set out (pdf) to answer this question: What determines the success of a Broadway musical? Uzzi and Spiro began by poring through a data set that included 2,092 people who worked on 474 musicals from 1945 to 1989. To determine how good each production was they considered metrics such as reviews and financial success. They also controlled for things like talent and economic and geographic conditions to ensure that the big New York City musicals didn’t flub the data.

What they found was that successful productions relied on two components: “The ratio of new blood versus industry veterans, and the degree to which incumbents involved their former collaborators and served as brokers for new combinations of production teams.” In other words, productions that worked found a balance between strong social ties and weak ones, rookies and veterans, familiarity and novelty. They weren’t flooded with a group of likeminded people but neither was everyone a stranger to each other. Uzzi and Spiro hypothesized that the reason intellectual diversity was important is because “small world networks that help to create success or failure in Broadway musicals… face liabilities in the realms of innovation and collaboration that impede their creating new, successful musical hits… too much small-worldliness can undermine the very benefits it creates at more moderate levels, due to a decrease in artists’ ability to innovate and break convention.”

What’s alarming about their conclusions is that a plethora of psychological data suggests that most of us balk when we are given the chance to connect with people who might not share similar intellects. Consider a study (pdf) done back in 2007 by Paul Ingram and Michael Morris at Columbia University. The psychologists gathered a group of executives and had them attend a cocktail mixer where the psychologists encouraged the executives to exchange ideas, network and meet new people. Like good behavioral scientists, Ingram and Morris weaseled microphones on all the nametags to record what was said. Prior to the “mixing” the executives stated that they wanted to “meet as many different people as possible” or “expand their social network,” but the Ingram and Morris found just the opposite. “Do people mix at mixers? “ they asked in the concluding remarks of their study, “The answer is no… our results show that guests at a mixer tend to spend the time talking to the few other guests whom they already know well.” Or, as Jonah Lehrer somewhat sarcastically puts it in a recent post, “investment bankers chatted with other investment bankers, and marketers talked with other marketers, and accountants interacted with other accountants.”

Ingram and Morris’ study should be taken as a warning: If we want to broaden our intellectual horizons it’s important to remember our natural tendency to drift towards and eventually connect with only likeminded people. Stories of innovation and discovery throughout history illustrate how important this point is. My favorite, which doesn’t get told enough, is the discovery of Cosmic Microwave Background Radiation (CMB), a key piece of evidence that changed our understanding of the origin of the universe forever.

The story begins in Holmdel New Jersey at Bell Labs where Arno Penzias and Robert Wilson were experimenting with a horn antenna originally built to detect radio waves that bounced off of echo balloon satellites. After spending some time with the antenna they ran into a problem. It was a mysterious hissing noise – like static on the radio – that persisted all over the sky, day and night. The duo went to great lengths to eliminate the hiss – they even washed bird droppings off of the dish – but it was all to no avail. Meanwhile, at Princeton University just 60 miles down the road, Robert Dicke, Jim Peebles and David Wilkinson were trying to find evidence for the Big Bang in the form of microwave radiation. They predicated that if the Big Bang did in fact take place it must have scattered an enormous blast of radiation throughout the universe much like how a rock thrown into a lake creates ripples that broadcast outwards. With the right instrumentation, they believed, this radiation could be all over the sky, day and night.

It was only a matter of time before serendipity set in and a mutual friend at MIT, professor of physics Bernard F. Burke, told Penzias about what the researchers at Princeton were looking for. After that, the two teams exchanged ideas and realized the implications of their work. It turned out that the hiss that Penzias and Wilson were trying so hard to get rid of was precisely the radiation that the Princeton team was looking for. A few calculations and a published paper later landed Penzias and Wilson the 1978 Noble Prize in Physics; the rest of us are still repeating the benefits of a more complete understanding of the universe.

The story of CMB reminds us that when it comes to solving difficult problems a fresh set of eyes, even one that comes from a different field, is vital. The CMB story shows itself in one form or another many times throughout history. The world’s great ideas are as much about other people as they are about the individual who makes it into the textbook. As Matt Ridely explains in a TED lecture in a slightly different context, “what’s relevant to a society is how well people are communicating their ideas and how well they are cooperating not how clever the individuals are… it’s the interchange of ideas, the meeting and mating of ideas between that [causes]… innovation.”

There is a wonderful website called that facilitates what Ridley calls the meeting and mating of ideas. The framework of InnoCentive is quite simple: “seekers” go to the website to post their problems for “solvers.” Problems range from the “Recovery of Bacillus Spore from Swabs,” to “Blueprints for a Medical Transportation Device for Combat Rescue,” and multi-billion dollar companies like General Electric and Procter and Gamble often post them with cash prizes up to $1 million.

The amazing part is that it’s working. A study (pdf) by researchers at Harvard Business School found that about 33 percent of problems posted on InnoCentive were solved on time. Why does InnoCentive work? The same reason that successful Broadway plays do and CMB was discovered: intellectual diversity. If an organic chemistry problem only attracted organic chemists it tended to be troublesome. However, if a biologist got involved with that same problem then the chances were greater that the problem was solved. The implications of this should make you think: solvers were at their best when they were at the margins of their fields of expertise.

Maybe it sounds obvious to suggest that a proper mixture of minds is important for accomplishing tasks, but remember the lesson from Uzzi’s and Spiro’s cocktail party study: it’s really hard to not surround yourself with people like you. Don’t hang out with too many opposites though, we don’t want another Spider Man: Turn Off The Dark.

%d bloggers like this: