It may not have been apparent five or six years ago, but crowdfunded campaigns today provide developers great and small with exciting new avenues for generating public interest in video game projects. Among their most powerful features are their ability to bring communities of like-minded gamers together to support a common cause, as well as the ability to give gamers a greater sense of being involved with a project’s development as they form close relationships with developers. However, some crowdfunding projects also fall short of their goals and end up disappointing fans. The University of Pennsylvania and Kickstarter collaborated on an independent study earlier this month – titled “The Kickstarter Fulfillment Report” – which delves some of the psychological underpinnings behind what makes backers satisfied on Kickstarter, and why backers feel that many projects fail to deliver what they promise.

In this study, which began back in March of this year, Kickstarter asked Professor Ethan Mollick from the Wharton School of the University of Pennsylvania (which focuses on business and entrepeneurship) to survey 456,751 backers about “project outcomes and backer sentiment.” That is, Kickstarter wanted to survey how backers felt about supporting projects that users felt were “failures.” The study’s criterion for a failed project: “Projects were deemed failures if backers answered that they never expect to get the promised reward, or that they did receive a reward but it wasn’t what they were promised.” Mollick states in a disclosure that he was not paid by Kickstarter to conduct to study, and that he performed the research independently. His full report, titled “Delivery Rates on Kickstarter” and published on December 2nd this year, can be read here.

Mollick gathered data from backers who had supported 65,326 funded projects spanning April 2009 through May 2015. For each of these projects, “The set included all projects that raised more than $1,000, half of those that raised between $250 – $1,000, and a quarter that raised less than $250.”

His first finding: 9% of Kickstarter projects fail to deliver rewards. To clarify himself, Mollick states that a “failed” project does not mean that it never finished. A failed project was instead judged on its ability to deliver appropriate backer rewards.

For example, if a creator successfully finished a film funded through Kickstarter but backers didn’t get a copy of their DVD, the project was counted as “failed.”

While “8% of dollars pledged went to failed projects” and “7% of backers failed to receive their chosen reward,” backers generally felt positive about the projects they supported. According to Mollick, “65% of backers agreed or strongly agreed with the statement that ‘the reward was delivered on time’.” In this segment of the study, Mollick concluded that “Project backers should expect a failure rate of around 1-in-10 projects, and to receive a refund 13% of the time,” and that ultimately, neither failure nor fraud seems to be prevalent on Kickstarter. He is careful to note that his findings are only intended to apply to Kickstarter campaigns; other sites (like Indiegogo, for example) may use different funding paradigms that can affect how backers feel about their projects.

Mollick’s second finding: Failure rates are consistent across categories. By categories, Mollick refers to Kickstarter’s fifteen creative categories under which their projects fit (Art, Comics, Crafts, Dance, Design, Fashion, Film & Video, Food, Games, Journalism, Music, Photography, Publishing, Technology, and Theater). Failure rates for every category gravitate toward a stable 9.95%. To refine his results, Mollick split the term “failure” into three defined categories: Broad (at least one backer considered the project a failure), Middle (at least half of backers agreed the project was a failure), and Narrow (all backers agreed the project was a failure).

A chart for the failure rates can be seen below:

From https://www.kickstarter.com/fulfillment

Mollick explicates the 9.95% failure rate in his paper:

However, given that individual complaints are not uncommon, this is likely too harsh a definition. If instead, we classify projects where at least half of backers considered the project as a failure (which I will refer to as the “middle definition” of failure), the rate drops to 8.6%, and if we take the strict definition that all backers should consider the project a failure, the failure rate is 5.6%.

He concludes this section by stating that even these numbers might over-estimate failure rates, “as a project may have failed to deliver its reward, but still succeeded in accomplishing the goals of the creator.”

His third finding:
Projects that raise less than $1,000 fail the most often. Mollick does not give any reasoning on why smaller projects tend to fail the most in the eyes of their backers.

From https://www.kickstarter.com/fulfillment

His fourth and final finding: Failure does not always mean the project’s creative content turned out poorly. Backers seemed to be more forgiving than one would expect. About 73% of backers who supported a “failed” project said they would back another Kickstarter project. However, only 19% said they would back another project by creator whose project failed.

Mollick also found that “in 15-20% of cases, backers reported that failure was handled well by creators.” Of these backers, 13% reported that they received refunds or compensation from the creator. 17% stated that they understood why the project failed, meaning that the project creators clearly explained the reasons behind their missteps. This does not, of course, demonstrate that the backers were happy with the “failure” of the creators to appropriately distribute rewards.

Mollick’s conclusion: failure rates for Kickstarter projects hover at a relatively low 9%, with a possible range of about 5%-14%. Failures tend to be non-predictable as well, and often arise for reasons that neither the backers nor the creators could have foreseen. Moreover, backers can expect to receive a refund or compensation for a failed project around 13% of the time. He closes his study by stating that creators will need to account for these unpredictable failures when developing their projects, the most important function being the ability to keep lines of communication open with their backers.

Kickstarter thanked Mollick and its backers for participating in the survey. They then added that backers should not expect the site to act as a “store,” with guaranteed exchanges of goods between backers and creators. Kickstarter is a creative space that fosters innovation, and with innovation comes the potential for disappointment or failure. Mollick’s findings are likely reassuring for Kickstarter, though, since the relatively low failure rate means most of the site’s projects deliver satisfying products.

Creators of all kinds – including video game developers – would do best to keep the shadow of failure in mind when they launch crowdfunding campaigns, and they should be ready to maintain close communications with their communities if failure does arise. If there is one lesson crowdfunding campaigns have taught us as gamers, it’s that the game development process itself can bring communities and developers closer to one another as collaborators who conjure up fantastic worlds together.