image: kickstarter

Learning from Kickstarter and predictive models

Kickstarter’s transparency in providing detailed data about its projects provides the basis for many kinds of predictive model, and which are worth thinking about.

A few months ago, my friends at BigML came up with a machine learning model based on 17,000 campaigns that illustrated the importance of factors such as the number of people who make donations (backers) and the size of the minimum amount required (Kickstarter campaigns are only considered a success when the minimum amount set by the would-be recipient is reached, only after which are donations actually released).

Now, three researchers at the École Polytechnique Fédérale de Lausanne have developed a model able to predict with a reasonable level of accuracy (76%) which Kickstarter projects will reach their goals, and which not, just by examining the first four hours of their development. Around 44% of projects on Kickstarter currently achieve their objectives; so being able to predict from the first moments of a project’s launch is very useful for managing any related activities. The conclusions are interesting, mainly because they are so unsurprising, but also for their possible application in other areas than crowdfunding. One variable the second study suggests is activity on the social networks, notably Twitter.

There is room for some discussion on whether this is the right variable: my opinion is that it doesn’t offer a full perspective and that it would be necessary to assess mentions in blogs and other media, which in the final analysis are very influential when it comes to generating confidence and attracting backers to a project. Nevertheless it establishes one thing at least: the social networks are highly influential in establishing the success of just about any initiative, something that anybody who creates content and tries to assess its impact will know. But would a campaign on Kickstarter still make it if it receives a positive mention on a highly popular page once it is already underway? There are many examples of feedback: on many occasions, and particularly in the case of successful campaigns, when social network coverage comes about as a result of success because the news is traction that boosts the project: something that creates a clear bandwagon effect.

Causality is, as any researcher knows, a treacherous business. It is not entirely clear if being discussed on the social networks leads to success, or if success leads to being talked about on the social networks, but there is certainly some ongoing relationship between the two.

All of which leads me to wonder how many companies would be able to apply predictive models of this type to the launch of their products or services. In reality, the majority of companies that I know tend to carry out a post-mortem analysis of their projects: with the event or campaign over, they gather some data in the hope of learning something about them, but little else. The ability to act on variables related to a project tends to be very limited, when not actually zero.

In an environment characterized by what we now call real time, such business intelligence tends to be delayed. This is either because of the time required to collect and collate the information, to then analyze and transform it into something readable, or to respond to it and make decisions, all of which create delays, or latency, that most of the time make it impossible to change course.

It is noteworthy that in a small Kickstarter project, participants tend to compile many of the variable needed to predict with success after just a few hours, based on the pattern of donations, the amounts donated, the success in relation to incentives, mentions on the social networks, etc, while bigger companies with more means are not able to come up with this kind of analysis. In the case of the social networks, companies either give no importance at all (except when there is a communication breakdown) or simply see them as an end in itself (along the lines of “yippee, we’re a trending topic”) but which is not related to the real objectives of the project. On occasions, the lack of analysis is borderline pathetic, such as when a project is passed off as a success simply because it has been mentioned frequently on the social networks, although the briefest of analyses reveals that most comments have been markedly negative.

So, Kickstarter can be seen a source of inspiration, and the availability of tools for machine learning and predictive models has never been wider. But do we understand the environment sufficiently well to be able to build productive models of this type, and that can be applied in real time to our company? If the answer is no, then you know the answer: read more.