If You Want to Be Creative, Don’t Be Data Driven
Bill Pardi

Very good article. The title ‘If You Want to Be Creative, Don’t Be Data Driven’ is true! I’ve been caught out by users of the tools our company produced expecting ‘it’ to do it all for them. As a user you bring your the data to the party, what theories do you have about the data, the company supplies an environment to experiment and vastly deepen your understanding. There is no ‘automatic’ tool that will make sense of you complex data just like that and magically give you actionable intelligence. This is such a common misconception.

What we are really describing here is good science/engineering informed by data. For the machine and data to be ‘autonomous’, a bottom up approach is required, find features in the data, put those features together, see if patterns occur that correlate with an acceptable answer (often not well defined). You can do that, but the search space is likely so huge, it will never terminate. Deep learning is making some inroads here, but still has a long way to go. So, the choice is top down (engineering), or bottom up (vast search space). There is perhaps a middle way, have ‘lower level’ theories that can be applied at intermediary levels, how do you find, store, match and apply these ‘intermediate’ theories, that is an open question. So there are opportunities to be creative whichever approach is taken.

My take is the goal of AI/machine learning is to reach a critical mass of intermediate ‘theories’ (and the ability to learn new ones) then apply them when appropriate. As a human, at the moment, there are so many possibilities, there is still a big part for human engineering. The biggest creative challenge is to engineer partial solutions, the ability to experiment, and the ability to apply them, so machines can do it too. How do we create the meta-engineering ability for machines to be creative too…