A new era of programmer, who will create the software that powers an artificial intelligence revolution, will look more like a sports coach instructing an athlete or an architect designing the plan of a skyscraper, than the image your mind conjures up when picturing a “software developer”. Systems designed to learn and react to changes in their environment won’t be built by computer science graduates writing lines of code, logical instructions for a computer to execute in a precise order, into a text editor. The next generation of software won’t involve any code at all. Creating software that can solve human problems requires a more human way of teaching computers how to find solutions. The artificially intelligent systems of tomorrow (a tomorrow a lot closer than you probably think) will be taught the same way we teach a toddler to read, or a teenager to drive a car, often with a lot of practice, and a lot more patience.
Since the first programming languages developed over sixty years ago, the method for creating software has remained relatively unchanged. There are reasons however, as to why providing computers with clearly defined text instructions became the status quo for so long. Despite many ideas still used in AI research (such as the modelling of artificial neural networks) being first developed as early as the 1940’s, the computing power available at the time was staggeringly less than is available in even today’s mobile phones. Hardware capable of effectively simulating such networks at scale was still decades away.
Restrictions in computing power available at the time meant that if you wanted to solve a problem, your program had to be designed to be maximally optimised, reducing the amount of work the system had to perform. This need for simplicity led to the explosion of synchronous programming, the conceptual act of breaking up a problem into its most basic steps, like you would do when planning your driving route home. The hardware industry followed suit, with a voracious desire for more cycles driving the technological economy towards what we know today as Moore’s Law. The idea that the number of transistors, and the resulting processing power that could be manufactured into a microprocessor, would double every 18 to 24 months.
The microprocessor, and computing industries flourished over the coming decades. As systems became faster, software became more complex, and it appeared as though no problem was exempt of a solution. There was an ever growing elephant in the room however, as scientists struggled with a particular set of problems. Ones that were hard to clearly define as instructions, that had to deal with unknowns, or changing environments. While computers excelled at things like finance, document creation and mathematical repetition, they coincidentally had trouble with things that humans excelled at. Our ability to deal with large amounts of new and unknown information was grossly underestimated, and could not be solved by simply throwing more processing power at the issue. This branch of problems got its own name; Artificial Intelligence.
“It would appear that we have reached the limits of what it is possible to achieve with computer technology, although one should be careful with such statements, as they tend to sound pretty silly in 5 years.”
- John von Neumann
Solving these kinds of problems requires a radically different approach to software development. One where you design systems that can’t really do anything well, but are able to learn a vast array of solutions through training. What is particularly interesting, is that while many different AI algorithms have been developed and theorised over the years, they seem to be converging on a small set of simple, elegant building block ideas. This is primarily thanks to recent advances in newer hardware architectures like the GPU, with its ability to perform thousands of small tasks at the same time (parallelism). Certain algorithms, like neural networks are able to scale up well this way, beating alternative approaches. It might sound counterintuitive, but the more powerful AI algorithms have become, their code has actually dropped in complexity. This can be seen for example in Baidu’s speech recognition system which uses an end to end neural network, foregoing many layers of audio preprocessing that was utilised by other approaches.
This convergence of principles is leading towards a “modularisation” of AI components that can be joined together in any number of combinations like Lego blocks. Constructing systems from a basic set of building blocks requires much less coding in the traditional sense, as the blocks themselves only need to be built once. This results instead, in a much more organic design approach, where developers try different combinations of components in order to achieve systems that are better at learning. A side effect of this process, is that most of the time spent developing intelligent systems is now dedicated to the training phase. The analogy would be that right now AI developers are creating babies. Systems that alone are useless, but with years of nurturing guidance and training, can become masters. Think for a second about how long humans undergo this sort of induction into the world. As a comparison, right now the most most advanced image and speech recognition systems take on the order of days or weeks to train.
So where are we right now? People all over the world are creating “experimental” neural networks in different languages, based on different ideas (some more like the brain than others), training them for specific tasks, and more or less throwing the finished products away when they’re done. The next evolution of AI will be the standardisation of some of these ideas, so that people are able to combine already trained systems, in order to achieve greater capabilities than first envisioned. A perfect example to showcase this process is recent work done at Stanford (http://cs.stanford.edu/people/karpathy/deepimagesent/). These researchers connected a neural network that could identify objects in images, with another that was able to understand and build sentences, resulting in a new system that is able to build coherent descriptions of photographs.
This work took a lot of effort, but with the right tools and standards for sharing and combining systems, one can imagine projects like this being done with ease, in a much shorter timespan. We are already beginning to see new software development careers (Google, Baidu, Facebook, etc.), that involve much less coding, and instead repurposing and combining common components to create new types of learning systems.
The most exciting part of this trend is soon to come. For at the moment, all learning systems are trained usually on very specific digital data. However, when more people get involved, and the iterative process of combining and improving intelligent systems accelerates, the input/output bottlenecks will open their floodgates. What this means is that instead of only being able to train systems by interacting via the keyboard, we will start to be able to communicate and teach systems in a more human way. This includes things like communicating with machines via speech, or simply showing them how to do something in the same way we show another human, by example. If you consider the vast possibilities for different sensors, and the “internet of things”, near infinite options for interacting with machines opens up. In the end, this all boils down to communication.
This is the kind of scenario I predict will become the norm for future software developers. Imagine going online and sifting through a marketplace of components, like visual and behavioural sub systems that have been pretrained to perform at a specific level. These could be combined, forked, or extended, possibly integrated into new robotic hardware, to help with a specific need, say farming. All of this work done without writing a single string of code. Extrapolate a little further into the future, and you get something like this:
I want to emphasise that this is not the distant future, but is beginning to emerge right now. Soon, with tools like ones we are building, everyone on the planet will essentially become a coder, as we slowly start interacting and communicating with machines in the same ways we do already with everyone else.
