“We didn’t build aeroplanes that flap their wings.”
Talking to Nigel Toon, CEO of Graphcore, on the future of computing and machine intelligence.
“You can’t *discover* that the brain is a digital computer. You can only *interpret* the brain as a digital computer,” said John Searle, the eminent philosopher and neuroscientist, weighing in on the debate of whether you can reproduce human intelligence. According to Searle, we don’t yet fully understand intelligence. Neuroscientists are still relatively early in their exploration of how it works; while the working definitions of human thinking, emotion and consciousness, are still pretty rudimentary. For Searle, and many experts in his field of research, replicating intelligence in its entirety is not yet possible. As Luciano Floridi, the Professor of Ethics and Information at Oxford University puts it, “Alan Turing himself himself knew that intelligence was not definable. In his famous paper he concludes that the question, “can machines think?” is essentially meaningless. We have no idea what thinking means.”
And yet when you look at the images that our portfolio company, Graphcore, built last February to show the deep neural networks for computer vision, image recognition, and inference training, you’d be forgiven for thinking they depicted “natural” processes. Like a digital imprint of the human brain in multi-colour, they visually describe the process that machine learning takes to train and understand. The images are actually computational graphs. As Nigel Toon, the CEO of Graphcore, explained:
“Sometimes we take inspiration from nature. often because we are using different materials we need to create things in different ways, there are different trade offs and compromises. That’s true here too, our brains are molecular structures, and yet we’re building computers in silicon. We can take inspiration from those natural structures, and we can try to understand them; but we are implementing this process using different materials and techniques so we need different solutions. That’s true in this case. ”
“The reason the machine learning activity looks biological, is because we are trying to limit the communication between the points on the graph. This is because that takes energy. This is actually exactly what your brain does; it tries to minimise the communication necessary between your neurons to make them more efficient, and so you get a clustering effect. When we run software on our processor, we are doing the same thing. This isn’t a molecular approach, we are building using silicon. We didn’t build aeroplanes that flap their wings.”
While we’re still a long way from general intelligence, systems which are all knowing and can replace human intelligence, we are not far from building systems which can problem solve in intelligent ways and build rapid improvements in areas as diverse as personalised medicine and transport. The developments, according to Toon, will be fundamental to almost every industry.
“Systems that can remove drudgery, that’s what we’re really excited about. Things like autonomous cars, more detailed language understanding, scalable legal research, understand how to deliver drugs better or fight cancers, all of these are potential applications.”
Building models that can train, infer and predict, requires the computer to build the capacity for judgement. Things may have a context which might not be immediately obvious. This, says Toon, is at the heart of building really good machine learning models.
“Let’s say I ask you, ‘Who is the Prime Minister of the UK, and what kinds of clothes do they wear?’ That’s quite straightforward. I then tell you that the prime minister is Theresa May. From this you would infer that she is a woman and that she probably doesn’t wear the same clothes that, say Donald Trump wears. I didn’t say that in the words, but for a system or a machine to understand that, it has to understand that conceptual leap. These are capabilities that machine learning systems have to understand; the ability to understand context, to recall situations, to learn techniques of how to solve problems.”
Yet in order to fulfill the potential of machine learning, we need to fundamentally change the way that we build, and interact with, computers.
“For 70 years now people have told computers what to do, step by step, in a programme. We’re now reaching the point where we can train computers and they can learn and get better through machine learning. This is potentially transformational for compute. But this kind of work has a fundamentally different workload; that’s the computational challenge. It’s not like the desktop apps or the web hosting that we’ve traditionally built. It’s not even like High Performance Compute or the more advanced graphics processing, that we’ve built graphics processors for. This needs a completely different approach.”
Crucial to building machine learning applications is the hardware that you need to build it. Semiconductors are key. Moore’s law is coming to an end in a literal sense, because the exponential growth in transistor count cannot continue. But from the consumer perspective, the Law simply states that “user value doubles every two years”. The challenge is then an engineering problem at its core: how do you build a semiconductor that can manage a totally different workload in a more powerful way than has ever been done before?
“Moore’s law is actually still continuing, what has changed is that we are now power limited. We’ve put so many transistors onto these chips, and crammed them in so tight, that we need to be very careful in the way that we manage power inside the chip. We can’t just keep making the chip go faster and faster as it will burn too much energy. But what we can do is put down many parallel processors. The challenge then becomes how you get those parallel processors to co-ordinate, co-operate and share data. How do you get the transistors to communicate in a sensible and highly parallel manner? That’s what we are doing at Graphcore: creating a parallel processor that is designed for this workload of machine learning, which is easy to programme and which is much more efficient. At Graphcore we can deliver something between 10 and 100 times the compute speed, compared to current processors like CPUs and GPUs for this class of problem.”
Over the past 3 years, the team has grown substantially and Toon, together with his co-founder Simon Knowles, have built out a very talented team of engineers in Bristol, where the company is based. By building both the software to enable developers to continue coding using the methods they are used to, and the hardware needed to facilitate the speed of development, the team are set on accelerating the progress made in machine learning.
“This is what we are building here at Graphcore, a new compute platform for the future of computing”.
Today, we are excited to announce that we will be investing, once again, in Graphcore, alongside Atomico, Amadeus Capital, Robert Bosch Ventures, C4 Ventures, Dell Technologies Capital, Foundation Capital, Pitango and Samsung Catalyst Fund.
We are also delighted that AI experts Demis Hassabis (DeepMind), Greg Brockman (OpenAI), Ilya Sutskever (OpenAI), Pieter Abbeel (UC Berkeley/OpenAI), Scott Gray (OpenAI) and Zoubin Ghahramani (University of Cambridge, Chief Scientist at Uber) join as angel investors.
