Podcast: Machine Learning Realities and Opportunities

Everyone’s talking about machine learning and how it can be applied to solving some of today’s top business problems. CTOs know that it’s becoming an imperative for building successful software products. But is it really the answer and, if so, how can companies over come the many execution gaps that exist? In this first episode of a two-part series, Georgian Partners’ Madalin Mihailescu talks with Jeremy Barnes, the Founder and CEO of Datacratic to get some unique perspectives about both the technological and business sides of machine learning.

You’ll hear about:

  • Jeremy’s definition of machine learning (5:42)
  • The other areas of pattern recognition fall under machine learning (6:46)
  • Why businesses should care about machine learning (9:17)
  • How machine learning is evolving as a tool for solving business problems (12:45)
  • Realistic applications for machine learning today versus expectations (13:39)
  • The complexities of applying machine learning (16:27)
  • The current state for ensuring model quality in production (19:16)
  • Incorporating safe guards into systems (21:12)

https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/243166823&color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false

***

Transcript

Jon Prial: Machine learning, is this the panacea for every company with lots of data? Let’s hear from some experts and find out what’s real and what you need to focus on.

Welcome to “The Impact Podcast.” I’m Jon Prial.

I’m here with Madalin Mihailescu.

Mads, as one of the key technical resources in the Georgian Partners Impact Team, you talk to all our CTOs on a regular basis.

Madalin Mihailescu: I do, indeed, John.

Jon: How often do you have to talk about machine learning and artificial intelligence?

Madalin: Not every day, but it’s been happening more frequently over the past year. Most of our CTOs recognize, today, that machine learning is becoming imperative when building software products. I would say there is still an execution gap that we, at Georgian, now are focusing on.

Jon: Excellent. Then, for this podcast, who did you talk to and why?

Madalin: I’ve got together with Jeremy Barnes. He’s the CEO at Datacratic, which is a machinery company out of Montreal.

Jeremy is actually a good friend of ours, of Georgian’s, and he has quite a unique perspective on both the technology side of machine learning and the business side. I figured getting him to share some of his thoughts would be beneficial for both the CTO and the CEO of a software company.

Jon: That’s great. Well, with a space like this, with hype everywhere, I’m looking forward to getting everything cleared up. Let’s listen to your conversation.

Jeremy Barnes: It’s good to be here, Mads. Just, before we get started, I’m happy that the VCs outside the Valley are taking interest in machine learning, that we’re not leaving all the exciting stuff happen on the other side of the continent or the other side of the border. It’s good to be having this conversation.

Madalin: Great, Jeremy. Well, with all the high end research that happens in Toronto and Montreal, we cannot leave all the interesting business aspects of machine learning to the Valley, right?

Jeremy: Or all the good people to flee south, as well.

Madalin: All right, Jeremy. Maybe tell us a bit about yourself. Don’t skip the fact that you hail from Australia and now live in Montreal, and that you’re now at your second business in the machine learning space.

Jeremy: Yeah, sure. I’m the Founder and CEO of Datacratic. We’ll be talking about Datacratic a bit more later on, so I won’t talk too much about it now.

My career has been 15 years doing machine learning. At the end of my university career, I did a thesis on machine learning my final year. That was what set that direction up. That was in Australia, obviously.

I then went and cofounded a startup in April of 2000 in the Valley, which was not the best time to cofound a startup in the Valley.

We spent a while looking for finding there. I’d actually moved to Montreal basically for the tax credits. It was the only place where the company could work. That company was a computation linguistic startup. It was also one of the early companies to have machine learning as the central naval around what was built.

The company was disambiguating tweets, search queries, Web pages, conventional ads, things like that, and making them into the context which was here information or entities found in say Wikipedia or the other linguistic resources there. The goal there was to get a semantic understanding of text.

Interestingly enough, that company we ended up employing two of the winners of the Netflix Prize yet before they won a Netflix Prize, one of the inventors of the Amazon’s first recommendation algorithm. We had some good talent that went through the company.

I eventually left the company once the technology was done. We had a very, very good technology especially for the time, but we hadn’t identified the market for it. I considered that my job there was done, and spend a year doing data science competitions looking around for what to do next.

I eventually founded Datacratic, how to determine that especially in Montreal where I’m from or where I am. In fact, for family reasons, there was no job for me that I didn’t create myself, and so I started a second company Datacratic. That was in 2010 and I have been here ever since.

Madalin: Got it. That’s a great background. It seems you are a veteran of the machinery space and machinery apply to business. Maybe let’s start a bit with some basics. I guess thinking as I said back, it’s quite interesting for me at least to talk to companies and investors today and hear the term, “Machine Learning,” quite a lot in most conversations I’m having and even my teammates have.

You are probably seeing it as well. Everyone really talks about it. How would you define machine learning?

Jeremy: Machine learning is basically a way of learning models or even programs from data rather than constructing them by hand. It’s a little bit like asking a computer to write a program for you to do something which is very, very complex where you have some data about what the input is and what the output should be.

You don’t have any possible hope of your hand constructing a program to do it even if you have really good programmers or really good scientists. From a technical perspective, it’s that.

From a more business perspective, it’s used, primarily, to identify a relationship between what you know now, like pixels in a picture or perhaps a customer’s browsed or something like that, and some kind of an outcome that you don’t know, like what the number plate on a car that your customer buys in the next two weeks. Those relationships, you’re going to use them to automate decision‑making or to inform human decision‑making processes, depending on what the weight of those decisions are being made.

Madalin: Interesting. If you look up the spectrum of associating inputs with outputs in a more automated way, this whole space of pattern recognition, would you say that all the algorithms from simple logistic regression to more complex deep learning and deep networks fall under this machinery umbrella, or would you split them?

Jeremy: I think that you can define machine learning in a pretty broad sense, and of which, even running a regression in Excel is machine learning. You can narrow the definition a little bit more. I think, probably, a narrow definition is a little bit more useful. I would say that machine learning, it has components of both art and science to it.

The art is about, you take a business problem you have in the real world or some kind of problem you’re trying to solve, and you map that into the form that a machine learning algorithm can solve. You map some data, and you map some kind of a representation, that you map a prediction onto the thing you’re actually trying to figure out.

The science part is more about using available algorithms, data, and technology platforms to make that solution viable. Importantly, you need to do it with a little bit of rigor, to have some confidence that your machine learning system is actually going to solve problem.

If you were combining the art and the science properly, you come up with a solution which you can, in some sense, you can have certainty. You are mathematically bound to certainty, but some certainty, that under certain conditions, yet solving a problem in an optimal or a close to optimal way. Whereas, if you’re just blindly applying algorithms without thinking about any of those things, you lose the ability to understand in what context this solution is optimal or even what problem it’s solving.

It doesn’t mean you can’t solve problems with your simple algorithms applied blindly, but it does mean that it’s much harder to reason about your work problems that’s not solving. I’d say the machine learning involves actually applying some kind of a scientific method to what you’re doing, and that involves a fair bit of art, not just an application of algorithms or downloading a toolkit, and applying it to your data’s file.

Madalin: Got it. From a business standpoint, why do you think businesses should care? Walk us through one example, where machine learning can solve a problem, that just writing a piece of software without applying machine learning techniques would not work.

Jeremy: Sure. That’s something which is really exciting, because the number of places in which that’s true, in which machine learning makes things possible that weren’t before, is starting to get very large. I would say that if you look at it from a Macro perspective, what’s happening with machine learning, at the moment, is it’s providing a way for new businesses to opening new fronts of competitiveness, especially with the larger entrenched businesses and we’ve to enter more industries there.

A lot of that is because these upstart businesses are able to gain efficiency from using data and using that to apply to real business problems. If you have a look across all the industries, that kind of thing is happening.

For example, if you have a look at the lending industry, your payday lending industry, there are companies now, that are much smaller and much more nimble that are starting up. They are able to offer your better rates and have a much more effective business, because they’re using data.

Essentially, they’re using this extra front which has opened up about how data is used, to exploit weaknesses that some of the incumbents have, and that’s allowing them to enter the market and disrupt those industries. That’s happening all over the place. That’s happening with everything from transportation to agriculture to insurance where larger companies are having difficulties figuring out their data strategy fast enough to see off these new companies that are starting up.

Madalin: How do you define AI today? What’s your definition of AI today? AI has been a moving target and the term comes with a lot of baggage. It seems today AI is being associated quite strongly with machine learning. How would you define it?

Jeremy: Machine learning is a lower goal than AI. Machine learning is a set of techniques, which have a mathematical relationship between the input and the output, and how we automatically be creating algorithms with that. AI is talking about intelligence.

I think that the times of things that you can do with take learning are much closer to AI in that we can start to deal with some of the earlier parts of cognition like visual recognition, ability to separate words and to match them up into the context and things like that.

Intelligence implies some higher level of understanding. I think we are already scratching the surface of intelligence. I think we are creating good building blocks for artificial intelligence at the moment. I don’t think that we’ve really achieved anything you could say is intelligent nor I think that that’s going to happen very, very fast over an hour. I think there’s a lot of work that still needs to be done there.

Madalin: Got it. It’s really about starting with that business objectives and architect. It’s something that uses machine learning with that business objective in mind including perhaps goals in terms of accuracy, in terms of your false positives and false negatives that already do that to your business case.

In healthcare, you might have different objectives than, for instance, in advertisement as an example, is that fair?

Jeremy: Exactly. You might have a very skewed risk in that making a wrong decision in one direction will lose you a bit of money. Making a wrong decision in the other direction can get you sued. Those things if you don’t find a way to pass them down into the machine learning layer, then you are putting your business at risk by applying machine learning in an incorrect manner.

Madalin: If I look at 2015, it truly has been a breakout year and everyone is really super optimistic. I talked to people in the industry, investors, and even researchers. Researchers are by and large pessimistic in nature. What do you think is real today and what is promise when it comes to machine learning and the applicability of machine learning to business problems?

Jeremy: I think it’s been quite a ride this year as you say. A year ago, when I would talk about what the algorithms we are using or I try to go into a little bit more depth about how we take machine learning to solve a problem, people would say to me say, “Hey, hey, hey, wow, Dr. Spoke. Stop talking about matrices or whatever those things are.” Like, “Give to me in language I can understand.”

Now when I talk to someone they say, “Your solution is like so behind the times you didn’t even say tenths or watts.” The language has changed a lot. I think that people have decided that machine learning is actually a viable and exciting solution not this mad scientist type thing that is now for.

Madalin: Exactly.

Jeremy: With respect to your question, which is what’s real about machine learning and what is there which is still promise or still unrealized?

Certainly the advances we’ve made have been around a certain number of set of techniques in particular techniques called, “Stochastic Grading Percent,” which are ways of with sufficient amounts of data of training really, really complex models that can emulate some of the human characteristics of the sketch.

That’s what’s made these huge advances in image recognition, in speech, in natural language processing is the sudden realization that the people who designed neural networks all those years ago were actually right. There just wasn’t enough computational power and enough data to train them yet.

Now, we suddenly discover there’s a lot more data, there’s a lot more computational power and they think what, these things actually work. The form is very similar to about what they were a long, long time ago.

It’s really the confluence really of these mathematical techniques and these advances in computer architecture and data storage which have lead to these explosion that is happening.

That being said, there are a lot of other things which haven’t been explored. There’s a lot of push in that direction of making those grading discern algorithms run faster. There is still a lot of work to be done with representational aspect of how do you deal with uncertainty in those algorithms, how do you deal with things that have more structure than you can represent in a fixed length spectra of numbers, things like that.

I think that there’s still a lot of promise that hasn’t been realized in machine learning in the way in which those algorithms are applied or mutated to better match real world type situations, because any image is a fairly concrete representation.

There are lots of phenomena in the world that we need to be able to work with which aren’t nearly as concrete as that or where there’s a lot of missing information. We need to be able to deal with those as well.

I think what we have is a set of foundational building blocks that are very, very powerful that we can use as part of machine learning solutions. I don’t think we’ve gotten to the point where any problem is trivially solvable with machine learning even if there is a lot of data available.

That’s ignoring entirely the solution of how do you get that data and how do you effectively make a solution that leverages, but that’s your cost effective and solves the problem you are trying to solve with.

Madalin: What do you think complexities lay when it comes to applying machine learning to solve a business problem? I guess the first one is really having that comprehensive dataset and assuming that problem is solved. What would be the other complexities?

Jeremy: I think there are complexities that would…You could break it down to two main categories. There are system complexities, which are around the system and how it interacts with the problem you are solving. Then there are technical complexities, which are about how you make the actual system works.

I think the biggest complexities really are in the first category, because once you have trained a machine learning model, it’s assuming things about the data coming in, the data that you trained it with in some ways that constrain its view of the world and that constrains you.

It expects a data that comes in later on to the exactly the same. An example of that is I don’t know if you’ve been following Google’s work on image classification. They have a bit of a problem with one of their inception models, which is a really powerful, classify, used to do image recognition and image [inaudible 19:00] . That system, when applied to ordinary people’s photos, came out with fairly insulting predictions for those people about what was in the photo itself.

The reason for that was in the end that that system didn’t have enough data about people in it. It had too much data about animals. It just so happened the dataset had been collected with less emphasis on people. According to the system, it was much more likely there would be an animal in the photo than a person and it made predictions based on that.

That’s a system‑wide complexity, because that isn’t about the model itself. The model did what it was supposed to, which was distinguish between different classes of animals. But in the context that that model was deployed, there’s a much wider set of assumptions, which, [inaudible 19:55] the assumptions that were in the data.

That’s an example of the kind of complexity that you get when you start to apply machine learning on more real‑world problems and particularly when you start to try to extrapolate from data in order to make broader predictions than this set of data which was available while it was tracked.

Madalin: I like the way you went through the whole machine learning landscape and applied to businesses. I’m wondering if it’s fair to ask you ‑‑ or even if there is such a question ‑‑ what is machine learning?

Jeremy: Talked to that a little bit before. I would say what machine learning isn’t is a blind application of tools to data and you shake out and see what happens at the end. Machine learning is, in the end, it’s a tool which is used as part of the process which is solving some kind of a problem.

In the same way that if you put a monkey in front of a keyboard and you got them to type, they might eventually type a valid program. You can’t say what program is going to do at the end. You have to have it directed by some kind of a goal, [inaudible 21:22] activity.

I think just the mere fact of using machine learning algorithms does not mean that you’re doing machine learning, at least in a way that’s going to be profitable. You do need to understand where the art is that you’re bringing and how you understand the problem and you map it to some kind of a machine learning solution, and you do that in such a way that you understand what the problem you’re solving is and that your solution does actually solve the problem.

[music]

Jon: Thanks to Mads and Jeremy for a great discussion. It’s clear that it’s more than data and it’s more than just having a tool and it comes down to how you put it all together. We’ll be covering that in part two of Mads’ interview with Jeremy and we’ll dig into these issues in more depth. Thanks for listening. This is Jon Prial for the Impact Podcast.

Show your support

Clapping shows how much you appreciated Georgian Partners’s story.