Those working in AI must remain versatile while specialisation increases

What skill sets do you need to work with a world class team of artificial intelligence (AI) developers? I spoke to Misha Bilenko, the head of Yandex’s machine intelligence and research (MIR) group, who told me about the trends that he’s observed:

“It’s definitely diverse, even in terms of skill sets, but as the field is exploding you inevitably get more specialisation. Depending on the specific problem, you will need specific skills.”

“Some people will have the specific skills required to work only on a speech recognition system, or a recommender system, or an image classification system, and then somebody who is just trying to analyse log data will have other specific skills, as will someone doing sales prediction for a company.”

The developer who provides you with that top 10 list is not solving AI in the glamorous, highly technical sense of the word, but what they’re building is doing an amazing amount of things.”

“Even though we could generalise all these roles as being in the AI sub-industry, the skill set may be dramatically different between these specialities, and the differences are becoming more pronounced.”

“To give an example, one thing that recently has emerged is a clear distinction between data scientists and engineers, where previously the lines were somewhat more blurred.”

“However, there’s lots of hybrids and it is certainly the case that developers who are hybrids — in terms of having some set of base skills and then some depth skill — they are usually the ones that are most productive.”

“Being totally narrow makes you less effective in certain scenarios. A developer may need to pull a bunch of big data, and do some analysis quickly — not as a core part of their function but to enable them to train.”

“For example, let’s say you’ve got people working with neural nets: they should have enough diverse base skills to be able to process some data, then clean it, and then get it in to your system.”

“The ease of use of new tools broadens the capabilities of people whose core function may not be a neural net expert, or a machine learning expert, but rather a data scientist, or a specialist in optimising some recommendation stack who often produce the best results.”

“A lot of the time the problems businesses want to solve are manifested in a very specific way to a very specific vertical. In that sense, it’s not a singular AI requiring a specific skill set that needs to be created.”

“Take the example of a recommender system, which will provide a user with a list of the top 10 pubs nearby. What is essential is getting that list to your customer when they are standing on a street corner at 10pm, with a certain group of friends, and knowing that they will have certain preferences or restrictions.”

“The developer who provides you with that top 10 list is not solving AI in the glamorous, highly technical sense of the word, but what they’re building is doing an amazing amount of things.”

“The application giving you that recommendation can process a huge amount of data about gazillions of people who were in the area, it knows about you and your processing history, and you can usually talk to it!”

“In a sense it is very, very, very clever, in terms of the final list that it produces, which is actually pretty useful and there’s a high probability that you will end up in one of the choices that it provides you.”

But a lot of the time customers…just want to have the best recommendations for them, and the developers who get them that list will probably possess a variety of base skills, as well as their specialisation.”

“A system providing top 10 tailored recommendations is intelligent in every sense of the word, but it is not a general AI in the science fiction sense of the word, which would take highly technical skills to design and create.”

“But a lot of the time customers don’t care about that, they just want to have the best recommendations for them, and the developers who get them that list will probably possess a variety of base skills, as well as their specialisation.”

“But, it is the case that there is a talent gap. Which is why Yandex is fun to work in, because in Russia it’s nice to be the big kid in the job market. We’re working a lot with the universities and we’re trying to make sure that we can utilise the amazing talent pipeline that exists in Russia to compete.”

Misha Bilenko is now head of the Machine Intelligence and Research (MIR) department at Yandex, after a decade working at Microsoft. Yandex is a technology company that builds intelligent products and services powered by machine learning. It is one of Europe’s largest internet companies and the leading search provider in Russia.