The exciting challenges of developing an AI product
AI is quite the buzzword these days. It is being heralded as a new industrial revolution that will put people out of work. Once you spend some time with AI you will realize it is just software, a different kind of software. Machine Learning is the science of helping computers discover patterns and relationships in data instead of being manually programmed. It’s a powerful tool for creating personalized and dynamic experiences, and it’s already driving everything from Amazon product recommendations to autonomous cars.
At Bicycle AI, we are building an AI powered sidekick for customer service organizations that suggests responses to customer queries. While the space has been explored in the past, newer and more powerful tools make better solutions possible. These however present unique challenges.
The team
I met some my colleagues for the first time in San Francisco as we all flew in to San Francisco to attend Y Combinator’s 2017 winter batch. Others I’ve met only over Hangouts scrum meet sessions. These we held at late at night since 10AM in Bangalore is 9:30PM in the Valley. Crazy time difference aside these 3 months in the US really helped me to explore product management for an AI driven product. I’ve learned a lot about AI and its applications in YC.
Right from the beginning, I noticed that there were two distinct teams in engineering; a traditional engineering teams and the AI team. Our CTO and co-founder, Arpit Mohan leads the more traditional engineering efforts with 2 back-end engineers and one front-end developer. Arvind Singh, on the other hand heads the AI team of 3.
Coding vs Deep Learning
The big difference between these two approaches is how consistent the outcome of developments will be. On a traditional system, if you make a teal button with a radius of 5, it will always show up the same way to almost all your users. It will show up grey for that one user still using a black and white CRT monitor, but you can still design with most of your users in mind. With an AI product, especially for one driven by natural language. If we take the buttons example, based on the users’s data set, that button might look wildly different with different deep learning models applied.
Combining models for a user is a way to get a higher success rate, but it will never be as reliable as a hard coded button. You might remember the Google Photos fiasco that affected a proportion of users. Google Photos however still remains one of the best examples of an AI driven mass market product. The nature of variability in the system also means they are harder to test internally.
These are some of the challenges when developing an AI product. How do you set measurable targets and how do you break down what needs to be done by each team? After all, providing value to users is why our company exists. New features and improvements are derived from user stories and users don’t care which team does what in order to improve their experience. Most times it comes down to everyone pitching in.
The design challenge
Then there is the challenge of designing for a product that has ambiguity and variable usefulness at its core. Interaction between human and AI is a rather new field, giants like Google are only beginning to explore it. This means there are no established best practices or clear paradigms to follow. This challenge is also an opportunity and it makes working on an AI product exciting.
Again, users don’t really care what you are using in your product to make it work as long as it helps them. Using AI and deep learning as buzzwords can do more harm than good as users would attach wild assumptions to it. So how do you design for building trust in the user? How do you show for example how good the machine thinks a result is?
One option that most products use is exposing confidence scores to the user. Confidence scores are given by the deep learning algorithms to indicate how close the result is to what we set the model to aim for. They are between 0 and 1. I don’t think this number means anything to the user though. How much better is 0.69 confidence than 0.62 confidence? Would you check the slightly less confident answer more? And how confident is the machine in its confidence scores to begin with? Surely we can do better. We just need to figure it out from scratch since it is such a new area.
Sounds interesting, right?
Working on an AI product is full of interesting challenges on every front. How do you manage a product with variable outputs? How do you work together with different branches of engineering to deliver value to the customer? How do you combine the latest advancements in machine learning with age old methods to solve a problem? How do you design for a platform agnostic solution that has a variable output? How do you develop the front-end for something like this?
These are just some of the questions we are trying to find answers to. If you have encountered these, let us know what you think. What other challenges do you think would an AI product have? Now and in the future.
Do you find these challenges interesting? If so, join us, let us work on the together. These positions are also available remotely:
If you’d have any questions, pointers or just would like to chat about AI products and such, drop me an email at matt@bicycleai.com
