Working with Enterprise AI Vendors
I’ve spent the past 18 months working with and for a few different enterprise AI software companies. During that time I noticed some recurring themes while working with prospects and customers. I thought it would be helpful to share them with those of you on the outside that are starting conversations or implementations of intelligent systems.
I just need some AI
Artificial intelligence for enterprise systems is rapidly advancing. It’s exciting and people in innovation roles worry about getting left behind. This can lead them to seek out AI solutions without understanding the actual value or business problems that it could solve.
Using “AI” anywhere in marketing material can result in plenty of sales leads. I’ve been on quite a few sales calls where the prospect’s priority was to learn about AI solutions. They don’t always have a particular need or problem to solve, or even know why they’d want the product. I’m all for using vendors to do your research but you’ll make your research more productive if you do a little homework first. If it’s your job to understand AI and machine learning potential for your company, I recommend building a high-level understanding of key technologies. It will help you better communicate your needs and understand the potential within your environment. You’ll also be better able to compare vendor solutions.
My resource list is a great place to start. As a simple exercise, you can start with the Predict, Automate, Classify framework to identify some potential opportunities.
It’s not all Watson commercials
IBM has great technology and an enormous marketing budget so of course, the commercials are going to make AI seem like it can solve anything with ease. I do like that these examples show targeted solutions to real-world business problems. Most people don’t realize that there’s a lot of data, effort, and time that goes into building this level of intelligence and automation. You don’t buy an AI product then sit back as it starts automating process and decisions for you. Even the best AI offerings aren’t going to magically solve your business problems.
It’s all about the data
Now that you know that it’s not all magic, let’s review what it does take. Without data, AI and machine learning systems are useless. Unless you’re buying an AI toolkit or framework, the vendor will need access to your data. For cloud providers, this also means data may exist outside of your control.
Most companies will already have a security team ready to hand over a multiple page questionnaire. But this usually comes later in the sales process. At a minimum, here are some questions to prepare for your early discussions with vendors:
- What does it mean from an auditing and compliance perspective for your data to be sent to and managed by a 3rd party?
- How long does the vendor keep your data? Does the data have to persist for on-going model training or can it be purged soon after initial load?
- How is sensitive or personally identified information handled?
- Will your data be used to train aggregate models across multiple customers? There are some fine lines here. Most vendors will keep your data isolated to your account and services. However, there is also value to both vendors and customers to leverage common patterns across customers. An example would be improving natural language understanding in NLP systems. Knowing the language patterns and terms used in one company is helpful, but knowing how entire verticals or industries uses them could help build better understanding systems for all customers.
- What are the controls on vendor employees accessing your data?
- What is the process for requesting the deletion of all company data if service is terminated?
It’s not just about the data
In marketing, it’s common to hear the phrase “content is king”. For data science, I prefer to focus on “context is king”. Yes, it’s true a good team of data analysts and scientists can do some interesting and useful things with the right amount of data. Without understanding the processes that generate and consume the data value could be limited.
You should think about any additional related data that could add value to the training. For example, if you’re sharing a bunch of ticket transactions or chat logs, it might be helpful to also include the context of the users involved in those transactions. Often there are patterns around a user’s location, department, or role that could help improve the performance.
Whenever you’re sharing data with an AI vendor, be sure you are both on the same page on how best to use the data and context to train models. If they don’t ask you about your business process, you should ask them why.
Where did this result come from
The whole purpose of AI-based solutions is to have machines learn how to make decisions, predictions, and perform actions. How they learn can range from simple regression algorithms which are easy to explain, to complex neural networks. Neural networks figure out their own way to learn and continue evolving, so explaining how they made a decision is not usually an option.
The point is that you, or the vendor, won’t always be able to identify how the system has learned and how it delivered a particular result. Have a discussion with the vendor to understand the expectations of how you can make sense of the results and troubleshoot when results aren’t as expected.
Integrations
The range of integration details will vary based on your needs and the type of solution you are implementing.
Some things to think about:
- Beyond the process of getting the raw data to the vendor for training, what other types of integrations are needed by either you or the vendor?
- Is data transferred on a scheduled basis or is there a need to implement process event triggers?
- For transaction-based integrations, how are integration delays or offline scenarios handled?
Measuring performance
While it might not always be clear how the system is doing its job, it is important to know that it is doing a good job and hopefully improving over time.
Make sure you understand the right business metrics that allow you to track system performance and return on investment. You and the vendor should be in agreement on what success looks like. Even if the vendor’s platform has a reporting dashboard you should identify ways to report and track performance that is closely tied to your business process activity.
Hopefully, this helps you be a more informed buyer and improves your relationships with AI vendors. Be sure to follow the K Means What series for more information and tips.
Already working with AI vendors? What have you learned?
