Architecting the Right AI System for your Problem — Part 1

As the amount of ingested data increases for companies, different businesses need to convert these ever-increasing sources of data into new insights that help their users in different ways — but there is no one-size-fits-all solution. In this series, we explore you, as a Head of AI, Data Science, VP of Data Science, or similar can think about approaching this problem.

In the wild, we always hear many people say “data is the new oil”. Data is the new oil, but only the data through which insights can be extracted. Recently, the cost of storage and compute has become cheaper with services like AWS Glacier and Elastic Compute resources have become faster and easier to access cloud services. But much like how oil had challenges with the processing, ownership, and operationalization to make it useful to society, useful data usage has challenges. And just like the consequences of wrongly utilizing oil created the climate crisis, we will be hearing more about the ethical consequences of wrongly utilizing the data and algorithms in the decades to come. We will not get into the ethical consequences of data science in this blog and humanity may be early in feeling the pain from it but we’ll come back to that in another blog.

In Part 1 of this series, we will lay the framework for how businesses can get value out of data.

As practitioners, we hear a lot about multiple tools in the market and this makes it difficult for data science teams to choose given the noise. Andreessen Horowitz created a great blog titled “Emerging Architectures for Modern Data Infrastructure” aiming to summarize and categorize them, including a report containing data infrastructure reference architectures compiled from discussions with dozens of practitioners and startups.

Overall Architecture Diagram laid out in “Emerging Architectures for Modern Data Infrastructure”

Check it out to get a sense of the startup tool ecosystem in data analytics. We’ll refer to it periodically in this series, but just knowing the startups or tools that are available doesn’t help us as leaders decide which tools to use.

In this Artificial Intelligence in Practice blog we attempt to provide an unbiased point of view on which tools would best serve the needs of the company and tips on the decision process from our conversation with other practitioners. Assume the following situation: you are hired as the VP of Data Science in the company and you are given a task to figure out the data strategy to make your company successful. Now, as you join the company, you start with understanding the business problem required to be solved. The first step is to segment the problem into one of two buckets,

Architect AI system based on the problem

1. Providing business analytics to internal and external stakeholders. For example, the growth or business team in a high growth startup wants to understand MAU or when users are taking orders on the app and how COVID has changed user behavior.
2. Solving operational problems with deployed AI models. For example, the Product team at a fintech company that disburses loans wants to forecast if a person can be approved for a loan based on their past behavior.

Now for (1) providing business analytics, there are two kinds of solutions you can set up for the success of the company you joined,

1. Business Intelligence Tools. E.g. Tableau, Looker
2. Business Intelligence Tools + Artificial Intelligence modeling Tools. Avoid setting up both unless really necessary for the business

For (2) solving operational problems with deployed AI models, you can break it down as

1. Artificial Intelligence modeling Tools. E.g. AWS Sagemaker
2. Business Intelligence Tools + Artificial Intelligence modeling Tools. Avoid setting up both unless really necessary for the business

Hence, given the problem for any company, the three possible solutions for data infrastructure boil down to:

1. Business Intelligence Tools. E.g. Tableau, Looker
2. Artificial Intelligence modeling Tools. E.g. AWS Sagemaker
3. Business Intelligence Tools + Artificial Intelligence modeling Tools. Avoid setting up both unless really necessary for the business

Once you know which path you want to take, each one of these solutions can be solved and broken down in a number of ways.

Stay tuned for the next parts of this series where we will share the complete list of tools applicable for each and a few real-world examples.