How To Build A Successful AI PoC
Turn Your Artificial Intelligence Ideas Into Working Software
Read the original article on Sicara’s blog here.
Building an AI PoC is hard. In this post, I will explain my thought process to make my Artificial Intelligence PoCs succeed.
“What if my alarm could use traffic information to wake me up just in time to go to work?” We’ve all thought of resorting to AI to solve one of our problems. The goal of a Proof of Concept (PoC) is to test whether it’s worth investing time into it. Building a PoC is hard, but it’s even harder to build an AI PoC because it requires a large set of skills.
When building an AI PoC, data science only represents a small fragment of the work, but it’s one of the most important skills.
It’s easy to find great tutorials on how to solve a specific task like how to build a detection algorithm to park your car or how to deploy a flask app to the cloud. But it’s a lot harder to design a solution to your specific problem, mainly because you need the hindsight to reformulate your problem into a standardized task.
In this post, I will explain my method for achieving this.
First, I will start with a review of what an AI System looks like.
Then, I will describe my 3-step process to design an Artificial Intelligence.
Finally, we’ll see 2 examples, a simple one and a complete one with a python implementation.
Overview of an Artificial Intelligence System
As an example, I will take a system which classifies documents. It answers to “What kind of document is this?” with classes like an “electric invoice” or a “to-do list”.
AI workflows consist of 5 steps:
- receiving the question: “What kind of document is this?”
- adding complementary data on the user or the context: “What type of documents does the user have?”
- using the data to answer the question: “Which type does this document belong to?” by “This is an energy invoice”
- storing the result: adding the new documents to the database
- answering the client’s question: “This is an energy invoice”
You can break this down into 3 tasks, or semantic blocks:
- Handling the client: receiving the question, making him wait…
Example: an HTTP server - Data conciliation: communication with the “company knowledge base” to add or receive relevant data.
Example: communication with a database - AI Block: the AI itself which answers the question with a context.
Example: expert system, SVM, neural networks…
You can find great tutorials on how to architect your server or your data conciliation layer on the web. The simplest solution for an AI PoC in Python is using Flask and a SQL database, but it highly depends on your needs and what you already have. Here is a tutorial on using Flask with SQLALchemy.
We are going to focus on designing the AI itself.
Designing The AI Block
AI tasks can involve multiple heterogenous inputs. For example, the age and the location of a user or a whole email discussion.
AI Outputs depend on the task: the question we want to answer. There are a lot of different tasks in AI. You can see some of the usual tasks in computer vision in the image below.
Thinking of ways to build an AI seems complicated as soon as you venture out of the standardized inputs and tasks.
To wrap my mind around the complexity of building an AI, I use a 3-step process.
Step 1: Browsing the relevant inputs
First, gather all the inputs you suspect are capable of answering the task at hand and select those that are self-sufficient in the majority of cases.
When testing an AI idea, it’s easy to get greedy and think about solutions that include a lot of inputs: the location of the user may give me an insight into what their next e-mail will be, for example. The truth is: it’s just so easy to get lost in mixing various inputs with different meanings or nature and end up delivering nothing.
Stick to simple, self-sufficient inputs when building your AI.
Step 2: Vectorizing the data
The second step is to preprocess those inputs, to make those usable for various algorithms.
…
Read the full article on Sicara’s blog here
