Building Blocks of AI
Ingredients of your AI Recipe …
There are hundreds of articles floating around on the internet talking about which framework to use, what kind of data to choose, which architectures to pick and so on and so forth, But what isn’t there is Prequel and Sequel to that story.
Few of the most compelling questions when one takes up a challenge is —
What kind of AI model can solve such a problem?
Where do i get the data from?
How can i make it available to the user?
Whether any such solutions exist in the market?
And a lot more.
In today’s article we will take a look at the —
How, Why, Which, Where, Whats of an AI solution from ground up.
Without wasting much time let’s dive in into the questions.
What kinds of problems can AI solve?
If broadly considered, AI can tackle 2 major types of problems — Classification and Regression
Given the problem statement, if it can be categorised into either of the categories, then Voila, AI can solve your problems with mind blowing speed and accuracy.
If we were to simplify the answer, then if your problem statements requires any sort of predictive modelling or statistical output then AI is your goto.
As a sub part to the answer, there is a limited range of data types that it can handle, but the catch is if the input can be converted into a numpy array (numbers), then sky is the limit.
For more details into the question you can read a reference article here!
Why should I choose AI?
As compared to conventional technologies which run on template based engines or extensive mathematical modelling, AI runs on statistical modelling and multi-variate relation mapping. Leveraging such algorithms helps the system to provide state of the art accuracies and with extreme speeds.
The number of variations that AI can incorporate with minimalistic training is just phenomenal. Because the system has the capacity to understand not only spatial structure but also context and features of the input data, it can cater to larger variations and still provide great results.
Another one of the major advantages of AI is that, it has self evolving pattern recognition system, which learns and grows over time as it is exposed to higher volumes of samples, over and above that it doesn’t even require any pre-configuration to guide it through the process. Making the same model scalable and re-usable across multiple problem statements.
Last but not the least AI is as close as we get to imitating the high levels of intelligence demonstrated by humans as an overall evolved species.
What are are the requirements of building neural models?
One of the most amusing facts about AI based solutions is that their underlying requirements and workflow primarily remain the same across engagements. Let’s take a look at some of the basic fundamental requirements for developing the core solution —
Identifying the problem (as per question 1)
Understanding the Data availability
Data gathering
Data cleaning and pre-processing
Selection of resource (Cloud vs Open Source)
Selecting supporting frameworks and programming languages
Training & Testing
Understanding scalability and usability
Deployment for user consumption
Model versioning and tracking
Of the many mentioned requirements some of the most important include understanding the user impact it creates and the ease of consuming the same.
Many a times it happens that if the problem hasn’t been identified accurately it leads to under-performance or unsatisfactory results.
Where do I gather data from?
One of the biggest challenges faced by AI solutions is accumulation of dataset. For problems which are faced by people on a larger scale mostly there are huge volumes of open dataset available, but that also implies that tons of people have tried resolving the problem statement as well.
The challenge arises when one is trying to solve a niche problem or build a custom solution. In such cases it’s difficult to find already existing data suited to the requirement. There are two approaches to tackle such a situation based on the availability.
Suppose there is data already available, in such a case all that is needed is data and feature engineering to understand the relevance of segments of data pertaining to your problem statement.
Whereas if there is no data available whatsoever, which is the case in most instances. The best approach is to create small volumes of synthetic data to give a pre-emptive training to the model, simultaneously one needs to set up a data gathering pipeline from the data feed that is supplied to the system while the users consume the application.
This way your system learns, enhances and grows over time, without actually having the need to train it over large volumes of gathered data. To read more about this approach you can find the article here!
How can people use my solution?
Last but not the least, once everything has been build, if people cannot use the solution to make their everyday lives easier, then all of the effort goes in vain. This is where a new realm of topics of discussion opens up — MLOps.
Building a neural model is one-third of the task, another major segment of the complete process is deploying it as a consumable API for people around the globe to be able to use it.
A model creates its impact only when the world has access to it. Maintaining high accuracies, keeping up with trending data, managing model versioning are some of the trivial concepts taken care by MLOps.
Conclusion
At the end of the day, based on your problem statement some of the granular details of the requirements may change, but at a high level, no matter what kind of problem statement you pick if it has AI, then you are going to have to answer all the above mentioned questions.
I hope this article finds you good and helps you develop impactful solutions for the society. 😁
