Image Created with Nightcafe

Fast Prototyping of Artificial Intelligence Applications

Use Intel AI Reference Kits and Open-Source Software for Fast Prototyping

Whether you’re an established enterprise or a startup just getting off the ground, prototyping is an essential tool to verify that a proposed solution solves the identified problem, justify investment in future product development, and recruit supporters to your idea. The Intel AI Reference Kits enable developers to compose a prototype without writing too much boilerplate code, while providing a clear path from ideation to an attractive demo.

What Is an AI Reference Kit Prototype?

An “AI Reference Kit Prototype” comprises distinct processes that have been reimagined — transforming them from legacy implementation to machine learning solutions powered by the Intel hardware and software stack with integrations of popular open-source tools (Figure 1). An example we will visit later is introducing automated visual quality inspection to replace manual inspection of production line products.

Figure 1. Stack of components used in one of the AI reference kit

Each component of the prototyping framework starts with an AI Reference Kit and/or Edge AI Reference Kit refactored to enable a pseudo-microservice architecture (Figure 2). Refactoring the original kit makes integrating the final product into more complex applications easier.

Figure 2. Diagram of the adaptation flow of AI Reference Kits

This framework is ideal for simultaneously prototyping multiple parts of a workflow, showcasing how an optimized end-to-end solution could be made with Intel components to address a complex industry challenge. Let’s explore the framework below.

The AI Reference Kit Prototyping Framework

The framework is a simple set of steps for developers to follow as they leverage the AI Reference Kits to build prototypes. We will review these steps within the context of the AI Kit Prototype for the Pharmaceutical Manufacturing Business, which serves as the exemplar for this framework. The code and a step-by-step breakdown of the prototyping framework can be found here.

• Demand Forecasting Module: employs a time series prediction CNN-LSTM model optimized for Intel 4th Generation Xeon Scalable processors using Intel Extensions for TensorFlow, ensuring accurate demand projections for various products across multiple locales.
• Predictive Asset Maintenance Module: uses an XGBoost classifier with the Intel Extension for Scikit-Learn to preemptively flag equipment that needs service.
• Visual Anomaly Detection Module: Based on VGG-16 or Padim models, visual anomaly detection capabilities are embedded to quickly determine product quality via visual inspection, leveraging technologies such as OpenVINO and Anomalib.
• GenAI Chatbot Module: Complementing these is a generative AI chatbot powered by GPT4all-J LLM and RAG, tailored for interactions related to robotic maintenance situations.

All components are optimized for the Intel 4th Generation Xeon Scalable processors.

After deciding that we wanted to address inefficiencies in the pharmaceutical manufacturing lifecycle, we explored the AI Reference Kits and Edge AI Reference Kits to identify use cases that closely resembled the problem we wanted to solve with our prototype (Figure 3).

Figure 3. In collaboration with Accenture, Intel offers a series of downloadable AI reference kits to the open-source community to help enterprises accelerate their digital transformation journey. These kits are built upon the AI application tools that Intel provides to data scientists and developers.

We identified the Demand Forecasting, Visual QA/QC, Predictive Maintenance, and Chatbot reference kits. We cloned the appropriate repositories to gain access to the source code (Figure 4). For example, to clone the Predictive Asset Maintenance Kit, run git clone https://github.com/oneapi-src/predictive-asset-health-analytics.git.

Figure 4. Reference kits selected to help build our prototype AI application

The reference kits are structured as command-line Python scripts. Consequently, we refined these scripts into Python modules and libraries, preparing them for seamless integration with the APIs we planned to develop. For example, we refactored the Predictive Asset Maintenance Kit, originally designed to predict when powerlines need to be maintained. We pivoted the original use case into a predictive tool for maintaining robotics equipment on the pharmaceutical production line. Repurposing the original reference kits is where you should expect to spend the most time. The time savings come from leveraging the boilerplate code already in the kits.

The following steps involve building API endpoints, preferably using FastAPI or another Python API tool, and setting up the frontend using Streamlit. For example, here is the FastAPI script used to create and deploy the endpoints supporting the machine learning logic on a uvicorn server:

import uvicorn
from fastapi import FastAPI
import logging
import warnings
from predict import inference
import pandas as pd

from fastapi import FastAPI
from model import TrainPayload, PredictionPayload
from train import RoboMaintenance

app = FastAPI()

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
warnings.filterwarnings("ignore")


@app.get("/ping")
async def ping():
    """Ping server to determine status

    Returns
    -------
    API response
        response from server on health status
    """
    return {"message":"Server is Running"}

@app.post("/train")
async def train(payload:TrainPayload):
    """Training Endpoint
    This endpoint process raw data and trains an XGBoost Classifier and converts it to daal4py format.

    Parameters
    ----------
    payload : TrainPayload
        Training endpoint payload model

    Returns
    -------
    API response
        Accuracy metrics and other logger feedback on training progress.
    """
    model = RoboMaintenance(payload.model_name)
    model.process_data(payload.file, payload.test_size)
    logger.info("Data has been successfully processed")
    model.train(payload.ncpu)
    logger.info("Robotic Maintenance Model Successfully Trained")
    model.save(payload.model_path)
    logger.info("Saved Robotic Maintenance Model")
    accuracy_score = model.validate()
    return {"msg": "Model trained succesfully", "validation scores": accuracy_score}

@app.post("/predict")
async def predict(payload:PredictionPayload):
    
    sample = pd.json_normalize(payload.sample)
    results = inference(data = sample, model = payload.model, num_class = payload.num_class)
    return {"msg": "Completed Analysis", "Maintenance Recommendation": results}

if __name__ == "__main__":
    uvicorn.run("serve:app", host="0.0.0.0", port=5003, log_level="info")

We built three separate endpoints that respond to client requests through a standard REST API:

• /ping endpoint: returns “Server is Running” when the server is up.
• /train endpoint: receives training parameters and data and returns the location of the trained XGBoost classifier.
• /predict endpoint: receives the location of the trained classifier and raw data and returns maintenance classification.

We leverage the Streamlit to build a multi-page web application (Figure 5). Each component of the application has its frontend Streamlit script. The script for each pharma manufacturing frontend component can be found here.

Figure 5. Our prototype web application

The frontend and backend are connected using HTTP protocols. The endpoints are hardcoded into the frontend scripts but can be adapted to accommodate various deployment environments. If you intend to make this change, you can adjust the URL value in the Streamlit scripts:

URL = 'http://robot_maintenance:5003/train'
DATA = {'file':data_file, 'model_name':model_name, 'model_path':model_path, 
                  'test_size': test_size, 'ncpu': 4}
TRAINING_RESPONSE = requests.post(url = URL, json = DATA)

To make changes to the UI components, we recommend visiting Streamlit’s API Documentation.

As far as deployment goes, the framework leverages docker and docker-compose to manage the containerization and simultaneous deployment of multiple services. For example, we configure the predictive maintenance image in the following dockerfile:

FROM public.ecr.aws/docker/library/python:3.8

# copy assets over to image
COPY /src /robot_maintenance

# set the working directory
WORKDIR /robot_maintenance

# install dependancies
RUN pip3 install --user --no-cache-dir -r requirements.txt

# set PATH
ENV PATH=.local/bin:$PATH

# exposing endpoint port
EXPOSE 5003

ENTRYPOINT ["python", "serve.py"]

For more complex deployments, consider leveraging a container management tool like Kubernetes.

We use Python 3.8 from the AWS ECR Docker library. The config copies the ‘src’ folder to a ‘robot_maintenance’ directory in the image and sets it as the working directory. Dependencies are installed from ‘requirements.txt’ without caching. The PATH environment variable is adjusted, port 5003 is exposed, and the container runs ‘serve.py’ on startup.

Using docker-compose, in the absence of an object store or database, we bind a local directory to serve as a data layer across the application. This is not typically done in a production setting but works well for prototyping. The following docker-compose config file shows how this tool configures each of the four services and the frontend for deployment:

version: '3'
services:

  medication_demand_forecast:
    build:
      context: ../medication_demand_forecast/
      dockerfile: Dockerfile
    ports:
      - 5001:5001
    volumes:
      - <local directory>:<target container directory>
    restart: on-failure

  medication_qaqc:
    build:
      context: ../medication_qaqc/
      dockerfile: Dockerfile
    ports:
      - 5002:5002
    shm_size: '20gb'
    volumes:
      - <local directory>:<target container directory>
    restart: on-failure

  robot_maintenance:
    build:
      context: ../robot_maintenance/
      dockerfile: Dockerfile
    ports:
      - 5003:5003
    volumes:
      - <local directory>:<target container directory>
    restart: on-failure

  supportbot_chatbot:
    build:
      context: ../supportbot_chatbot/
      dockerfile: Dockerfile
    ports:
      - 5004:5004
    volumes:
      - <local directory>:<target container directory>
    restart: on-failure

  app_frontend:
    build:
      context: ../app_frontend/
      dockerfile: Dockerfile
    ports:
      - 5005:5005
    volumes:
      - <local directory>:<target container directory>
    restart: on-failure

For efficient setup and deployment from the command line, we leveraged the Linux make utility. You can find the make file for this prototype here.

Summary and Discussion

The article discusses the AI Reference Kit Prototyping Framework, powered by Intel’s AI Reference Kits and open-source software, designed to streamline AI prototype development. The framework aids in transforming traditional components into machine learning solutions optimized for Intel hardware and software. This framework allows developers to efficiently prototype various workflows, as illustrated with the AI Reference Kit prototype for the Pharmaceutical Manufacturing Business.

I encourage you to join the Intel Developer Cloud, start an instance, clone the repository, and launch the Pharmaceutical Manufacturing prototype. You can start exploring the AI Reference Kit Library and Edge AI Reference Kits for opportunities to build meaningful prototypes using this framework.

Don’t forget to follow my profile for more articles like this!