Released ailia SDK 1.2.16

We are pleased to introduce version 1.2.16 of ailia SDK, a cross-platform framework to perform fast AI inference on GPU or CPU. You can find more information about ailia SDK on the official website.

FP16 support for Vulkan

FP16 mode is now supported when using Vulkan to speed up inference in environments with limited memory bandwidth, such as mobile GPUs.

For example, when inferring YOLOX-S with Mali-G78, it takes 156 ms with FP32, but 117 ms with FP16.

Vulkan 1.2 or later is required to use FP16. Vulkan 1.3 is the standard for Android 13 or later, so it can be used as standard on those devices.

Lower memory footprint

The memory usage was reduced by freeing up CPU memory after weights are transferred to the GPU. In addition, the algorithm allowing memory reuse for cuDNN has been improved to allow more GPU memory to be efficiently reused.

Support for ONNX over 2GB

Support for loading external pb was added, allowing for loading ONNX over 2 GB.

Voice processing optimization

Performance of the voice processing model has been improved by optimizing the processing of Concat, which has a small number of channels, and Conv1D, which has a large kernel size. As a result CPU inference for Crepe is about 2.6 times faster, and the CPU inference for RVC is also speeded up by approximately 20%.

Support for opset17

LayerNormalization in opset17 is now supported. This allows inference of voice models using this operator, such as models exported with slightly older RVC.

Addition of ailia.js

The ailia.js library is now officially supported to perform inference in thee web browser. More details on ailia.js can be found below.

ailia.js: a library to execute ML models inside the browser

New models available in ailia SDK 1.2.16

Thanks to the support of external pb explained above, we now support much larger models to run.

SegmentAnything: high-quality segmentation model by Meta

ailia-models/image_segmentation/segment-anything at master · axinc-ai/ailia-models

Segmentation by coordinates ( Source: https://github.com/facebookresearch/segment-anything/blob/main/notebooks/images/truck.jpg)

BLIP2: high-precision image captioning model

ailia-models/image_captioning/blip2 at master · axinc-ai/ailia-models

$ python3 blip2.py

BLIP2 example input (Source: https://github.com/salesforce/LAVIS/blob/main/docs/_static/merlion.png)

### Output ###
singapore merlion fountain

Whisper Large V2, V3: speech recognition models over 2 GB

ailia-models/audio_processing/whisper at master · axinc-ai/ailia-models

$ python3 whisper.py -m large-v3

Multilingual E5 Large: latest text embedding model

ailia-models/natural_language_processing/multilingual-e5 at master · axinc-ai/ailia-models

$ python3 multilingual-e5.py -m large

ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.