ailia SDK Now Available Through Flutter pubspec

ailia SDK now supports installation using Flutter’s pubspec. This allows you to easily integrate the ailia SDK into your Flutter applications.

About ailia SDK

ailia SDK is an AI inference engine that allows you to easily integrate various AI models published on ailia MODELS into Unity. The applications developed with it can run on Windows, macOS, iOS, Android, and Linux.

ailia SDK

ailia MODELS

About Flutter

Flutter is a cross-platform application development environment developed by Google. It allows for unified development of applications for Windows, macOS, iOS, Android, and Linux using the Dart programming language.

Flutter — Build apps for any screen

About pubspec

The pubspec file is the package management file in Flutter. By registering the library URL in the pubspec file, you can easily add libraries to your application.

Installing ailia SDK via pubspec

Add the following to your pubspec.yaml file. Then, run flutter pub get to install the necessary libraries.

ailia:
    git:
      url: https://github.com/axinc-ai/ailia-sdk-flutter.git

  ailia_audio:
    git:
      url: https://github.com/axinc-ai/ailia-audio-flutter.git

  ailia_tokenizer:
    git:
      url: https://github.com/axinc-ai/ailia-tokenizer-flutter.git

  ailia_speech:
    git:
      url: https://github.com/axinc-ai/ailia-speech-flutter.git

For macOS, to access the license file, set com.apple.security.app-sandbox to false in macos/Runner/Release.entitlements and macos/Runner/Debug.entitlements.

Usage of ailia SDK with Flutter

Please refer to the following repository to have access to ailia MODELS samples in Flutter.

GitHub — axinc-ai/ailia-models-flutter: ONNX Model Library for Flutter

Currently, the following four models are supported, with more to be added in the future:

• Object identification using ResNet18
• Object recognition using YOLOX
• Speech recognition using Whisper
• Text embedding using MultilingualE5

When launched, the screen will look like this:

Select a model from the list and press the plus button to download the model and enable inference.

For example, Whisper can be executed with very short code, making it easy to implement speech recognition in your Flutter application.

import 'dart:io';

import 'package:flutter/material.dart';
import 'package:wav/wav.dart';

import 'package:flutter/services.dart';
import 'package:ailia_speech/ailia_speech.dart' as ailia_speech_dart;
import 'package:ailia_speech/ailia_speech_model.dart';

class AudioProcessingWhisper {
  final AiliaSpeechModel _ailiaSpeechModel = AiliaSpeechModel();

  void _intermediateCallback(String text){
  }

  Future<String> transcribe(Wav wav, File onnx_encoder_file, File onnx_decoder_file, int env_id) async{
    _ailiaSpeechModel.create(false, false, env_id);
    _ailiaSpeechModel.open(onnx_encoder_file, onnx_decoder_file, null, "auto", ailia_speech_dart.AILIA_SPEECH_MODEL_TYPE_WHISPER_MULTILINGUAL_TINY);

    List<double> pcm = List<double>.empty(growable: true);

    for (int i = 0; i < wav.channels[0].length; ++i) {
      for (int j = 0; j < wav.channels.length; ++j){
        pcm.add(wav.channels[j][i]);
      }
    }

    //_ailiaSpeechModel.setIntermediateCallback(_intermediateCallback);
    _ailiaSpeechModel.pushInputData(pcm, wav.samplesPerSecond, wav.channels.length);

    _ailiaSpeechModel.finalizeInputData();

    String transcribe_result = "";

    List<SpeechText> texts = _ailiaSpeechModel.transcribeBatch();
    for (int i = 0; i < texts.length; i++){
      transcribe_result = transcribe_result + texts[i].text;
    }

    _ailiaSpeechModel.close();

    return transcribe_result;
  }

}

ailia-models-flutter/lib/audio_processing/whisper.dart at main · axinc-ai/ailia-models-flutter

Additionally, by using Multilingual E5, you can easily implement Retrieval-Augmented Generation (RAG) in Flutter. The ailia SDK provides ailia Tokenizer, which allows you to easily obtain embeddings from Japanese text.

ailia-models-flutter/lib/natural_language_processing/multilingual_e5.dart at main ·…

Setup of the license file

To use the evaluation version of the ailia SDK, a license file is required. The license file can be downloaded with the following code. This code is included in ailia MODELS Flutter.

import 'package:ailia/ailia_license.dart';
await AiliaLicense.checkAndDownloadLicense();

ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.

ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.