BlazeHand : A Machine Learning Model for Detecting Hand Key Points
This is an introduction to「BlazeHand」, a machine learning model that can be used with ailia SDK. You can easily use this model to create AI applications using ailia SDK as well as many other ready-to-use ailia MODELS.
Overview
BlazeHand is a machine learning model that detects key points of the hand. Since it can detect detailed hand movements, it can be applied to gesture recognition.
Detected landmarks follow the following structure.
BlazeHand architecture
BlazeHand consists of two models, BlazePalm
and BlazeHand
. After detecting the hand position from the input image with BlazePalm, keypoints of the hand are detected from the hand image with BlazeHand.
The hand detection by BlazePalm is very demanding if it is processed every frame, it may also lose track of the hand. Therefore, in the first frame, BlazePalm performs the detection process, and in subsequent frames, it calculates a slightly larger Rectangle (ROI) from the key point of the hand detected by BlazeHand, and applies BlazeHand to that Rectangle to move the Rectangle. This enables fast and robust recognition.
BlazePalm, the hand position detector, is a simple SSD-based detector with a similar architecture to BlazeFace.
BlazeHand, which detects key points of the hand, has an architecture similar to FPN. For training, real world images were used as well as computer generated synthetic images.
The output of BlazeHand contains 21 set of (x, y) coordinates, relative depth, plus 2 flags: Hand Presence
, which indicates the probability of hand presence in the input image, and Handedness
, which indicates whether the hand is left or right.
BlazeHand usage
The following commands runs the model using the web camera input.
python3 blazehand.py --video 0
Here is the kind of result you can expect.
ax Inc. has developed ailia SDK, which enables cross-platform, GPU-based rapid inference.
ax Inc. provides a wide range of services from consulting and model creation, to the development of AI-based applications and SDKs. Feel free to contact us for any inquiry.