Smile detection for image moderation

Assessing the attractiveness of images by capitalizing on Smiles

Nadir Trapsida
Dec 15, 2020 · 8 min read

What is under the hood?

Face Detection

Single face detection using Dnn model
Multiple faces detection using Dnn model
Large faces detection using Dnn model
Single and multiple faces detections using Dlib HOG model
Large faces detection using Dlib HOG model
Face detection after combining Dnn (red) and HOG (green)
Jaccard index computation visual from Source
Face detection after the Jaccard index filter

Landmarks

Dlib face landmarks output points Source

Geometric criteria

Mouth opening computation using distances
Mouth direction computation using a linear function
Mouth position computation using geometric distances
Smile detection final result with bounding boxes

Test it yourself

curl --location --request POST 'http://sportvision.api.decathlon.com/v2/smile-detector/predict/' \
--form 'file=@/<IMAGE_ABSOLUTE_PATH>'
{
"coordinates": [198, 51, 333, 217],
"landmarks": [
[205, 89],[198, 108],[193, 130],...
],
"smile_score": 0.6
}

Next steps

Example of facial actions units from Source

Further reading

Decathlon Technology

Empowering The Sport Tech Community