PyTorch and Mars Petcare: Formulas, photos & feces

Authors: Mark Parkinson, Next Generation Technologies Director, Andre Dourson & Michael Fitzke, Next Generation Technologies Senior Leaders, and Robert Wiggall, Global Pet Testing Manager at Mars.

At Mars Petcare, we are inspired every day by our Purpose: A Better World for Pets. We want to improve the lives of pets all over the globe because we know that they are your family and companions. Pets and humans can positively effect each other’s wellbeing, and we reflect this belief with our dog-friendly office policy that encourages Mars Associates to bring their dogs into the office. And it’s not just in our offices that we manifest that belief!

The Next Generation Technologies team at Mars is always looking for ways to apply latest technology to make products and services to create better lives for people and pets.

One of our latest projects specifically focused on providing healthy diet for pets — by taking a look at their feces. The consistency of dog poo is an important health consideration; it’s something every dog does every day, and every owner must deal with it! Often, pet owners only link the extremes of poo consistency to health implications. Owners expect a perceivable ‘healthy’ consistency of their dogs’ waste, and they want to see, collect and dispose of poo in the most convenient way.

Feces consistency is an important consideration when ensuring pet food products meet pet owner expectations. Mars developed a standardized classification chart (shown below) where feces is graded by a human scorer on a scale ranging from 1–5. The scale ranges from extremely hard (dry, crumbly) feces (score 1) to a liquid consistency/diarrhea with no defined shape or structure (score 5).

Notice: The link includes images of feces, open at your own discretion.

Click on this link to see the images and formal scoring system developed by our researchers.

This process is time consuming and the human scorer has a level of subjectivity. To eliminate this inconsistency, allow more rapid capture of feces quality, and enable more accurate assessment, we came up with a solution that leverages PyTorch. This library can be used to train computers to interpret and understand feces using artificial intelligence.

Over the years, we have compiled a large library of labelled images of dog waste from feeding trials conducted at our facilities around the world, but we also reached out to our pet-loving Associates to help as well to build our model. We asked them to send us images of their dogs’ feces to help develop our model. The support was overwhelming, and we received images from hundreds of Associates all over the world.

At the end, we were able to collect a dataset of 12,000+ feces images that were manually labeled by domain experts into 17 classes. We used this data set to train machine learning models to classify the images. Naturally, we used artificial neural networks, which are well known to perform extremely well in the field of computer vision.

We did an architecture search over several models from the torchvision model zoo (A library of artificial neural networks). The main model of our final system is a ResNet-50 that was pre-trained on the large image dataset imagenet. Internally we coined the fully trained network TurdNet.

As the dataset was heavily imbalanced with a very limited number of samples for the most extreme ends of the scale, we used data augmentation techniques such that the model would generalize better (flipping, rotation, zoom, etc.). We also generated synthetic samples (leveraging a generative adversarial network) for the most extreme scores to have a more even distribution within our entire training set.

We trained the model using several regularization technics (weight decay, dropouts, discriminative learning rates), and we also used a modified variant of the Focal loss function (https://arxiv.org/abs/1708.02002) as we wanted to force the neural network to learn more from the less represented classes.

We implemented a specific metric named “accuracy in a range,” to validate the quality of our predictions within a range around the actual value. We managed to reach 92% for our accuracy in a range metric, which is far above the human scorers evaluated on the same metric (43%).

The model was deployed as an application programming interface (API) and accessible through a small PowerApps tool. See the image below:

Leveraging a model to provide image classification enabled us to build a powerful application, allowing us to do smarter business by improving consistency, accuracy and timeliness of pet backend (pun intended) performance.

PyTorch was crucial to enabling us to rapidly prototype, demonstrate technical viability and create at pace. Our world class scientific research underpinned by our Digital Technologies data science teams leveraging PyTorch is a winning combination.

Mars Petcare is hiring! See opportunities here: https://careers.mars.com/us/en/c/it-digital-jobs.