Using ML.NET in Xamarin projects
A little more than a month ago I asked my followers on Twitter what they would want me to write about:
As it turns out, AutoML and Xamarin were the most requested combination. So, let’s talk about it!
If you’re trying to learn more about machine learning, but don’t have a special degree and/or experience there’re always several ways to start:
- theory (algorithms, creating models, learning about tools, etc),
- jump-start playing with pre-built models, APIs and SDKs (cognitive services, AI building blocks from Google, etc.) or
- combine both approaches.
When I’m asked about tools for beginners, I recommend Cognitive Services. Microsoft provides great documentation, there are lots of samples online, and you can access the services through a REST API, so they can be implemented in all kinds of applications.
If you’re a .NET developer who has already checked out Cognitive Services, and want to have a more customized experience, or you just want to build ML models based on your custom datasets, then ML.NET can be the right tool for you. I first heard about ML.NET at Build when it was released; since then it has gotten a slew of new features and improvements. I’m not going to go into details about ML.NET since they’re all covered in the docs, but I want to highlight a couple of things:
- ML.NET has been used internally by many Microsoft teams for more than 8 years in products such as Bing Ads, PowerBI, Azure ML Studio, etc.
- ML.NET is made for developers — with or without a data science background
- With ML.NET you can consume other popular ML frameworks (TensorFlow, ONNX, Infer.NET, and more)
Building new models, figuring out the best approach, and the most efficient model type can be very confusing especially when you’re just starting out, but AutoML is designed to help you to solve those problems. There are three ways to use automated ML:
- Using a graphical user interface with the ML.NET Model Builder
- On the command line with the ML.NET CLI
- Via an application with the automated ML API
Let’s dive into the first approach:
As a sample, I decided to create a very simple messaging app with just two fields: recipient and message and a send button. The goal of this app is to use a machine learning model to decide if the message a user is trying to send might be offensive and warn the sender — an idea was shared with me by one of the attendees at Xamarin Dev Summit (https://xamarindevelopersummit.com/). Side note: if you’re that great person and you’re reading this post, please, remind me your name, so I can give you proper credit.
I started with creating a blank Xamarin.Forms app in Visual Studio. I switched to Visual Studio 2019, but VS 2017 can be used as well.
I’m sure you know the structure of a Xamarin.Forms solution. You can add support for UWP, TvOS, WatchOS, etc, but I just added Android and iOS projects.
Then I added ML.NET Model Builder Visual Studio extension from the Visual Studio Marketplace (note: you need to do it only once).
Now with all the tools ready, it is time to think about data for the machine learning model. I usually use Kaggle for my data needs. This time I was also able to find a dataset with labeled text messages: 1 for offensive and 0 for neutral. The dataset wasn’t very well maintained but I thought it would be good enough for testing purposes. If you’re building production grade models, definitely make sure that the data is correct, unbiased, and the dataset is well maintained. Data is the base of every machine learning model; it’ll only know and learn from data you provide.
The next step was to create a machine learning model using Model Builder and the dataset. You can find the detailed instructions here. The model builder is checking several types of models and uses the best one for your dataset, and also gives you a recommendation on how long you need to train your model for. I chose “Sentiment Analysis” scenario, uploaded data file and selected “label” column for prediction. That’s pretty much it — Model Builder did everything else for me.
As a result, Model Builder added 2 more projects to my solution: ML.NET model project and a Console App for testing the model.
I tried to make it work with Xamarin without additional manipulations, by just using ML.NET in the shared code to access the model and provide predictions. Unfortunately, I was getting System.DllNotFoundException:’CpuMathNative’ error. I ended up reaching out to Cesar De la Torre Llorente — Principal PM for ML.NET. As it turned out, ML.NET doesn’t support ARM processors right now due to native C/C++ code is not being ported to ARM. It would be absolutely amazing if they added ARM support soon so we can enjoy the benefits of ML.NET in Xamarin and IoT projects without additional middleware. If you think the same, please, reach out to me or Cesar directly (you can find his info here or just comment/vote for the issue on GitHub), so his team can prioritize it.
I still wanted to finish my project at this point, so I created a Web API to wrap ML.NET functionality as a workaround. It’s a simple API with only one controller used to return a prediction based on the MLModel. In our case, “Prediction” is just a Boolean value indicating if the message is rude/offensive or not. I decided it would be easy to host the Web API on Azure, so I just created an Azure Web App right from my Visual Studio (Publish dialog) and deployed my code there.
From the shared code in Xamarin, I was passing the text of the message to the API and getting a prediction back. Then, I was using SMS class from Xamarin.Essentials to either send the message to the recipient or display an alert dialog with a warning that the message could be rude or offensive. Users can then confirm if they still want to send it.
if (prediction)
{
var sendText = await DisplayAlert("Warning", "The message you're about to send might be rude and/or offensive. Are you sure you want to send it?", "Yes", "No");if (sendText) await SendSms(text, recipient);
}
else
await SendSms(text, recipient);
The Xamarin.Essentials library provides cross-platform APIs so you don’t have to write platform-specific code. Really easy.
I’m definitely disappointed I couldn’t use ML.NET in a Xamarin.Forms app directly and had to figure out a workaround. That said, I’m glad I was still able to use AutoML, and build a high efficiency model using a custom dataset to create a POC for something that could potentially be a very useful app.
You can find my code for this proof of concept project on GitHub: https://github.com/Veroni4ka/messaging-app
