ARTIFICIAL INTELLIGENCe
AI with ML.NET for text classifications
Developing the first AI app in C#
Recently, Artificial Intelligence (AI) has been gaining popularity at breakneck speed.
OpenAI’s ChatGPT was a breakthrough in Artificial Intelligence, and the enthusiasm was huge.
ChatGPT triggered a trend towards AI applications that many companies followed.
You read and hear about AI everywhere. Videos and images of celebrities performing strange dance moves appear, or you hear interviews and songs by artists who have “actually” been dead for several years.
The exact functioning of Artificial Intelligence will be explained below. As developers, we are used to thinking in categories and breaking problems down into small steps, and this is exactly how all Artificial Intelligence works. They are based on so-called models that have been trained for their respective areas of application.
Example:
If you ask ChatGPT for a picture, a separate model such as DALL-E must be used.
If you ask for a poem, ChatGPT uses a language model.
ChatGPT is nothing more than a ChatBot that uses different models for the task at hand.
This is where I would like to start and program a small Artificial Intelligence that classifies a customer review, which is available as free text, as positive or negative.
As a .NET developer, Microsoft offers a low-threshold entry into the world of machine learning with ML.NET. This free open-source framework makes it easy to integrate ML models into new and existing .NET applications — without any knowledge of Python, R, or other ML libraries. ML.NET already contains many ready-made models for common use cases and can be easily integrated into the existing .NET app lifecycle.
Basics of machine learning
The term “machine learning” describes the approach of teaching computers to learn patterns and relationships from data instead of explicitly programming them with rules and program logic. The aim is to enable the machine to “learn” independently from sample data and apply this knowledge to solve similar problems.
There are essentially three types of learning:
Supervised learning
Supervised learning is a method of machine learning in which a model is provided with training data consisting of input data (e.g. texts) and the corresponding desired outputs (e.g. classifications such as ‘positive’ or ‘negative’). Using these examples, the system learns to recognize patterns that relate the inputs to the outputs. This enables it to later predict correct outputs for new, unseen inputs. A classic application example is object recognition in images.
Example:
An application example for supervised learning with ML.NET could be the classification of product reviews as ‘positive’ or ‘negative’. The procedure would be as follows:
// Loading the training data with reviews and labels
var data = mlContext.Data.LoadFromTextFile<ReviewData>(pathToFile);
// Creating the processing pipeline
var pipeline = mlContext.Transforms.Text.FeaturizeText("ReviewText", "Features")
.Append(mlContext.BinaryClassification.Trainers.SdcaLogisticRegression("Label"));
// Training the model
var model = pipeline.Fit(data);At this point, we have a trained model that can classify new reviews.
Unsupervised learning
Unsupervised learning is a machine learning approach in which the model is only provided with unlabelled input data, e.g. text or images, without any associated target variables or classifications. The system must independently recognize structures, patterns, and correlations in this raw data. A typical area of application is clustering, in which similar input data is summarised into groups. Possible examples are the identification of customer segments based on marketing data or the grouping of news articles according to subject areas when analyzing text.
Example:
Unsupervised learning can be used with ML.NET for text clustering, for example:
// Loading the text data
var data = mlContext.Data.LoadFromTextFile<ArticleData>(pathToFile);
// Create the text featurisation pipeline
var pipeline = mlContext.Transforms.Text.FeaturizeText("ArticleText", "Features");
// Train the clustering model
var model = mlContext.Clustering.Trainers.KMeans(pipeline, numberOfClusters: 5).Fit(data);
// Predict the cluster membership for new articles
var predictions = model.Transform(data);In this way, for example, news articles can be automatically categorized into thematic clusters such as politics, business, sports, etc.
ML.NET focuses mainly on supervised and partially unsupervised learning. ML.NET does not currently offer any specific support for reinforcement learning, in which an AI is trained through rewards in a simulated environment. This learning paradigm is used in particular for strategy-finding tasks, such as games or the optimization of processes.
Reinforcement Learning
Reinforcement learning is a learning paradigm in which Artificial Intelligence is trained by trial and error in a simulated environment. In contrast to supervised learning, there are no complete training examples with input-output pairs. Instead, the system receives a score (reward) for each action, which indicates how “good” this action was. By maximizing the accumulated rewards, the AI learns to find the optimal strategy for a task. This principle of reinforcement learning is used in particular for tasks that require a complex strategy to be found. Examples include board games such as chess or Go, but also applications for robot control, process optimization, or autonomous driving.
Example:
A classic application example is a chess player agent. Here the environment could be a chessboard, possible actions are move movements. The agent then receives an evaluation for each move, e.g. +1 for a sure pawn win, -5 for an impending piece loss. Through many training games, the AI learns which moves to lead to a win in the long term and thus maximizes its cumulative reward.
Set up the development environment
To get started with ML.NET, we need the .NET Core or .NET Framework environment in version 4.6.1 or higher. ML.NET can be used in Visual Studio, Visual Studio Code, or the command line.
The first step is to install the required NuGet packages of ML.NET:
Install-Package Microsoft.ML
This main package contains basic functionality such as the ML.NET context, data transformations, and catalogs with models and tasks. For additional functionality, other packages such as Microsoft.ML.Vision for image processing or Microsoft.ML.Recommender for recommender systems can be added.
After installation, we can create a .NET Core or .NET Framework console application in Visual Studio and start development.
Optionally, integrated tools such as the Model Builder interface in Visual Studio or the ML.NET CLI can be used to visualize data and create models. I’ll come back to this later.
Creating the first ML model
In the following, the basic concepts of ML.NET are explained step by step using an example:
The classification of customer reviews is based on free text.
The result should be to categorize these ratings as positive or negative.
This is a classic supervised learning problem.
We create a new C# console application and import the ML.NET NuGet packages.
Next, we load the data set into the working memory. In our example, we need a TSV (tab-separated file) file for this. This is represented by a simple class in our console application:
This class stands for each line within the TSV file.
public class ReciewData
{
}Now we add properties for each column of the TSV data:
public class ReviewData
{
public string ReviewText { get; set; }
public bool Label { get; set; }
}
Unfortunately, this is not enough to load this TSV file and teach ML.Net how to reference this file.
We need attributes for this: LoadColumn plus the index of the column.
public class ReviewData
{
[LoadColumn(0)]
public string ReviewText { get; set; }
[LoadColumn(1)]
public bool Label { get; set; }
}This is a simple mapping from the ReviewText property to the TSV file ReviewText column.
Now let’s take care of the storage paths for the files and save them in our application:
First, we create the path to our training file, the TSV file (Review.tsv).
string baseDirectory = @"..\..\AI";
string subDirectory = "TrainingData";
string trainingDataFilePath = Path.Combine(baseDirectory, subDirectory, "Reviews.tsv");We will use this file to train the model that we want to use for classification.
Now we define the path to the model that we want to train. This model is saved as a ZIP file.
string modelPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "model.zip");Now we need a variable for our MLContext. The MLContext is comparable to an EF DbContext. So, whenever you want to work with machine learning, you have to interact with the MLContext, similar to the DBContext in EF Core.
Now we are ready to load the data. We can manipulate it as we need it to train the model according to our requirements.
Now we build our model:
For this, we need a specific type from ML.NET. The IDataView:
var trainingDataView = mlContext.Data.LoadFromTextFile<ReviewData>(trainingDataFilePath, separatorChar: '\t', hasHeader: true);Here I can now use a function from the MLContext called LoadFromTextFile. `<ReviewData>` is now used to project the data into our initially created class. The path to the training file must now be passed to this function. As a bonus, we can specify further options here, such as the separator from the TSV file and whether our training file has a header or not (we have a header, so I set this parameter to true).
Now that we have loaded the data into a DataView, we can start preprocessing. Preprocessing is a means of creating what is called a pipeline. This can later be used to perform predictions when we have incoming data (e.g. our customer score). To do this, we need to extract the data from the data view we have created (IDataView) and then transform it so that it can be used as part of the machine-learning process.
Now let’s create our pipeline. This has a specific return type: IEstimator<ITransformer>:
/// <summary>
/// Creates the machine learning pipeline for review classification.
/// </summary>
/// <param name="context">The ML context.</param>
/// <returns>The machine learning pipeline.</returns>
public static IEstimator<ITransformer> CreatePipeline(MLContext context)
{
return context.Transforms.Text.FeaturizeText("Features", "ReviewText")
.Append(context.BinaryClassification.Trainers.SdcaLogisticRegression(labelColumnName: "Label"))
.AppendCacheCheckpoint(context);
}Context.Transforms.Text.FeaturizeText is a text transformation component in ML.NET that is used to convert text data into numerical features.
“Features” is the new column name that contains the resulting numerical characteristics.
“ReviewText” is the name of the column from the TSV file that contains the text to be converted into numerical characteristics.
Now the training of the model begins:
`.Append` adds the next step to the pipeline. The model is trained with `Context.BinaryClassification.Trainers.SdcaLogisticRegression`, which defines the algorithm with which the model is to be trained — in this case the SdcaLogisticRegression algorithm. “Label” is the column or the name of the column that represents the target variable to be predicted. In our example, it is the classification of whether this evaluation is positive or negative.
Let’s summarise this again:
This command builds a pipeline that performs the following steps:
- text featurisation: the text from the “ReviewText” column is converted into numerical features and stored in the new “Features” column.
- model training: The numerical features in the “Feature” column are used to train a binary classification model with the target variable in the “Label” column.
Now we have created our pipeline and can start training the model.
var pipeline = CreatePipeline(mlContext);
var model = pipeline.Fit(trainingDataView);We use the `pipeline.Fit(…)` command to train the model on the data from the TSV file. The model is then stored in memory.
However, to save the model and not have to create and retrain it each time, we can use the `Save()` method. This allows us to save the trained model in a specific file path. In this way, we can simply load and use the model when needed without having to retrain it each time.
There is an option in MlContext where we can find the Save() function:
context.Model.Save(model, schema, modelPath);Firstly, we transfer the trained model and the schema from the IDataView from the TSV file, and then the path where we want to save the model.
We have now trained our model and saved it to the hard disc.
Now we need another class that represents the trained DataSet, i.e. the results from our model (prediction class). To do this, we create another class that has a similar structure to the class for the DataSet from the TSV file (ReviewData).
public class ReviewPrediction
{
[ColumnName("PredictedLabel")]
public bool PredictedLabel { get; set; }
}The attribute defines that the name of the “PredictedLabel” column is to be used in the MLContext to read the predicted value there.
To connect the model to the prediction class, we now need a PipelineEngine. We create this engine again from the MLContext.
var predictionEngine = mlContext.Model.CreatePredictionEngine<ReviewData, ReviewPrediction>(model);
We use the generic types TSrc (Source) and TDst (Destination) for our input and output classes, “ReviewData” and “ReviewPrediction” (“Good rating”, “Positiv”).
The individual components and their interaction should now be understandable: Why we need to convert texts into numerical features, why we need to prepare data so that the computer can understand it, and that our “ReviewData” class is our source class and “ReviewDataPrediction” is our target class.
Now that we have completed the preparation steps, we can move on to the prediction phase.
Let’s now take care of the code that performs the prediction and returns the expected target variable.
In our example, the predicted result is a Boolean value.
The first step is to enter a score, which we then write to our “ReviewData” input class.
var input = new ReviewData { ReviewText = inputReviewText };The variable “ReviewText” is a string variable that contains our review.
We now pass this input variable of type “ReviewData” to the pipeline that we have just created.
As a result, we receive our “ReviewPrediction” output class and can now access the target variable from the model via the variable.
// Classify the new rating and display the result
var prediction = predictionEngine.Predict(input);
Console.WriteLine($"The classification for the rating ‘{ input.ReviewText}’ is: { (prediction.PredictedLabel ? "Positiv" : "Negativ")}");If the model had good training data, we should now get the correct predicted results.
It seems to work.
Of course, this is only a small example, and the model is not perfect, but if you train it well, it should ideally always return correct results.
The following extensions have been added to the example to retrain the model if inputs were incorrect or unknown.
When starting the application, you can choose between adding new data to the model and direct training or you can create new comments.
In the following article, I will take a closer look at image recognition and prediction.
Summary:
Although this article is limited to text classification, on closer inspection, and with a little creativity, the approach described opens up numerous possibilities. For example, the recognition of a certain classification could be used to trigger further processes automatically.
Overall, it can be said that Artificial Intelligence has enormous potential to support us in many areas of everyday life and to optimize processes. Text classification is just one example of the many possible applications. It can be assumed that more and more areas will benefit from the possibilities of AI in the future and that this will open up new opportunities for companies and private individuals alike.
To fully exploit this potential, it is important that we are open to new technologies and that we address the possibilities and limitations of AI. This is the only way we can ensure that Artificial Intelligence is used responsibly and for the benefit of all in the future.
Links:
