Using AI to automatically estimate the status of a façade
Whenever a new property is submitted to Casavo’s specialists attention, it undergoes a sequence of steps which we summarize as eligibility evaluation process. This process aims to determine whether a given property is eligible for acquisition, namely whether Casavo could make profit from buying, renovating and replacing it on the market.
The list of factors that determine whether a property is eligible or not is long. Just as an example, the following are only few impactful aspects our specialists take into consideration:
- Price
- Location
- Renovation status
- Whether the property is in a building with an elevator
- … many more
It goes without saying that the evaluation of all these conditions, when performed on a large number of properties, is tedious, costly and time-consuming.
In order to facilitate and speed-up the whole thing, at Casavo we are working towards the automation of the eligibility evaluation process, by providing our specialists with automatic feedback about the properties that are submitted to their attention.
An attention element in the process is the status of the building the property is located in.
Indeed, Casavo tends to prefer acquiring houses located in buildings whose façade is in a good condition, mainly for two reasons:
- They are easier to sell
- Casavo cannot directly renovate the exterior of a building
These rationales lead Casavo to discard property whose façade shows signs of degradation, such as cracks, graffitis or peeling paint.
This is why we came up with a tool which can automatically evaluate whether the façade of a building is in good condition or needs renovation, just by looking at a picture of it.
The tool, named after the Pokémon Cloyster, is a Machine Learning based system, which has been trained on a dataset containing pictures of façades. By looking at the features of each picture, Cloyster is able to tell whether a façade is good or bad looking.
Let’s dive into the functioning of the system!
What kind of data do we need?
As you may know, Machine Learning models need data. In this case, our data is composed of pictures of façades.
Therefore we needed a way to collect as many images as possible. And not just that, we needed labeled images, namely for each picture collected we also wanted to know the condition of the façade depicted.
The Internet is full of pictures of façades, you can just google for pre-collected datasets.
However, there are a couple of reasons why those were not enough for our experiments:
- They were not labeled, namely we could collect a lot of pictures of façades, yet we lacked the information about the condition of each of them.
- Public datasets contain pictures of houses and buildings taken in the US, where the architecture and design of the houses is considerably different from the cities where Casavo operates.
- By just skimming through them, we found that most of the pictures we could collect were skewed towards façades in good condition, creating a problem of class imbalance.
It was clear that we had to put much effort into our data collection process.
In particular, we needed new sources of data, as well as labeling them.
And this is where the power of our whole Tribe Brokers came into play!
While the Machine Learning team was surfing the net, hunting for relevant pictures and labeling them (one by one!), the rest of the Tribe took part in the data collection by physically going around in the city they lived in and taking pictures of façades, also providing information about their status.
By doing so, we managed to put together a consistent amount of labeled data which we could employ to train our model!
We need to go deeper…
Let’s talk about the juicy part: the algorithm behind the magic.
The task we were dealing with was a Computer Vision one. Nowadays, the typical approaches for Computer Vision tasks all rely on the use of Deep Learning, a family of machine learning methods based on neural networks.
What’s fascinating about Deep Learning is that you can compose your own neural network as you like or need, by choosing from a wide range of building blocks. For image-related problems, you usually go for Convolutional Neural Networks, a class of neural networks that extract features from pictures by applying convolution operations on the input.
Nonetheless, neural networks are quite complex models, with tons of parameters. From one side, these parameters allow them to be extremely flexible and capable of learning even the most difficult tasks. On the other hand, in order to tune each of these parameters, you need a lot of data. Which we didn’t have, as most of our data collection was being carried out manually.
Luckily enough, Deep Learning developers can leverage a wide range of pre-trained models. These are neural networks which have been trained on other tasks and have gained a consistent amount of knowledge which can be exploited to solve the specific problem a developer deals with. In our case, we started from MobileNetV2, a well-known architecture pre-trained on ImageNet, a huge datasets of pictures coming from various domains.
Once you have your pre-trained backbone, which is exploited to extract features from our house pictures, we just attached a binary classification head to it, and only trained its parameters to tell good façades from degraded ones.
Our technological backbone
In Casavo, the standard for Deep Learning development is Pytorch Lightning, a framework that reorganizes standard Pytorch code in order to “decouple science from engineering” (quoting their own repository). Pytorch itself is a widely used library to develop Deep Learning applications, due to its completeness, versatility and vast community. Lightning enhances PyTorch capabilities by removing boilerplate code and by modularizing it, making ML-applications more scalable.
As mentioned already, when you deal with Machine Learning, you also deal with data. And dealing with data might be tricky, because of their size, location, and occasional change. This is why we make use of Data Version Control (DVC), a tool that helps us keep track and save different versions of our data, just like we do with code. DVC came in handy when we wanted to re-run old experiments with the data they were originally supposed to see, but was also helpful to track the evolution of our dataset as long as our project advanced.
What is not-so-sexy about neural networks is that they need a lot of computational power, which makes them hard to train on our laptops. This is the reason why developers mostly rely on cloud resources for training their algorithms.
In order to automate all the tiresome settings of cloud instances, we adopted Continuous Machine Learning (CML).
CML is an open source library that implements CI/CD in machine learning projects. We used it to spin up new cloud instances, launch training pipelines and shut instances right at the completion of the training phase. This is done smoothly and with simple workflows which can be triggered, among others, by GitHub actions.
Likewise, it is utterly important to keep track of the outcome of all the runs we launched, as to organize experiments, track their metrics and compare them in an effective way. For this, MLflow came to help. MLflow is an easy-to-use platform that exposes a set of APIs for logging the results of your experiments. It allows you to record hyperparameters, training metrics and save models.
After logging, MLflow can be interrogated to fetch the best performing model, according to a user-specified metric, and use it for inference purposes.
Last, but not least, we wanted Cloyster to be readily available within Casavo, to let people use it. For the deployment phase, we leveraged BentoML, a framework that enables a fast serving of your ML algorithms at scale.
Given that it natively supports Pytorch, BentoML is extremely easy to use. With BentoML, you can create services exposing APIs, which can be then interrogated for inference. Services can then be automatically containerized, as BentoML is able to generate a docker image including all the needed dependencies. After image generation, we simply deployed it in our production environment.
At the moment, Cloyster is up and running and can be used to automatically skim tons of pictures in search of façades which might be in need of renovation.
Conclusion
Cloyster is just a single example of how we exploit technology at Casavo. Developing tools like this enables us to streamline internal processes and serve our customers effectively.
As you might have understood from this article, our developers try to keep the pace of technological advances and are not afraid to try new libraries, tools or frameworks, as long as there are significant improvements in the value we deliver to our customers.
Nonetheless, the development of such tools does not merely rely on the skills of our Data Scientists. Their success depends on the capabilities of many people, ranging from our Product Managers, who come up with new ideas to enhance Casavo’s business, passing through our Software Engineers, whose core job is actually putting the bricks together to build our products, and finally coming to our Platform Engineers, who make sure the tools we develop are always available and usable.
Join us 🚀
Our Product & Tech team is continuously growing!
If you like our business and are interested to know more about us, feel free to check our open positions →. We’re always looking for awesome talents!
