Today, one of the components of uKit AI, an intelligent website redesign system, is going public. It’s a prototype module for evaluation the beauty of web pages. It combines a neural network and decision trees to imitate the reaction of an ordinary visitor to athe website’s design.
In the future, such a module will evaluate the work of the generative design algorithm, the key element of uKit AI, which will design pages without human involvement relying on the available content and “knowledge” of the difference between a non-effective website and the one aimed at increasing the conversion rate.
The current WebScore AI version reflects the view of an average Internet user about the website appearance. Although, we can create other options, for example, it’s possible to rate a website usability.
Websites used for training the system. First of all, we’ve collected 12,000 sites and online stores created in different years on various platforms and in different languages. The main task was to get enough visual gradation examples, from pretty bad websites to very good ones. This way, we’ve shown the system what it can come accross in the modern web.
Each gradation is measured with a scale, and this scale is supposed to be understood by an ordinary person whose opinion we try to model. So, we came up with an idea of the ‘from 1 to 10’ scale , which is used in our service.
People who are imitated by WebScore AI. We needed two things to form a dataset (a set of data for a training model) from a variety of websites:
- the signs by which the system will determine if the website is attractive;
- the evaluates (marks) made with the help of our scale for a certain amount of websites. They will become a model for the system.
Someone should put these initial evaluates. Such a “teacher”, or a group of “teachers” , to be more specific, will greatly affect how the model will work.
To gather a focus group, we conducted a preliminary selection of candidates on 1500 website examples. A routine work, but a responsible one and requring great focus. The preliminary selection helped us to eliminate unsuitable candidates and also to exclude the “controversial” (when someone rates it as 1 and the other one as 10) websites from the sample.
At first, we experimented with evaluation methods.
For example, we offered to evaluate one website at a time, then two websites at the same time, or to choose one out of two, the most attractive one. The approach where the respondent saw one single website and evaluated it worked best. We used it to evaluate 10.000 of the remaining websites.
A person evaluated whether a website is beautiful or not. How will the machine do this? You and I need only one look to form an opinion on the overall beauty of something. But we know that the devil is in the details.
The website visual attractiveness signs which will guide the model is a key moment for the whole project. We asked uKit website builder design team for a hand, their work is used as a basis for hundreds thousands of websites, and millions of people see it. Together we have compiled an expanded list of features that professionals pay attention to when developing a website design. And then tried to cut it, leaving only the most important ones.
As a result, we got a checklist of 125 quite different yet significant criteria grouped into fifteen categories. For example, the list has: adaptation to popular screens, a variety of font sizes, purity of colors, length of headings, proportion of the images on the entire page, and so on. What’s left to do is to train the model by using these rules.
Create an algorithm. What is a ‘teaching model’ exactly? It’s the construction of an algorithm that is based on a given set of characteristics and can evaluate the selected website. It is desirable that the system’s evaluation and the average teacher’s evaluation share a minimum gap in their final evaluations.
We’ve decided to use gradient boosting method over the decision trees, because it’s one of the most popular and effective approaches. Using basic algorithms, it constructs a set, the overall result of which exceeds the results of any separate algorithm.
Moreover, adding each subsequent basic algorithm, it tries to improve the quality of the answers of the entire set.
To accelerate and ease the process, we used the CatBoost library from Yandex which allows building a gradient-based booster in so-called “oblivious decision trees” ensuring good training capabilities of a model from the start and a quick transition to providing predictions (estimates) for new objects.
Adding a neural network. When the basic algorithm was ready, we decided to conduct an experiment: will the results improve if we add a neural network? Actually, we already knew how to ‘look at’ a website and its design, and now we decided to give the system some kind of a ‘magnifying glass’ which it can use to reveal even more details.
We chose one of the most popular networks, resnet50, it’s known as a good algorithm for extracting high-level features. And we’ve learned how to get 1000 additional attributes for website evaluation. As a result, the system now characterizes a URL by the total of 1125 features and finds the website’s ‘place’ on a 10-point scale. The process takes up to several dozens of seconds, that’s why we consider to speed up the model by reducing the number of signs while keeping the quality of evaluation on the same level.
First results. The model trained this way could make 3 times more accurate estimates compared to those of individual ‘teachers’.
We can say that the model surpassed its first teachers since the focus group estimates differ from the average one more greatly than the estimate of the neural network. Now we put the algorithm into the network for further training. And you can become its teacher as well.
