Investigate user behavior, using Azure application insights

Azure Application insights is a powerful monitoring tool, how powerful? Well, it helped us figuring out an anomaly in our application activity and almost find the user who is responsible for that weird behavior.

It all started on a nice spring evening, with an Azure Monitor email alert in our inboxes:

Presidio is an open-source tool to recognize, analyze and anonymize personally identifiable information (PII). Using trained ML models. In addition to an OSS library we also have a demo site for Presidio we maintain. This alert was one of several defined for the site, saying the number of requests the server received was greater than usual. First thing we did is to check the monitors, starting from several days before the alert until the moment the alert started, that’s where we found this:

The first instinct will usually be to scale out, so the requests will be distributed to more servers. This is a good idea in case the machines CPU or memory is suffering due to the new traffic or we are getting to some limit on the amount of open connections on a single machine, in our case it wasn’t so, the server wasn’t even flinching:

The next thing we did was to ask ourselves why? Why suddenly there’s a spike? Did the site start to get that much attraction? Is this real user activity? If so, why didn’t the memory/CPU change or even, increased at all?

The graphs won’t be able to answer these questions for you, but the logs will. Viewing the requests logs between the increase timeline, we saw a weird user behavior:

Either someone really likes our favicon or something else is going on here. We suspected the latter.

This behavior didn’t last long, about 6 hours later the alert and traffic were gone, but don’t worry, they came back the next day and the day after, each time for 5–6 hours:

At this point, we were still curious about two things:

1. Who is this person?
2. Why are they so fascinated with our favicon?

First question (who is it?):

The right query will give us the right data of what we are looking for:

Great, we found the user! User HBu2e, simple!

Or not so much, as can be seen below, the client_ip was 0.0.0.0, we had the client city and state but not their IP so we couldn’t block them.

We started investigating to why we can’t see the user IP and very quickly arrived to the Azure monitor documentation:

Makes sense. By default Azure protects you from improper collection of user data and complies with world wide regulations such as GDPR. You can find more guidelines about data stored in Log analytics and applications insights in here.

If you do choose to enable this feature, you need to do it carefully by using two methods:

1. Be aware it is on and make sure to turn it off.
2. Have a low log retention.

We did not get to use the feature or block the user IP, but incase we wanted to, we would block an IP using access restriction for Azure App Service.

2. Second question (why favicon?):

At first we thought it might be a crawler but then again, why would it only access the favicon, if it was a crawler bot, it did a really lousy job.

We tried to consult experts and ask our long time friend google, all we came up with is an article on how websites can track incognito user using the favicon, which was not relevant.

So this question will need to remain open for us and for you.

Thanks to Nava Vaisman Levy, Avishay Balter and Tamir Kamara.