Turn on Application Monitoring
Once I was handed a simple application. No big deal, Angular frontend, .NET Core API backend, SQL Server for database, and couple interfaces to Exchange and ServiceNow.
However, this application was attending the whole company and the number of incidents was high at the beginning of operations of this application, generating some noise.
When checking the available documentation, among other things, I could notice that tests were not fully stressed. The happy paths were ok, but there was a lack of scenarios and stress tests.
“That’s one of the reasons we have a high number of incidents”, I thought.
When started analyzing incidents, another surprise: where are the logs?
Well, no logs. In fact, the documentation pointed to a log file, but when checking that, it was just ASP.NET tracing, which no much useful information for most of the incidents.
So, investigating incidents were pretty challenging, relying only on checking source code and trying to reproduce scenarios. Furthermore, the application was using hard delete on database actions, which lead to some issues being impossible to reproduce or gather further information.
Application Insights
That was when I heard about Microsoft Application Insights and then I decided to implement it on the backend because it sounded like a quick win.
Setup was easy, simply adding the NuGet reference and creating a new Application Insights on Azure Portal. Grab the Instrumentation Key and put on appsettings.json property created for that.
Once I published a new version in production, it was amazing to start seeing the requests coming in, all that telemetry and connections information. All of it almost effortless.
Show me the metrics
With all this excitement, I decided to cross that information with some of the most weirdo incidents.
One of them was a complaint from several users that some functionalities were not working properly on Internet Explorer, but it was working fine on Chrome. Curiously I was always informed during the handover that the system was focused on Chrome, but without any specific reason.
But as soon as I started collecting telemetry, something showed up:
The server was throwing 404 responses (page not found) for several GET calls. And not by coincidence, when checking the source code…
No GET operations! Only POST!
“That’s a good start!” I thought. Next stop, FrontEnd source code. Where are we calling those methods using GET?
Bingo! There was a condition (don’t ask me why!) there was checking if the browser was Internet Explorer, forcing the call to be done by GET.
Issue solved
BitBucket branch created, Jira issue solved, fix patched on the code and new bugfix release sent to production.
After applying the patch in production, I couldn’t wait to check the monitoring data again. The results were pretty good!
Note that the purple area represents the total requests and blue area of the failed requests. After the patch was applied it is possible to see an incredible reduction in the number of failed requests. Job’s done! (at least for this incident type!).
Now users are using Chrome or IE, as they wish, as they should.
This case clearly represents a cycle on DevOps where the monitoring has a fundamental role to provide useful information as input to solve incidents.
After that, I started to monitor the exceptions of the application and also created custom tracing on the code, which helped me with the investigation of other issues related to integration. Furthermore, those tracing were helpful in cases where data was hard deleted.
Find your tool
In this case, I used Application Insights, from Microsoft, that could be easily integrated into the .NET Core API BackEnd. But there are several other solutions, with different flavors and platforms.
But the important message here is to start monitoring your application. Especially if you are starting an application from scratch, be sure to consider a monitoring solution when designing and architecting it and don’t miss the opportunity to identify where you can have good tracing.
It can save your life sometimes, as it saved mine!
As usual, don’t forget to follow, comment, clap, and check the other articles of DevOps, Cloud & IT Career Publication!
Marcello Marrocos is a DevOps and Cloud advocate, passionate about technology and software development. He is an Azure certified professional in DevOps, Developer, and Architecture. In his spare time, he writes for DevOps, Cloud and IT Career publication on Medium. You can find him on Twitter and LinkedIn.
