Monitoring & Observability
What is Monitoring
Monitoring is the systematic process of collecting, analyzing and using information to track a programmer's progress toward reaching its objectives and to guide management decisions.
Why Monitor ?
Some of the advantage which we have like we want to analyze some long-term trends and compare speed, cpu, ram as well as utilizations. for example latency is a startup so it want to troubleshoot.
There are many reasons to monitor a system, including:
Analyzing long-term trends
Comparing over time or experiment groups
Alerting
Building dashboards
Conducting ad hoc retrospective analysis (i.e., debugging)
Primary reasons why we do the monitoring
93% of the time we set up monitoring to alert on our issues. Issues can be anything like server issues, network issues, process issues and different things we also want to diagnose so we set up a monitoring we want to track SLA/SLOs service level agreement or service level objective we want to track it whether we are able to achieve those which is committed to the client or not. Also we want to identify area of optimizations and to ensure reliable release.
Monitoring Plan
When you get into the monitoring plan you need to know, some of the basic questions like ,
- What to monitor (which objectives does your project have)
2. How to monitor. (Define indicators & sources of verifications, SoVs such as interviews, studies, tests etc.)
3. Who will collect the data. (Disseminate responsibilities for monitoring)
4. When to monitor. (When will you collect the data/the results)
So these are the primary questions you must be having in your mind.
What to Monitor
We want to monitor like a system level checks. So what are the system level checks its like cpu, memory, disk, load, network. We also wants to monitor the process level checks and application level checks. That’s means that process running or not, this process threading information, memory utilization of that particular process and stuff like that. Also we want to check that message level checks like active client number, number of request, number of responses rate and check of the api level like api call and find api errors.
We want to monitor for availability of the applications. we want to monitor things for the security monitoring and do the data validations and we want to monitor the performance monitoring, Integration monitoring and check availability whether system is up and running or not.
Overall we monitor application services, operating systems network and all these systems are produce metric data. That is what monitoring system capture that all these matrices and store it at some place and from there we do the visualizations.
Flow of Monitoring
If you want to set up a monitoring, you need to know what are the flow for it.
You have to find the information, share the information, review the meetings and changes review indicator to monitoring. In nowadays also we have one process which is called continuous monitoring in which define the what you want to monitor continuously, establish implement, analyze, report, respond review and update.
Condition Monitoring benefits.
Some of the benefits which we have is smart replacement, recover revenue, asset availability, improve workforce, decrease planned maintenance, IOT, decrease service loss and lower risk. These are the some of advantages.
Why is monitoring hard?
In nowadays, we have lots of data sources. So that monitoring is not something we are doing today, We’ve been doing monitoring from last 10 years. We’ve been lots of data sources. That’s one of the reason like monitoring is hard.
Lets assume your applications can accessible from the different people, different devices, different isp’s, different web servers, different app servers and different database servers. But few people is 300milliseconds and for a few people 1.5 seconds to wait. them you need to know what causing the response time is of 1.5 seconds, Because one person is happy from the same setup, one person is not happy for the same setup.
So answering, we were setting monitoring which we have doing from last 10 years. So primarily we put the monitoring into the two bucket
- White Box Monitoring.
- Black box monitoring.
I hope to talk more about it in the future.
Thank you.
