All you need to know about performance in one dashboard
In this agile-development-software era, everything need to be done fast and fine. Your software engineer team would be so hectic to build new feature, tweak this and that, and suddenly your systems have grown so large and complex. Managing those systems is already hard, let alone knowing that they are working well. Keeping track of your application performance can be painful sometimes, and that’s also what we encounter here in blibli.com.
As our system grow big, we currently have to deal with 250 microservices. Analyzing those services one by one will take a toll (ain’t nobody got time for that, huh). It would be so nice that we can know the current performance of those distributed yet integrated microservices with ease, our software engineer will be helped, our boss will be amazed, everyone is happy. So, what if I told you that we can make that happen? What if I told you that we can tell you all the performance of those 250 microservices with just a few taps of your fingertips? That’s what we (Blibli’s Performance Team) has been done, introducing PEDAS (Performance Dashboard) for blibli’s microservices.
What PEDAS can do?
Well we are basically lazy to analyze performance of those microservices one by one, but we also a perfectionist that want our job done right. So with PEDAS, you basically can :
- Performing Backend Performance Test
Our integrated dashboard in PEDAS will allow you to run API testing using JMeter in just one click. You can run your predefined JMeter test script as load testing tools automatically and the process will be streamed to PEDAS using websocket. Our system will also read the result file (.jtl) and compile the aggregation result and show the test report to user.
- Performing Frontend Performance Test
By using Sitespeed.io as base engine and created an automation tools that will run sitespeed by a scheduler (4 times in a day). The result will be aggregated, so we can create a graph report of frontend performance trend for each url that have defined before. We also get all nice feature provided by sitespeed like, har waterfall, video, coach for every job, even it can give you some advices to improve the performance of your frontend system. User can also use custom request test by just providing the url, script to run, speed of test and the system will do the rest (test).
- Calculating Performance QIP
PEDAS was able to integrate with other services like, New Relic and JIRA. By this integration, we can get 90 percentile response time from all endpoint in the defined microservices (from New Relic) and then create JIRA issue ticket automatically if any endpoint has 90 percentile response time above the threshold (2 second). PEDAS will also calculate score of each services per squad (team), so the squad can know their overall backend performance score. For frontend integration, we aggregate the test result that has been run before (using sitespeed) and create JIRA issue ticket automatically if any url have exceed the threshold (onload time above 4 second in slow 4g throttling speed)
- Provide Performance Trend
PEDAS has a nice graph that will show performance trend of blibli.com for every week. So we can know if there are any improvement or degradation in our system performance. We also get this data from New Relic
- Incident Management
A nice service provided by PEDAS that will record all of incident that happened in blibli system. The recorded incident will be aggregated to build blibli’s knowledge of incident.
Used technology : Vue ( Vuetify ), Python ( Django ), GO ( Gin ), Docker, NSQ, JMeter, Sitespeed, Lighthouse, New Relic, Jira, Rundeck, Jenkins,
Reposted from my linkedin article : https://www.linkedin.com/pulse/all-you-need-know-performance-one-dashboard-abdus-salam/
