Why do you need one more utility for data aggregation and streaming?
Dive into problem
Several years ago I started develop SIP server. The first problem I encountered — I didn’t know about SIP.
The good way is learning about SIP by studying theory, but I don’t like studying — I like investigating!
Therefore, I started from investigation of simple SIP call.
But, the next problem that I encountered, was how many servers (or micro-services) need for doing simple SIP call — approximetly 20.
20 servers! It means, that before you will hear anything in the ip-telephone you need to trace more than 20 servers, and each server will doing work with your call!
How to trace one SIP call? You have several ways:
- Setup ELK stack in your micro-services environment and investigate logs after sip call
- Via ssh get any information that you need
- Write own utility for investigation
As you understand, we will talk about own utility )
Data Aggregation Utility — why is it necessary?
What’s wrong with first two variants?
ELK stack looks good, but:
- What if you want looks, for example, at tcp dumps and ELK don’t aggregate them?
- What if you don’t have ELK?
From other side, via ssh and command line you can do anything, but, what if ou need to aggregate data from over 20 servers and run several commands on each server? This task is turning to the bash/powershell nightmare.
Therefore, several years ago, I wrote the utility, that can:
- Aggregate and streams data via command-line commands from multiple servers at the same time
- Each aggregation session is saved into separete folder. Each aggregation command is saving and steaming into separate file
- Data Aggregation Sources are simply and can be used repeatly.
Meet Daggy — Data Aggregation utility
Short screen cast demo — https://youtu.be/oeNSwv9oYDc
Getting started with daggy — https://docs.daggy.dev/getting-started
Github — https://github.com/synacker/daggy
