Your favorite star’s movie trailer released on YouTube and five hundred thousand fans, including you, are watching it simultaneously.
How does the system work?

When you open the YouTube app and click on play icon, you are subscribing to the paticular video for that session.
Then the server will seamlessly stream the data packets to all the subscribers in the ordered manner. Since it is 500K users, the server has to stream the one video to all of them in a different time (of course, everyone doesn’t started watching at the same time)
Following three are the main components of the YouTube application topology.
- Video file saved in a hard disk
- YouTube app is a video player playing video
- The tricky part is the server, where all the distribution are happening
We know that the YouTube is not a small organisation. They invest a lot and lot on the develop and retain the server to ensure seamless video streaming for any number of users.
Finally, what is the job of the YouTube video streaming server?
It get the data from one place in a format and disseminating to many (too many) subscribers in same format.
Let’s turn out to the data streaming.

I’m well familiar with the stock market business. Data streaming is one of the pillars of this business. Let me explain a scanerio.
Data Streaming in stock market business.
When a single stock of Tesla (TSLA) is getting sold, there will be a change that appears in all the prices and statistics in the market. Tesla stock is listed in NASDAQ (US stock market). So, NASDAQ will publish these details from their system to the rest of the world by their own protocol.
They will publish the details as a data feed. Such as, what is the current value of Tesla, change that happened to whoever invested in Tesla, what is the impact and change that happened in Apple stock, Google stock and etc. This feed consists of 1000’s of information because of a single purchase of a stock.
There are many trading happening every second so that NASDAQ is publishing all the information as data feed. It is typically 1 million pieces of information for every sec.
I’m trading in London, and I’m only interested in Tesla data, a bank in Switzerland need accumulated data of Tesla and Google for the last 2 hour, Singapore Stock Exchange wanted all the data feed in real-time, a stockbroker in New York looking for all the Tesla’s data and Apple’s data together.
So every subscriber party having their own interest from NASDAQ. But NASDAQ can’t work for their interest. So NASDAQ only publishes all the information via its own protocol.
So, let’s put a smile on the face of all the interested parties. Build a data streaming network, and that will work for everybody’s interest. Where NASDAQ will publish the data to the Data Streaming Network, and any authorised parties can subscribe and get their interested details.

So what is the job of a data streaming network?
1. Real-time (low latency) performance: Instant update of the data wherever the subscriber located and whatever the network bandwidth
2. Uninterrupted experience: Must detect drop-offs, redeliver messages, and process missed messages in a completely seamless manner
3. Multi Protocol support: Understand and distribute multiple protocols
4. Security: Control who can send or receive data while integrating standard encryption protocols such as TLS, AES
5. Global Distribution: International points of presence
But there are no third-party tools or an open-source tool to build an entire data streaming network. Engineers choose to use open source tools like Kafka and go for the experiment with expensive supports, such as every protocol has to have a connector with Kafka to communicate with Kafka. There is a significant bottleneck in global distributions in such open-source tools. And also, though they choose such open-source tools, it is not possible to reach the ideal real-time performance with growing data quantity and real-time requirements.
So is there any tools for accomplish the requirement of data streaming network?
Leorix is a platform to provide all the data streaming capability at a glance. Technically they are using a unique adaptive data compression and adaptive data distribution for accomplish all the data streaming capability in a way never before exist.

Another Big Question!
Data stream network is build by data streaming platform. It is a network for streaming the data from one location to another.
We all know, NYC to Singapore via Dubai transit flight is sometime much cheaper than direct flight. Similar like, getting a data from NYC data center to Singapore data center via London data center may faster than direct transfer due to network latencies.
While we are proposing low-latency & real-time, the data streaming network have to ensure the fastest delivery of the data. How it is working?
We would need to go to second layer of the data streaming network.
That will be continued..!
