Streaming with IPFS
Static and Live Videos streaming with IPFS and HLS
Current Scenario of Streaming over Internet
In the last few years, video-supported applications, and especially video streaming, have become quite popular. Many providers started publishing their content such as news, series, and movies on their dedicated Web sites or for this purpose used dedicated sharing services such as YouTube, Hulu in the US, and many others.
In the above mentioned services, video streaming is usually based on the HTTP and TCP, and the video player is embedded in a web browser. TCP is currently the most widely used transport protocol in the Internet but it is commonly considered to be unsuitable for multimedia streaming. The HTTP and TCP are general purpose protocols and were not specifically designed or optimized for streaming media delivery.
Bandwidth is precious. When we have a lot of people requesting the media over the internet, there is lot of bandwidth that travels across the internet. These connections are uni-cast and they stream to each individual person. If there are 500 people watching, each person has to get a copy of that media. In this case, bandwidth increases with number of people. For 500 people we need 500 times the bandwidth.
To tackle this issue, companies like YouTube are dependent on CDNs. Basically they are fixing this issue by adding lot more of computers and by moving the distribution centers nearer to the mass users.
To make matters more complicated, the difference between yearly price drop in storage (40%) and bandwidth (26%) is almost double.
All this means that we will have more people, streaming more data on channels that are not scaling as fast. This creates congestion that at one point in time cannot be solved by just adding more hardware.
Imagine you are in a lecture together with 100 people and you all watch the same video. What happens is that this video has to be fetched from the closest nodes of Google, streamed to each of the student’s laptops and repeated 100 times. Instead of students, who have an identical copy of the video, sharing it with each other, we propagate large amounts of data, long distances, multiple times. Inefficient, but this is how HTTP works and it is creating large congestion problems on the backbone of the Internet.
Today the Internet is a centralized system that naturally has a single point of failure. This point of failure can be misused to disable access to the content for an entire country. Also, these traditional streaming services are under the authority of one or few big organizations. The Quality of service and censorship rules are decided by them and the users need to follow that without any questions. Making it a centralized system of power and authority.
What is IPFS?
IPFS or InterPlanetary File System is an open-source protocol and network designed to create a content-addressable, peer-to-peer method of storing and sharing hypermedia in a distributed file system. It aims to make the web faster, safer, and more open.
How can IPFS help us in Streaming?
IPFS is becoming a new major subsystem of the internet. If built right, it could complement or replace HTTP.
In IPFS streaming we don't need to push the content to every user. All you have to do is, push the content to the IPFS gateways. Anyone who wants that content can pick it up from there. IPFS gateway caches the content locally. So more number of gateways, more content sources.
IPFS provides persistence storage of those live streamed media content, so that if someone misses the live streaming, they can always come back to find a saved copy of it. The network automatically deletes duplicates and tracks version history. With our interface we also allow users to schedule the content for live streaming and broadcasting. Users can record the media and automate the system to stream that media on the given time for the specified duration.
IPFS helps to resolve congestion and overly controlling governments by distribution. Instead of locations, IPFS addresses point directly to the resources and it makes sure that this data comes from the closest sources.
This means that if a classroom full of students would watch the same video, they would fetch it from each other instead of any central location. This would make streaming a 4k video bufferless.
How IPFS Streaming Works
1. Getting the Content
WebRTC provides web browsers and mobile applications with real-time communication via simple application programming interfaces. With WebRTC we transfer the webcam content to node server as WebM with VP9 codec every 5 seconds.
2. Converting to HLS
Once we receive the content chunk at server side, using FFMPEG we convert that chunk into HLS format. It creates m3u8 file and ts files. We add these files into a folder specifically dedicated to this session of live stream. ‘
3. Adding the files to IPFS
On every update in this dedicated folder, we need to add / re-add that folder to IPFS and republish the IPNS. After every update content changes and hence hash of that folder changes. But we don’t need this, we need a static hash which can reference to this ever dynamic folder of HLS files. For this purpose, we are using the IPNS. A name in IPNS is the hash of a public key. It is associated with a record containing information about the hash it links to that is signed by the corresponding private key. New records can be signed and published at any time.
4. Play it on the Client Side
HTTP Live Streaming (HLS) is one of the most popular protocols used to stream video today. Over the past few years, HLS has become a standard protocol for web video, and with good reason. HLS is pretty simple to set up, it’s free to use, and it’s supported on a wide range of devices.
Anyone on the network can request the content with the help of IPNS hash and play it on the browser with HLS player. HLS player takes the ‘master.m3u8’ file and plays the chunks in defined order.
To summarize, with the help of IPFS we want to reduce the mammoth amount of bandwidth used by the current and traditional streaming technologies and provide a persistent content storage, better quality streaming. Also, avoid keeping and transferring so many copies of content over the internet and instead start addressing the media by its content. This IPFS based streaming system will also allow us to reduce the unwanted censorship of content and automate the streaming process whenever necessary to make it easier for everyone.