New way of sharing files across devices over the web using WebRTC
There have been various methods of sharing files on the web. You upload a file to a server and share a link using which other people can download that file. While this is a tried-and-tested method of sharing data in general, let’s try to tackle this problem with a more “device-to-device” sharing approach rather than “device-server-device”.
Thinking about this problem, I kept coming back to technologies such as Bluetooth, NFC, WiFi-direct sharing. While I love these technologies, honestly they were a bit underwhelming in terms of speed, range and overall experience in general.
On the contrary, Web enjoys being connected to a global network and having access to the full potential of the internet speeds of the device. By implementing a better experience, I wondered if there was any way to combine the benefits of the web with file sharing ideologies of Bluetooth. I got excited when I learned about this underrated feature on the web that allows us to do exactly this and today I would like to share it with you 😄
WebRTC
WebRTC stands for Web Real-Time Communication. WebRTC provides a set of APIs that make it possible to make realtime device-to-device(peer-to-peer as how it’s called) connections to transfer audio, video, or any general data over the web.
Theoretically, a “peer-to-peer” connection allows for direct data transfer between two devices without a need for a server to persist the data. Sounds cool right… perfect for our case? Sadly though this is not how WebRTC works! 😕
While WebRTC does create a peer-to-peer connection, it still requires a server 😲. The server (known as signaling server) is used to share information about the devices that are needed to implement this connection with each other. These details can be shared through any traditional data sharing methods. WebSockets is preferred here as it reduces the latency in sharing this extra information in a large network for establishing a connection.
So, putting it simply, the server helps in establishing the connection, but once the connection is established, it should no longer have access to the data shared between the connected devices.
Let’s try to implement a file-sharing mechanism between two devices with WebRTC. Since the original APIs of WebRTC make it tedious to implement the connection, I will be using a wrapper library — simple-peer, along with a tiny library download.js which makes it simple to initiate file download programmatically for the examples.
How WebRTC creates a connection (on a technical level)
Ok, so this is going to get confusing, hang tight! Out of all the devices in the network, there must be at least one device which initiates the connections by generating the signal data to be sent to the signaling server. This peer is known as initiator and in simple-peer { initiator: true } is passed to the constructor when an initiator peer is created.
Once we get the signal data of a peer, this data should be somehow sent to other nodes via a signaling server. The other nodes receive this data and try to establish a connection with the initiator. During this process, these nodes also generate their signal data and are sent to the initiator. The initiator receives this data and tries to establish a connection with the rest of the nodes.
And after this some cool magic…… Booooom……. Devices connected! ✨
A peer’s signal data can be accessed by listening to the signal event. To establish the connection using this signal data, other peers call signal() and pass the signal data as the argument. Example: somePeer.signal(peer1Data)
Don’t worry if you couldn’t get through the above workings of WebRTC and how simple-peer abstracts it. It confused the hell out of me when I just started dabbling around with WebRTC 😖. You’ll be fine in the upcoming sections.
Sharing a file with WebRTC (using simple-peer) 😎
<html>
<head>
<title>Sharing files using WebRTC</title>
</head>
<body>
<input type="file" id="file-input">
<script src="https://cdn.jsdelivr.net/npm/simple-peer@9.5.0/simplepeer.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/downloadjs/1.4.8/download.min.js"></script>
<script src="file-sharing-webrtc.js"></script>
</body>
</html>/**
* Peer 1 (Sender)
* Peer 2 (Receiver)
* This example has both the peers in the same browser window.
* In a real application, the peers would exchange their signaling data for a proper connection.
* The signaling server part is omitted for simplicity
*/
const peer1 = new SimplePeer({ initiator: true });
const peer2 = new SimplePeer();
/**
* Implementing the WebRTC connection between the nodes
*/
// Share the signalling data of sender with the receivers
peer1.on('signal', data => {
peer2.signal(data);
});
// Share the signalling data of receiver with the sender
peer2.on('signal', data => {
peer1.signal(data);
});
/**
* Connection established, now sender can send files to other peers
*/
peer1.on('connect', () => {
const input = document.getElementById('file-input');
// Event listener on the file input
input.addEventListener('change', () => {
const file = input.files[0];
console.log('Sending', file)
// We convert the file from Blob to ArrayBuffer, since some browsers don't work with blobs
file.arrayBuffer()
.then(buffer => {
// Off goes the file!
peer1.send(buffer);
});
});
});
/**
* Receiver receives the files
*/
peer2.on('data', data => {
// Convert the file back to Blob
const file = new Blob([ data ]);
console.log('Received', file);
// Download the received file using downloadjs
download(file, 'test.png');
});If you try the above code in the browser(or visit the deployed example) and select some image file(preferably below 100KB), you’ll see that it immediately downloads it. This is because the peers are in the same browser window and transfer is instantaneous. ⚡️
Now open the Console in the devtools, to check some cool stuff (shown in the image below). Notice the size property. The size of the data sent and received is the same. This shows that we were able to transfer the entire file in one go! 🎉
Let’s start chunking the files 📦
In our previous example, if we choose a large file (above 100KB) the file most likely wouldn’t get transferred, this is because of some limitations of WebRTC channels.
Tiny files can be sent over WebRTC in one go, but for larger files, it is a good idea to divide our files into smaller chunks and send each chunk accordingly. Both ArrayBuffer and Blob objects have a slice function which makes this easier.
We keep chopping our array buffer, till there’s nothing left to chop! 🔪
Let’s discuss the implementation of chunking a little bit. The slice function takes a starting and ending byte position and returns a new array buffer which has a copy of all bytes in the given range. This new array buffer acts as a chunk. Then we update the existing array buffer by removing the chunk we separated earlier. This keeps going till we make our array buffer empty (i.e. chunked the entire file).
Now the question is what should be the size of each chunk? There’s no hard limit for this. Each browser has a little different implementation of WebRTC and data fragmentation internally. Some improvements in newer browsers don’t play well with older browsers 😑. From a lot of searching, I have come to assume that a safe max limit is 16KB for each chunk. Let’s add chunking to our existing code.
/**
* Peer 1 (Sender)
* Peer 2 (Receiver)
* This example has both the peers in the same browser window.
* In a real application, the peers would exchange their signaling data for a proper connection.
* The signaling server part is omitted for simplicity
*/
const peer1 = new SimplePeer({ initiator: true });
const peer2 = new SimplePeer();
/**
* Implementing the WebRTC connection between the nodes
*/
// Share the signalling data of sender with the receivers
peer1.on('signal', data => {
peer2.signal(data);
});
// Share the signalling data of receiver with the sender
peer2.on('signal', data => {
peer1.signal(data);
});
/**
* Connection established, now sender can send files to other peers
*/
peer1.on('connect', () => {
const input = document.getElementById('file-input');
// Event listener on the file input
input.addEventListener('change', () => {
const file = input.files[0];
console.log('Sending', file);
// We convert the file from Blob to ArrayBuffer
file.arrayBuffer()
.then(buffer => {
/**
* A chunkSize (in Bytes) is set here
* I have it set to 16KB
*/
const chunkSize = 16 * 1024;
// Keep chunking, and sending the chunks to the other peer
while(buffer.byteLength) {
const chunk = buffer.slice(0, chunkSize);
buffer = buffer.slice(chunkSize, buffer.byteLength);
// Off goes the chunk!
peer1.send(chunk);
}
// End message to signal that all chunks have been sent
peer1.send('Done!');
});
});
});
/**
* Receiver receives the files
*/
const fileChunks = [];
peer2.on('data', data => {
if (data.toString() === 'Done!') {
// Once, all the chunks are received, combine them to form a Blob
const file = new Blob(fileChunks);
console.log('Received', file);
// Download the received file using downloadjs
download(file, 'test.png');
}
else {
// Keep appending various file chunks
fileChunks.push(data);
}
});If you try the above code(or view the deployed version), you should see there’s no difference in how the app works from the user’s end. But internally, we are now chunking and sending larger files too⚡️. Amaaaazing right?
Benefits of chunking
While it may feel that chunking is just some extra code to make stuff complicated, we get following benefits that can help in improving our file sharing app. 😇
- Large file size support — As mentioned earlier, this is the primary reason we implemented chunking.
- A better way to interpret the amount of data transferred— By sending a file in chunks, we can now show the data such as percentage of file transferred, rate of file transfer, etc.
- Detect incomplete file transfers — Situations, where a file could not be transferred completely, can now be caught and handled differently.
What’s next?
Now that we have a simple file sharing app built using WebRTC which also uses chunking, we must now start considering stuff that would make our application production-ready. I won’t be covering all of them in detail here, but give some general pointers on how we can improve our app.
- Signaling server — In the examples above, both receiver and sender were on the same browser window. This won’t happen(or I should say shouldn’t happen) in a real-life scenario. Hence, consider making it truly device-to-device using a signaling server.
- Sharing multiple files — Sharing multiple files can be implemented easily based on the above examples.
- Adding some fallback when WebRTC fails — When WebRTC connection fails, a different fallback method of file sharing can prevent the hassle of retrying connections.
Well, that’s it from me for today, WebRTC is much more than just file sharing, so I would encourage you to go deeper into it if you found this interesting 😄
On a side note, I have developed a full-fledged web app that uses WebRTC and WebSockets to share files across devices. It’s called Blaze ⚡️ and is open-source. I would love to see your contributions and feedback for the app. Share some ❤️ on it’s GitHub repo.
