Resumeable uploads in Golang

Recently I was working on a media streaming platform which allows users to share bulky content.On the platform, users can share files as huge as movie files or documentaries! To ensure users had uninterrupted uploads,I faced two main challenges:

• Creating resume-able uploads on the client side to allow users to cancel or resume uploads. Also works better on low internet connection zones because the file is uploaded as small chunks.
• Handling chunked uploads on the go-lang server side

Implementing resumeable uploads on the client side is quite straight forward. Having used it before, I jumped on to my favorite up-loader library resumable.js. The configuration of resumable.js is easy, thanks to the open source community.

First create a division where uploads will be dropped or selected from.

Embed resumable.js and target your go-lang API upload endpoint as illustrated easily on the documentation.

Bingo!!

Your small html page is now ready to perform resumable uploads. Resumable.js chunks the file via the HTML5 file APIs and uploads all the chunks independently.

To develop a server side endpoint requires the understanding of how resumable.js works. The endpoint must accept a GET and POST request that are handled in this manner:

• GET — verifies if the chunk resumable.js is asking about is already uploaded
• POST — Accepts a chunk and uploads it

The go-lang implementation would look like this

The chunked parts i.e part1,part2…part(n) are now in your temporary folder. The second challenge presents itself, recombining the chunks back to the original file.

Note that I have left out the section that recombines the uploads in the upload handler. This is because you might consider implementing this as an asynchronous task/job.

After quite some trial and error sessions, mostly involving memory management, I went back to the go-lang file api where I realized I could solve my problem with go-lang file functions. The approach is to create an empty file then write each chunk at the right index i.e where it was originally.

In case the page is not rendered by the same go-lang app(You are running it on an independent Vue js or React app), make sure you enable CORS.This can simply be done with this super cool library

Done!

Resumabe.js has alot of cool apis and events you can play with such as showing progress of uploads and many more.Happy coding