CFS: Tips and Tricks, for meteorJS
by Pranab @ www.zipboard.co
We’ve written a lot of code over the past few months. For the most part it has been smooth sailing. However, like any other application, at zipBoard, we did get stuck at a number of points, and had to hack our way to move forward. Our solution has grown to thousands of lines of code, and many of these hacks have now gotten deeply buried into our source pool. In this post, however, I have made an effort to jot down some of the recent issues that we faced with the CFS package, and our ways to get around them. Before I start, I must mention that CFS is an excellent package, and it’s a one of a kind solution for complete file management for meteor applications. However, it still needs a bit of work to make it production ready for commercial applications.
Problem 1: Callback on completion of File Upload — At this time, there is no way to find out if cfs has completed uploading a file, and this file is available on the server for further use. This is where the meteor community has struggled the most with the cfs pacakge, and there are several posts, both on GitHub, and Stack OverFlow with potential work-arounds.
In our use-case, our users upload zip files. Once the zip file is uploaded, we want to automatically unzip this file, and show its contents to our users. There is an isUploaded flag that is published in the CFS documentation, but unfortunately it does not guarantee that the file has been completely uploaded, and available on the server. Along with isUploaded, there are several undocumented flags in the package, for instance, hasUploaded, and hasStored. We tried all of them, however, none of them gave us a definitive response on the completion of upload. We tried a number of other hacks based on suggestions on GitHub, and Stack OverFlow, but unfortunately none of them worked for our specific use-case. After spending many hours on this issue, we were able to make this to work using the below hacks:
The first one is suggested by ncubica on GitHub. His suggestion is to query the file url from the database. The idea is that the file url gets generated only after the file has completely uploaded, and stored on the server. Ingenious, and it worked for us on our sample projects.
yourFsCollection.find({_id : _id}).observe({
changed : function(file, oldFile){
if(file.url() !== null){
uploading.onFinish(file._id); //this is your callback
}
})The second approach was our own invention, as our code-flow required us to fire this code from the html template. In the below code, once the isUploaded flag is set to true by the CFS package, we consequently keep calling the unZipNow helper method, till the time the isExtracted flag is set to true. Now, unZipNow keeps pinging the location where the file is supposed to be uploaded till the time it finds this file, and consequently unzips it. And when we get done with unzipping the file, we go and manually set the isExtracted flag to true, and hence, stop the unZipNow helper from firing up. Mother of all hacks, but, fortunately or unfortunately, this is the ONLY way through which we should get things to work consistently.
{{#if isUploaded}}
{{#if isExtracted}}
<span ... > <span ... >
{{else}}
<!--Hack - Keep Pinging the unZip Function,
till the File is Available after isUploaded is True-->
{{unZipNow this.original.name}}
{{/if}}
{{/if}}Problem 2: Responsive Progress Bar for Upload — At the face of it, everything looks simple: just install the Meteor-cfs-ui package, and write the below code, and you are done.
{{#each images}}
{{#unless this.isUploaded}}
{{> FS.UploadProgressBar}}
{{/unless}}
{{/each}}No, it’s not! We quickly realized that the minimum step size on the cfs upload progress bar is 2 MB. This meant that if we uploaded a 1 MB file, for instance, the progress bar jumped from 0–100% in one go
The solution turned out to be pretty simple though — there is a uploadChunkSize variable inside the FS configuration object that we were able to dynamically set based on file size. Once again, an undocumented variable, but we were able to get to it easily by dumping the config object.
//Hack for More Responsive Progress Bar
//Max Upload Chunk is set to 2097152
if(fsFile.original.size < (2097152)*10) {
var chunkSize = fsFile.original.size / 10;
FS.config.uploadChunkSize = chunkSize;
}
uploadFile(fsFile, fullFileName, projectId);So, now it works like magic, and everyone on our marketing team loves it! It shows 10 steps for all files less than 20 MB, and a step for every 2MB for files greater than 20 MB.
Problem 3: Cancel Upload — What if the user wants to cancel a file upload after it has started. A pretty simple use-case, and pretty much every application in the modern era supports it. However, we did not find any clean way to do this through cfs. We wrote the below hack to make this to work.
var list = FS.HTTP.uploadQueue._processList.fetch();
_.each(list, function(item) {
item.queue.cancel();
});Somewhere inside HTTP, we found the uploadQueue object, and inside it the cancel() method. We were able to cancel uploads, however, with one caveat: once we call cancel() on the upload queue, it got corrupted and subsequent uploads did not work. Unfortunately, till date we have not found a work-around for this. At this time, we force-refresh our application if the user cancels an upload. This refreshes the queue, and the upload functionality of the CFS package starts towork again.
If you would still like to dig down deeper into this issue, these posts on GitHub (1) , (2), and (3) might give you a couple of hints.
That’s it folks! Using the above tricks, we were able to put together a good looking file manager for our customers. It’s pretty cool that the end-users always sees a brilliant application, and shall never know the blood-bath that has taken place in code to make it to work 😉 Check out zipBoard, it’s a visual feedback tool for websites and web-apps. We are really proud of what we have put together; feel free to leave your comments and suggestions.
by Pranab @ www.zipboard.co
