Multiple simultaneous uploads in Android

We at Bolo are committed to building world class products for the Indian people. If you are passionate about the India story, do hit us up at hello@theboloapp.com, we have multiple openings across the board.

By Harjot Singh Oberai

Bolo is a video Question & Answer platform that allows users to ask and answer questions in Hindi. Answers are short videos less than 1 minute each and questions range from a variety of topics including news, career counselling to beauty & fashion.

It’s also super-easy to answer a question: Just record a 1 minute selfie video on the Bolo App and upload the same on the App itself. Due to this convenience, a large number of users end up answering multiple questions, one after the other. Bolo App shows them an upload progress notification for each of the upload in progress.

Now, the challenge happens to be that majority of our users belong to Tier 2 and 3 towns and generally use low-end android devices. This combined with poor internet connectivity usually results in taking a few minutes for all the uploads to complete.

Once the user would go to home screen or play with another app and app goes in background, the android system would kill Bolo App. This led to frequent upload failures. As a result users were often left frustrated and would complain about it.

In technical terms this happened because the upload was tied up to the application and once the application got killed, the uploads would stop as well. Thus, a way had to be found which allowed:

Uploading files to a server, in background, independent of the application state and showing the upload state as a notification (progress, success or error). Also the ability to handle multiple uploads simultaneously.

Three pieces of the puzzle have standard android implementations, fourth one is the missing piece here

Upload Files in background : A simple asynchronous method, can be implemented using AsyncTask or something fancier like rx-java.

Multiple uploads simultaneously : A service to handle all requests and run asynchronous tasks from it as requests come in.

Show state of the notification : A simple callback associated with every asynchronous task. Associate a notification with every task and update it from inside the callback.

Independent of the application state : Have to find a way out for this

Background Tasks independent of Application state

So we need to make sure the upload tasks are running in background, even if the application is killed.

It makes sense to upload files independent of the application, as the application has nothing to do with this operation. No callbacks are associated to update the application state or UI, so its only fair to expect upload tasks to be completely independent.

AsyncTasks, rx observables, Evernote’s android-job, Android’s internal JobScheduler all of these do not work in this scenario, as all these run the tasks on background threads but these background threads are bound to the application itself. If application is killed, the thread dies with it.

This leaves only one option, Services. Normal services are still bound to an android activity, what we are looking for is something called Foreground Service

Android foreground service is an android service object. But it always runs in foreground, this can avoid service object being recycled by android system when the OS does not have enough resources.

So basically, foreground service is the only thing that can get us close to achieving our goal, but there is one problem:

A foreground service must have a notification associated with it, and this notification is shown, while the service is running in the foreground.

An easy way out would have been to show a static notification saying Uploading Files and removing the static notification once all uploads have succeeded.

But this clearly is bad design choice. The users are already overwhelmed with notifications on their smartphones, adding a redundant notification to show that files are being uploaded, when the user already has upload progress notifications showing as well — not cool.

So we took the high road and worked out something to fit the use case.

The Solution (The high road)

Here’s the gist of the solution :

If a service is running then startForeground() can be called anytime to set the service as foreground and associate a notification with it. The notification does not have to be the same every time. We can associate a new ongoing notification with the service at anytime to make the service foreground.

The upload notification has the following states :

1. Progress : An ongoing notification, whenever there is an update in upload progress. First notification to be shown for an upload request.
2. Success : A simple non-ongoing notification that replaces the progress notification when the upload succeeds.
3. Error : A simple non-ongoing notification that replaces the progress notification when the upload throws an error.

The solution architecture :

1. For each new request generate a unique upload ID.

private int generateUploadID() {
	int uploadId = Math.abs(new Random().nextInt());
	while (uploadIDs.contains(uploadId)) {
		uploadId = Math.abs(new Random().nextInt());
	}
	uploadIDs.add(uploadId);
	return uploadId;
}

2. Maintain a map for the upload data and this upload ID.

3. Create a notification for this request and associate it with the upload ID, by maintaining another map.

• The ID for the notification will be the same as upload ID.

4. Start the foreground service every time a new request comes (The “associated” notification is changed).

private static Map<Integer, UploadData> uploadDataList = new HashMap<>();
private static Map<Integer, Notification> notificationMap = new HashMap<>();
// New Request
uploadData = /*Data to be uploaded*/
uploadId = generateUploadID();
uploadDataList.put(uploadId, uploadData); // Keep the upload data for this upload ID in a map
notification = createUploadProgressNotification(uploadId, uploadData);
notificationMap.put(uploadId, notification); // Keep track of notification for this uploadID
startUpload(uploadID, uploadData);

5. Update the notification through the callback from asynchronous tasks.

• If we update with a progress notification, that is ongoing, we don’t have to worry.
• If we have an error or success, then we replace the notification associated with the ID, with a notification that is NOT ongoing, but if this notification is the “associated” notification at the time, then we cannot just use the NotificationManager to change the state to NOT ongoing. We will have to pass the “associated” tag to some other upload that is still in progress and can handle being ongoing.

void onProgressChanged(int uploadId, ...){
    updatedNotification = /* create updated notification and show it */
    notificationMap.put(uploadId, updatedNotification); // update the notification corresponding to uploadId
}
void onError(int uploadId, ...){
    switchForegroundNotification(uploadId); // Switch the foreground service to some other notification that is ongoing.
    errorNotification = /* create error notification, that is not **ongoing** */
    notificationMap.put(uploadId, errorNotification); // update the notification corresponding to uploadId
}
void onSuccess(int uploadId, ...){
    switchForegroundNotification(uploadId); // Switch the foreground service to some other notification that is ongoing.
    successNotification = /* create success notification, that is not **ongoing** */
    notificationMap.put(uploadId, successNotification); // update the notification corresponding to uploadId
}

6. In case of Error or Success, pass on the “associated” tag to some other ongoing notification by iterating through all the IDs.

7. Call startForeground(), with the upload ID of choice and the mapped notification along with it.

private void switchForegroundNotification(int uploadID) {
    for (int uID : uploadIDs) {
        if (!errorUploadIDs.contains(uID) && uID != uploadID && videoUploadDataList.containsKey(uID)) {
            startForeground(uID, videoForegroundNotificationMap.get(uID));
            return;
        }
    }
}

8. When all uploads are done, just stop the foreground service by calling stopForeground().

Summary:

Multiple uploads running in background using standard android background task solutions leads to failures. Using foreground service we can solve this problem and provide user with low end device and slow internet a seamless experience.

India is rapidly moving forward with 100s of millions of Indians coming online for the first time every year. The product challenges that they face are unique in the world with slow internet speeds, weak devices and dozens of languages.

We at Bolo are committed to solving these problems and are passionate about building world class products to propel India forward. We have multiple openings across tech and ops, if you are among those who feel strongly about the India story, do hit us up at hello@theboloapp.com