Going deeper, paging from network and database in the MAD skills series

Welcome back! In the last episode, we integrated a Pager into our ViewModel and used it to populate the UI using a PagingDataAdapter. We also took considerations for adding indicators for load states and retrying if there’s an error.

This time, we’re dialing things up a notch. Till now we’ve been pulling our data directly from the network which works only in the best of circumstances. We may sometimes be on a slow internet connection, or have lost connection entirely. Even if our connection is good, we certainly don’t want our app to be a data hog as re-fetching data every time you navigate into a screen is wasteful.

The solution to these issues is to have a local cache we pull from, and refresh only when necessary. Updates to the cache should always hit the cache first, and then be propagated to the ViewModel. This way the local cache is the single source of truth. Conveniently for us, the Paging Library has this covered with a little help from the Room library! Let’s get into it!

Creating a PagingSource with Room

Since the data source we’ll be paging through is going to be from our local database instead of the API directly, the first thing we want to do is update our PagingSource. The good news is we barely have to do much. The little help from Room I mentioned earlier? Turns out it’s a bit more than that: getting a PagingSource from a Room DAO is as simple as adding a definition for it on the DAO!

In the GitHubRepository we can now update the construction of the Pager to use the new PagingSource:

The RemoteMediator

That’s all well and good, but we’re missing something. How does the local database ever get populated? Enter the RemoteMediator; it’s the class responsible for fetching more data from the network when PagingSource runs out of items to load from the database. Let’s see how it works.

A key thing to note about the RemoteMediator, is that it is a callback. The result from the RemoteMediator is never returned to the UI as is, It’s just the way Paging notifies us as the developer, that the PagingSource ran out of data. It’s our job to update the database and tell Paging there’s new data in the database. Similar to the PagingSource, the RemoteMediator is generic on its query parameter and result type.

Let’s take a closer look at the abstract methods in the RemoteMediator. The first method is the initialize() method. It’s the first call made to the RemoteMediator before any loading has begun and returns an InitializeAction. The InitializeAction is either LAUNCH_INITIAL_REFRESH, which will cause the load() method to be called with a refresh load type, or SKIP_INITIAL_REFRESH which will cause the RemoteMediator not to refresh unless the UI specifically requests it. In our case, since repo stats may update often, we return LAUNCH_INITIAL_REFRESH.

Next is the load method. The load method is called at boundaries defined by the loadType and the PagingState where the load type may either be a refresh, append or prepend. It is responsible for fetching the data, persisting it to disk and informing of the result which can either be an Error or Success. If it’s an Error, the load states will reflect it and the load may be retried. If it is successful however, the Pager needs to be notified if more data can be fetched or not.

Since the load method is a suspending function that returns a result, it’s important that the UI is able to accurately reflect the status of the work being done. In the last article we touched briefly on the withLoadStateHeaderAndFooter extension and saw how we can use it to display loading header and footers. A closer look at the name of the extension reveals a type, the LoadState. Let’s go over this type some more.

LoadState, LoadStates and CombinedLoadStates

Since paging is a series of asynchronous events, it’s important that the UI reflects the current state of the data being fetched. In paging, the loading status of Pager is represented with the CombinedLoadStates type.

Like its name implies, this class is a combination of other types that convey loading information. These other types are:

LoadState: A sealed class that fully describes the loading status:

• Loading
• NotLoading
• Error

LoadStates: A data class containing LoadState values for:

• append
• prepend
• refresh

Typically, the prepend and append load states are used to react to extra data fetches, while the refresh load state is used to react to initial loads, refreshes and retries.

Since the Pager may be loading from a PagingSource or a RemoteMediator, the CombinedLoadStates data class has two LoadState fields, one for the PagingSource called source and the other for the RemoteMediator named mediator.

As a convenience, CombinedLoadStates also has refresh, append and prepend fields similar to LoadStates, which will reflect the LoadState of the RemoteMediator or PagingSource depending on your Paging configuration and other semantics. Be sure to check out the docs on the behavior of the fields in different scenarios.

Using this information to update our UI is as easy as collecting from the loadStateFlow exposed by the PagingAdapter. In the case of our app, we can use it to display a loading spinner on first load.

We start collecting from the Flow, and use the CombinedLoadStates.refresh field to show a progress bar if the Pager isn’t loading and the existing list is empty. We use the refresh field because we only want to show the large progress bar when we launch the app the first time, or because we explicitly trigger a refresh. We can also check if any of the loading states have errored out and notify the user.

Recap

Thanks for reading along! To recap, we:

• Paged from the database as a single source of truth
• Used a RemoteMediator to feed the Room based PagingSource
• Updated the UI with progress bars based on the load states from the PagingAdapter’s LoadStateFlow

We’ll be wrapping up this series in the next article, so stay tuned and see you soon!