Android Paging: Efficient way to populate recycler view dynamically

Src: io18 Paging Talk

List UI is a very common thing in Android application. Pretty much every app has a list of some data shown to users. We use the recycler view to show this data in scrollable behavior. When using architecture components we store this data in the ViewModel (to survive configuration changes) in the form of LiveData(to maintain dynamicity). That data is fetched either from a local database or from some API.

The problem with the above approach is that it loads the entire data and creates objects for all the entries every time the data is fetched or updated. But as a user can see only a small amount of data on a screen at a time, so fetching the entire data is just wasting the memory and also the battery and bandwidth.

A better approach should be to fetch data dynamically when it is required, i.e., as a user scrolls the list.

The Paging Library helps you load and display small chunks of data at a time. Loading partial data on-demand reduces the usage of network bandwidth and system resources.

It provides us with 5 benefits:

  1. Convenience: It provides you data in the form of LiveData so we can observe changes in the data and update the UI without needing much change to the code.
  2. Multiple Layers: It works completely fine with the local database backed by a server. It has callbacks to ease the process.

3. Performance: It does not do any big chunk of work on UI thread or any unnecessary work. Hence, it does not hinder performance and is fast and efficient.

4. Lifecycle aware: It will not do any work if the user is not on the screen related to the data.

5. Flexible: Keeping in mind the different databases and APIs it is easily configurable according to the use case.

Paging Library Architecture:

The paging library consists of 5 components:

Src: Paging Codelab
  • PagedListThe Paging Library’s key component is the PagedList class, which is a collection that loads chunks of your app's data, or pages, asynchronously. It can be used to load data from sources you define, and present it easily in your UI with a RecyclerView. As more data is needed, it's paged into the existing PagedList object. If any loaded data changes, a new instance of PagedList is emitted to the observable data holder from a LiveData or RxJava2-based object. As PagedList objects are generated, your app's UI presents their contents, all while respecting your UI controllers' lifecycles.
  • DataSource and DataSource.Factory — a DataSource is the base class for loading snapshots of data into a PagedList. A DataSource.Factory is responsible for creating a DataSource.
  • LivePagedListBuilder — builds a LiveData<PagedList>, based on DataSource.Factory and a PagedList.Config(the configuration file for paging).
  • BoundaryCallback — signals when a PagedList has reached the end of available data so that more data can be fetched from the server in case of DB backed up by the server.
  • PagedListAdapter — a RecyclerView.Adapter that presents paged data from PagedLists in a RecyclerView. PagedListAdapter listens to PagedList loading callbacks as pages are loaded and uses DiffUtil to compute fine-grained updates as new PagedLists are received.

Implementation:

To start using paging, replace all the List data to PagedList in the ViewModels and its observers in activities and fragments. If there are any models responsible for updating UI, change them also.

val dataList: LiveData<List<data_model_object>> = ...
val dataList: LiveData<PagedList<data_model_object>> = ...

viewModel.dataList.observe(this, Observer<PagedList<data_model_object>> {...})

The PagedList loads content dynamically from a source. If the database is the main source of truth for the UI, it also represents the source for the PagedList. If your app gets data directly from the network and displays it without caching, then the class that makes network requests would be your data source.

A source is defined by a DataSource class. To page in data from a source that can change—such as a source that allows inserting, deleting or updating data—you will also need to implement a DataSource.Factory that knows how to create the DataSource. Whenever the data is updated, the DataSource is invalidated and re-created automatically through the DataSource.Factory.

The Room persistence library provides native support for data sources associated with the Paging library. For a given query, Room allows you to return a DataSource.Factory from the DAO and handles the implementation of the DataSource for you.

@Daointerface RepoDao {
@Query(<query>)
fun someQueryMethod(): LiveData<List<data_model_object>>
fun someQueryMethod(): DataSource.Factory<Int, data_model_object>
...}

To build and configure a LiveData<PagedList>, use a LivePagedListBuilder.

//get the dataSourceFactory by making a call to dao function
val data = LivePagedListBuilder(dataSourceFactory, DATABASE_PAGE_SIZE).build()

Besides the DataSource.Factory, you need to provide a PagedList configuration, which can include the following options:

  • The size of a page loaded by a PagedList
  • How far ahead to load
  • How many items to load when the first load occurs during the creation of pagedList by DataSource.
  • Whether null items can be added to the PagedList, to display placeholders for data that hasn't been loaded yet. As soon as the data becomes available the ViewHolder gets populated with that and the view is updated with a nice cross-fade animation. If you have it disabled then your scroll bar jumps whenever data loads and that gives rise to bad user experience.
    But there are some problems with placeholders:
    a) All the items should be of the same size otherwise the animation looks weird.
    b) The adapter must handle null items.
    c) DataSource must count items (room does that implicitly).

You can have a separate config object to add configurations.

val config = PagedList.Config.Builder()
.setPageSize(30) //mandatory to provide
.setInitialLoadSizeHint(50) //default: page size*3
.setPrefetchDistance(10) //default: page size
.setEnablePlaceholder(true/false) //default: true
.build()
val data = LivePagedListBuilder(dataSourceFactory, config).build()

Note: The DataSource page size should be several screens' worth of items. If the page is too small, your list might flicker as pages content doesn't cover the full screen. Larger page sizes are good for loading efficiency, but can increase latency when the list is updated.

To bind a PagedList to a RecycleView, use a PagedListAdapter. The PagedListAdapter gets notified whenever the PagedList content is loaded and then signals the RecyclerView to update.

class RecyclerViewAdapter : PagedListAdapter<data_model_object, RecyclerView.ViewHolder>(DATA_COMPARATOR)

Also notice that now the data item is nullable and we have to update the onBindViewHolder to make it nullable. You can work with that data by applying checks if you want.

val item:data_model_object? = getItem(position)// optional        
if (item != null) {
(holder as itemViewHolder).bind(item) }

If your entire data is on a local database then we are done and you do not need to perform this step. But if your local DB is backed up by a server then we should find to way to ask for more data from the server only when there is no more data in the local DB. By providing all the data to the UI from the DB itself we follow the single source of truth principle.

For this, we use BoundaryCallback which gives two methods to perform some task when there are zero items in the list or the last available has been loaded so we need more data.

For this create a class extending the BoundaryCallback class and provide it the service and the local DB instances. Then override the methods to update the DB with your server data.

class sampleBoundaryCallback(
private val service: ApiService,
private val db: localDb,
private val query:String
) : PagedList.BoundaryCallback<data_model_object>() {
override fun onZeroItemsLoaded() {
loadDataFromServerAndStoreInDB()
}

override fun onItemAtEndLoaded(itemAtEnd: data_model_object) {
loadDataFromServerAndStoreInDB()
}
}

Then add the callback to LivePagedListBuilder:

val boundaryCallback = SampleBoundaryCallback(service, db, query)

// Get the paged list
val data = LivePagedListBuilder(dataSourceFactory, config)
.setBoundaryCallback(boundaryCallback)
.build()

Wrap up:

Now that we added all the components, let’s take a step back and see how everything works together.

The DataSource.Factory (implemented by Room) creates the DataSource. Then, LivePagedListBuilder builds the LiveData<PagedList>, using the passed-in DataSource.Factory, BoundaryCallback, and PagedList configuration. This LivePagedListBuilder object is responsible for creating PagedList objects. When a PagedList is created, two things happen at the same time:

  • The LiveData emits the new PagedList to the ViewModel, which in turn passes it to the UI. The UI observes the changed PagedList and uses its PagedListAdapter to update the RecyclerView that presents the PagedList data. (The PagedList is represented in the following animation by an empty square).
  • The PagedList tries to get the first chunk of data from the DataSource. When the DataSource is empty, for example when the app is started for the first time and the database is empty, it calls BoundaryCallback.onZeroItemsLoaded(). In this method, the BoundaryCallback requests more data from the network and inserts the response data in the database.

After the data is inserted in the DataSource, a new PagedList object is created (represented in the following animation by a filled-in square). This new data object is then passed to the ViewModel and UI using LiveData and displayed with the help of the PagedListAdapter.

When the user scrolls, the PagedList requests that the DataSource load more data, querying the database for the next chunk of data. When the PagedList paged all the available data from the DataSource, BoundaryCallback.onItemAtEndLoaded() is called. The BoundaryCallback requests data from the network and inserts the response data in the database. The UI then gets re-populated based on the newly-loaded data.

You can find a sample project implementing Paging here. I will also make a video tutorial for paging on youtube and will add the link here.

Feel free to comment if there are doubts or if you feel anything needs to be corrected😀.

IO18' Paging library launch

Android Codelab

Android Developers documentation