Pagination in Android with Paging 3, Retrofit and kotlin Flow

Photo by BENCE BOROS on Unsplash

Haven’t you asked how does Facebook, Instagram, Twitter, Forbes, etc… let you “scroll infinitely” without reach a “end” of information in their apps? Wouldn’t you like to implement something like that?

This “endless scrolling” feature is commonly called “Pagination” and it’s nothing new. In a brief resume, pagination help you to load chunks of data associated with a “page” index.

Let’s assume you have 200 items to display in your RecyclerView. What you would do normally is just execute your request, get the response items and submit your list to your adapter. We already know that RecyclerView by it’s own is optimized for “recycle our views”. But do we really need to get those 200 items immediately? What if our user never reach the top 100, or top 50? The rest of the non displayed items are still keep on memory.

What pagination does (in conjunction with our API) is that it let us establish a page number, and how many items per page can we load. In that way, we can request for the next page of items only when we reach the bottom of our RecyclerView.

Non library approach

Before paging 3, you could have implemented Pagination by adding some listeners to your RecyclerView and those listeners would be triggered when you reach the bottom of your list. There are some good samples about it, here is a video with a vey detailed explanation (it’s on Indi, but it is understandable anyways).

A real PRODUCTION ready example is on the Plaid app. Look at their InfinteScrollListener class.

Android Jetpack Paging 3 and Flow

Today you gonna learn how to implement Pagination by using paging 3 from the android jetpack libraries. For my surprise the codelab of Paging 3 was one of the most easiest I have ever done. Florina Muntenescu did a great job with each step of the codelab, go check it out and give it a try. If you want to go straight to the sample code, check this pull request and see step by step how I implement paging 3 to this project.

ChristopherME/movies-android

Step by Step

For use Paging 3, first we need to add the dependencies in our app level gradle file:

dependencies {
  def paging_version = "3.0.0" //current version at the time

  implementation "androidx.paging:paging-runtime:$paging_version"

  // alternatively - without Android dependencies for tests
  testImplementation "androidx.paging:paging-common:$paging_version"

  // optional - RxJava2 support
  implementation "androidx.paging:paging-rxjava2:$paging_version"

  // optional - RxJava3 support
  implementation "androidx.paging:paging-rxjava3:$paging_version"

  // optional - Guava ListenableFuture support
  implementation "androidx.paging:paging-guava:$paging_version"

  // Jetpack Compose Integration
  implementation "androidx.paging:paging-compose:1.0.0-alpha08"
}

We gonna need basically 4 classes in order to implement our “infinite scroll” functionality.

• PagingData — A container for paginated data. Each refresh of data will have a separate corresponding PagingData. This is what you are going to return from your repository (If using the repository pattern).
• PagingSource — a PagingSource is the base class for loading snapshots of data into a stream of PagingData. This is where you are going to retrieve your items from a service (usually Retrofit) and return them wrapped into a LoadResult type.
• Pager.flow — builds a Flow<PagingData>, based on a PagingConfig and a function that defines how to construct the implemented PagingSource.
• PagingDataAdapter — a RecyclerView.Adapter that presents PagingData in a RecyclerView. The PagingDataAdapter listens to internal PagingData loading events as pages are loaded and uses DiffUtil on a background thread to compute fine-grained updates as updated content is received in the form of new PagingData objects.

Image from paging 3 docs.

Ok, now that we are some kind of familiarized with the classes, let’s start! We are going to make an app that load movies, like Netflix. We are going to use TMDB API( For use the API don’t forget to register and request your developer API KEY).

1. Movies service with retrofit 🎬

Let’s define a simple retrofit service that return a list of movies. (Retrofit implementation it’s off this topic therefore not mentioned on this blog).

2. Implement the PagingSource 📖

In order to build our PagingSource, first we need to indicate the type of the paging key and the type of data to load.

PagingSource<Type_Of_Paging_key, Type_of_Data_to_load>

In this case the type of paging key is Int, as our page index is number based. And our type of data to load is of MovieResponse type.

We need to get that information from “somewhere”, that’s where MovieService enter the game. Here is how our MoviesPagingSource finally look.

The implementation of this class is usually going to be the same as the presented for almost every use case. For more info checkout the codelab.

We have to override two methods here. load() and getRefreshKey().

• load(params: LoadParams<Int>) — This function will be called by the Paging library to asynchronously fetch more data to be displayed as the user scrolls around. The LoadParams object keeps information related to the load operation. We are going to use the params.key for get the current page index. If this is the first time that load is called, params.key will be null. In that case, we will have to define the initial page key with the TMDB_STARTING_PAGE_INDEX constant. Finally, params.loadSize is the requested number of items to load.
• getRefreshKey() — The refresh key is used for subsequent refresh calls to PagingSource.load().

3. Build and configure PagingData 🛠️

We create our PagingSource successfully. But now we need a way to emit flows of PagingData. For that task, we’re gonna need a PagingData builder.
In our repository or dataSource we must create an instance of Page which is going to accept a PagingConfig and a pagingSourceFactory.

This remote data source is later injected into a repository. If you only use a repository and don’t have a remote or local data source, then you probably should create the Pager directly on that class.

The PagingConfig have two params, the network page size and the enablePlaceHolders. Which in this case is going to be false because we don't need a “shimmer” effect on this sample. Later, we need to define our pagingSourceFactory, which is going to be our MoviesPagingSource .

4. Request your data in the ViewModel / Presenter

We are almost there, now we just need to call our getMovies() method in our ViewModel / Presenter.

5. Integrate it in your UI ⛓️

So far we are setting up how to obtain the data. But now it’s time to render the UI. We are going to use the PagingDataAdapter from the library which is another implementation of the ListAdapter class.

MovieAdapter, MovieDiffCallBack & MovieViewHolder

And finally, in your fragment / activity

With that you have a functional “infinite scrolling” effect. There are some missing things though. Show empty view, display a progress when we are requesting more items, etc. You can find how to implement all that in the paging 3 codelab (I really encourage you to do it) or checkout the sample project mentioned at the beggining.

Final thoughts

I liked the paging 3 library, but I feel like it attaches you to their implementation and you loose flexibility. In the main branch of this project, I use the Either sealed class for wrap my interfaces methods and handle the possible failure or success data by my own, I couldn't do that anymore due to the need of return a PagingData<UiObject>. Also, I implement a safe retrofit wrapper for execute the calls on the Dispatchers.IO context and avoid to repeat the same thing in all my remoteDataSources, but that was gone too because the retrofit service execution is done in the PagingSource and have to explicitly return a LoadResult , as you can see the exceptions are handled there too. Yea, you could modify the call extensions and make them return a LoadResult, but what about other uses cases like a simple login? 🙃

👉 Checkout the branch feature/movies-paging3 for the complete implementation.

Try the library and tell me how is it going for you!

See you later! 👋