Android’s Jetpack Paging 3: Understanding its history and evolution by building a generic paging component
Written by Levy Schiavetti — Android Developer at AMARO
Skimming History
Even though Aza Raskin, the infinite scroll creator, regrets his own great contribution to the world of technology, it’s hard not to recognize that it really made our experience easier, and especially more fluid. Since then, it became possible for a user to see hundreds, thousands of posts, products or any kind of data presented as a list without having to hit a next or see more button and waiting for the page to refresh.
It took no more than a while for companies to see a huge opportunity and hence for developers to start writing code from this idea, creating their own paging components, each with their own specific need, but all sharing the same purpose, which is:
“Monitoring a UI scrolling component state, keeping track of its current page number and other data in order to trigger data requests and handle load and others indicators”
Speaking Android, it would require for us to watch a RecyclerView’s Adapter position through a ScrollListener interface to trigger new API or database requests. We would have bunches of code to control the requests similar to this:
override fun onScrolled(recyclerView: RecyclerView?, dx: Int, dy: Int) {
super.onScrolled(recyclerView, dx, dy)
val visibleItemCount = layoutManager.childCount
val totalItemCount = layoutManager.itemCount
val firstVisibleItemPosition =
layoutManager.findFirstVisibleItemPosition() if (!isLoading() && !isLastPage()) {
if (visibleItemCount + firstVisibleItemPosition >= totalItemCount && firstVisibleItemPosition >= 0) {
loadMoreItems()
}
}
}
Not that simple to implement and gets trickier as other features and needs start coming along.
In the previous code we find methods such as: isLastPage, isLoading, findFirstVisibleItemPosition, loadMoreItems and others. All of them are required to calculate if it’s time to call the data source again for more data. But too much for a view to calculate and process, when it could only be showing data, as it should. It’s clearly a downside to have this amount of logic on the view.
Ideally, when following some of the good practices on separating concerns, it’s not desirable to have that on our project. We usually want our ViewModels or Repositories to take care of that whereas the View only displays what has already been defined or calculated.
Jetpack’s Paging Library
Among other interesting new features, Google announced at its 2018 I/O, the first version of Jetpack’s Paging Library. From that moment on, Views taking care of the current page or calculating when to trigger requests were no longer necessary. In fact, not only Views, because most of the calculations were reduced everywhere, leaving most of the responsibility to this special library.
At the time of this writing, the library has three versions so far. Regardless of the version, its main components and their responsibilities remaining basically the same and consist of:
- The DataSource, responsible for managing database or network requests.
- The Adapter(s), in charge of keeping track of the view state, requesting more data to the DataSource when necessary, all under the hood.
But since the first version came out, the Paging Library has changed swiftly. A few methods have been deprecated, several classes renamed and performance increased. It now supports Flow and LiveData.
It’s been a significant improvement so far, and made our lives even easier now.
The previous versions
When it comes to the implementation on our side, the Data Source may have suffered the main changes. Its first version used to contain three methods whose purpose was to handle the logic regarding the different moments of the scroll:
loadInitial()
Triggered when the data was loaded for the first time and responsible for bringing the initial amount of data to the view.
loadAfter()
Whenever user was scrolling down and view detected it was time to fetch more data, this method was the one in charge of handling the next requests
loadBefore()
Depending on your paging mechanism, you could choose not to keep pages stored after a certain point, and thus, if the user decided to scroll back up, this was the method triggered, responsible for fetching previous data that was not available on cache anymore.
It is certainly much cleaner than managing all by ourselves on our views, adapters and repositories but still, it felt like it was a complex and verbose implementation as it’s possible to see:
override fun loadInitial(
param: LoadInitialParams<Long>, callback:
LoadInitialCallback<Long, Article>
) {
/*
1. Handle Loading or whatever necessary before the request
2. Perform initial request of data
3. Handle success or error result
4. Handle Loading or whatever necessary after the request
*/
}override fun loadBefore(
param: LoadParams<Long>, callback:
LoadCallback<Long, Article>
) {
/*
1. Handle Loading or whatever necessary before the request
2. Calculate the previous index
3. Perform the request of previous data
4. Handle success or error result
5. Handle Loading or whatever necessary after the request
*/
}override fun loadAfter(
param: LoadParams<Long>, callback:
LoadCallback<Long, Article>
) {
/*
1. Handle Loading or whatever necessary before the request
2. Calculate the next index
3. Perform the request of previous data
4. Handle success or error result
5. Handle Loading or whatever necessary after the request
*/
}
Google sensed that there was much room for improvement, many other aspects that could be simplified and ease the implementation for us. The result is Paging 3, which besides being able to work with Flow and LiveData, currently makes it really comfortable to use a paging mechanism on our projects. As a single sample, the previous Data Source implementation turned into a single method:
override suspend fun load(params: LoadParams<Int>): PagingSource.LoadResult<Int, Response> {
/*
1. Handle whatever necessary before the request
2. Calculate next and previous index
3. Perform request
4. Handle success or error result
5. Handle whatever necessary after the request
*/
}Paging 3: Creating a flow generic paging component
While presenting the main functionality and components of Paging 3, the idea is to build a generic paging component that may be reused all around the same project, with different types of list.
As mentioned, the data component, responsible for performing all the network or database requests, needs to extend from the PagingSource abstract class. Our component will also be abstract so other repositories may use it:
abstract class PagingRepository<Response : Any> : PagingSource<Int, Response>()The superclass demands two methods to be implemented:
getRefreshKey()
Returns the next key that will be used to load the subsequent request. This method is triggered before load() and sent to it via the LoadParams parameter.
As the key to be used in our component, we may use the state.anchorPosition, which will return the most recent accessed index in the list. When the step type to be used as a page indicator is an integer, state.anchorPosition should be the return value.
override fun getRefreshKey(state: PagingState<Int, Response>): Int? {
return state.anchorPosition
}load()
Its purpose it’s to load data by calculating the next and previous indexes and using them to perform the network or database requests. It should return the data within a PagingSource.The data should be returned through a LoadResult.Page when success, and LoadResult.Error when something went wrong.
To build a LoadResult.Page, we need to provide the next key, in case the user continues to scroll down, a previous key, in case the user loads data that has been previously discarded, and finally, a list containing the data to be displayed:
LoadResult.Page(
data = result,
nextKey = nextKey,
prevKey = previousKey
)As expected, the list will be retrieved by the request. The only thing to add is that, if anything goes wrong, we need to emit a LoadResult.Error. So the whole load() code will be wrapped in a try/catch block:
} catch (e: Exception) {
LoadResult.Error(e)
}As for the next and previous keys, they’ll have to be calculated by making use of the key parameter (received from getRefreshKey()).
To calculate the next key, we consider the returned list. If everything went well with the request and we have more items to show, then the key will be key + 1. In case there’s no more items, we may return null and the library will understand that it’s the end of the list and stop the requests:
val nextKey = if (list.isEmpty()) {
if (currentPage == 0) {
throw UnexpectedException()
} else {
null
}
} else {
currentPage + 1
}Similar to this, the previous key will be calculated such as:
val previousKey = if (currentPage == START_PAGE_INDEX) null else currentPage - 1It’s important to remember that load() only gets triggered once a Pager is created. A Pager is an entry point for paging. Besides a PagingConfig, the class responsible for carrying the details regarding the paging behavior, a Pager needs to provide the PagingSource that will be used to perform its requests:
private fun getDefaultPagingConfig(): PagingConfig {
return PagingConfig(
prefetchDistance = pageSize / 2,
pageSize = pageSize,
enablePlaceholders = false
)
}Having declared the PagingConfig, the Pager constructor will look like this:
return Pager(
config = getDefaultPagingConfig(),
pagingSourceFactory = { this }
).flowMost of the logic is done by now. What we need now is to provide a way of triggering the first request from the class that is going to inherit, which in this case will be a repository. Declaring an execute() method and passing the api request as a parameter may work well enough:
private lateinit var call: (suspend (currentPage: Int) -> Flow<List<Response>>)protected fun execute(
call: (suspend (currentPage: Int) -> Flow<List<Response>>)
): Flow<PagingData<Response>> {
this.call = call
return Pager(
config = getDefaultPagingConfig(),
pagingSourceFactory = { this }
).flow
}
Gathering all the code, except for a few details, this is going to be the whole abstract class:
import androidx.paging.Pager
import androidx.paging.PagingConfig
import androidx.paging.PagingData
import androidx.paging.PagingSource
import androidx.paging.PagingState
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.single/**
* Generic PagingSource component that handles performing paginated api requests.
*
* [Response] is the domain return type for the api call
*/
abstract class PagingRepository<Response : Any> : PagingSource<Int, Response>() { protected open val pageSize: Int = DEFAULT_PAGE_SIZE
private lateinit var call: (suspend (currentPage: Int) -> Flow<List<Response>>) private fun getDefaultPagingConfig(): PagingConfig {
return PagingConfig(
prefetchDistance = pageSize / 2,
pageSize = pageSize,
enablePlaceholders = false
)
} protected fun execute(
call: (suspend (currentPage: Int) -> Flow<List<Response>>)
): Flow<PagingData<Response>> {
this.call = call
return Pager(
config = getDefaultPagingConfig(),
pagingSourceFactory = { this }
).flow
} override fun getRefreshKey(state: PagingState<Int, Response>): Int? {
return state.anchorPosition
} override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Response> {
val currentPage = params.key ?: START_PAGE_INDEX return try {
val result = call(currentPage).single()
val nextKey = if (result.isEmpty()) {
if (currentPage == 0) {
throw UnexpectedException()
} else {
null
}
} else {
currentPage + 1
} val previousKey = if (currentPage == START_PAGE_INDEX) null else currentPage - 1 LoadResult.Page(
data = result,
nextKey = nextKey,
prevKey = previousKey
)
} catch (e: Exception) {
LoadResult.Error(e)
}
} private companion object {
const val START_PAGE_INDEX = 0
const val DEFAULT_PAGE_SIZE = 20
}
}
And as an example, here is a repository using the new PagingRepository. It gets a list of products and pass it back to the view:
import androidx.paging.PagingData
import kotlinx.coroutines.flow.Flowclass ProductRepositoryImpl(
private val productApiDataSource: ProductApiDataSource,
) : PagingRepository<Product>(), ProductRepository { override val pageSize = PRODUCT_PAGE_SIZE override fun getPagingProducts(): Flow<PagingData<Product>> {
return execute { currentPage -> getProducts(currentPage) }
} fun getProducts(
currentPage: Int
): Flow<List<Product>> {
return flow {
emit(
productApiDataSource.getProducts(
currentPage = currentPage,
pageSize = pageSize
)
)
}
} private companion object {
const val PRODUCT_PAGE_SIZE = 48
}
}
The final code presents a base class that could be used not only with the Product type, as in the last example, but any type at all. It should be really useful, for instance, in a large Clean Architecture project, where all repositories that contain requests using the paging mechanism may be able to perform their paging calls by only wrapping their API call with the execute method.
