PagedList BoundaryCallback applying Single Source Of Truth concept in Kotlin

Implementation of Paging library BoundaryCallback (MVVM, Room, Retrofit, PagedListAdapter, etc)

I want to share my experience and knowledge (correct me if you find any errors :)) that i acquire during some personal projects i developed.

Firs of all, for those who are still learning about it, PagedList is a library from Android Jetpack that helps to load and display chunks of data at a time. Normally, this reduces usage of network bandwidth, systems resources and contributes for user friendly apps, where the user can scroll down a RecyclerView without the need for waiting the data to load. All of this could be achieved by listening the scroll of RecyclerView and act accordingly, but with this would be difficult to maintain efficiency and logic. I won’t enter in details about the implementation of the typical PagedList solution, where you use a DataSourceFactory and a DataSource class with all 3 methods (loadInitial, loadAfter and loadBefore), like you can see on the image above. Instead, i will be using the PagedList.BoundaryCallback class. If you need, you can check my repository.

Also, as i referred before, we will be applying Single Source of Truth. Sometimes in your app you will want to save some information locally instead of always making new requisitions to the REST APIs, or even if different REST APIs endpoints are returning the same data, if we don’t check for consistency, our UIs could show confusing information. For this reason, it is common to fetch the data needed from a single source, for example, our Room database. This is called Single Source Of Truth concept. Again, i won’t post all my room code because you can find plenty of examples online.

Also, i developed this project using flickr API to fetch images for an image gallery, according a search tag provided by the user.

The purpose of the following GalleryBoundaryCallback class is to fetch the necessary data and to send it back to repository class. With this approach, Repository has only one job, that is asking BoundaryCallback for information, not caring about how or where the data comes from. After this, repository will send data back to ViewModel that will warn the view with the help of LiveData.

So, let’s start coding :)

My build.grade(:app) imports look like this (despite i will only show PagedList code, i will post all my imports):

def paging_version = "2.1.1"
def room_version = "2.2.4"
//ViewModel and lifecycle to work with coroutines
implementation "androidx.lifecycle:lifecycle-extensions:$arch_version"
implementation "androidx.lifecycle:lifecycle-viewmodel-ktx:$arch_version"
implementation "androidx.lifecycle:lifecycle-livedata-ktx:$arch_version"
implementation "androidx.lifecycle:lifecycle-runtime-ktx:$arch_version"

//paging
implementation "androidx.paging:paging-runtime-ktx:$paging_version"
testImplementation "androidx.paging:paging-common-ktx:$paging_version"

//glide
implementation 'com.github.bumptech.glide:glide:4.9.0'
implementation "androidx.swiperefreshlayout:swiperefreshlayout:1.0.0"
kapt 'com.github.bumptech.glide:compiler:4.9.0'

//room
implementation "androidx.room:room-runtime:$room_version"
kapt "androidx.room:room-compiler:$room_version" // For Kotlin use kapt instead of annotationProcessor
// optional - Kotlin Extensions and Coroutines support for Room
implementation "androidx.room:room-ktx:$room_version"
// optional - RxJava support for Room
implementation "androidx.room:room-rxjava2:$room_version"
// optional - Guava support for Room, including Optional and ListenableFuture
implementation "androidx.room:room-guava:$room_version"
//Navigation Component
implementation 'androidx.navigation:navigation-fragment-ktx:2.2.1'
implementation 'androidx.navigation:navigation-ui-ktx:2.2.1'
implementation 'androidx.lifecycle:lifecycle-extensions:2.2.0'

//dagger2
def dagger_version = "2.25.4"
implementation "com.google.dagger:dagger:$dagger_version"
kapt "com.google.dagger:dagger-compiler:$dagger_version"
kaptTest 'com.google.dagger:dagger-compiler:2.25.4'

//gson
implementation 'com.google.code.gson:gson:2.8.6'

//retrofit
implementation 'com.squareup.retrofit2:retrofit:2.6.1'
implementation 'com.squareup.retrofit2:converter-gson:2.6.1'
implementation 'com.squareup.retrofit2:converter-moshi:2.6.0'
implementation 'com.squareup.retrofit2:converter-scalars:2.3.0'
implementation 'com.jakewharton.retrofit:retrofit2-kotlin-coroutines-adapter:0.9.2'

And this is how my BoundaryClass looks like:

class GalleryBoundaryCallback @Inject constructor(
    private val galleryRemoteRepository: GalleryRemoteRepository,
    private val dao: GalleryDao,
    private val scope: CoroutineScope)
    : PagedList.BoundaryCallback<PhotoRoomModel>() {

    var lastRequestedPage = 0
    var tag: String = ""

    private val connectionError = MutableLiveData<Pair<Boolean,Int>>()
    val networkErrors: LiveData<Pair<Boolean,Int>> get() = connectionError
    private val requestState = MutableLiveData<Boolean>()
    val networkState: LiveData<Boolean> get() = requestState

    private var isRequestHappening = false

    /**
     * Database returned 0 items. We should query the backend for more items.
     */
    @MainThread
    override fun onZeroItemsLoaded() {
        requestAndSaveData(tag)
    }

    /**
     * User reached to the end of the list.
     */
    @MainThread
    override fun onItemAtEndLoaded(itemAtEnd: PhotoRoomModel) {
        requestAndSaveData(tag)
    }

    /**
     * every time it gets new items, boundary callback simply inserts them into the database and
     * paging library takes care of refreshing the list if necessary.
     */
    private fun insertItemsIntoDb(results: MutableList<PhotoRoomModel>, initial: Boolean) {
        Log.e("PostsDataSource ins: ", "Inserted on local")
        dao.insertAll(results)
    }

    override fun onItemAtFrontLoaded(itemAtFront: PhotoRoomModel) {
        // ignored, since we only ever append to what's in the DB
    }

    private fun requestAndSaveData(tag: String) {
        Log.e("PostsDataSource page: ", "" + lastRequestedPage)
        if (isRequestHappening) {
            return
        }
        scope.launch {
            try {
                isRequestHappening = true
                requestState.postValue(true)
                val response = galleryRemoteRepository.getSearchResults(lastRequestedPage, tag)
                requestState.postValue(false)
                if (lastRequestedPage == 0) {
                    insertItemsIntoDb(response, true)
                } else {
                    insertItemsIntoDb(response, false)
                }
                isRequestHappening = false
                lastRequestedPage++
            } catch (exception: UnknownHostException) {
                isRequestHappening = false
                val retryPair = Pair(true, lastRequestedPage)
                connectionError.postValue(retryPair)
                Log.e("PostsDataSource", exception.message!!)
            }
        }
    }

    suspend fun removeItemsFromDb() {
        Log.e("PostsDataSource", "Removed items from localDatabase")
        dao.removeAll()
    }

}

As you can see in class declaration, i am using Dagger 2 for dependency Injection. I am injecting the class GalleryRemoteRepository, where i have all my methods that will call retrofit service, my GalleryDao for Room and finally a coroutineScope, responsible for calling my suspend methods inside GalleryRemoteRepository. I used 2 liveData that allow keep track of the current progress of the BoundaryCallBack, in case the user swipes the RecyclerView too fast, showing a ProgressBar in this case or report some error. The only problemi faced using this BoundaryCallback class was that it was not possible to keep track of the last page fetched automatically, so you need to keep track by yourself (i used lastRequestedPage variable). This might come in hand if for some reason the user turned off Internet. If you want, you can show a snackBar with some text and an option to “retry” the search.

PhotoRoomModel is the Room class where i declared my Room database parameters, as you can see below.

@Entity(tableName = "gallery", indices = arrayOf(Index(value = ["id"], unique = true)))
data class PhotoRoomModel(

    @SerializedName("id")
    @PrimaryKey
    val id: String,
    @SerializedName("url_q")
    val url_q: String? = null,
    @SerializedName("url_l")
    @Nullable
    val url_l: String? = null
)

There are 3 main methods, responsible for keeping track of the current Recycler page scrolled: onItemAtEndLoaded, onItemAtFrontLoaded and onZeroItemsLoaded. As name suggests, they are called each time the last, first or zero items are loaded into RecyclerView “page”. This RecyclerView page is configured on repository class, and in my case, is configured to load or fetch 20 items at a time from the REST API. I chose to just use onZeroItemsLoaded and onItemAtEndLoaded methods, and i just call my main method “requestAndSaveData”, that has the responsability of calling my GalleryRemoteRepository for fetching more data from API.

When we first launch an app that will call BoundaryCallback class, onZeroItemsLoaded will be called for fetching data from API. By default, BoundaryCallback always have a margin of 3 pages, so onZeroItemsLoaded will fetch the first 3 pages (60 images url) of API and store them on my RoomDatabase. After we scroll our RecyclerView to 19th item, onItemAtEndLoaded method will be called and the 4th page will be fetched from the API. Right now you might be wondering, “what if i used my app one time, closed it, stored the first 4 pages on my Room database and the open the app a second time?” Well, PagedList library has this awesome functionality where you can add “DataSource.Factory” to the Room Dao method, so now your repository can only observe your Room database for new data.

@Query("Select * FROM gallery")
fun getAll() : DataSource.Factory<Int, PhotoRoomModel>

My repository class looks like this:

class GalleryRepository @Inject constructor(private val database: GalleryDatabase,private val galleryBoundaryCallback: GalleryBoundaryCallback,private val coroutineScope: CoroutineScope) {

    companion object {
        private const val PAGE_SIZE = 20

        fun pagedListConfig() = PagedList.Config.Builder()
            .setInitialLoadSizeHint(PAGE_SIZE)
            .setPageSize(PAGE_SIZE)
            .setEnablePlaceholders(true)
            .build()
    }

    fun observeGalleryImages(tag: String, lastRequestedPage: Int, localSearch: Boolean): RepoResult {
        lateinit var dataSourceLocalFactory: DataSource.Factory<Int, PhotoRoomModel>
        lateinit var data : LivePagedListBuilder<Int, PhotoRoomModel>
        coroutineScope.launch {
            try {
                dataSourceLocalFactory = database.galleryDao().getAll()
                if (lastRequestedPage == 0 && !localSearch) {
                    galleryBoundaryCallback.removeItemsFromDb()
                }
            } catch (exception : Exception) {
                Log.e("RoomError", exception.message!!)
            }
        }
        galleryBoundaryCallback.tag = tag
        galleryBoundaryCallback.lastRequestedPage = lastRequestedPage
        val networkErrors = galleryBoundaryCallback.networkErrors
        val networkState = galleryBoundaryCallback.networkState

        if (localSearch) {
            data = LivePagedListBuilder<Int, PhotoRoomModel>(dataSourceLocalFactory, pagedListConfig())
        } else {
            // Get the paged list
            data = LivePagedListBuilder<Int, PhotoRoomModel>(dataSourceLocalFactory, pagedListConfig()).setBoundaryCallback(galleryBoundaryCallback)

        }
        return RepoResult(data.build(), networkErrors, networkState)
    }
}

In this class i am configuring my PagedList.Config.Builder and observing my Room database data using “LivePagedListBuilder”. Also, using a coroutineScope i am also loading some data right after a search is done by the user, and if user has Internet connection, deleting my Room database data, so that it can be updated if there are any changes on back-end side. Also, if user starts the app without Internet connection, i will only search on my Room database, without the BoundaryCallback class. My RepoResult class looks like this and it’s only used to observe data changes from ViewModel:

data class RepoResult(
    val data: LiveData<PagedList<PhotoRoomModel>>,
    val networkErrors: LiveData<Pair<Boolean,Int>>,
    val networkState: LiveData<Boolean>
)

Finally, from my ViewModel i observe the liveData values like this:

class SearchResultsViewModel @Inject constructor(private val galleryRepository: GalleryRepository) : ViewModel() {
    var localSearch : Boolean = false

    private val queryLiveData = MutableLiveData<String>()
    private val queryLastRequestedPage = MutableLiveData<Int>()

    /**
     * Val responsible for start searching for queried tag
     */
    private val repoResult: LiveData<RepoResult> = Transformations.map(queryLiveData) {
        galleryRepository.observeGalleryImages(it, queryLastRequestedPage.value!!, localSearch)
    }

    /**
     * Val responsible for listening Paging SingleSourceOfTruth results from BoundaryCallback
     */
    val repos: LiveData<PagedList<PhotoRoomModel>> = Transformations.switchMap(repoResult) { it -> it.data }

    val networkErrors: LiveData<Pair<Boolean, Int>> = Transformations.switchMap(repoResult) { it ->
        it.networkErrors
    }

    val networkState: LiveData<Boolean> = Transformations.switchMap(repoResult){ it->
        it.networkState
    }

    /**
     * Search galleryImages based on a query string.
     */
    fun searchRepo(queryString: String, lasteRequestedPage: Int) {
        queryLastRequestedPage.postValue(lasteRequestedPage)
        queryLiveData.postValue(queryString)
    }

    /**
     * Refresh request on swiping screen
     */
    fun refresh(tag: String) {
        searchRepo(tag, 0)
    }
}

This code is simple to understand, as i’m only listening for any changes on data fetched and also starting the search when the user clicks search button on my fragment. I also added a swipe to refresh option.

And that’s it. I’ve implemented an image gallery that allows user to search for any tag, and the app will provide the user an infinite list of images related with that tag. I skipped most of my code, because as i said before, the purpose of this was to show how to implement Single Source Of Truth using PagedList.BoundaryCallback, but if you have any doubts or questions, feel free to ask. Also, if you find something wrong, please leave a comment (i’m also learning :D). Hope you like it.