Creating A NewsApp in Android Paging Library, Retrofit, Room, ViewModel
Acknowledgement
The data is collected from NewsApi.org
Github Repository Url
If you want the code, here is the link. If you want to read the description, continue reading.
Update 12/11/2020: I have updated the code with online and offline support. Please refer at the bottom part of the story.
Preview
Description
Hello there!
So I was struggling with this android pagination stuffs for quite some time now. You see, most of the tutorials are either really old(you don’t want to see them), some only show you the paging library with networking support only, some only show database support, some don’t show a loader at the ena of recyclerView and so on.Finally, last night I was able to code a sample pagination app successfully with networking, database support, a little loader at the bottom and an extra row at top(you know, for displaying stories/my day/ whatever you call it). So finally after understanding all these things properly, I wish to keep all of it in this project. I hope you’ll like it.
We’ll be using android paging library 2.1.2 as it is production ready. Though the paging library 3.0 alpha is released, it’s still in alpha. So it might not be a good idea to use it in production. Hence the version 2.1.2.
Add News Api Key
we need this key to fetch data from newsapi.org. In production, we want to keep these keys secret. In this case, I’ll be putting this in local.properties file as it is ignored by the git. If you keep it in some other file, add it in the .gitignore file.
So in local.properties (OR your desired file), add this, may be at the end:
newsApiKey=123_your_top_secret_key_789Now open your app gradle file, and:
def getNewsApiKey() {
Properties properties = new Properties()
properties.load(project.rootProject.file('local.properties').newDataInputStream())
return properties.getProperty("newsApiKey");
}Then add this line in default config:
android{
// ... other lines
defaultConfig{
// ... other config
buildConfigField "String", "NEWS_API_KEY", "\""+getNewsApiKey()+"\"" // <-- Add this line
// ... other config
}
}Now create a file, say , Const.kt. We can create a constant string and initialize it with the actual api key like this:
object Const {
const val BASE_URL = "https://newsapi.org/v2/";
const val API_KEY = BuildConfig.NEWS_API_KEY;
}Add Retrofit Client, Room
We’ll be using retrofit to use our api, and Room database library to work with sqlite. I created some models according to api response.
Add RecyclerView Adapter
This is just a typical recyclerView code. However, we’ll be extending PagedListAdapter<Article, RecyclerView.ViewHolder>. I also created some viewHolders to show 1. Stories at the top, 2. actual news items in the middle, 3. a footer at the bottom showing a loader.
Now to keep things clear, rename position to uiposition (ie, onBindViewHolder(holder: RecyclerView.ViewHolder, position: Int) to onBindViewHolder(holder: RecyclerView.ViewHolder, uiPosition: Int) etc).
And inside, we define,
dataPosition = uiPosition - OFFSET_KOUNT;
OFFSET_KOUNT = number of header rowsSuppose, we have N items loaded from database / network. But in the ui, we have (1 header for story) + (N data items) + (1 footer) = N+2 ui items. So we need to make a proper mapping between our uiPosition and actual data position. So in this case, ith data will go to (i+1)th uiPosition.
Similarly, if we had OFFSET_KOUNT=m headers, then dataPosition = uiPosition - m would be the relation. For now, the OFFSET_KOUNT = 0. Later, I'll change it and we'll see how to implement it.
Connect RecyclerView with Database
We need to create a pagedList of Articles from database. For that we write this method:
private fun initList() {
val config:PagedList.Config = PagedList.Config.Builder()
.setPageSize(30)
.setEnablePlaceholders(false)
.build(); liveArticleList = initializedPagedListBuilder(config).build(); }
And,
private fun initializedPagedListBuilder(config: PagedList.Config):
LivePagedListBuilder<Int, Article> { val database:NewsRoomDatabase = NewsRoomDatabase.getDatabase(this);
val livePageListBuilder = LivePagedListBuilder<Int, Article>(
database.newsDao().articles,
config); livePageListBuilder.setBoundaryCallback(NewsBoundaryCallback(database));
return livePageListBuilder
}
So now we create NewsBoundaryCallback.kt file. It extends BoundaryCallBack class. Whenever, we reach at the end of recyclerview, this callback is triggered. It will handle network call to fetch new data.
When we reach to the end of our pagedList, we need to make a network call, insert the data inside the database and show it to out recyclerView. This is a complicated task. Luckily we have PagingRequestHelper.java to help us. It is written by the same dudes who wrote the paging library. For some weirdo reason, it is not inside the library itself, so we need to copy/download it from github, and use it.
If we run our code, we’ll see that there is nothing. So next we’ll connect with network to actually populate the database and show it to recyclerView.
Use Boundary Callback to Trigger the API
Open the NewsBoundaryCallback.kt file and add the following code:
/** override methods */
override fun onZeroItemsLoaded() {
super.onZeroItemsLoaded(); helper.runIfNotRunning(PagingRequestHelper.RequestType.INITIAL){
helperCallback: PagingRequestHelper.Request.Callback? ->
var call:Call<NewsApiResponse> = api.fetchFeed("bbc-news", Const.API_KEY, pageNumber, Const.PAGE_SIZE);
call.enqueue(object: Callback<NewsApiResponse>{
override fun onResponse(call: Call<NewsApiResponse>, response: Response<NewsApiResponse>) {
if(response.isSuccessful && response.body()!=null) {
Log.e(TAG, "onSuccess");
val articles: List<Article>? = response.body()?.articles?.map { it };
executor.execute{
db.newsDao().insertAll(articles?: listOf()); // ?: listOf(); -> if null create emptyList
pageNumber++;
helperCallback?.recordSuccess();
}
}else{
Log.e(TAG, "onError");
helperCallback?.recordFailure(Throwable("onError"));
}
} override fun onFailure(call: Call<NewsApiResponse>, t: Throwable) {
Log.e(TAG, "onFailure");
helperCallback?.recordFailure(t);
} })
};
} override fun onItemAtEndLoaded(itemAtEnd: Article) {
super.onItemAtEndLoaded(itemAtEnd);
helper.runIfNotRunning(PagingRequestHelper.RequestType.AFTER){
helperCallback: PagingRequestHelper.Request.Callback? ->
var call:Call<NewsApiResponse> = api.fetchFeed("bbc-news", Const.API_KEY, pageNumber, Const.PAGE_SIZE);
call.enqueue(object: Callback<NewsApiResponse>{
override fun onResponse(call: Call<NewsApiResponse>, response: Response<NewsApiResponse>) {
if(response.isSuccessful && response.body()!=null) {
Log.e(TAG, "onSuccess");
val articles: List<Article>? = response.body()?.articles?.map { it };
executor.execute{
db.newsDao().insertAll(articles?: listOf()); // ?: listOf(); -> if null create emptyList
pageNumber++;
helperCallback?.recordSuccess();
}
}else{
Log.e(TAG, "onError");
helperCallback?.recordFailure(Throwable("onError"));
}
} override fun onFailure(call: Call<NewsApiResponse>, t: Throwable) {
Log.e(TAG, "onFailure");
helperCallback?.recordFailure(t);
} })
};
}
Basically we have to write some codes in onZeroItemsLoaded() and onItemAtEndLoaded() methods. We have PagingRequestHelper helper. Inside Helper.runIfNotRunning we conplete this lambda. This is where we write our networking code. Just some trivial retrofit call. On success, we insert the result in our database. If we run our code, we'll see something like this!
Add a Footer / Loading
Now we want to add a loader to make things a bit pretty. Create an enum
enum class LoaderState {
DONE, LOADING, ERROR
}This will help us to keep track of our loading states. In the main activity, create a LiveData object. LiveData helps us with some trigger stuffs. Whenever the liveData value changes, it will auto-trigger a change. So all we have to do is, define what changes will take place (in this case, show / hide a loader).
So in main activity, we add this:
liveLoaderState.observe(this, Observer<LoaderState>{
newState -> newsFeedAdapter?.setLoaderState(newState);
})Now open the adapter, and make these changes:
override fun getItemCount(): Int {
return OFFSET_KOUNT + super.getItemCount() + isLoading();
} /** private methods */
fun isLoading():Int {
if(state.equals(LoaderState.LOADING))
return 1;
return 0;
} /** public apis */
public fun setLoaderState(newState: LoaderState) {
this.state = newState;
notifyDataSetChanged(); // <-- this tells the adapter to update the ui
}
Finally, we make a change in out NewsBoundaryCallback class by passing a liveLoaderState: MutableLiveData<LoaderState> parameter. Create a field to hold on to this reference. Now inside onZeroItemsLoaded() and onItemAtEndLoaded() we will simply update it's value like this:
var call:Call<NewsApiResponse> = api.fetchFeed("bbc-news", Const.API_KEY, pageNumber, Const.PAGE_SIZE); liveLoaderState.postValue(LoaderState.LOADING); // <----- call.enqueue(object: Callback<NewsApiResponse>{
override fun onResponse(call: Call<NewsApiResponse>, response: Response<NewsApiResponse>) {
// check if successfull
liveLoaderState.postValue(DONE);
// ...
} override fun onFailure(call: Call<NewsApiResponse>, t: Throwable){
liveLoaderState.postValue(ERROR);
}
}
Simply run the code, and you’ll see the loader. Now it might be hard to see the loader if your network connection is good. If you can’t see it properly, try this: open Const.kt file and change PAGE_SIZE = 1, this will allow you to see the loader(but don't try this on your production app!)
Add a top Row for Stories
You may want to add one or two extra rows on top. Say, facebook / instagram stories, stuff like that. Open the NewsFeedAdapter, and add val OFFSET_KOUNT = 1; Then on bindViewHolder add this at the top:
override fun onBindViewHolder(holder: RecyclerView.ViewHolder, uiPosition: Int) {
var dataPosition:Int = uiPosition - OFFSET_KOUNT; if(uiPosition < OFFSET_KOUNT) { // <-- Add this block at top
// top row //
var storiesViewHolder: StoriesViewHolder = holder as StoriesViewHolder; //
storiesViewHolder.bind(images); // <-- Add this block at top
} else if(0 <= dataPosition && dataPosition < super.getItemCount()) {
var article: Article? = getItem(dataPosition);
var articleViewHolder:ArticleViewHolder = holder as ArticleViewHolder;
articleViewHolder.bind(article);
}else {
// bind the footer
}
}
And in getItemViewType, add :
override fun getItemViewType(uiPosition: Int): Int {
var dataPosition:Int = uiPosition - OFFSET_KOUNT;
if(uiPosition == 0) {
return STORY_TYPE;
} else if(dataPosition < super.getItemCount()) {
return ARTICLE_TYPE;
}
return FOOTER_TYPE;
}Finally, save and run the code. You should see a title Stories at the top.
Now this is too boring. So I used newsapi to fetch the top headlines. I only took their photos. Then I added a horizontal recyclerView to show them. Now it looks much better. Now it looks like this:
MVVM
Finally we are gonna use view model in our project. All the data so far are in the main activity class. But under certain situations, say the user is frequently rotating the device, the activity recreates itself and so it the same data needs to be loaded again and again. This is bad. So we simply move all our working data inside ViewModel / AndroidViewModel as data inside the viewModel can survive beyond activity lifecycle. For large projects, it is a good idea to keep the data in a Repository class and create an object of that Repositorty inside the VideModel class.
So I simply created NewsRepository, moved all the data inside there. Then create a ViewModel, and create a NewsRepository object. Finally, in the main activity, I am accessing the data through the viewModel. Check out the codes.
Force Refresh on Swipe
Now we want to add a swipe forced refresh.
Note that, I’m not sure how to use newsapi to ask for specific / new items. That's why I am gonna delete the existing data from my app. But I suppose in production, one should properly check and request for just the necessary portions of data. So suppose, I have the news from id = 1234 to id = 2345, or maybe greatest timestamp = todat 10:00 am. Then I should ask the api to give me news with id > 2345 or timestamp 10:00 am, things like that.
That being said, to refesh news, first put your recyclerView inside a SwipeRefreshLayout. Next in your main activity, add this:
swipeRefreshLayout = findViewById(R.id.swipeRefreshLayout);
swipeRefreshLayout.setOnRefreshListener(SwipeRefreshLayout.OnRefreshListener {
// todo: refresh
newsViewModel.newsRepository.forcedRefresh();
swipeRefreshLayout.isRefreshing = false; });
Now go to the NewsRepository.kt file and add:
fun forcedRefresh() {
val handlerThread = HandlerThread("dbHandlerThread");
handlerThread.start();
val looper = handlerThread.looper;
val handler = Handler(looper);
handler.post(Runnable {
database.newsDao().deleteAll();
});
newsBoundaryCallback?.pageNumber = 1L;
var dataSource = datasourceFactory.create();
dataSource.invalidate();
}There are some other changes, such as converting some local variables into private fields, and some private fields into public ones for convenience.
If you have any questions, feel free to ask me and I’ll try my utmost to answer you.
Update: 12/11/2020
Use a Factory Pattern to Properly Support Online-Offline modes
Previously, the data was read from database. When end of data were reached, a network call was triggered to fetch new data. It was a bit awkward. So after studying and experimenting for some more time, I came up with a factory pattern to properly manage the online offline modes.
So first, we’ll be creating a class named NewsDataSource that extends PageKeyedDataSource<Long, Article>. Here, we use Long as our news api uses number type to indicate next page. If your api uses something else (say, String), then you should use PageKeyedDataSource<String, Article> somwthing like that.
Now you have to complete loadInitial, loadBefore, loadAfter. If you google for paging library tutorial, this is the thing that pops up everywhere. Just follow one if you need details. My personal favourite is https://www.raywenderlich.com/6948-paging-library-for-android-with-kotlin-creating-infinite-lists. So I’m gonna skip this part.
Once you’re done with it, we’ll start making the data factory. Create a class named NewsFactory. This is the most important part of the code:
/** Private methods */
// offline data
private fun offlinePagedListBuilder(): LivePagedListBuilder<Int, Article> {
val livePageListBuilder = LivePagedListBuilder<Int, Article>(offlineDataSourceFactory, config);
return livePageListBuilder
}// online data
private fun onlinePagedListBuilder(): LivePagedListBuilder<Long, Article> {
val dataSourceFactory = object : DataSource.Factory<Long, Article>() {
override fun create(): DataSource<Long, Article> {
val newsDataSource = NewsDataSource(context, liveLoaderState);
dataSource = newsDataSource;
return newsDataSource;
}
};
val livePageListBuilder = LivePagedListBuilder<Long, Article>(dataSourceFactory, config);
return livePageListBuilder;
}/** Public apis */
fun getLivePagedArticles(type: Int): LiveData<PagedList<Article>> {
if(type == Const.OFFLINE) {
Log.e("NewsFactory", "NewsFactory->OfflineDataSet");
val livePagedListArticles: LiveData<PagedList<Article>> = offlinePagedListBuilder().build();
return livePagedListArticles;
}else if(type == Const.ONLINE) {
Log.e("NewsFactory", "NewsFactory->OnlineDataSet");
val livePagedListArticles: LiveData<PagedList<Article>> = onlinePagedListBuilder().build();
return livePagedListArticles;
}else{
// todo: if you have other sources, maybe place them here...
Log.e("NewsFactory", "NewsFactory->OtherDataSet");
return MutableLiveData(); // empty live data :/ some weirdo kotlin thing...
}
}
Once done, go to NewsRepository class, and initialize your Live<PagedList<Article> > in this way:
fun initList() {
var lastNewsUpdateTimeMillis:Long = sharedpreferences.getLong(Const.LAST_NEWS_UPDATE_TIME, 0);if( System.currentTimeMillis() - lastNewsUpdateTimeMillis > (Const.ONE_DAY_IN_MILLIS) ) { // once dailythis.liveArticleList = newsFactory.getLivePagedArticles(Const.ONLINE);
var editor: SharedPreferences.Editor = sharedpreferences.edit();
editor.putLong(Const.LAST_NEWS_UPDATE_TIME, System.currentTimeMillis());
editor.apply();
}else{
this.liveArticleList = newsFactory.getLivePagedArticles(Const.OFFLINE);
}
}
Now if you run it, you will see that each day, on the first time, the data will load from network call, and then it will be stored in the database. After that, it will load data from database only. You can see the evidence yourself if you run the code and check the log prints.
Finally, for the force refresh part, I did this:
fun forcedRefresh() {
val handlerThread = HandlerThread("dbHandlerThread");
handlerThread.start();
val looper = handlerThread.looper;
val handler = Handler(looper);
handler.post(Runnable {
database.newsDao().deleteAll();
});
liveArticleList = newsFactory.getLivePagedArticles(Const.ONLINE);
newsFactory.dataSource?.invalidate(); // <------ This is the important line
}This will delete the old news, and restart loading data from network. Although this is not the ideal way. I really don’t understand how to tell the newsapi to give me only the latest news. So I did this.
Thank you for reading.
References
2. https://blog.mindorks.com/implementing-paging-library-in-android
3. https://proandroiddev.com/8-steps-to-implement-paging-library-in-android-d02500f7fffe
