Server-side pagination and filtering with Angular 6

Recently I have been porting a fairly large front end code base from AngularJS to Angular 6. This code base manipulates large collections of items, which are served by a REST API built on top of the awesome Django REST framework.

One thing I struggled with was how to handle server-side pagination and filtering in a reliable fashion while adhering to the Don’t Repeat Yourself (DRY) philosophy. After a fair bit of experimenting, I settled on a solution which is generic enough that I’ve used it throughout my code.

The full code for both the backend server and the frontend application are available on github.

Scenario and goals

Let’s say we manage a very large herd of ponies, and we want to build a web application which allows us to view the ponies, search for ponies by name and filter them to only display those who are available for a ride at any given time. Why ponies you ask? Well, they are both cute and a recurring meme in the Django community!

Design constraints:

• Filtering and pagination must be performed on the server side. As we are browsing a large number of objects, simply downloading all objects and performing filtering and pagination on the client side is not an option.
• Filtering and pagination code must be re-usable. We have big plans for our pony browsing app, and plan to expose many collections of objects. We don’t want to rewrite our pagination logic for every new collection.
• Follow pagination links provided by the server. For now we are using a simple limit / offset style pagination, but we plan to switch to cursor-based pagination in the future as our herd grows. To accommodate this, we must make sure that the frontend application never builds URLs for next / previous pages itself.
• Use reactive style programming. Reactive programming allows you to explicitly manage the flow of data in a declarative way, which greatly simplifies the state machine of your application.

The examples below are based on the Django REST framework’s LimitOffsetPagination, but can easily be modified to use other pagination styles.

Generic filtering and pagination helpers

The first step is to write a generic Page type which represents a single page of results. In our demo application, we shall manipulate pages of ponies, but it works with any kind of object.

Next we will define a queryPaginated method to retrieve a page of results from the server. Its two mandatory arguments are the HttpClient which will make the HTTP requests, and a baseUrl from which the collection of objects can be queried. The third argument is optional and can be:

• Absent. This can be used to retrieve the first page of results with no filtering criteria.
• An object containing filtering query parameters. This can be used to request the first page of results with filtering criteria.
• A full URL. This can be used to follow next / previous links provided by the server.

The pony model and service

With this generic code in place, we can now actually describe our Pony model and write a service which allows us to retrieve ponies from the server. The PonyService has a single method called list() which returns an Observable with a single page of results.

As you can see the service is very short since most of the logic is encapsulated in queryPaginated. This makes it trivial to write many such paginated services without code duplication.

The pony list component

Now that we have a means of fetching pages of ponies, it’s time to display them! We will write a PonyListComponent to do just that.

The main features of the interface are:

• A search bar, to perform a textual search on the pony names. We wish to debounce the search input, to avoid sending a storm of requests to the server while the user types.
• A dropdown menu, to filter only available or unavailable ponies.
• A list of results, with available ponies in green and unavailable ponies in grey.
• A paginator, to navigate between the different pages of results.

This a usecase where reactive programming really shines, and as a result the code for the pony list component is fairly short. The component has three member variables:

• filterForm is a form containing the filtering criteria: textual search on the name and availability.
• page is an observable which represents the current page of results retrieved from the REST API.
• pageUrl emits the URL of the next or previous page of results when the user clicks the next or previous buttons. This is triggered by calls to the onPageChanged method.

In the constructor, we wire up page in response to changes in the filtering criteria and next / previous pagination requests. We use the following RxJS operators

• debounceTime to debounce user input.
• startWith to trigger the initial request with the empty search criteria.
• merge to watch both the filtering criteria and request the the next / previous page.
• switchMap to make the call to PonyService.list(), which triggers the actual HTTP request. The beauty of switchMap is that if the filtering criteria change while an HTTP request is outstanding, the request will be cancelled automatically before sending out the new one. No out-of-order responses!
• share so that multiple subscribers can listen for the results. In our usecase we have two subscribers : the list of results, and the paginator widget.

The HTML for the pony list is fairly straightforward:

For the sake of clarity and reusability, the paginator has been split out into a separate component.

Running the code yourself

The full code for both the server and client sides is available on github, with instructions on how to use them:

• pony-pagination-server is the Django backend server which provides a paginated REST API.
• pony-pagination-app is the Angular application which provides the user interface.

If you found this article useful, or if you have suggestions on how to improve it, do please leave a comment!