Understanding Django Rest Framework Filters

credits goes to flaticon.com

Let us say, we have a requirement for collecting Weather reports of all States and Cities for a particular date, including latitude and longitude of the City. The JSON object of Weather report can be shown as below for storing into the database.

{
   "id": 1,
   "date": "1985-01-01",
   "latitude": 36.1189,
   "longitude": -86.6892,
   "city": "Nashville",
   "state": "Tennessee"
}

We want to create a POST request for storing this JSON object into the database. For now, let us use SQLITE for this. Let’s start by creating a Django project ‘weather_app’ and app ‘rest_api’. I am using Pycharm for this, you can use your favourite code editor.

Click on File -> New Project -> select Django on the left and provide the name as “weather_app” -> click on “create” button.

create a new Django project

Once you have created you get the project structure as shown below.

Now let us create django app ‘rest_api’ from terminal. Open the “terminal” in Pycharm which you can see at the bottom and type the following command.

python manage.py startapp rest_api

start a Django app

Once you have executed the above command you can find the rest_api app in your project folder as.

rest_api application added

We will install djangorestframework for writing the APIs. Open the terminal in Pycham and execute the below command. Make sure you are in the virtual environment.

pip install djangorestframework

Let us add the “rest_api” and “rest_framework” to the “INSTALLED_APPS” section in weather_app/settings.py file.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'rest_framework',
    'rest_api'
]

Now that your Django recognises that you have an application with name rest_api and you are also using djangorestframework.

Let us start creating our Weather model in rest_api/models.py file.

class Weather(models.Model):
    date = models.DateField(blank=False)
    latitude = models.DecimalField(decimal_places=4, max_digits=7, blank=True)
    longitude = models.DecimalField(decimal_places=4, max_digits=7, blank=True)
    city = models.CharField(default='', blank=True, max_length=25)
    state = models.CharField(default='', blank=True, max_length=25)

We are taking the decimal_places as 4 and max_digits as 7 for both the latitude and longitude as the no of digits after a decimal can be 4 and the total no of digits can be not more than 7 for a particular latitude and longitude.

We are also creating a date field that automatically gets created whenever we are storing a new Weather record into the database. date field tells to add the current date in the format YYYY-mm-dd.

We have created the model, let us create a serializer for our model so that it takes for serializing and de-serializing the Weather data when sent over the HTTP GET or POST calls.

create a file serialisers.py in rest_api application.

from datetime import datetime
from rest_framework import serializers
from rest_api.models import Weather

class WeatherSerializer(serializers.ModelSerializer):
    class Meta:
        model = Weather
        fields = '__all__'

Makemigrations and migrate the database. Open the terminal and run the following commands.

python manage.py makemigrations rest_api
python manage.py migrate

DB migrations

Let us create few Weather records using Django Shell. Open Django shell by typing the following command in the terminal from Pycharm.

python manage.py shell

Django shell

Import the WeatherSerializer and create the objects by using dummy data.

First weather object created in db

Let us create few more records by modifying one of the values in data dictionary. It can be either date, latitude, longitude, city, state . I recommend to change date and city.

Creating 4 new records

We are going to use class based views which automatically give the power of re-using an existing view. You can visit the following the link for understanding more on Class Based Views.

Introduction to class-based views | Django documentation | Django

We are going to use generic class based views which give the flexibility of writing extensible views as per our requirements. The generic views provided by REST framework allow you to quickly build API views that map closely to your database models.

I highly recommend to read the following link for better understanding of generic views in rest framework.

Generic views

Create a view in rest_api/views.py for Weather model to either list all the Weather records from db or to create a Weather record in db. These two can be done by extending our class view to generics.ListAPIView from djangorestframework.

from django.shortcuts import render

# Create your views here.
from rest_framework import generics, status
from rest_framework.response import Response

from rest_api.models import Weather
from rest_api.serializers import WeatherSerializer


class WeatherListView(generics.ListAPIView):
    queryset = Weather.objects.all()
    serializer_class = WeatherSerializer

    def post(self, request):
        serializer = WeatherSerializer(data=request.data)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

This one single class view will allow you to list all the Weather records available in the db by GET request and also to create a new record into db using POST request.

Map a URL to the current view in weather_app/urls.py as

from django.contrib import admin
from django.urls import path

from rest_api.views import WeatherListView

urlpatterns = [
    path('admin/', admin.site.urls),
    path('weather/', WeatherListView.as_view())
]

Now, let us run the project and check the POST request.

python manage.py runserver

Open a browser and navigate to the URL: http://127.0.0.1:8000/weather/ This actually makes the GET request call which results in getting the entire list of Weather records present in the DB.

If you scroll down the page, you also have the provision for making a POST request to create a new record.

POST request for Weather model

This is the beauty of ListAPIView, it allows to make GET and POST request in the same VIEW beautifully without writing any new code.

Now that, we have a view to get all the Weather records and also to create a new weather record. Let us start with the requirements now.

Imagine, we have to accept an optional QueryStringParameter date in the format `YYYY-MM-DD` while making a GET call like for example,

/weather/?date=2019-06-11

When this param is present we have to get the records matching with the provided date and filter the remaining records. We can solve this by overriding the get_queryset() in the WeatherView in rest_api/views.py as

class WeatherListView(generics.ListAPIView):
    queryset = Weather.objects.all()
    serializer_class = WeatherSerializer

    def get_queryset(self):
        date = self.request.query_params.get('date', '')
        if date:
            return Weather.objects.filter(date=date)
        return Weather.objects.all()

    def post(self, request):
        serializer = WeatherSerializer(data=request.data)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

Now, when you want to filter the records that doesn’t have the date value as 2002–10–20, we can simply open a browser and navigate the following URL which gives the records having the date with 2002–10–20.

http://127.0.0.1:8000/weather/?date=2002-10-20

This is so easy, but, we have a problem here. What if tomorrow we want to filter records by the field city? We update the get_queryset() method as follows:

def get_queryset(self):
    date = self.request.query_params.get('date', '')
    city = self.request.query_params.get('city', '')
    objs = Weather.objects.all()
    if date:
        objs = Weather.objects.filter(date=date)
    if city:
        objs = objs.filter(city=city)
    return objs

Fine, what if the other day we want to filter the records by field state? We have to again update the get_queryset() method. This is strongly coupled towards the requirements. If the next day we are adding a new field to the model and want to filter to the records by that field we have to update the get_queryset() method. Every time we want to filter by a new field we have to re-write the get_queryset() method. This is not a good programming. For solving this, we have a package called Django-filters. Let us install it into our project and add it to the INSTALLED_APPS in the settings.py file.

Open a terminal from Pycharm and run the following command.

pip install django-filter

Add the application to the INSTALLED_APPS in weather/settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'rest_framework',
    'rest_api',
    'django_filters'
]

Set a default filter backend django_filters.rest_framework.DjangoFilterBackend in the weather/settings.py

REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': ('django_filters.rest_framework.DjangoFilterBackend',),
}

Let us remove the get_queryset() method written in the WeatherListView and so the VIEW will be as

class WeatherListView(generics.ListAPIView):
    queryset = Weather.objects.all()
    serializer_class = WeatherSerializer

    def post(self, request):
        serializer = WeatherSerializer(data=request.data)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

Let us write a FilterSet class that extends the django-filters Create a new python file model_filters.py in the rest_api application and enter the below code.

import django_filters
from rest_api.models import Weather


class WeatherFilter(django_filters.FilterSet):
    class Meta:
        model = Weather
        fields = "__all__"

If you observe, the WeatherFilter class is extending the django_filters.FilterSet class and the model is associated to the Weather model. This makes Django to add the capability of filtering on all the Weather model fields. If you want to apply the filtering on only specific fields we can change the fields value to list of the field names as

class WeatherFilter(django_filters.FilterSet):
    class Meta:
        model = Weather
        fields = ('city', 'state')

This tells to apply filter only on city and state. Refer to this guide for understanding the filters more.

Also, set the filterset_classfield in the WeatherListView for making Django recognize that this VIEW is using a custom filter WeatherFilter.

from rest_framework import generics, status
from rest_framework.response import Response

from rest_api import model_filters
from rest_api.models import Weather
from rest_api.serializers import WeatherSerializer
import django_filters.rest_framework as filters


class WeatherListView(generics.ListAPIView):
    queryset = Weather.objects.all()
    serializer_class = WeatherSerializer
    filterset_class = model_filters.WeatherFilter

    def post(self, request):
        serializer = WeatherSerializer(data=request.data)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

The purpose of including the filterset_classfield is that, now our view can automatically filter the records based on the field provided in the GET URL.

Please note that the spelling of the field provided in the GET URL must match with the field names in the Weather model in the rest_api/models.py

Lets start playing with the Django filters now. Open browser and hit the below URLS and check the output for each.

1. Filter by date

http://127.0.0.1:8000/weather/?date=1985-04-23

records filtered by date field with value 1985–04–23

2. Filter by State

http://127.0.0.1:8000/weather/?state=Texas

records filtered by state field with value Texas

You can also filter by city, latitude and also longitude fields.

We can also filter by combining multiple conditions. For example we want all the records matching state as Texas and city as Plano, we can simply make a GET call as below, provided we must have the respective records in the database:

http://127.0.0.1:8000/weather/?city=Plano&state=Texas

records filtered by both state and city

Lets have an interesting requirement. What if the user wants to filter the records based on multiple values of state ? For example, how can we get the records matching state value to both Texas and California ?

For this we can write a method in the WeatherFilter and assign it to the state field explicitly as

class WeatherFilter(django_filters.FilterSet):
    state = django_filters.CharFilter(method='char_filter')

    def char_filter(self, qs, field_name, value):
        value_lst = value.split(',')
        return qs.filter(state__in=value_lst)

    class Meta:
        model = Weather
        fields = "__all__"

In the char_filter() the value is the actual value sent by the user from GET request. We are expecting the user will send the list of states separated by comma as California,Texas and that’s the reason we are splting the value by comma in the first line of char_filter. qs.filter(state__in=value_lst) will actually search for the records matching the state for every value present in the value_lst.

Open the following the URL in the browser and you get the desired output.

http://127.0.0.1:8000/weather/?state=California,Texas

records filtered by state with multiple values

We can also write a separate method for each field and assign it to the respective field for dynamically filtering based on the values provided.

The entire code for the project can be found at my github link which includes the requirements in the README.md file.

For more understanding on the django restframework filters refer to the following pages.

https://www.django-rest-framework.org/api-guide/filtering/
https://django-filter.readthedocs.io/en/stable/guide/rest_framework.html

For more courses, you can visit my personal website.

Have a great day!