Python decorators in production

Piyush Chaudhari
Jan 3 · 5 min read

Introduction

In this blog, I will be sharing my experience about extensively using decorators in the production machine learning inference service and how it helped write clean and extensible code. This is a step-by-step story about how I started using decorators in a machine learning inference service, and as more requirements came in how did decorators help me write clean and concise code. I will also try to explain each and every situation with code examples so that the readers can relate to it.

To understand the basics of decorators you can read a nice and detailed blog

We will consider predict API, of the ML inference service to walk through the experience of using decorators. For a given text and language as input, the predict API is responsible for predicting the tag or label with respective probabilities. The predict API does this using an already trained ML model.

The First iteration of predict API

Use Case

The first use case was that the input request JSON received by the predict API should have the required features keys for the API to process the request correctly and if the required features keys are not found then we should send an error message.

Solution

We wanted to add request payload validation to the predict API without actually modifying any code in the predict function itself and the best solution we could think of was writing a decorator and decorate the predict API with it.

We implemented the verify_required_keys decorator. It validates the presence of multiple required parameters in the JSON input. The implemented decorator will be executed before the predict API call and the predict API will be executed only if all the required keys are available in the JSON input.

From the above code example, we can see that the function f is the predict API function passed as an argument to the verify_required_keys decorator. The function f is called only when the required keys are not missing else the decorator terminates the request and returns the appropriate error.

Attaching a decorator to an API supports the “ API should be open for extension but closed for modification” design principle.

The Second Iteration of predict API

Use Case

After customer adoption, we started seeing errors for some of the requests. The investigation revealed that there were requests where the required keys were present but the text was empty and because of this predict API was not able to process the requests. For the predict API to work, the text is a mandatory parameter, and also the value of the text should be non-empty. This being a severe problem we had to quickly fix this because it was directly impacting the customers.

Solution

The solution to solve this problem was easy as we were already aware of decorators and we knew that we can just hook up a decorator with the predict API to check if the text is empty or not and we did exactly the same. We wrote a decorator named validate_empty_text and applied it to the predict API.

Decorator Chaining

A very useful feature of decorators is that it supports the chaining of multiple decorators i.e multiple decorators can be applied to a function. Multiple decorators are applied to a function by stacking the decorators on top of each other as shown in the above code snippet. One very important concept to understand here is the sequence of execution of the decorators.

The decorators are executed from bottom to up. In the above code snippet, the decorator verify_required_keys which is to the bottom will be executed first. And then the validate_empty_text decorator which is on top of it will be executed.

Now after applying the validate_empty_text decorator to the predict API, when the predict API is called it will first call the verify_required_keys decorator, it will verify all required keys are available and then call validate_empty_text, this decorator will check if the text is non-empty and on success will call our predict function.

These two features, the simple syntax of decorator chaining, and a fixed sequence of execution of the decorators make them a very powerful tool in a developer's toolbox!

The Third Iteration of predict API

Use Case

There were complaints from the customer that the results of the prediction are not relevant to the query text. To be able to solve this problem we needed insights into the relevance of the prediction for the query text. To get the answers we decided to log the query text and the prediction results of the Predict API.

Solution

To add this functionality to our predict API we already knew what could be a clean solution and yes it was attaching a decorator to the predict function.

We have attached the log_api_request_text_and_response decorator to the predict API, with this an additional behavior is attached to the predict API. Now when the predict API is called, After executing the bottom two decorators our newly attached decorator will be executed. In this first, our API function f will be called and the decorator will log the request and response payload of the request. This type of decorator where the function call is executed first and then the decorator logic is executed is called Post decorator.

Pre and Post Decorators

Decorators in which the decorator logic is executed first and then the call to the actual function is made these types of decorators are called Pre decorators

In the above code, a Pre decorator is defined in which the decorator logic is executed first, and then the function call fis made.

Decorators in which first the actual function call is made and then the decorator logic is executed based on the response of the function call are called Post decorators.

In the above code, a Post decorator is defined in which the function call f is made first and then the decorator logic is written based on the result of the function call.

Summary

Over the years programmers have experienced that there is always a high chance that modifying code in a function can introduce bugs. To avoid this, decorators help us to add functionality to a function without modifying the actual function code. Decorator chaining is a very useful feature where multiple functionalities can be added to a function. Decorators also give us the flexibility to add functionality before or after the actual function execution. This flexibility enables a programmer to write complex logic with simple constructs.

helpshift-engineering

Engineering blog for Helpshift