Implementing Log Decorators in Python

Logging is one of the crucial steps in software development. While working on any project, we need to record the details in order to debug and analyse our code, this is where log decorators comes into play. It will not log the details, but also save us a lot of time, help to minimize lines of code and most importantly will help us to maintain cleaner and modular code.

I researched a lot about implementing log decorators and struggled to resolve a lot of use cases which came while implementing it in one of my project and hence came up with a related appropriate approach.

Defining the get_logger() function :

We create our first file, log.py in which we define a get_logger() function. Here, we will create and return a logger object using theLogging module in python.

In get_logger(), we will pass two parameters - log_file_name and log_sub_dir, which we want to set. Then we will create a log path if it does not exist. Create a logger object using theLogger() function of theLogging module.

get_logger()

After creating a logger object, we need to add a formatter and level in order to specify the format for our log text file.

Add Formatter in get_logger():

add formatting in get-logger()

We will create a FileHandler and specify our own log format using CustomFormatter().

So, Why are we using the customFormatter class?

The logging decorator, logs the file and function name where it's defined not where it's used.

For example, if we are using a decorator to log the file main.py and defined the log decorator in log_decorator.py. Then, in our log file even though we are logging main.py, it will take function name and file name of log_decorator.py from where we are calling.

To get the actual file name and function name we have defined our own custom class named as customFormatter by overiding their values.

custom_formatter()

Now, let us set the logging level.

Setting level:

There are various levels in python in which one can log details. When a logger is created, the level defaults to NOTSET.

LEVEL      NUMERIC VALUE
CRITICAL : 50
ERROR    : 40
WARNING  : 30
INFO     : 20
DEBUG    : 10
NOTSET   : 0

Only messages of or above the selected severity level will be emitted by whichever handler or handlers service the logger.

Let us set our level to DEBUG.

add level in get-logger()

Now, let us define our decorator in log_decorator.py

Inside the decorator, we will define a wrapper class in which we will call our function which we want to log.

log-decorator()

We will create logger_obj by calling get_logger(). After that, we can simply log the function details using logger_obj. Here, we are logging start of execution the function, it’s return value and an exception if it occurs.

Logging function arguments

Let us also log arguments which are passed to function. For this, we will first format the arguments in a representational way using string functions. Then we will add it to a logger.

Format-Arguments

At this point, the log will record the function and file name of defining class (log-decorator) rather than used class.

Let’s understand with help of an example: I have created a demo file calculator.py which will log the details of add(). I have added a decorator on top of add() in order to record the details.

Note that, here we have also added custom logs specific to a function by creating a logger object for a class.

calculator.py

The output log file will be :

log file output — add()

Using extra argument in logger :

Now in order to get the correct function and file name, we will use the‘extra’,in-built argument of the logger object. Using this parameter, we can get the used function and file names.

using extra argument

We are getting file name using getframeinfo() routine of module inspect and function name using __func__.name and store it in extra_args variable. In logger_obj we will pass extra_args as an extra arg, which will override the names using customFormatter class.

Now, the output log of add() will be :

log file output — add()

Let us see an example where an exception will be logged :

calculator.py

The output log should record exception divide by zero.

log file output — divide()

In this way, we can log any function which we want to just by adding a decorator on top of it.

Here is the complete implementation.

References :

Please refer: Python logging cookbook for more details.