Part 2- A Beginners Guide to Time profiling in Python.

Hello folks, welcome back. If you are joining back from my last blog then pretty much context has been set about time profiling, If you have not yet gone through the last blog, I highly recommend you to do so, though it is not mandatory.

Using %lprun for time profiling.

In the previous part of our blog, we learned ipython magic methods i.e.%timeit and %%timeit. We have used them to time profile code in python. In this part of the series, we will be learning more about line_profiler and use it for better time profiling.

%timeit works well when we need to analyze the total time taken by the block of code. However, it becomes repetitive when we use it for each line of code. As a good programmer, we will look for a more smart approach.

For above-mentioned problem, we can use an external python module named line_profiler. It is used to time profile a code line-by-line.

To install line_profiler using pip, we can run pip install line_profiler in the terminal. It works best when your block of code is defined in a file, not in a notebook or some interactive namespace.

We will be using ipython shell for all our demonstrations.
For using line_profiler package

• Run %load_ext line_profiler in the ipython shell.
• Then run %lprun -f <name_of_function> <name_of_function>()

Let’s see an example of line_profiler, then we will understand how it works.

For the demonstration, we have created a python file named line_profiler_demo.py in the home directory and added the following block of code inside my file.

def multiply_by_two(nums):
    new_num = list()
    for num in nums:
        new_num.append(int(num) * 2)
    return new_num

Let’s open ipython shell in the home directory for time profiling.

It is pretty evident that we need to create a function in a python file then import it in ipython shell for time profiling that function with line_profiler.

We have generated a list of 1000 random numbers using random python package and saved this list to a local variable named random_numbers. We have then passed this list to our custom defined function multiply_by_two(random_numbers).

For profiling, we have written %load_ext line_profiler in the ipython shell as we are using an external python library i.e. line_profiler.

We have then called
%lprun -f multiply_by_two multiply_by_two(random_numbers).

Let’s try to deeply understand the output of %lprun now.

• At the beginning of the output, Timer unit gives us the unit of measuring the execution time and we can see that it is being reported in microseconds.
• Total time gives us the overall time taken in the execution of the whole block of code.
• Line column shows the line number of the code.
• Hits column shows the number of times that a particular line has been hit while executing the code.
• Time column shows the time taken by a particular line of code.
• % Time column shows the relative time percentage of each line of code.
• Line Content shows the content of each line of code.

We can easily analyze that in our function, line number 4 i.e. new_num.append(int(num)*2 is taking the maximum percentage of time i.e. 61.3%.

Function used in this demonstration is a very simple python function. However, while writing complex functions containing multiple lines of code, line_profiler helps us a lot in identifying the bottlenecks of the function with respect to execution time. One can identify that particular line and optimize that line of code for better performance.

Kudos!!! We learned another new thing about time profiling. Let’s meet in the next part of this series and learn some more ways of profiling in python. Till then goodbye and keep learning.