Code, Format, Repeat #1

Simple Python Profiling with the @profile Decorator

One weird trick for pain-free performance profiling

Ryan Kuang
Apr 16 · 3 min read
A line profiler hard at work // Illustration by Cathy Yang

#Uncountable is hiring! To help us accelerate the future of materials development, check out our Careers page.

Oway of analyzing the performance of any given function or program is to use profilers during its execution. Profilers can help us understand timing, memory usage, and other pertinent information about code. The key to using profilers is to determine which portion of the code is slow or computationally expensive and assist the process of catching errors for optimization.

The never-ending profiling process (Illustration by Josh Wagner)

One particular library we use often at Uncountable is the Python line profiler.

This library is helpful when digging into a specific function to find the exact lines that take the most time to run. This is especially useful in diagnosing large functions and finding the hotspots that cause performance issues, which may not be apparent to the naked eye. For example, a single line that uses a function from an external library can trigger a lot of hidden computation in the background. This function call can take a lot of time to run, although it may appear to be simple to use. Without the profiler, we wouldn’t know what was slowing down the code during our diagnosis of poor performance.

The line_profiler can be installed simply using pip:

$ python -m pip install line_profiler

We can naively use the line profiler, but the library limits its uses to the command line, which makes using it in flask more inconvenient. Instead, we can create the following decorator:

In addition, we can also create a helper function to print out the profile statistics:

To use the line profiler, we will import our decorator and apply it to the function we wish to diagnose.

After running our function, the console should output the profiler results to our program’s standard output stream in the following format:

The code is printed along with information generated by the profiler, which includes:

  • Line number
  • Hits, the number of times the line is run
  • Time: the total amount of time this line executed across all hits
  • Per Hit: the average amount of time
  • % Time: The percentage of time spent on that line relative to the total amount of recorded time spent in the function.

In our next blog post on profiling, we will walk through some common optimization examples. [u]

#Uncountable is hiring! To help us accelerate the future of materials development, check out our Careers page.

Ryan Kuang was previously at the University of Chicago, where he received degrees in Economics and Computer Science. At Uncountable, he works as a fullstack engineer on platform development. In his free time, he is a massive foodie, enjoys hikes, and dabbles in crypto.

Uncountable Engineering

Uncountable Engineering Blog

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store