Code, Format, Repeat #1
Simple Python Profiling with the @profile Decorator
One weird trick for pain-free performance profiling
#Uncountable is hiring! To help us accelerate the future of materials development, check out our Careers page.
Away of analyzing the performance of any given function or program is to use profilers during its execution. Profilers can help us understand timing, memory usage, and other pertinent information about code. The key to using profilers is to determine which portion of the code is slow or computationally expensive and assist the process of catching errors for optimization.
One particular library we use often at Uncountable is the Python line profiler.
This library is helpful when digging into a specific function to find the exact lines that take the most time to run. This is especially useful in diagnosing large functions and finding the hotspots that cause performance issues, which may not be apparent to the naked eye. For example, a single line that uses a function from an external library can trigger a lot of hidden computation in the background. This function call can take a lot of time to run, although it may appear to be simple to use. Without the profiler, we wouldn’t know what was slowing down the code during our diagnosis of poor performance.
The line_profiler can be installed simply using pip:
$ python -m pip install line_profiler
We can naively use the line profiler, but the library limits its uses to the command line, which makes using it in flask more inconvenient. Instead, we can create the following decorator:
In addition, we can also create a helper function to print out the profile statistics:
To use the line profiler, we will import our decorator and apply it to the function we wish to diagnose.
After running our function, the console should output the profiler results to our program’s standard output stream in the following format:
The code is printed along with information generated by the profiler, which includes:
- Line number
- Hits, the number of times the line is run
- Time: the total amount of time this line executed across all hits
- Per Hit: the average amount of time
- % Time: The percentage of time spent on that line relative to the total amount of recorded time spent in the function.
In our next blog post on profiling, we will walk through some common optimization examples. [u]
#Uncountable is hiring! To help us accelerate the future of materials development, check out our Careers page.
Ryan Kuang was previously at the University of Chicago, where he received degrees in Economics and Computer Science. At Uncountable, he works as a fullstack engineer on platform development. In his free time, he is a massive foodie, enjoys hikes, and dabbles in crypto.
