tenchijinblog
Published in

tenchijinblog

Tenchijin Tech Talk #2 — Python Optimizations

I will go over some Python optimization tips in today’s tech talk for Tenchijin. These tips are primarily for computation cases, and does not reflect a general use of Python. Note: Each function run are in separate files to reduce amount of benchmark errors, but I will put them all in 1 code section for each topic for this article.

There are some packages required to run this demo:

Remember to separate each function into different files. Use for loops over while loops in general, since while loops are slower. They are generally good for waiting for something instead of iterating through something.

Generators can replace some iterators that you will usually encounter. Here is a link to a good example on when and how to use generators: docs.

I have seen some people use the if something; return this; return that format before. More times then not, they are using this format thinking that skipping else statement would make it return earlier than with the else statement. After testing it myself, I can conclude that the margin of error is big enough to say that there is really no difference in performance to do any of these types of returns.

Making a sum function in Python is slower than the built-in sum function, as the built-ins themselves are written in C.

When it comes to arithmetic, numpy will generally be faster than the builtins in Python.

Note that the numpy’s arange function is actually slower at building a list than python’s (3.9.1) built-in range , and this is due to the type of objects that are being made.

However, using the arange function in numpy’s sum function will outperform the built-in’s range sum combo.

Recap

You can take advantage of python’s threading capabilities to run functions in parallel with each other. This can either speed up certain calculations by splitting the load up, and or by splitting up the work of several different functions that are required for the result. I would assume that this is worth it at a big scale, but smaller functions would be better off as simple code.

Python versions will have certain speed differences in some builtin functions, thus creating a difference in most of your code. You can check out the official benchmark website by Python to see the differences of the major process intensive functions there.

Usually memory isn’t too big of an issue nowadays, but sometimes when dealing with something like cache; a small optimization can lead to a huge change.

Enter dataclass with dunder slots, this will be one of the most memory efficient data structure you can use. What __slots__ applies to a class is a list of attributes that can only be in a class, and nothing less or more. My example shows mutable attributes with dunder slots:

You might notice that the dataclass and regular class have the same memory size. The difference between these 2 is that dataclasses have a few perks, these include: less boilerplate, faster attribute access speed, and other parameters as seen in the docs.

I have mostly shown examples by running a single file with a single function, this is because function order in python has an effect on it’s speed. Your own PC can also affect the speed, since everyone’s PC runs different processes. If you are doing some intensive process in the background while running a python program, it will be slower than someone else’s PC where they are running almost nothing in the background.

This might be somewhat subjective, but Python is supposed to be a language of zen. You can even import the zen of Python by using import this . Python can do some crazy one-liners, but sometimes that is not the best choice as it is not readable. I think this language is already sparse enough in it’s syntax by having short worded syntax and less clutter with no brackets and semicolons. It is pretty sparse that being very explicit with your code is fine to do, because it is usually the case that the code you are writing does not need to be optimized by 5 milliseconds (at least it is less common for a web dev). The usual case is that it is more important to worry about who is going to be reading and maintaining your code next, and if they can pick it up easily or not.

Not really readable?
More readable than the above?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store