Django: Performance optimization and Pitfalls
Hi I’m Patrick, a programmer in AnyMind Group.This article was written assuming you are already familiar with programming and have some experience.
Every programming language have their own standards. Zen coding, principles, unwritten rules they call it — communities create these to standardize coding, making it more readable and making it possible to maintain the codebase for years, heck a decade! But optimizing and planning have their own pitfalls specially if executed prematurely.
The main objective upon development is to write a code that works, achieve the criteria and to function as it should be. Then coding standards, efficiency and performance should follow.
Let’s start!
Clean your code
MAKE IT A HABIT. Clean code makes the system maintainable specially for your colleagues. Some tips on how to keep your code clean:
- Adding document string for functions, complex logic, edge cases and even adding notes and references is good practice and can help developer readability.
- A method should only be called once per process. If you need to use its result in multiple scenarios, store it on a variable and use this variable instead.
- Checking for unused code and deprecating unused functions, variables, classes.
- Avoid redundancy. Don’t rewrite code unnecessarily and don’t redefine variables. Keep the code on point.
- Reading good articles and books should help improve and exercise good coding habits. i.e. Zen of Python, Two Scoops of Django, etc.
Querying and Database Optimization
Most of the time, queries are the bottlenecks in terms of speed and performance. Maximizing the methods provided making your queries short and efficient is a good practice.
- Retrieving data from the database consumes a lot of time, specially if dealing with a thousands if not millions of rows of data.So that’s why the lesser the number of queries, the faster the performance and the faster the response specially if dealing with APIs.
- Use select_related and prefetch_related on queries with relational fields will improve the performance.
- Use Q object for complex queries. Practice annotate and aggregate method for efficient data computation.
- Practicing defer( ) and only( ) to only retrieve needed fields.
- Executing the computation in queries is faster than executing it in python.
- Make sure to pick the correct database for your needs.
Being Lazy is good
- Laziness avoids pre-computation and only is executed when needed or required. For instance, QuerySets are lazy. QuerySets will not fetch data when you create, pass, or combine them.
- Certain operations force the QuerySets to prematurely fetch the data. Avoiding this scenario help the system and data computation become faster.
- Practice using keep_lazy() decorator. It forces Django to only execute the function when necessary.
Celery
- Use the vegetable! For async process, batch processes or functions that take too much time to execute, we can use celery. Specially on APIs that requires heavy computation and results to timeout responses.
- celery-beat is a nice library for periodic tasks. It is being used for computations that requires to run at a scheduled point in time.
Caching data
- You should cache data that is not expected to change or update (specially derived ones) and using cached_property() decorator.
- Maximizing Django’s caching framework and implementing it should improve the performance drastically.
Premature Optimization
“Premature Optimization is the root of all evil.”
I agree to the sentiment. Meticulous adjustments, micro tuning and overthinking leading to improper designs. The result is a code that is difficult to modify, maintain, and read. This is one of the common pitfall in programming and data science.
- Note: optimizing could be addicting, to the point that we are bypassing the designs and structures of the code.
For many programmers…the more they know, the more ways they will be tempted to prematurely optimize.
Yes! with great knowledge comes with dangerous premature optimization. Ohh wait..
Then how to avoid premature optimization?
import this
Tadah! yup don’t forget. The Zen of Python. Simpler is better than complex. Yes, yes this principles. Keep in mind.Easiest , maintainable, readable, easy to test.
Note: For your codebase, if you plan on optimizing functions specially complex ones — don’t forget to write your unit test first before refactoring!
Conclusion
Optimization if done wrong could lead into disastrous outcomes and can produce more harm than good. Following a coding standard, making it a habit and code reviews, then avoiding premature optimization should improve system efficiency and performance.
