Extending the Language to make better software
Paul Graham talks about how they extended Lisp using macros to make it better. They were able to deliver software at a speed their competitors couldn’t have imagined. And they used Ansi Lisp which is like a language nobody knows. But the closest thing we have to Lisp that people do know is Python. Except for Lisp style Macros and it’s speed python has all the rest and it is almost as fast as interpreted lisp(pretty slow). I have been reading a lot about the advanced features of python. Generators, Iterators, Decorators, Named slices etc and trying to come up with a way to get the things in order. Then I started to wonder since python doesn’t have the macros is it possible to expand python particularly in the context of web development to make it a super powerful development tool. Then I realized that expanding the language is what creating a framework is and that I have been already using a framework called Django for that. It automatically structures things and gives me a lot of helpful function and other constructs that I can just call and get my work done. But now I am thinking since django is a framework how can I make the best use of this framework. Surely it is possible to extend django and make the best use of it. I can take some hints from it’s development version and modules which basically extend django. But django does offer a lot of things to extend itself which is super cool. I am going to divide the django web development into three parts and then discuss how I plan to extend django in the next two days to make it super fast to develop in.
The model layer accesses the database. The primary function of this layer is to just get me the right data without taking too much time. How does django extend this layer ? The most important part of extending this layer is Models. The first thing I am going to do is to put all the business logic in the model. Business logic is like knowing that photos with status 11 are deleted photos and photos with status 5 have to be moderated by the admin. So yeah putting business logic as well as putting operations that have to be done always. Like changing the SEO url every time the name changes. Things like that. The other thing is inheritance in models. Defining my own abstract models will lead to simpler code and more uniformity. So for example I define one abstract model for Review, Messages and comments. And then I can use that everywhere in all three. This can lead to homogeneity and helping people know which problems are already solved. The third thing to do is to optimize queries which can only be done using managers. So defining custom managers that give me the results I want and prefetch stuff that is important. Later I can have a manager which automatically handles cache invalidation and cache loading as well as cache multi gets. I can also have a manager which has functions for carrying out some common tasks rather than having to write the query all the time. I also need to index common fields as well as SEO URL fields because I will making queries in database from them using the index_together command. Also I need to optimize my queries in general using more advanced F() and Q() expressions.
The views layers consists of the url conf as well as the views architecture. The url conf has to be hierarchical and it needs to be as few as possible. I have noticed the url confs blot up very quickly when you are developing new features. So we need to urls that use the same template underneath with one view function and pass a dictionary variable with that for differentiation. The other optimization in views is to implement views as classes and implement general functions as mixins. Implementing views as classes doesn’t really have any inherent value but when you start including mixins you start extending the language having already made features that you can just put in that. That would be a real big thing. But I am not sure where to start here as most of the things I need are already there. Probably a search class that also implement indexing. So I could make indexing a mixin among other things. Only then it offers some real benefits. I can’t see how making some custom middleware will make my life easier except in logging. It might help me in logging everything but I would need to defer that logging to some scala based service or my system will slow down. Also I don’t have that much disk space right now. FlatPages might be useful but only marginally. So I’ll skip that for now. I already have a sites framework in place for sitemap generation.
I want to make a mini CRM out of the admin layer. One use of this is that. The other use is giving non-tech people access to some of the tables that they can change and then manipulate to get right. There are many other things that I found were really missing in the django admin. I have found a good drop-in replacement for admin interface called django-xadmin and it looks very promising. The other thing to do in admin layer is to define management commands for common actions. So if you are expected to do some action more than 5 times through manage.py shell you should probably write a management command for that so that it becomes simple. I think all this will pay off unless I have to migrate away from django which is going to be almost impossible.
Despite all that to make your code really work there’s like a tonne of things that can be used. Two of the things that I really like are content types and signals. Signals allow you to attach event handlers so that you can sync between many systems and content types is just an ORM thing and only helps solve some problems when you want to resolve the type of a foreign key or reference after you have created the table. It can be useful to store a foreign key of two or three models in the same field. Honestly I don’t think it matters if your table has three id fields instead of one because storage is cheap and it also makes things a little difficult to read for the reader but I am sure it would be helpful in some other cases. So I am going to read it and wait for the perfect fit for content types. I also need to study postgres as it is a very feature rich database and it can help solve a lot of problems and help in optimization and reducing code complexity.