Rewriting the whole project in 2 weeks

With quite a good amount of time on my hands, on April 22, 2012, I released the first stable release of Imagination Framework, which is a reusable software component (RSC) and aspect-oriented programming (AOP) framework for Python, as the result of study and hand-on experience with Spring Framework and Symfony Framework. It started off as a pet project with making it simple. There were some classic design faults I made but it works well in general.

After a four years of honeymoon period, when I started working at Statflo, Imagination Framework has been used extensively and my friends and I pushed the framework to the limit upto the point where one of the design faults happened to cause issue and force the code to be redesigned.

So, while we decided to implement the workaround, I decided to rewrite the framework within two weeks. In the end, I made it in time for the new project begins. The span of two weeks for a relatively unknown small framework seems possible but writing a framework which contains a subset of the functionalities of JavaBeans is no simple task.

I have a few lessons I got from four years of experience.

Good Planning and Impact Assessment

As the framework is used by many active projects of mine, I have to carefully plan what I have to do to fix the problems, technical impact assessment, project coordinations, and the rollout plans.

While it tooks a few days to figure do all of that, it allowed seamless transition without interupting existing projects.

Keep it simple to set up

In the initial design, the framework is designed to be atomic and modular and while many things have to work together, I decided not to write anything to facilitate putting it together for a simple use.

While it maximized the openness of the API, I found that it was such a pain to set up or even work with the framework without the documentation. The code some times unnecessarily complex in order to maximized the openness of the API. For example, I decide to make an XML configuration loader an extension of the framework, instead of treating it the first-class citizen of the framework.

So, when I work for the second version (this rewrite), I decided to rely on the configuration loader to handle everything, including initializing the core of the framework, in contrast of the previous version that developers are required to manually initialize a core component, configuration loader, parameter convertor etc., and put all of them together.

Keep it light and simple

In the previous version (version 1), for no good reasons beside showing my prowess in Python, there were many unnecessarily hacks and wrapper all over the places. It looked cool and I was happy at the time. However, later on, I started questioning, “what the hell have I done?” Because, I hardly understand myself from four years ago. I have attempted to rewrite this framework a few times but without time and necessity, I gave up the idea of doing until recently as my friend discovered a bug that only rewriting can fix.

Now, light and simple code design makes it easy to maintain and work with. It also make the code behavior predictable.

Not rushing to release

In the past few years, I learn to adapt the concept of “if it is not ready, let’s not release it.” approach. While the code rewrite finished within two weeks, I decided to slowly test out with tests and real uses for the next four weeks. While the test writing phase was pretty short, I found out that the real uses provide more feedback as tests are just simulation to confirm the functionality. This turns out to be the smoothest release of all time.

Quick Takeways

In this rewrite, I found a few interesting things.

First, my experience in trials and errors prevent myself from repeating my own mistakes, such as over engineering, and allow myself to quickly focus on and solve the problem.

Second, in code design, the user experience is very important. The code should be designed to encourage people to use, not to be afraid of. It should be friendly but controllably extensible. It should be understandable without thinking too much of it. It should be predictable. Imagine that your girlfriend or boyfriend can use your code without complaining about your code complex.

Next, good planning and technical impact assessment allows seamless transistion. This requires hightly maintainable code to be able to pull off this kind of stunt.

Lastly, rushing is never a good thing. Software engineering and development can be undeniably complex as long as the software is still working. Rushing it without testing will increase the risk of the death-on-arrival. The agile development methodology tends to increase feedback cycle and allow continuous improvement. However, it was not to allow developers to be relaxed and deliver broken software.

In the end, the product of two weeks for full rewriting and extensive tests, I ended up with a faster, lighter, simplier, more maintainable Imagination Framework 2.0 and I learned and practiced tons of stuff in the process.

Imagination 2.0 is now available via PyPI. Please check it out and leave any comment if you have one.

For more reading: