Rebuilding 200,000 lines of code from scratch: A review of a major software project

It’s been two months since the stable release of SupportPal v2, the major update to the on-premise, self-hosted support help desk software that was formerly known as ArcticDesk, and we thought it would be interesting to review aspects of the development process from the start to where we are today.

The development process was long, by far the largest project that we’ve ever worked on — a process which began in early 2015 and took almost 15 months to complete.

The project started with a difficult decision: should we stick with the existing code base and improve on that, or rebuild the product from scratch? Sticking with the existing framework would have reduced the gap in the release cycle but could have slowed development going forward. Starting again meant being able to take advantage of new technologies but would take significantly longer.

In the end, we decided to start again.

Changing the tools we used

When the decision to rebuild the software from scratch was taken we also reviewed all the tools we used, leading to a complete change across the board.

We initially started to develop v2 on the Zend framework, but found it a little too rigid for our liking. Searching for alternatives, we came across Laravel, a framework that was quickly growing in popularity. Within hours of trying it, we decided it was what we wanted to go with.

Previously we were using SVN for version control, but in recent years Git has become more popular. Compared to SVN, Git allows us to commit code offline and offers easier branching/merging. Gitlab was an excellent choice for self-hosted repository management.

Atlassian produces great tools for software companies, and JIRA Agile was a big improvement in issue tracking for us. We also began to use HipChat for communication instead of Skype, and Confluence for documentation compared to a heavily skinned Mediawiki.

A more recent tool added to our arsenal was Rollbar, which allows us to optionally receive diagnostic data from production installations and resolve issues faster. Previously a customer would have to contact us if they came across a problem.

Commit numbers and patterns

The original v1 code base contained close to 125,000 lines of code, and the final v2 code base at release was over 200,000 lines — larger due to the additional features added. No code was copied over from v1, it was all written from scratch over the course of over 3000 commits. Below are some statistics and graphs based on these numbers.

The first few months after starting the development process, the number of commits were relatively low as we would usually commit larger blocks of code together. The commits started to become smaller the more we reviewed what was changing, signalling a change in mindset. Our busiest month was January 2016, ahead of the first beta which was released in February. The average number of commits throughout the whole period was 8.5 commits a day, including weekends.

We don’t have a fixed working schedule and do work on weekends, but as you can see above, most of our coding is done during the week. Our numbers are fairly consistent until Friday, when they start to drop off, possibly as thoughts of the weekend start to creep into the mind.

We’re not early birds at SupportPal, with only just over 10% of our commits coming before noon. We tend to be most productive in the late afternoon and early evening period.

Strategies going forward

The transition from working on a new and unused piece of software to improving existing, used software was also interesting.

Previously, we mostly cared about implementing as many customer suggestions as quickly as possible. Later, it became more about maintaining that speed of development while ensuring our changes didn’t cause any regressions.

Since the stable release, we have begun to use merge requests more prominently when implementing both bug fixes and improvements. Merge requests have to be approved by another developer before the changes are merged in to the working code base. This allows us to review changes in greater depth, ensuring existing unit tests pass and that there are adequate new tests.

Final thoughts

Though it has only been just over two months since we released v2, it is clear for us that the rewrite has been a very clear success. What was at times a tough path, is now worth it, seeing the final product being used in the real world.

The reception from our existing customers has generally been positive, over 60% of our user base have switched to the new version so far. While a few are reluctant to switch right now, as is expected with any major update, it is clear the new version is significantly better and a worthwhile update. Starting again gave us the opportunity to carefully redesign and re-engineer each part of the software.

Since the original stable release, we’ve already released two maintenance releases with many improvements and new features. Our new toolset means that we can develop new features faster, while preventing regressions, and we’re excited to see what we can achieve in the coming months.

For those who are thinking about starting a rewrite themselves, it’s a really tough decision and not one to take lightly. What turned out to be a success for us at SupportPal, might well become a failure for another company. It requires dedication and persistence throughout the whole period, and even then it’s not guaranteed to work out.

Show your support

Clapping shows how much you appreciated Jay Shah’s story.