How I build software: Enough design to clear my mind

In this series of posts I’m reflecting on how I typically build software using the project I’m working on as an ecologically-valid example.



Following on from the last post where I set out the vision and motivations for my project, now I’ll talk about why I needed to design the system and how I started to do so.



As was a theme in the last post, the purpose of design is mostly to clear up my mind — to turn feelings about how the system will work, into visual designs that can be reasoned about, resulting in mental clarity and less reliance on my memory — so I have confidence to start building.





I blogged previously about diagrams if you want more information.

Mental clear out

Even though I had a vision and list of features, I wasn’t comfortable to sit down and start coding — too many unknowns. From experience, I know that coding in this state would leave me with scatter brain and lots of erratic hacking. Instead, I wanted a theoretically-plausible design to guide me.



I sketched out a few diagrams and continued to revise them over the course of a week or two. Over the same time period, and often guided by question marks in the design I did research about related topics (such as remote akka).



Let me show you just one example of how the diagrams helped me get a fundamental decision right, and prevented me from spending weeks of my spare time writing unnecessary code.

Instant payback from the diagrams

One big question I was contemplating was how I was going to transmit messages across the network. My gut reaction (ergo emotion) gave me a strong feeling that json over http was the best option by a mile.



After I created the first version of the diagram I could visually see that I was going to be communicating between two akka applications using json over http. At that point I felt like (especially with the misspelled “you’re”):

Like Erlang, akka has it’s own protocols and delivery mechanisms for sending messages between remote instances that are part of the same cluster — this is a much smarter solution to my problem. The updated version of the diagram (below) shows this

.

click image to view full size

I just saved myself a LOT of wasted effort (weeks of spare-time coding). Life-lesson: don’t trust your instincts or emotions.



You can see other significant decisions on this diagrams that give me lots of time to research and re-consider them before I start to code. Although code can always can be changed, it’s cheaper to fix problems the earlier I find them.

A few more diagrams, much more mental clarity

Just to give a full picture of what I considered on this project to be “enough design to clear my mind”, here are the remaining 3 diagrams.

click image to view full size

click image to view full size

click image to view full size



As you can see, there aren’t pages and pages of highly-constrained diagrams — just enough to make me feel confident that I can start to build the system with a general outline in mind.



All of these diagrams were created with guidance from the vision, motivations, and high-level traits to ensure they meet the project’s goals. For me it would be very difficult to do all of this just from working memory as I sit at a computer coding.



I also find that ideas come to me when I’m in the gym, in the kitchen, or in any other situation where I’m thinking about something totally different. That’s good, because a bit of design like this doesn’t force me to think about everything when I’m sat at my desk.

Helps bring people on board

These diagrams make it easier for other people to understand the system. Be that people wanting to use it, or people wanting to help me build it (get in touch if that’s you). I blogged before about using diagrams to on-board team members.



Often I think to myself how much easier it would be for me to use an open source project if there were a few diagrams about how it worked. So I know there is value here.



I can also use these diagrams to collaborate in other ways, too. For example, If I need help building the system I can use these diagrams as the basis for conversation with other developers.

I’m now comfortable to make a plan and then start coding — exciting

These few diagrams have given me a lot of mental clarity, because I’m comfortable that what I’m about to build, theoretically, should work as well as meeting all of the requirements.



Remember, though, that the design is not fixed. Even better, any time I do make a high-level change I have to update the diagrams. This forces me to see how changes fit into the bigger picture. Down at the code level and without any diagrams, I’m at the mercy of my cognitive abilities to perform all this work.



Having a basic design means I can break up the project and finally start to plan how I’m going to start to carry it out; sounds like my next post.