Greenfield Software Projects — A Pessimist’s Outlook
TL;DR: Technical decision making in software projects is a challenging endeavour, especially if it’s being developed for someone else, viz. a customer or a product, and even more so if it’s a greenfield one. Technical aspects to consider for such a project are endless and often anxiety inducing. Below is an opinionated & pessimistic outlook on the topic.
Greenfield projects, or projects starting from scratch, or those that do not have a prior precedent, are unique in their appeal and challenges. Developers are drawn to such projects like moths to fire as there is immense learning potential. But there are caveats too, like any Type-1 decisions, some choices will stick for the life of the project. Being a conservative technologist (not in a political sense) I take a different and not so ambitious route to executing project delivery.
Following are the Top Three (among many) aspects I think are important for such projects and should be ruminated upon.
Language/Framework
Tech Lead: What tech stack seems good for this use case?
Team Member: Reactive Akka Streams with Kafka and Cassandra! And of course a Vue based frontend. That’s the only option!
Tech Lead: Hmm…It’s a simple web based TODO list app. Any thoughts on a server side rendered Rails App? Just throwing an idea around.
Team Member: You will embarrass me in front of my friends.
It’s easy to get sucked into lengthy discussions around language and framework choices for a greenfield. Rails? Spring Boot? Java? Akka? Kotlin? Each has its strength and weakness. Sometimes teams over analyse and bicker to no avail, especially in a Microservices-hype era which promise eternal novelty in choice for tech stacks, one should be extra careful. Even Google, the holiest of tech companies, has a very strict policy around choice of languages in most of their mainstream projects. Java, Python and Go are the choice of languages for most projects.
In consultancy projects, you would be rather safe to pick something that’s widely used and has a good community (on GitHub, StackOverflow, blogs, Tutorials etc.). Client projects are not opportunities to further learning goals; not to say it’s bad to be ambitious, but there’s a thin line between bravado and bravery. Delivery is the king in consultancies and ‘cool, shiny’ stuff should not jeopardise it. A true pessimist would play safe and pick tools that solve a general problem well, something like Java with Spring Boot (I know, dull) or Django with Python.
Engineering Methodology
Project management is often frowned upon by engineers, and I have probably done this more than anyone else. Although, with experience I have learnt (sometimes through bitter encounters) that a well-contemplated execution plan is important.
“Plans are useless but planning is indispensable” - Dwight Eisenhower
It need not be a PMP certified plan/tool, it could be as simple as a spreadsheet based approach. But there has to be a process in place. As an XP evangelist, I recommend using XP in conjunction with either Scrum or Kanban. Most tools support these execution strategies (e.g. Jira, Trello, Kanbanize, Asana, Taiga etc.).
One can also consider small semi-autonomous teams owning their piece of application and driving it end to end. It’s difficult to achieve this in traditional enterprises though. It’s of utmost importance to design a team flow — aka tools, communication channels, bug reporting, rituals etc. It may seem dull but helps in longer term. A pessimist embraces the dull to prevent long term risks. A pessimist sweats in peace to prevent bleeding in war.
Architecture/Infrastructure
This is perhaps the most important of decisions for your greenfield project. There is just so much fluff out there that it’s impossible to be sure of your architecture choices. ‘Cloud’ has become both ubiquitous and ominous at the same time. Your app does not need an Event Driven/CQRS architecture. People tend to throw “But Google and Neflix do it”. Sure, but you are not FAANG, and they didn’t achieve it overnight or even over-year. It’s a general tendency to try and copy the technology results of FAANG, but not their processes.
“Perfect is the enemy of Good”
While Microservice architectures have had their share of success, it’s important to understand the failure cases (which have been many). Starting with a complex architecture never helps. Microservices require a huge upfront investment, and the results are seldom guaranteed. A strong DevOps culture is a pre-requisite, and cultures don’t change because the CTO read Martin Fowler’s blog.
Agile, Kubernetes or DevOps won’t fix a rotten org culture.
As a pessimist, it may help to start simple with integration points. E.g. starting with a non-IAC approach. Your application may not need Kubernetes at its inception. Twitter & LinkedIn started out as Rails apps, Facebook as PHP and Google was a Python spaghetti, but these choices made them iterate quickly and succeed. A pessimist would start conservatively and iterate fast before adding the complexities of Serverless & Kubernetes (etc).
Like all things in life, there is no one-size-fits-all solution to executing projects. Playing safe does not mean cutting corners on software quality or team ambitions. It just means taking the path of least resistance by making safe choices. It may not be alluring, but it works!
