No/Low code Driven Development (NLDD)

TL;DR — How much code is too much for a Product-at-scale?

https://unsplash.com/photos/LqKhnDzSF-8

Prologue

A Software Product’s Wheel of Progression

Software engineering is about building products that reside on the digital highway (On-premises, private/hybrid/public cloud, etc.) and eternally serve customers. I am not using the word “eternal” in exaggeration. I am using it to justifiably summarize the aspirations of the organization that wants the product out in the market.

Most products emerge out of three key junctures of evolution — Design, Architecture and Development (testing included). They then slowly but surely get into the flywheel of Continuous Integration/Continuous Delivery (CI/CD).

While design and architecture remain fairly stable once they go through the early rounds of iteration (Agile methodologies do mean there is scope for change in design and architecture as well, but let us keep the rate of change at around 10% for those), the story with development though, is a completely different ball game.

Development, with most organizations, is a daily chore. New features, enhancements, bug fixes, patches — the list is just the tip of the iceberg.

Revamps also do pay a visit, probably a year into the launch, and rightly so because technology is fast-paced and multi-faceted. Consequently, it keeps you on your toes and typically every three months, new versions of technology stack come out that look radically different and sometimes makes you feel obsolete using the older versions of the same technology.

This pace brings with it some baggage — as the team develops the product everyday, there is a positive progression for the Product and the features it can offer, while there is a negative progression in the amount of code that gets into the code base.

Source-code’s Wheel of Progression

The source code (aka code) goes through its own journey in phases.

Phase I: Lean Origins

The code typically begins as a clean, lean and readable artifact(probably, but not necessarily a monolith) that follows every rule in the textbook — Functions/methods with readable names and readable variables, detailed comments, SOLID boundaries for the entities (eg. well-defined Class boundaries for a Java Project) and test-coverage along with detailed code reviews.

Phase II: Artifact in growth mode

Over the next few weeks (or sprints), the code evolves into a sizable artifact with multiple contributions from the team.The usual characteristics of the code at this point would be :

• Different styles of coding at different places in the code
• Some methods that may be missing comments.
• Some code with not-so-readable variable names.
• Reasonable but lower test coverage etc.

Phase III: Goliath in-the-making

A quarter in, the code would most likely have become a substantially big code-base, with the team beginning to consider, if not done so already, splitting up the code into microservices. Also, the team having experienced a few bugs down the line, would start looking at ways of improving the testing practices and employing tighter CI/CD checks and compliance.

Phase IV: A Herculean Artifact

After probably an year, we observe a classic case of the team having too little control over the day-to-day flow of code commits. Also, with velocity-to-market always taking priority, the team now looks at automating tests, bringing in new standards and assigning accountability to individuals to maintain the hygiene of parts of the code-base.

Is This Evolution Ideal?

Ideal or otherwise, this pattern of evolution is fairly typical and happens across Products in the software community and the industry. Nevertheless, it brings with it a few caveats:

• We saw that the code took birth as a “poster-boy”, with all the rules being honored. But, for no major fault of the team, the code soon transcended into an artifact that is not quite easy to govern.
• With a diverse team involved, it is not uncommon for various distinct practices to influence the code-base. But that inadvertently amounts to an unfriendly artifact for anyone coming in new, or wanting to contribute (eg. Innersource enthusiasts)
• Automation sure evolves hand-in-hand, but is never found to be adequate enough to avoid P0/P1 bugs.
• With the code-base having gotten considerably big, any revamp or tech debt efforts being mooted are not going to be a cakewalk.

Can this situation be avoided ?

Predominantly yes, with a paradigm called No/Low-code Driven Development (NLDD). The feasibility and possible options of adapting this approach may change from case to case, but it should still be possible for most Product Development teams to embrace NLDD.

Enter: No/Low-code Driven Development (NLDD)

So, what is NLDD? We have all possibly heard of Test Driven Development (TDD) and Domain Driven Development (DDD), two popular paradigms being actively discussed and adapted by the community today.

No/Low-code Driven Development (NLDD).is yet another potential candidate that can be seamlessly adapted (and we will see how).

To the best of my knowledge, this paradigm(NLDD) is being coined for the first time in this post and I am hopeful that it will receive traction and adoption, on the basis of the value proposition it brings to the table.

Circling back finally to the crux of the topic — What are the tenets of NLDD ?

I strongly believe some things are best explained with examples or if possible, with narratives. I would prefer choosing a narrative.

Succinct Inc: A narrative

There exists a(n) (imaginary) product “Succinct” that helps consumers intelligently add reminders and manage them.

The Product Development(PD) team at “Succinct Inc” has decided on the following set of requirements for the product:

• Java/NodeJS will be the preferred core language of coding for the API layer.
• There will be three microservices: An API Gateway, A mid-tier API layer for User Reminder Management and User Identity management.
• MongoDB will be the preferred persistence store.
• The Team would like to have fine-grained logging and would like to have Observability elements added-in, for defining SLOs and other metrics and sent over to an Observability vendor for monitoring.
• The Product would be presented to the consumer as a ReactJS website that can help users manage their reminders. The Team would like to expand to a Mobile app down the line.

There are various ways in which the Succinct PD could have gone about the cycle of evolving the Product.

The Usual Route

The usual way to build Succinct would be to build core-java microservices, with say, JAX-RS as the API layer, build their own data access layer supporting all CRUD operations on MongoDB, write Helpers to register API metrics (like requests per min, latency etc) and then use a API gateway that is spoken to by a ReactJS component with API fetch calls to get data and display the necessary data on the product pages.

The NLDD route

Rather, if the Succinct PD team decides to lean towards a NLDD approach, this is how they would have gone about it.

The Team sits for a brainstorming meeting and comes up with the below development blueprint. Let’s call it the “Next Spring Trip on Cloud-9” (Something akin to a road trip in the Spring season, maybe?) pattern for convenience:

Destination: “Next Spring Trip on Cloud-9”

Frameworks/Libraries

Spring Cloud Gateway : No-code, config-only API gateway

Spring Cloud Sleuth : For Automated Distributed Tracing

Spring Cloud Config : For No-code centralised config management

Spring Data MongoDB : For No-code data access

Spring REST Controller : For Low-code REST API layer

Spring Cloud openFeign : For No-code REST Client abstraction

Spring Actuator with Micrometer: For out-of-the-box metric generation and vendor-agnostic portability of metrics.

NextJS : For low-code ReactJS front-end that connects to the API Gateway

(Optionally) Spring Boot DGS : For Low-code GraphQL API layer abstraction for access by a soon-to-come mobile client/app.

Note: This is by no means an endorsement for the Spring framework. There are various ways of choosing frameworks for NLDD and this is just one of those numerous ways and Spring is just a choice Succinct Inc(which is me, incidentally) made.

Policy checks

Just choosing frameworks doesn't cut the NLDD deal. There is more to it and this is how.

The “Think twice” Oath

Additionally, the team decides to always first look for an existing library in the open source world, evaluate its pros and cons, and to mandate a “think twice” rule before they start writing any line of code.

The “No-code” review

The team also agrees to provisionally have a manual “no-code” peer check during code reviews, so that the reviewers can suggest low/no-code alternatives to code snippets as a best practice and consistently cut down on any code getting into the ecosystem.

CI/CD: Line-diminisher check

The CI/CD pipeline is proposed to have a pre-commit hook to check the new number of lines added and have a “line-diminisher” rule that recommends a limit to the amount of new code added per code-commit.

“Care to Share” Policy

The Team will also decide, on a regular basis, to take stock of code chunks that are eligible to become shared libraries, so that they can be packaged into no-code supplements and re-used across components.

So, what’s the big deal?

That’s a very valid question to ask, given that the above list looks like a normal set of frameworks/libraries to choose from, from the whole gamut of available options.

While there are numerous other ways the Succinct PD could have chosen to develop their product, the PD chooses a carefully curated set of open-source Frameworks and Libraries as a starting point, which:

• Come packed with a full-feature set that solves ~95% of the needs “out-of-the-box”, without having to write a single line of code.
• Immense support through the developer community in the open-source arena.
• Lowest exposure to the nitty-gritty of the “non-business-logic” code. For example, Spring Data MongoDB, which we will see in detail in the next section.
• Obtain efficiency for free, by opting into one or more well-developed libraries that takes the “least resistance” route to solve a use-case.
• Place well thought out policy-checks in the SDLC workflow and scrupulously follow it, to inculcate the NLDD habits and keep the practice sustainable.

Diving Deep

Let me list out some of the very distinct features the Succinct PD gets from making the decision to take the NLDD path of “Next Spring Trip on Cloud-9” .

Spring Cloud Gateway (Learn more):

In addition to the default “route matching” that is typical of a Gateway, this cloud ready, no-code, config-first library provides the following luxuries out-of-the-box:

• Access Control
• Rate limiting
• Traffic Management
• And much more…

Spring Cloud Sleuth (Learn More)

Adds automatic tracing to incoming API requests, while also providing capabilities to add traces and spans without writing code.

Spring Cloud Config (Learn More)

Configuration is a key constituent of a project. Config has all the properties that a component needs, and most times there is a need to maintain common configs across components. This offering makes it a breeze to add centralized config to all components, without any code at all.

Spring Data MongoDB (Learn More)

This one is my favorites. You can write CRUD queries on MongoDB without any code using the “Repository” Pattern.

For example, for an Employee collection with an employeeID field, you can query a specific employeeID by just defining a method called findByEmployeeID(String employeeId) — Pure Magic!

Spring REST Controller (Learn More)

This is the most used Spring offering. Just annotate a method using one of the Rest Controller’s annotations and your API endpoint is ready-to-serve!

Spring Cloud OpenFeign (Learn More)

I can’t stop raving about the simplicity with which this library helps in connecting to other services, external or internal, with zero code. Very elegant and powerful at the same time!

Spring Actuator with Micrometer (Learn More)

Being an SRE-ian, I love the fact that there is a library called Spring Actuator, which, by just adding to your dependency tree, starts to track your API metrics like Latency, Requests per minute, 4XX, 5XX etc out-of-the-box!

Micrometer makes it even more productive by offering a facade to port the above generated metrics into any Observability vendor. Eg, DataDog, SignalFX, New Relic etc.

NextJS (Learn More)

One more of my favorites. This framework is taking the front-end (specifically the ReactJS) world by storm. A framework that gives you UI pages, APIs and data-fetching (SSR, SSG,ISR) with zero-code, A must try if you decide to build your UI with ReactJS

(Optionally) Spring Boot DGS (Learn More)
Netflix has recently open-sourced DGS. It is a long-awaited, much-needed framework in the java world offering to build Java based GraphQL APIs with just annotations.

Is NLDD sustainable at scale?

Like any other paradigm, there is a need for discipline and consistent hunger to keep pacing forward on the NLDD route progress at scale. Some of it is achievable through Policy checks like Think-twice, Line-diminisher etc mentioned above.

What would add nicely to this is if the leadership recognizes the value that it generates, both operationally as well as logically, with the team being able to churn out “more with less” pretty often.

Key Take-aways

The Succinct Inc narrative was an attempt at showcasing what a typical NLDD approach would result in.

Summarizing, here are the key take-aways that are evident from the above example, and holds good for any flavor of NLDD opted for by teams.

• Your code-base starts as a small artifact and remains a relatively small artifact for the foreseeable future, as long as you stick with NLDD tenets for development.
• You gain a lot of head-start from the frameworks you choose, partly because the community is adding consistent value into it, and partly because it is battle-tested by so many consumers of the framework(s).
• You get to use the most efficient solution for a problem, without needing to expend efforts on it yourself. In fact, you have an option of contributing back, if you have something to offer. In effect, it’s a beautiful cascading effect.
• The leaner your code-base, the better it presents itself to external/new/shared contributors who can start pitching-in at the word go..
• Your bugs are lesser in comparison to not following NLDD. This is simply because you are tapping into code that is tested and tuned by many others in the community and also due to the fact that the impact radius is pretty less as a consequence of low code.
• The approach has its flexibility. If you wish to, there is nothing stopping you from either ejecting out of the NLDD cycle at some point in the future, nor there is anything stopping you from creating a non NLDD component in the architecture, which can co-exist with other NLDD components.
• Your team can easily train developers who join from other companies, because the frameworks and libraries you use are most likely used in other companies as well, and because the code is less, there is no bottleneck to a faster ramp-up.

Epilogue

To be completely sincere, this may not be the most comprehensive account of what NLDD brings to the table, for the sake of staying concise and for brevity. But I do believe this is a reasonably good sneak-peek into the paradigm, to get teams to evaluate and start on this paradigm shift. I am hoping it catches up, and that it becomes the norm very soon.

Through the “No Code Syndrome” publication on Medium, I plan to pen down extension posts,when possible, where I list some honorable mentions — Frameworks,libraries and policies in the open-source arena that make NLDD a reality. Trust me, there are many!

PS: I truly appreciate any feedback and suggestions. So please do leave those in the comments if you have any!