Leveraging programming paradigms to reduce coding bugs for modern enterprise applications
In my role as a trusted advisor, I have been asked on many occasions by CTOs on how to reduce coding bugs in the development process.
The Cloud has given developers a boon in terms of infinite resources to tinker and provisioning of test environments that could be leveraged with automated testing tools as part of the CI/CD process. However, the test regime (unit-tests, integration-tests, etc) isn’t fool-proof as we still see the effects of software bugs that are constantly affecting service reliability. This article looks at various programming paradigms that can be utilised by the programmers to help reduce their coding bugs when developing/refactoring software.
Enterprise software tends to have a long life-cycle in the organisation, and usually, it is supported and maintained by groups of developers through change-requests; the developer adding new features to existing code or modifying the behaviour isn’t always the same developer that created the code in the first place. Developers usually develop using an imperative style to programming, which is what they were taught to code in.
To eliminate (or at least reduce) programming bugs, we need to understand how they were created in the first place. Issues stemming from the misunderstanding of the client’s requirements are being resolved with the Agile Development Manifesto; with shorter turnarounds to have the client’s input into the development output, we are assuring that the coding is towards what the client is envisioning for their business needs. What we need to understand is how coding bugs get created by developers, and what tools or changes that we can do to facilitate the prevention of such coding bugs in the first place.
One of the more common coding bugs that are encountered very often, stems from the situation when a programmer designs and implements his code to allow changes to programming states that are external to his code. In programming parlance, states are defined as variables or memory locations, used to indicate the current ‘condition’ or ‘configuration’ the application is in. Usually, these are globally accessible/shared objects or variables used to denote the state of the application. A program that changes states external to the code premise is said to have side-effects. As requirements become more reactive, programmers start using shared mutable states in the codes which leads to undesirable outcomes especially when programming in a group environment.
To mitigate such issues, Functional Programming (FP) paradigm is used to enforced immutability into the program design. The FP paradigm opposes Objected Oriented (OO) paradigm by making functions as the main focus instead of objects. FP tends to be declarative (as opposed to imperative) and helps remove unwanted side-effects and shared mutable states. By using FP, the coder is forced to think in a higher abstraction in solving the problem domain as deterministic functions instead of imperatively coding the algorithm with procedures and objects. This higher abstraction design tends to lead to cleaner codes and making changes easier for other developers. FP also makes it easier troubleshooting, to do unit-testing and integration testing.
Adopting the mindset of FP is more than just about eliminating mutability, but also the approach to solving problems. In the imperative style, the developer designs the steps required to achieve the desired outcome. In FP, the developer needs to decompose the problem into sets of functions, with the focus on what data is required and what transformation is to be done to the data through the functions. Instead of running through the bounded for/while loop in the imperative style, FP uses functions and recursions in the transformation of the data. The focus is not on the imperative algorithmic sequence but rather on the data transformation required and the functions that perform the transformation. FP emphasizes functions as the first-class elements for programming, and not classes or structures.
Other benefits for FP includes easier Unit-Testing and Debugging. As FP favours functions that return values based only on its arguments, the unit tests for the functions are relatively simple. There is no need to configure the right ‘state’ for the unit tests, as everything that affects the return values is in the arguments; you just need the edge-cases for the arguments. Debugging is also easier since you don’t have to worry about code-paths before reaching the buggy function. If the function’s return value is buggy, it will always be buggy, regardless of any external state. I have spent countless hours debugging imperative codes that were producing wrong results occasionally, only to find out that it depended on the code-path before reaching that section of code I was reviewing. With FP, the buggy function is always buggy and reproducible, thus making resolutions easier.
Also, since FP is stateless, it tends to favour concurrent programming easily. With today’s modern CPUs, we are given the luxury of multiple cores to leverage on. FP’s statelessness is easily deployed for multi-core computations or other thread-safe situations.
There are two common ways to adopt the Functional Programming paradigm.
An easier adoption methodology is by using the group’s current programming language of choice (e.g. Java, Python, JavaScript, etc) but changing the style of programming towards the FP paradigm. This involves strict discipline within the group and requires constant auditing through reviews to ensure that the imperative style is being deprecated. As the developers are familiar with the imperative programming language’s syntax and grammar, the learning-curve in this approach is the easiest and allows a phase-in journey. However, as the programming language itself allows imperative style, it behooves that the team’s QA process keeps constant vigilance to ensure the bad-habits are kept in check.
A more drastic approach is by adopting a Functional Programming Language itself (e.g. Clojure, Haskell, F#, etc) to replace the existing. These languages are by design Functional in essence and they make it difficult for the developer to go into an imperative mode of programming. The enforcement by the programming language itself provides a certain assurance and safety-net that precludes imperative programming. But the change is prohibitive with a steep learning curve especially if the development team is not well versed in the realms of declarative programming with functions as first-class citizens. It will take time before the team becomes productive with such languages and approach, but the investment is usually worth it with a better and more manageable codebase.
Other programming paradigms to note include Data-Driven Programming (DDP) and Flow-Based Programming (FBP) approaches. Enterprise applications (e.g., CRMs, ERPs, Accounting, Workforce Management, etc) all deal with data and processes that transforms them. In DDP, the data (and meta-data) set provides the business-logic and control-flow to process and transform the data through a generic ‘eval’-like code functionality. So if a programmer wishes to change the logic, the patching is done to the data itself, and not to the codebase. With no code-change, no incurrence of bugs into the codebase. So instead of modelling the application domain as a set of objects in OO style and algorithms in the code, DDP embraces the abstracting and modelling of the domain functionality through the design of datasets (including the embedded control and business logic). DDP, in essence, is like a homoiconic approach (data is code, code is data) to data-encapsulation and logic-design into the data-stream. An example of DDP is in Ansible or Kubernetes’ YAML file. The YAML file contains the declarative data for which the controlling code would use to change the domain’s functionality and behaviour.
Flow-Based Programming (FBP) paradigm, on the other hand, focuses on the transformation of data flowing through ‘black-boxes’ of functions. In this regard, it is very much similar to the FP paradigm but focuses on the flow-order of which the data is manipulated and transformed. The essence of the FBP paradigm precludes shared mutable states and thus helps in reducing bugs. At the same time, FBP promotes viewing the system from a data-flow/transformation holistic lens and helps in eliminating flow design issues. FBP changes the coder’s PoV when it comes designing complex data-applications. An open-source version (developed by IBM Emerging Tech) can be found here.
Summary:
In this document, I have listed several programming paradigms to help remove ‘state’ from the programming endeavour and through this approach, remove the commonly found issues found in misbehaving applications. Being ‘stateless’ is one of the factors in 12-Factor-Apps for developing modern scalable and distributed applications for the Cloud era, and by adopting a discipline through the paradigms, help to reduce such errors.
