Nasdanika Rules
This story provides an overview of Nasdanika Rules — a Java framework for building solutions to inspect data/models for compliance with a set of rules/standards/principles/guidelines.
During my IT career I’ve created quite a few “analyzers” for different artifact types — Java (sources and bytecode), YAML, Tibco Business Works, Chordiant Workflows, GitLab branches. Nasdanika Rules is a generalization of these experiences.
Contents
· Concepts
· A metaphor
· The Enterprise
· Applications
∘ Software development
∘ Document management
Concepts
Let’s take a quick look at how rule management and inspection (e.g. automated code analysis) works using the below sample code and its parse tree/reference graph:
a = 0
b = 1 / a
In this parse tree we have nodes of the following types:
- Method (or statement list) — the root node
- Assignment
- Division
- Numeric literal — 0 and 1
- Variable
- Variable reference
Let’s say we experience problems caused by division by zero. To address these problems we first establish and publish a rule: “Do not divide by zero!”. Communicating the rule to all people who write code might be enough and no further action might be required.
Now let’s assume that despite the fact that our rule was published and communicated to all interested parties we still see occurrences of division by zero leading to outages, customer dissatisfaction, etc. Therefore, we want to establish automated enforcement of the rule. There will be associated costs to build and run the enforcer, so we need to conduct a cost/benefit analysis and make a decision whether to proceed or not.
If we decided to proceed, in order to perform automated analysis and enforcement we need to :
- Load our source from some storage — local file system, version control repository, …
- Parse source to create a parse tree.
- Resolve variable references.
- Traverse the tree and when we encounter a node of “Division” type check whether its second operand is a numeric literal with value 0 or a variable reference pointing to a variable with zero value.
- Communicate findings to interested parties (stakeholders) — developers, management. This communication may be in different forms — developers need concrete occurrences to fix problems and possibly a way to inject fixing work into their workload so they don’t forge (say, create an issue in their backlog). Management need aggregated numbers and a way to track progress in addressing findings.
To generalize, we have a heterogeneous directed, possibly cyclic, graph with a containment sub-tree. We traverse the graph and perform inspection of nodes of specific type (division).
Many, if not all, real-life concepts can be represented as directed graphs with containment sub-trees. E.g. an organization has lines of business, departments, teams, … — tree. Elements of that tree interact, e.g. people from different departments work on projects — graph. As such, the above inspection process can be applied to organizations, source repositories (e.g. inspecting compliance with a branching strategy), software architectures and infrastructures (e.g. cloud deployments), documents.
In our simple example we have 5 types of nodes and one rule for division. In real life applications there might be hundreds of node types, millions of nodes and thousands of inspectors for different node types. With this level of complexity we’d need separation of concerns explained in the next section.
A metaphor
I love metaphors! So, below is one to explain a review approach used by Nasdanika Rules. However silly it might be, here it comes!
Imagine that you are a mayor of a city. Your mission is to ensure well-being of people living in the city. To achieve this mission you want to establish a practice of inspecting city buildings for compliance with building codes — plumbing, electric, A/C, you name it… There are hundreds of buildings in your city with thousands of “inspectable objects” like A/C units.
First you define a set of standards (rules) or adopt pre-existing standards, possibly with some modifications.
To establish an efficient inspection process you hire a group of inspectors each specializing in a certain area, e.g. A/C or plumbing. They don’t know the city and how to get to their “inspectable objects” of interest, but they know how to inspect them. If an issue is discovered, inspectors may either take actions to address the issues, or may prescribe actions to be taken by other parties.
To get the inspectors to their inspectable objects you hire a bus with a driver/dispatcher who knows how to drive through the city, enter the buildings and get inspectors to their objects of interest, e.g. an A/C inspector to an A/C unit.
With this approach the same inspector may inspect objects in different cities and buildings — it doesn’t need to know a particular city or building, just their inspectable objects. The driver/dispatcher may work with multiple inspectors — the driver only needs to know what a particular inspector is interested in.
If you need to make the process faster you may hire multiple buses with drivers/dispatchers (threads, processing nodes).
The Enterprise
In the previous sections we have defined at a high level the “What” and “How” of rule management and automated inspections. Now we’ll take a look at two “Why’s”:
- Why do you need them
- Why you might need a rules/inspection solution specific to your enterprise
To do so we need to define what an enterprise is. I like a definition by Chris Behrens: “A group of people and resources dedicated to achieving a mission”. I’ve modified it by replacing “people” with “stakeholders” and created an Enterprise Model, which is heavily influenced by TOGAF 10.
With the above definitions, you need rules/principles/standards and means to enforce compliance, as well as other resources, to achieve a mission of your enterprise. Your enterprise may be a part of a larger enterprise and contribute to the larger enterprise mission. In this sense rule management and enforcement is an enterprise with a mission to maximize efficiency of a larger enterprise, or in other words minimize “loss” due to consequences of non-compliance such as outages caused by division by zero. This, hopefully, answers why rule management and enforcement is needed.
One closing comment — rules do not enforce themselves. As my defensive driving instructor told me many years ago — “The only thing a stop sign stops is bird shit”. I mention it because in large enterprises some people say things “We’ll create a one-slider with seven bullets explaining our branching strategy and everybody will follow our guidance — there is nothing else we should do.” I’ve never seen it work this way, but maybe those people have different experience.
Now let’s talk about the second “Why” — why your org/enterprise needs org-specific rules and enforcement of those rules. After all, there are commercial and open source tools for, say, automated code reviews.
TOGAF defines a concept of Enterprise Continuum which is used for classifying architecture and solution artifacts/building blocks. The below picture is an extension of TOGAF’s enterprise continuum which includes organization units:
It is also worth mentioning that the real continuum is a directed graph. For example, an enterprise may adopt GitLab or Bitbucket as their Git repository. And a team in that organization would use GitLab but may have their team-specific conventions for, say, branching or tagging or accepting InnerSource contributions. That team may use Java > Spring Boot and use internal libraries provided by some other team in the organization. So, the team’s “context” would be a mix of several enterprise continuums and rules/guidelines to follow.
As you should have figured out by now, commercial and open source tools mentioned above belong to the “Industry” part of the continuum. While valuable, they don’t take into account organization specifics. In large organizations it means multiple segments of the continuum. These segments typically define hundreds if not thousands of customizations in forms of internal standards, policies, best practices. To my experience, is humanly impossible to be aware of all of them and keep up with them, especially taking into account that some of them are tribal knowledge. This is one reason why a formal rule/guidelines management and enforcement framework might be beneficial to an organization — it would provide a versioned source of truth.
Another reason is that it would allow the organization to shape their own enforcement tool instead of being shaped by external tools. This was the reason why I created Hammurapi many years ago — my organization adopted a code review tool which had hundreds of rules, but most of them were not applicable to our situation. At the same time it did not support customization. We needed rules/inspector to find resource leaks such database connections left open. But the tool did not support it. This situation reminded me of the drunkard’s search — we were addressing “problems” which weren’t really problems for us, but the tool had those rules, was expensive, and we had to justify the investment. It also reminded me of a joke: “Why are you taking so many pills, are you sick? No, I’m fine, I’m taking them because they expire soon”. So, beware of the sunk cost fallacy and a procrustean bed of third-party solutions!
Another point of view is inspired by the Kano model and Syndrome from The Incredibles. Industry solutions are “must have” non-differentiators — if you have them your competitors are likely to have them too. However wonderful these tools might be… “when everybody is super, nobody is”. At the same time org-specific rule enforcement solution is a “delighter” taking you ahead of competition — they don’t have it, but they may have something of their own. And in this sense if you decide to use just industry solutions you might be outcompeted, it makes an org-specific rule enforcement solution a “differentiator must have”.
Applications
This section outlines two of many possible applications of rules management and enforcement
Software development
A large organization may define standards/rules at multiple levels. Lower levels of organization would adopt specific higher-level standards, which in turn might be based on industry standards. With Nasdanika Rules all of it is done “as code” — rules are stored in a source control system, packaged as Maven modules. Documentation is generated is from those modules. So, at any point of time there is a single source of truth for a given org. unit. regarding what rules to follow. This approach provides the following benefits:
- Faster onboarding of new team members
- More efficient communication with “client” and partner teams
- Depending on your industry it might be a regulatory requirement to document all rules/processes and be able to provide evidence that they are followed to the regulators.
- And, yet to be tried out — you may implement RAG/Search/Chat on top of a body of rules. This Search/Chat would operate on a specific source of truth as opposed to enterprise search systems which pull data from all source of sources and that data might be outdated or not relevant.
For development teams using Java/Maven rules would be another Java/Maven module, similar to, say, microservices which they deal with every day.
Document management
Recently I was experimenting with Azure AI Document Intelligence using Azure Document Intelligence client library for Java (Javadoc, Maven Central). It has a well-defined “physical” model, which can be mapped to Ecore model, similar to, say, PDF model.
The Ecore model may extend the API model by, say, using Graph Node as a super-type for model elements (text fragments such as paragraphs). This would allow to add connections from, say, paragraphs to named entities found in those paragraphs.
Also, the Ecore model might be polymorphic — having different types of paragraphs depending on their role or position in the document. A type of a paragraph would be decided during loading based on the API paragraph data — content, bounding regions, …
Then, a physical model may be mapped to a logical model. Say, MT 700 Swift Message.
A set of rules may be defined to operate on a physical model, a logical model or both to capture potential issues with documents.
