Meanigful software. Preserving capabilities

Disposable and throw-away. Lean bias

Agile is heavily oriented towards product output. The lean analysis is focused on value flows in the organization, which results in the (incremental) output of usable products with the process and inventory as light-weight and disposable as possible.

Such thinking is easily (if not readily in many organizations) tipped over into seeing any inventory as a burden: any codebase outside of the product-specific repository, the software team outside of a given project, or the hardware infrastructure. Using open-source instead of reusable in-house code, outsourcing software writing, or putting all computing on the cloud are seen as go-to lean restructuring solutions.

Despite all the potential benefits of such an approach, it is prone to the bias of seeing capability as a liability. If something is disposable by design, it does not mean it was designed to be disposed of at the first opportunity. “Disposable” and “throw-away” are synonyms in plain English, but mean virtually opposite in the software development parlance: Say, building a throw-away prototype almost never is a good design, waste of time, and it goes against the grain of incremental development. Whereas one would call a class or utility disposable, if it can be easily moved around, displaced, or replaced, which means it has clear usage semantics decoupled from the product using it, rather than being monolythically welded into it through a bulky overfit interface. “Disposable” is almost a definition of “reusable” and from the experience, the most disposable components happen to be around for the longest time.

Disposable things require extra effort (which usually does not mean “extra time”) to design. However, once implemented, by definition they distance themselves from the final product. They are capabilities. But by virtue of being disposable, they fall the victims to the lazy approach of dropping them at the first opportunity, since they are so easy to drop to demonstrate short-term “savings” to the detriment of the project or whole organization.

Separating features and capabilities

Capabilities are demand- or product-driven. You may not need a good baker in a software team (or maybe you do for the team bonding).

You build or acquire a capability for a purpose. The common problem is that the capability required by a product gets built as an integral part of the product with no intention to use it for something else. Even if reuse is not a goal or possibility, such an approach almost inevitably leads to rigid coupled design: the capability is not separated from its use (implemented and used straight away) and therefore the semantics or the capability is not separated from the product semantics. As an immediate effect of it, testing becomes hard and is done at the feature level, with the capability buried in the feature implementation and therefore not accessible for direct testing.

Separating the feature from the capability, i.e. the capability deployment from the capability itself leads to decoupled, testable, and eventually reusable codebase. Such separation is actually the very first design step for a software artefact at any level from a small class to the large product (see also Architectureless design). For example, at the level of Scrum sprint, this separation is done when during the sprint planning the product owner and Scrum lead together formulate user stories on one hand and matching them “implementation” stories on the other.

(This step is closely related to, but orthogonal and complementary to separation of concerns, since the latter mostly deals with decoupling capabilities from each other. Also, obviously seeing code as two texts is at the core of it. Features themselves form capabilities for features of “higher” order — or rather features are capabilities deployed.)

The step of separating feature and capability is most commonly scrambled or skipped. From experience, people see it as an unnecessary mental overhead or splitting hair, but I almost never have seen that skipping this step would end up in a better design or in saving time.

Placing capability. Knowledge packaging. Third use case

To separate the capability from its deployment, it has to be placed somewhere. Thus, if we are talking about a software artefact, as a minimum, the product code repositories have to contain directories dedicated to capabilities:

• Those directories should be well-structured and well-named. It should not be a single folder called util with a flat bunch of a hundred of files.
• Features should not be mixed with the capabilities that make them possible.
• Capabilities do not need to be overly generic, either. A set of capabilities should form a cohesive mini-language in which the required features could be expressed.
• Nevertheless, the mindset of writing a standalone library serving multiple use cases should be preserved, to avoid overfitting to a specific case, so that it is impossible for the next person to figure out the motivation of design decisions. Although agile is against excessive and premature documentation, it may be really useful to think of writing documentation for the capability artefacts, once they are implemented: if it is hard to explain what they are and how to use them, it usually means that they are either exceedingly specific and overfit or poorly generalised so that there is a thousand knobs to turn to make them working, which by the way demonstrates that both problems have sloppy usage semantics due to the implementation bias as their common cause. One can say that after a product and a test suite, documentation is the third use case for an artefact (the second use case is the test suite).
• Capabilities not only get deployed to deliver features. Capabilities also package knowledge: they formulate the knowledge acquired and by the development team as if those capabilities are going to be reused elsewhere (even if they are limited to the specific project).

It is the software team that is the keeper of the software capabilities: not just the skills, but the artifacts; just like the only expression of the project that matters is the capabilities deployed in the product, the only meaningful thing happening to the team itself is building its capabilities and packaging knowledge. The company may outsource the software effort, but eventually somewhere there will be a team of software professionals doing the hands-on job and it is vital for its professional longevity not only to build personal skills inside of one’s skull, but having semantically structured placeholders for the capability enshrined in semantically crisp artefacts.

The main point is though that even if the team is temporarily assembled for a one-off stint, even if the product is too specific for any reuse, separating features from capabilities, separating building capabilities from their deployment, and deployment of capabilities from their placing is an essential step of software design.

Who owns the software capabilities?

Discerning between developing and deploying capability is a software design exercise done by the software team usually with a limited exposure to the product owner, since for him/her only the final deployment of capabilities as a product matters, while capabilities are implementation details.

The software team owns the software capabilities and packaged knowledge. The software team may consult various external and internal stakeholders to analyse and refine what capabilities need to be developed and packaged to further be deployed, but by any definition, the software team is the product owners of the capabilities.

Books on agile strongly recommend that there should be a single product owner. On the other hand, as strongly they recommend that a Scrum team does not have a single person in charge of the team, but instead self-organises. (See e.g. Craig Larman, Bas Vodde, Practices for Scaling Lean & Agile Development.)

There seems to be a contradiction here: although a single product owner is recommended for a product, the software team owns the capabilities collectively.

In my experience, the collective ownership of capabilities indeed often ends up in inadequate separation of the building and deploying of capabilities and the designs overfit to a specific product. In the projects for which I happened to run software teams, we always have had extremely constrained workforce and therefore clear-cut design and a very high degree of reuse have been life-savers for us. Thus, over the years we developed and retained a massive amount of highly reusable software artefacts. To achieve it, apart from the project-specific needs, I have had to put lots of effort in the capability retention, which at times looked like micro-managing the design process in a centralised way instead of letting the team to self-organise. This contradiction had bothered me for a long while, but now I think I have come to terms with it. I think in the agile literature, there seems to be some inconsistency between stipulating a self-organising team and the capability ownership.

I do not see a compelling reason why the capability ownership should be an exception: while sole product owner is expected for a product, I do not see why the capability ownership can effectively be collective. It seems that there needs to be a sole capability owner as a part of the software team. In a flat team, only the Scrum master seems to be vaguely well positioned to own the capability, however in a truly self-organising team, Scrum master’s role is meant to be rotated from sprint to sprint, while the product ownership certainly has to be long-term. Most importantly, owning software artefacts is not the Scrum master’s business. She/he is there to facilitate specifically Scrum process, not to set design priorities.

There is no use in conflating the roles of Scrum master and capability owner. The capability owner identifies (mostly internal) code users: the team members themselves, members of other teams or projects, etc; opportunities for synergy and reuse; well-structured knowledge packaging (not only in the code, but also in terms of the knowledge capturing, packaging, and sharing); etc. During the Scrum planning, she/he makes sure that the backlog artefacts not only satisfy user stories (deployment aspect), but also make a clear-cut separation between building and deploying capability. She/he does not have to be a team lead, but should have a sufficient authority to insist on a proper separation between building and deploying capability, even if it takes slightly longer in the short term and the product stakeholders are eager to cut corners. It will pay off anyway, quickly and generously.