The Effects of Non-Technical Systems on Software Projects

A look at how John Gall’s 1975 Systemantics still applies to software today

Kevin Brown
Jun 14 · 8 min read
Photo by Kelly Sikkema on Unsplash

Systems of all natures affect the outcomes of all engineering projects, but software engineers working with business systems must consider and anticipate the interactions of non-technical systems to a greater degree. Studies of software project failure have not indicated widespread technical deficiencies among practitioners, but instead issues related to understanding requirements, trust in solutions, and how they are rolled out. A software system typically exists within an environment of other systems each affecting the operation of one another in seemingly unpredictable manners. These systems can include other information systems, institutional or political systems, and social systems. Consequently, software professionals often require a level of appreciation for systems that go beyond their technical scope to ensure the success of their projects. The socio-technical nature of software systems has not been lost on the field; however, we can always benefit from greater sensitivity to the nature of systems themselves.

Enter Systemantics, the 1975 book by John Gall which earned Gall a law of his own: Gall’s Law. An entertaining read, it also provides important insight for information professionals. To this end, this article takes three major concepts from Systemantics and applies them to software engineering with the hope that it will enlighten practitioners on the travails of working with systems in a systems-dependent field.

Systems produce unexpected outcomes

However, delivering a technical masterpiece can only account for a single part of what a software professional can call a success. The software must be delivered within the context of a multitude of other systems that are largely non-technical. As Gall states, “Everything is a part of a larger system” (p. 133). For example, a CRM enables customer encounters to be tracked quantitatively; however, this has the potential to shift staff motivations to become overly focused on only those tracked metrics (how can you argue with quantitative results?). In isolation, this could negatively impact employee morale and result in reduced customer satisfaction. Indeed, the “‘success’ or ‘function’ of any system may be failure in the larger or smaller systems to which the system is connected” (p. 132). New software systems or modifications to existing ones need to be discerned with a healthy amount of caution and humility. Regardless the proficiency of those involved, “The mode of failure of a complex system cannot ordinarily be predicted from its structure” (p. 93).

W. Edwards Deming proposed a similar concept: an organization is itself a system with an aim to optimize the system as a whole, not individual components. With a software system being but one component, it is important to continuously relate its outcomes within the context of the whole. Simply fulfilling a specification is not enough, especially when another component of the system is being negatively affected.

Systems run best when designed to run downhill

When a software system is used as a primary force to impose change on neighbouring systems, an opposing force of some degree is almost a certainty. This is not to say that software cannot be a tool in a broader program to effect change, but expecting to make institutional change solely by using one system to affect another is unpredictable. For example, software systems are often used to enforce policies and procedures: A CRM to strongarm a sales process, an expense reporting system to enforce a high level of rigidity, and checklists in Electronic Health Records (EHR) to require doctors to ask a regiment of questions. And yet, each of these is naturally resisted by the social systems in which they interact. Denton writes of doctors needing to meaninglessly check through lists containing questions of “exercise-induced chest pain and feelings of anxiety” for a two-month old. Peter Drucker commented that, “our civilization suffers from a superstitious belief in the magical effect of printed forms.” If the value of the sales process is not already broadly understood independent of the software system, the expense report procedures not taken seriously, and the medical checklists not already agreed upon, then software will fail to enforce any of these. Even a system that happens to work today at tightly controlling a process might become too rigid in the future. As Gall also states, “Loose Systems Last Longer and Function Better” (p. 103).

Inertial forces are troublesome as they lie dormant out of the control of the engineer. As Gall states in the Law of Systems-Inertia, “A system that performs a certain function or operates in a certain way will continue to operate in that way regardless of the need or of changed conditions” (p. 86). For example, staff may be loathe to retrain, key users may be too busy to use the system to keep necessary data up-to-date, or staff may resent having lost control of their spreadsheet to an application from the IT department. Running downhill in this case often means simply being aware of the non-technical workings of an organization, and then designing the technical system to work with these tendencies (and human nature), and not against them. Integrating a new system within an existing one, or providing interim measures to keep control in the hands of the staff are both examples of running downhill with non-technical inertial forces.

Do it without a system if you can

The idea that the jump from a non-software problem to a software solution is unwise follows from Gall’s first bit of advice to those wishing to successfully manage a system: “Do it without a system if you can” (p. 100). The siren song sung by systems is unmistakable: “Systems are seductive. They promise to do a hard job faster, better, and more easily. […] But if you set up a system, you are more likely to find your time and effort now being consumed in the care and feeding of the system itself” (p. 100). It follows that if an adequate solution can be found without all of software’s complexity, then it should be preferred. Sometimes a change in procedures is all that is needed.

Gall’s namesake law states, “A complex system that works is invariably found to have evolved from a system that worked” (p. 80). Consequently, in the tardy employee problem where the solution to evaluate is the procedure itself, it would be much easier to first implement a manual solution. In this way adjustments can be applied much easier; moreover, the changes can be phased in slowly and in an understandable manner therefore minimizing opposing forces to the changes. Then, once the manual system has become too cumbersome, a true software problem has arisen. But if a manual solution does not work in the context of its surrounding systems, then there is little hope that a more complex software system has any chance of working either. Therefore, as a corollary to Gall’s law we can state, “A working software system is invariably found to have evolved from a working non-software system.

As Gall states, “A Complex System Designed From Scratch Never Works And Cannot Be Made To” (p. 80).

Conclusions

The third point (“Do it without a system if you can”) offers a further conclusion that is often understated: make sure that the problem being solved is actually a software problem. The intention of this article is not to set a hard-and-fast rule for determining what is a software problem; however, from the example above, creating a digital employee punch-in/punch-out system because pen-and-paper has become cumbersome is a much stronger case for software than creating the same system because supervisors are disinterested in staff behaviour. If there are steps missing between the problem statement and the recommendation of implementing a software system, then other options may need to be evaluated.

Finally, a common theme among all of this is the importance of having management involved in the development and deployment of software systems. Sharing concepts with W. Edwards Deming’s The New Economics, management’s job is clearly stated as “to direct the efforts of all components toward the aim of the system,” where the aim is ultimately to “achieve the best results for everybody” (p. 50). Software systems are components of the greater business therefore requiring management to ensure that advancement in software components translates to advancement for the business as a whole. This is especially true where it is necessary to make larger scale changes to business systems, or where a business problem must be solved immediately with a software solution. “Growth in size and complexity of a system […] require overall management of efforts or components” (p. 53). As discussed, in a software system these efforts and components are never only technical in nature.

(Originally published at https://dattivo.com/the-effects-of-non-technical-systems-on-software-projects/)

Management Matters

There's plenty out there for the C-suite.

Management Matters

There's plenty out there for the C-suite. What about the rest of us-the high potential managers & up-and-comers. The future C-suite. Real leadership & management advice for front- and middle-management. A publication focused on management matters, because great management matters

Kevin Brown

Written by

I am a Senior Software Engineer specialized in systems design/architecture and data engineering. https://www.linkedin.com/in/kevinpbrown/

Management Matters

There's plenty out there for the C-suite. What about the rest of us-the high potential managers & up-and-comers. The future C-suite. Real leadership & management advice for front- and middle-management. A publication focused on management matters, because great management matters