The Theory of Secure Software

Published in

theFramework

5 min readFeb 24, 2020

Tangram Flex is excited to share a four-part series on secure software methodologies adapted from a paper written by Lead Systems Engineer, Don Barrett. Follow Tangram Flex on Twitter and LinkedIn for updates.

Tangram Flex The Theory of Secure Software

From doorbells to submarines, software drives virtually every capability in the modern world. Almost everyone touches software in some fashion every day. Ask anyone if they want software to be ‘secure’ and you will get incredulous responses that all boil down to “of course!”

But what IS secure software? Not a complicated technical definition, but a simple working logic that the average person can get behind.

Secure software does what it’s supposed to do, and nothing else.

The Happy Path

Seems pretty simple, right? Unfortunately, reality is rarely that forgiving. The average software developer faces tight deadlines, shrinking budgets, and ever-growing requirements. On top of these hurdles, they need to verify that the software they develop is “guaranteed” to do what it’s supposed to do. In real-world application the mentality is often ‘as long as it works’. Testing ensures that the software functions as intended for most cases, with an acceptance that flaws found in edge cases will be patched sometime in the future. There’s little done to proactively find unanticipated behaviors in the code. “Wow, I didn’t know our software could do that!” is a surprising, and far too common, statement when reading bug reports. This mentality is especially challenging with embedded software, where many factors complicate the status quo.

Back to Basics

Every software component is built to provide functionality to someone or something. In simple terms, a software component is a black box that describes a function. Think back to the most basic function from high school math: f(x)=y . This equation describes a function (f) that takes an input (x) and returns an output (y).

Software components can be viewed in the same way. Consider a GPS component. The component is like a function: it receives input, such as timing signals from satellites, and outputs a latitude value. Just like a mathematical function, there are rules about these inputs and outputs. These rules are specified in the design stage of the software component. In the case of our GPS component:

When the GPS component receives correct timing signals from 3 or more satellites, it will output latitude values between -90 and +90
When the input to the GPS component is not correct or if the component calculates a latitude value outside of the specified range, it will output an error code

This set of rules forms an assume-guarantee relationship. Assuming the inputs are correct, the software component guarantees that it will produce the expected output.

Secure… In Theory

Assume-guarantee relationships provide a simplistic way to evaluate software components. First, we check our assumptions and then we evaluate the functional behavior of the software component. Does the output actually match the guaranteed behavior?

Creating assume-guarantee relationships helps engineers understand how the output of one component in a system becomes the input to another component. The assumptions of each software component depend on the guarantees of the components sending it data.

A system model can be created for evaluating the interaction of software components. This allows engineers to predict whether the system’s required functionality will occur. This same model can be used to test the opposite: how will the system behave if some guarantees are not met? What happens if one software component’s output is not the input required by the next components in the system?

In the “happy path”, all guarantees and assumptions line up perfectly in the model. Correct inputs to each software component will always result in correct outputs. The system will function properly and users will never experience any erroneous behaviors.

Happy Accidents

It is important to draw the distinction between erroneous behavior and emergent behavior. Erroneous behavior happens when the assume-guarantee contract is broken; when f(x) = banana instead of y. Emergent behavior is functionality that occurs that was unplanned by the designers. It can happen because of erroneous behavior, but it can also happen when f(x) = y and y is used in a new and unique way. Oddly enough, both can result in positive behavior for the user.

Let’s think back to our GPS component. GPS has become widely used in many applications beyond its original purpose of military navigation. A GPS receiver produces precision coordinates which are used by other software components to generate turn by turn directions. If the GPS receiver instead produced the turn by turn directions, that would seem positive- the user gets the directions without having to pay for the other components. However, there are many questions as well… Can the system as designed handle this behavior, since the components that are expecting to receive GPS position are now receiving something different? Where did the data come from to generate the directions? How accurate are the directions? And so it goes…

The bottom line is: can I trust the emergent behavior?

From Theory to Application

Modern systems are extremely complex, containing many software components. Engineers must focus on two critical elements to build trust:

Eliminating erroneous behavior
Maximizing user’s ability to creatively utilize the software.

The first element is addressed by understanding the assumptions and guarantees of the software, and verifying that they hold true. Requirement 2 is addressed by component-based engineering: building the system around modular components that can be developed, verified and implemented efficiently.

In this blog series we will explore how to achieve these goals. The next post in this series introduces model-based methods and correct by construction software. Later in the week we will talk about component architectures, code implementation, and more. Follow Tangram Flex on Twitter and LinkedIn for the latest.