Having said that, we thought about this challenge for a while and then decided that we would try to include security aspects in, in this case, an agile software development method.
A security-enhanced agile software development method (SEAP)
The model that we developed, “a security-enhanced agile software development method”, or SEAP, was applied for testing in the development of a cyber-physical system (i.e. an IoT system) for mobile money transfer that was developed at Ericsson. When Ericsson does software development, they usually use agile software development methods. And when they do that, they typically handle security and risk analysis through an external security manager, and only once every new system or every major system release. So this is something that is typically handled from the outside and only once every new system release.
In SEAP, we had the idea that we wanted to break down the security management role that normally is put as an external perspective at each new major system release. We had the idea to add that from the inside and break it down to four different competences. We kept the security management, or security manager. This is the person that binds everything together, who often leads the risk analysis work and makes sure that there is documentation, and who has a holistic view on the system that is being developed. Then we added the perspective from a security architect who reviewed the system security from an architectural point of view. We added security expertise, which was allowed to be really in-depth knowledgeable and nerdy, so to speak, about security questions in the development. And since this was a system that was intended for mobile money transfer, robustness was a particularly important criteria here. For that matter, we added a penetration tester who had the opportunity to test the system from the outside, the so-called pentesting methodology. So, the way to do this was to conduct risk analysis within the development teams instead of once every new major system release. This was done several times during development and it’s the documentation from this work that we have reviewed and analyzed.
A compilation of the main results
The study took place in 2015 and, in short, a compilation of the results look like this. The study involved 88 staff members at Ericsson. They were distributed among 8 development teams and 4 versions of the system. Version 1 here is particularly important to mention. In this version, Ericsson used the old, traditional agile software development method. We integrated SEAP in versions 2, 3 and 4.
Version 1 represents the typical normal case; this is generally how they do business. With the addition of our four security competences and an ongoing risk analysis process in the development teams, we could see that the number of corrected risk risks increased, the number of postponed risks decreased, and the number of unhandled risks also decreased. These were really good and interesting results. We could finally back up some of the work that many other researchers has previously set out to investigate.
SEAP found significantly more severe risks because of the more detailed and integrated risk analysis. Why? If we have more people thinking about security on a daily basis, I think it’s only normal that fewer severe risks are left unhandled. We also could see that the number of corrected risks increased five times, from 13 percent to 67 percent. In the SEAP, we left fewer risks postponed to future software versions (a decrease from 38 percent to 11 percent). The number of unhandled risks decreased from 50 percent to 21 percent.
On first glance, it seems like SEAP was much more expensive. We used 5 percent of the total project cost instead of 1 percent for the traditional agile software development method. Whereas SEAP might seem more expensive, it’s not entirely clear that that is the case: The integration of security in agile software development contributed to a higher level of time efficiency in the software development, which benefits product robustness and quality and a lot of other good stuff that consumers tend to appreciate.
Andreas Jacobsson leads the IOTAP project Intelligent Support for Privacy Management in Smart Homes (iSMASH).