A comparison of issues and advantages in agile and incremental development between state of the art and an industrial case — A retrospective
Kai Petersen and Claes Wohlin
Abstract: A decade ago, large-scale industrial agile software development was still in its infancy. It was an excellent research opportunity for an industrial Ph.D. project. As part of Kai’s Ph.D. studies, we published the article titled “A comparison of issues and advantages in agile and incremental development between state of the art and an industrial case’’ in the Journal of Systems and Software. In this blog, we do not just want to summarize the article. Instead, we discuss how the research affected both the industrial understanding of large-scale agile and how it influenced future research and hence which role the research played in the Ph.D. project. We present the story behind the study, ranging from the inception of the study to leveraging on the study results.
Inception
Setting the scene
We started designing the study and collecting the data in late 2007. It was the first study in the Ph.D. project being part-time employed at Ericsson and part-time at the university. The company identified the need for agility and was in the process of moving towards agile development. One central question was: How is it going? It was an excellent opportunity to conduct an empirical study investigating the advantages and issues of agile and plan-driven development (Ericsson’s old way of working). The topic was especially interesting at the time, as there was the question of whether agile development scales well. Hence, the context of investigation (development of large-scale telecommunication systems) was exciting from a research perspective, and at the same time very interesting for the company.
The research questions
In the study, we asked two research questions.
- RQ1: What are the advantages and issues in industrial large-scale software development informed by agile and incremental practices?
- RQ2: What are the differences and similarities between the state of the art and the case study results?
The first research question focuses on “How is it going?” when moving to agile development. The second question explicitly takes the literature into account, as it is interesting for companies to learn from findings in other contexts — the more similar, the better. Having formulated the questions, the next question was: where to get the data and how to analyze it?
Conduct
Collecting the data
The research started by digging through company documentation, which may be challenging to understand as companies often have their terminology and abbreviations. However, being an industrial Ph.D. student with part-time employment in the industry was an excellent start. It meant already being familiar with the terminology, and hence it was much easier to talk with the people at the company. Being embedded into the company was very beneficial both for industrial impact and research. The investigations of the documentation were complemented with discussions with colleagues.
We identified the people in dialogue with our champion at the company, and we wanted to make sure that we covered all the roles involved in the end-to-end process. We wanted at least two representatives per role. At the same time, three systems developed at the company were considered, which adds to the generalizability of the results. In total, 33 interviews were conducted. The interviews were recorded and transcribed. Besides getting interesting data, the interviews had another benefit: they helped to create a network in the company, which was valuable for the continuation of the Ph.D. project and the further research conducted after the completion of the Ph.D.
Analyzing the data
We analyzed the data conducting four steps that closely resemble the open and axial coding processes used in Grounded Theory. First, statements were grouped (clustered) and were associated with labels (in the paper called factors). After that, the labels were grouped (axial coding), and finally, the results were validated in two workshops.
Results
When reflecting on the results, we focus on the case study (RQ1). The advantages provided valuable feedback to the company, showing the benefits gained by moving towards an agile process. We identified eight advantages of agile observed at the company (Table 1) and twelve issues (Table 2).
Overall, it was found that in parts of the process, work became easier (e.g., communication in small teams), but the prior complexities in development were moved elsewhere (e.g., complexities in configuration management and management overhead for team communication and coordination).
We are looking at the results from two perspectives: (1) direct industrial impact (relevance), and (2) academic impact (the basis for further research).
Industrial impact
The issues were essential drivers for further research, as they indicated both research gaps and challenges the company was facing. Understanding the “real” challenges is a key success criterion when conducting industry-driven research [Garousi et al. 2016], and an integral part of technology transfer [Gorschek et al. 2006]. Hence, various solutions were successfully transferred to the industry. The transfer was successful due to the excellent understanding of the problem. Being a researcher embedded in the company helped, too. A selection of issues, from Table 2, are exemplified below.
CI11: Management overhead due to a high number of teams requiring much more coordination and communication, and CI12: dependencies between systems: Given the high number of teams and requirements that are specified, designed, implemented and tested across various systems, a first step towards managing the complexity is to understand the current situation. We proposed measures and visualizations of the development flow, taking into consideration that several systems are developed in parallel and have dependencies between them [Petersen et al. 2014][Petersen and Wohlin 2011]. These helped the company to proactively take informed operative decisions to improve their development flow and to initiate further improvement initiatives based on the measurement findings, such as value stream mapping investigations [Khurum et al. 14].
CI02: Design has free capacity due to the long lead-times in requirements engineering: When requirements arrive at the company, it is not easy to estimate the effort needed to develop them or even to assign them to a system. Given the complexity of a system, excellent knowledge of the system anatomy is needed. Thus, we proposed a new approach (FLOW, in combination with value stream mapping) to map the current and desired information flow to address CI02 [Ali et al. 16].
CI04: Test cycle length extends lead time: The proposition of the company was that reducing cycle times in testing would improve their overall velocity and with that, the ability to deliver value to the customer more frequently. We built a simulation model and used it in the context of value stream mapping to reflect on the impact of the length of cycle times on lead time [Ali et al. 15].
Academic impact
Given that the studies were conducted early in the adoption of agile development for large-scale software development, the research has spawned interest. Furthermore, to the best of our understanding, the findings were perceived as both relevant and valid. To be employed by both a company and a university helps in making the research both industrially relevant and being done in a rigorous way. The research has resulted in a number of citations, which are spread quite evenly over the last 10 years (see Figure 3), and hence the research still generates an academic interest.
Conclusions
We investigated the advantages and issues of agile software development used in a large-scale context. The study was of particular interest due to its large-scale industrial context.
The study was very beneficial as part of the Ph.D. project since it allowed to (1) get an excellent network in the company, and (2) identify research problems that were relevant to the industry partner. Given the benefits, we recommend starting all industry-academia collaborations with an exploratory study to understand the main challenges at the company, before conducting more solution-oriented research.
References
[Garousi et al. 2016] Vahid Garousi, Kai Petersen, and Baris Özkan. Challenges and best practices in industry-academia collaborations in software engineering: A systematic literature review. Information& Software Technology, 79:106–127, 2016. DOI: 10.1016/j.infsof.2016.07.006
[Gorschek et al. 2006] Tony Gorschek, Per Garre, Stig Larsson, and Claes Wohlin. A model for technology transfer in practice.IEEE Software, 23(6):88–95, 2006. DOI: 10.1109/MS.2006.147
[Petersen et al. 2014] Kai Petersen, Peter Roos, Staffan Nyström, and Per Runeson. Early identification of bottlenecks in very large scale system of systems software development.Journal of Software: Evolution and Process, 26(12):1150–1171, 2014. DOI: 10.1002/smr.1653
[Petersen and Wohlin 2011] Kai Petersen and Claes Wohlin. Measuring the flow in lean software development. Journal of Software: Practice and Experience, 41(9):975–996, 2011. DOI: 10.1002/spe.975
[Khurum et al. 2014] Mahvish Khurum, Kai Petersen, and Tony Gorschek. Extending value stream mapping through waste definition beyond customer perspective. Journal of Software: Evolution and Process, 26(12):1074–1105, 2014. DOI: 10.1002/smr.1647
[Ali et al. 2016] Nauman Bin Ali, Kai Petersen, and Kurt Schneider. Flow-assisted value stream mapping in the early phases of large-scale software development. Journal of Systems and Software,111:213–227, 2016. DOI: 10.1016/j.jss.2015.10.013
[Ali et al. 2015] Nauman Bin Ali, Kai Petersen, and Breno Bernard Nicolau de França. Evaluation of simulation-assisted value stream mapping for software product development: Two industrial cases. Information & Software Technology, 68:45–61, 2015. DOI: 10.1016/j.infsof.2015.08.005
Author Biographies
Kai Petersen received a Ph.D. degree in software engineering from the Blekinge Institute of Technology, Sweden, in 2010. He is currently a Professor at the Blekinge Institute of Technology and the University of Applied Sciences, Flensburg. He has authored over 100 research works in international journals and conferences. His research focuses on software processes, software metrics, lean and agile software development, quality assurance, and software security in close collaboration with industry partners.
Claes Wohlin is a professor of software engineering at Blekinge Institute of Technology, Sweden. He served as Dean for the Faculty of Computing for five years (2014–2018) and Pro-Vice-Chancellor for six years (2004–2010). Claes Wohlin has previously held professor chairs at the universities in Lund and Linköping. He received a PhD in communication systems from Lund University. In 2019, he was appointed Emeritus Editor for the Journal of Information and Software Technology after having served as Editor-in-Chief for 14 years and Editor for three years. Claes Wohlin received Telenor’s Nordic Research Prize in 2004 for his achievements in software engineering and the improvement of reliability for telecommunication systems. His research interests include empirical methods in software engineering, global software engineering, and software quality. In 2011, he was elected member of the Royal Swedish Academy of Engineering Sciences.
