Designing Digital Retail (Part 4): Orchestrating Product Development at Scale

This is the forth in a series of articles that explore how traditional retailers can move through the challenges of digital transformation. You can see all the articles in this series here.

Why orchestrate your product development?

In some companies and industries, the portfolio of digital products consists of a set of related but independent products with little need for integration of data and functionality. Traditional retailers have, until now, developed their digital channels with little integration between them.

However, as I have argued in part two, in order to compete in the arena of customer experience, retailers must move towards omnichannel retail (i.e. the total integration of all digital and physical channels to the customer). In order to become an omnichannel retailer, organisations must orchestrate the development of a set of highly integrated products and technology services in which integration leverage is critical (Cagan, 2018).

So, what is the value of orchestrating product delivery?

• Well-orchestrated development ensures that the business has more control over the delivery of their strategy, with each product contributing to outcomes of the overall strategy.
• Secondly, it ensures the best possible customer experience. As described in part 2, the empirical literature shows the value of integrated channels to both users and retail organisations. In a well-managed, coherent product ecosystem, each product positively contributes value to other products, rather than cannibalising other channels.
• A third value to the business is that harmonious product delivery ensures efficiency of software development operations, so that code and functionality is not replicated multiple times through the organisation.
• Furthermore it ensures that any dependencies between products are handled well, and that every line of code contributes to the development of the overall technology strategy of the organisation.
• A final value to the business is that harmonious IT infrastructure will enable the maximisation of operational efficiency. For instance, imagine a world where depot staff scan a pallet - and that data is made available to supply chain systems, store inventory systems, finance prediction systems etc etc.

When these factors are added together there are huge potential cost savings and revenue enhancements for the business. For example, Deutsche Banks’s ‘Executing 2020’ Strategy estimated that harmonising its IT infrastructure could potentially save 800 million euros per year in everyday running costs (Deutsche Bank, 2015).

Well-orchestrated development ensures that the business has more control over the delivery of their strategy, with each product contributing to outcomes of the overall strategy.

Managing Multiple Software Products and Channels

While there is significant work that develops an understanding of building individual software products, there is relatively less academic work on how to manage multiple products and channels, particularly in a retail environment. This management of product portfolios and the underlying technology infrastructure is a major challenge that retail organisations have to deal with as they move towards omnichannel retail. Developing maturity in this capability will enable retail organisations to evolve beyond multiple siloed digital channels, towards becoming fully omnichannel. Success at this activity will largely define their success in delivering competitive customer experiences.

One approach is Product family engineering, or software product line engineering (Bosch, 2000; Pohl et al, 2005) which suggests that an organisation should go through three phases as they develop their software product family. The first phase begins with a product management process, in which the business defines market strategy and vision, and defines the scope of the product family. In the second phase, the domain-engineering phase, technologists document the technical requirements of the different product lines, elucidating the common and variable requirements. This allows the business to create an abstract plan of the technical requirements of the whole organisation, together with a reference architecture for how these products might interact (such as information sharing protocols). Once this is complete, engineers begin building the first products, beginning with the previously identified set of common requirements. These teams build a set of reusable components which can be shared with other teams. These other teams will then only need to build the components that are specific to their products.

While this approach seems ideal in theory, it is rare that a business finds itself in a position to start building an entire product family from scratch. Furthermore, as outlined above, traditional large retail organisations have multiple products already in existence with often very little previous cross-product panning, and therefore few common components. It is therefore challenging to develop a product family using this entirely top-down approach to product development. Finally, this is a ‘Waterfall’ approach, so once executed, provides no direction about how an organisation can evolve their product ecosystem to meet the needs of the business, the market and customers.

Retailers must therefore use more approaches that can accommodate Agile practices and work effectively with more ‘evolutionary’ and iterative approaches to managing technology. Humble et al., (2015) took inspiration from the Lean Start-up movement (Reis, 2011) and the Toyota Production System (Ohno, 1978) to develop an evolutionary approach to the modern digital enterprise — known as Lean Enterprise. The Lean Enterprise approaches problems using a scientific methodology, in the sense that product teams develop hypotheses and then carry out simple actions to test them and take forwards the learning. There are a number of ways in which a Lean Enterprise differs from the traditional enterprise. Rather than planning all spending and activities in the coming year in a budget, the organisation sets out objectives and creates indicators to measure activities across multiple perspectives. Resources are allocated dynamically and the indicators are reviewed on a regular basis. Rather than defining the team activities for the coming year, they are given goals and empowered to work out how to achieve those goals. These teams also continuously work to improve processes through continuous measuring of key results which provide a feedback loop to drive the teams in the right direction.

In the Lean Enterprise, rather than defining the team activities for the coming year, they are given goals and empowered to work out how to achieve those goals.

A number of approaches for carrying out Agile software development at scale have been developed, including Scrum of Scrums (Sutherland, 2001) and the Scaled Agile Framework (Leffingwell, 2007). The Scaled Agile framework (Leffingweel, 2007) suggests a number of approaches for scaling software development in more hierarchical organisations. The primary suggestions are that teams should operate within a longer-term planning horizon, and that product owners should be more embedded in the business, to ensure they are delivering the business strategy. An organisation should create an enterprise technology and product roadmap which typically defines the next 12–18 months, and is updated regularly. Each product team must then create their own three-month planning horizon within the parameters of the larger roadmap. This model has been criticised for the way it constrains the autonomy of each team to ensure they comply with the organisations goals (Maximini, 2018). One core challenge here is that organisations have to manage this trade-off between top-down control and product team autonomy. By increasing top-down control, organisation gain more control over the overall product-family. However, at the same time, because teams are stripped of their decision-making power, there is a risk they become less effective at delivering successful products.

A number of researchers have studied how large organisations can be successful at scaling Agile practices. Dikert et al (2016) conducted a systematic review of the challenges of implementing Agile in a large organisation, finding that the four most important factors to success were customising the Agile approach to the organisation, management support, mindset and alignment, and training and coaching. The authors reported thirty different challenges and make twenty-nine separate recommendations for organisations, such as educating management on Agile, start with a pilot to gain acceptance, start with agile supporters, provide training on agile methods and recognise the importance of the Product Owner role. Kalenda et al (2017) found that resistance to change, an overly aggressive roll-out timeframe, quality assurance concerns and integration with non-agile business processes were the largest challenges to implementing Agile in a large organisation.

One of the great challenges of developing the software infrastructure of a large enterprise is in bringing together the ongoing operations of software maintenance and management (traditionally carried out by an I.T. team) with the development and launch of new products and services (traditionally carried out in ‘Waterfall’ software development projects). The Devops approach has been developed to bridge the gap between these silos. Under this approach, Devops Engineers test, deploy and monitor software and then feed back their learnings to the development teams who will use these insights to update and improve their software. Under DevOps, The goal is to have testing and operations happening simultaneously with design and development, enabling fast flow and high quality. DevOps encourages making small, fast iterations and deployments to reduce complexity and error and enable agile, fast software development and management.

Under DevOps, The goal is to have testing and operations happening simultaneously with design and development, enabling fast flow and high quality.

Conclusion

To build software at scale, there must be methods to manage complexity, facilitate communication and to help the organisation to manage the trade-off between team autonomy and top-down control. Companies must also find the right trade-off between investing in their underlying technology platform and giving teams autonomy to develop new products and applications. In the next two posts we’ll explore the value of architecture and the value of design for orchestrating retail

Up next: Designing Digital Retail (Part 5): The role of architecture

References

Jan Bosch, Design and use of software architectures: adopting and evolving a product-line approach, ACM Press/Addison-Wesley Publishing Co., New York, NY, 2000 https://www.amazon.com/Design-Use-Software-Architectures-Bosch/dp/0201674947

Cagan, M. (2018). Inspired. How to Create Tech Products Customers Love. Wiley, 2nd Edn

Deutsche Bank (2015). Executing Strategy, 2020. Deutsche Bank . Retreived from https://www.db.com/newsroom_news/Strategy_2020_-_Press_Presentation_engl._29.10.2015.pdf

Dikert, K., Paasivaara, M., & Lassenius, C. (2016). Challenges and success factors for large-scale agile transformations: A systematic literature review. Journal of Systems and Software, 119, 87–108.

Humble, J., Molesky, M. & O’Reilly, B. (2015). Lean Enterprise. How High Performance Organizations Innovate at Scale. O’Reilly

Kalenda, M. (2017). Scaling Agile Software Development in Large Organizations (Doctoral dissertation, master’s thesis, Faculty of Informatics, Masaryk Univ).

Kim, G., Humble, J., Debois, P., & Willis, J. (2016). The DevOps Handbook:: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. IT Revolution.

Leffingwell, D. (2007). Scaling software agility: best practices for large enterprises. Pearson Education.

Maximini, D., Maximini, & Rauscher. (2018). Scrum Culture. Springer International Publishing AG, part of Springer Nature.

Ohno, T. (1978). Toyota production system: beyond large-scale production. crc Press.

Pohl K., Bockle G., & Linden F. van der (2005). Software Product Line Engineering. Berlin, Heidelberg, New York: Springer-Verlag.

Ries, E. (2011). The Lean Startup: How today’s entrepreneurs use continuous innovation to create radically successful businesses. Crown Books.

Sutherland, J. (2001). Inventing and Reinventing SCRUM in five Companies. Cutter IT journal, 14, 5–11.