A Decade of Evolution in Architecting Reusable Software

Software reuse has largely delivered on the promise of enterprises building from shared services and libraries but this has required multiple waves of technology and process improvements that have incrementally built on one-another. While still a long ways from software development being systematized at the level achieved by other industries such as manufacturing, software development today is easier than it ever has been. The purpose of this blogpost is to provide a historical accounting of the major architectural and software reuse trends of the last decade (or so) to help clarify what each wave of innovation has contributed to current day practices. This topic was originally covered in my work at Intuit while preparing the IT function for adoption of internal API query language technologies as a means to answer the question of why such adoption was necessary and its relationship to past architecture approaches.

A few caveats before discussing each wave of architecture improvement:

  • This post summarizes architecture approaches with a focus on telling a story of evolution as it relates to reuse — it does not provide a comprehensive technical review
  • The application of architecture styles and their adaptations can differ significantly from one enterprise to the next
  • The degree of success enterprises have adopting architecture styles varies significantly
  • Architecture evolution is only roughly sequential
  • Services are only one way to build software — service-orientation has wide applicability but there are many types of software for which services should play little or no role

To a Service Oriented Architecture (SOA)

The ambitions of SOA were vast but appropriate. Unfortunately, many of the recommended best practices, standards, and technology stacks suffered problems in practice. Still worse, even in cases where SOA prescriptions were on target, the average enterprise lacked the competency to execute successfully.

While as the move to service oriented architecture in industry was chock-full of technology and standard changes, the real essence of the architecture approach was to prescribe the creation of reusable services for building applications across the organizational boundaries of enterprises. Prior to SOA, software development proceeded within organizational silos. Lack of coordination across an enterprise resulted in duplicative development, significantly driving up cost. Software costs were also high because organizations within an enterprise were buying horizontal application backend software from different vendors many times over than if backends were procured once for all.

In the days before SOA, there were few best practices and insufficient rigor put into the definition of remote interfaces for client invocation. Interfaces were typically hard for client developers to understand, they frequently combined dissimilar data and function into a single interface that met the requirements of a very specific point-in-time user experience, and interfaces did not adequately use abstraction to insulate client code from changes to backend applications.

What did SOA achieve? In terms of fixing problems from the days of Remote Object / RPC, SOA successfully drove enterprises to build services for reuse across organizational boundaries and many enterprises made inroads to consolidating duplicative backend applications. A handful of industry leaders were able to fully reap the benefits of building client software through significant service composition while other enterprise enjoyed more modest value-for-investment — the spectrum of success was long tailed. The industry as a whole benefited from attempts at SOA as it drove up competency in software governance, design, middleware (platform software), planning, collaboration across organizational boundaries, service discovery and delivery, etc.

SOA correctly prescribed designing service contracts to decouple consumers from dependencies to backend applications but enterprises generally didn’t do a good job of following this architecture tenet. Similarly, SOA correctly prescribed decomposing services according to non-overlapping business objects but here again industry struggled.

Critical SOA Limitations

The move from building software in silos to building services for broad reuse introduced problems for service clients. One-size-fits-all service contracts are generalized not optimized for specific clients, client software has to orchestrate sequences of service invocations, and client interactions with services is more chatty than in the days of remote objects / RPC. Furthermore, SOA advocated for processes and software infrastructure that turned out to be hinderances instead of enablers.

One Size Fits All Service Contracts are Generalized Not Optimized for Specific Clients. The operations of a reusable service are typically designed to meet the requirements of many clients, not one specific client as was more often the case in the days prior to SOA. A great virtue of SOA is reuse — the provider reduces the cost of development and maintenance by building a service once for reuse by all. The disadvantage is that now clients integrate with providers through interfaces where operations have request inputs and response outputs that may be irrelevant to any given client. Client developers are burdened with having to understand or at least ignore such inputs and outputs, which is likely to slow down development. Runtime performance of a service operation invoked for a given client use case suffers unnecessarily due to computation of unneeded inputs and outputs. Similarly, client performance suffers if required to process responses that have unneeded output.

Clients Burdened with Implementing Orchestrations. SOA advocates for clients to build their experiences through service composition, that is, by calling a set of reusable service operations mixed-and-matched according to the experience’s business requirements. Generally, reusable SOA services offer more fine-grain functionality than in the past to better support client mixing-and-matching and also out of necessity due to federated service ownership where clients build an experience from service operations offered by different providers. The amount of client-side code is driven up due to SOA service composition since more operation calls are required to build an experience and also because the outputs and inputs of such calls need to be chained together, typically requiring some amount of translation in between invocations. This is not a desirable outcome for client experience developers, especially if one sees them as the the customer and places high importance on their satisfaction, and also in cases where client software is resource constrained (e.g. mobile applications) and therefore can be overly burdened when asked to implement service composition.

Chatty Interactions Cause Poor Performing Client Experiences. A second problem related to asking client software to implement service composition is the runtime performance penalty of having to make many service operation calls across a network and back. A common problem encountered in SOA and even with REST based architectures is building experiences that pull together disparate information into single views, a use case that usually involves having to make many operation calls or REST API invocations. Each call roundtrip can be time expensive and performance suffers even more when calls are dependent on one another and therefore require sequencing.

SOA Prescribed an Ill-Advised Set of Practices. Unfortunately, a confluence of vendor self-interest, academic and insufficiently practical expert guidance, and industry lack of reuse experience led to many failed SOA initiatives. Proper coverage of this topic requires its own blogpost — a summary is provided here only to help tell the story of how architecture approaches have evolved from SOA to REST and from REST to microservices.

A commonly identified culprit of SOA failures is the people challenge of getting everyone to collaborate and coordinate as is required to achieve enterprise reuse. The infamous “mindset shift” of prioritizing the needs of the enterprise as a whole before those of the local organization remains difficult to achieve, indicating that self-interest endures as a strong motivator. So too does self-interest explain vendors pushing heavyweight SOA middleware and tooling in the quest for selling products as opposed to doing right by customers and delivering technologies with a clear track record of delivering value.

SOA best practices included many heavyweight processes and standards that were complex and burdensome to follow. Governance processes in particular were overly restrictive and punitive, frequently slowing down development without delivering clear value. Governance, inappropriately administered, generated organizational backlashes that swung the pendulum to no governance at all, for anything. The many WS-* (web service) standards that proliferated in the heyday of SOA were challenging to follow and in many cases unneeded. Sometimes standards were simply not well-designed. As a case in point, UDDI was an abject failure out of the gate that hindered service discovery, a necessary element of SOA solutions intended to connect service consumers with provider offerings.

To a REST Architecture

There are many references available to learn about the details of REST, the original work was done by Roy Fielding. Only a narrow subset of REST is covered here to explain key architectural improvements it contributed, specifically how to design APIs for uniform interface as REST dictates that remediates the problem of poorly designed SOA service operations.

SOA service operations tended to be poorly designed because the architecture style did not provide sufficiently detailed design guidance. From operation naming to designing request inputs and outputs, designers were left with too many decisions and this led to a lot of inconsistencies and operations whose behavior was difficult to anticipate and understand. When an operation was called, it was regularly unclear exactly how it affected system state. System state should have been modeled as key business objects but as was mentioned, in practice this was not done well.

A major contribution of REST was to provide a principled approach to design that, while at times was more pure than practical, still made great strides in regularizing the design of and making APIs (aka service operations) more easy to consume. Here are a few of the problems it solved:

SOA service consumers have difficulty understanding what state is affected by an operation. REST-based software designs model system state as a well-known and documented set of resources (business objects) that provide API consumers with a clear conceptual model for all data that is accessible via API. The conceptual model is a combination of resource datatype schemas and supporting documentation that includes the relationships between resources. API invocation takes place in the context of a resource, making clear the scope of the data that can be affected. APIs can act on individual resource instances using unique identifiers, resource instances of the same type can be affected in bulk, computations can be made against resource state where that state is unaffected, etc. For each of these use cases and more, REST prescribes design patterns where SOA did not.

SOA service consumers have difficulty understanding the expected behavior of an operation. REST provides several design constructs that give consumers and indication of the expected consequence / outcome / behavior of making API calls. First, practical REST provides a basic categorization for API types — Entity APIs provide CRUD operations on resource state, Utility APIs perform a computation that never changes resource state, and Task APIs orchestrate many API calls and as such may impact the state of multiple resources. Second, REST API behavior semantics must align to HTTP methods (get, put, post, delete, patch). Third, REST resource identifiers that determine the point of application of the API are descriptive URLs following standardized design patterns that help consumers understand what to expect from API invocation. The REST design pattern of an API being equal to an HTTP method plus resource identifier leads to better intuitive understanding of an API than the typical names given to SOA operations.

Hypermedia as the engine of application state (HATEOAS).

One prescription of REST is to provide a set of links in the response of each API call. These links include a resource identifier, HTTP method, and other metadata that client applications can use to determine the next API to invoke. In the grand vision of REST, client code traverses these links as execution proceeds, advancing application state. These links form a behavior graph that chains together typical application API call sequences. In practice, client use of HATEOAS links to drive application execution has not enjoyed broad adoption and therefore is not a part of most organization’s story of architecture evolution.

To a Microservices Architecture

The REST architecture style was available for designers to embrace back in the days when organizations were failing with SOA, although widespread adoption was several years in the making. What did not readily exist were alternatives to SOA for processes (planning, development, governance), runtime platforms, and development tooling. Nor was there a principle / philosophical replacement for what SOA espoused. For years, the failures of SOA sent industry back to the drawing board with organizations trying to figure out the next best way to develop software. What has emerged is a new approach for building software products from reusable APIs called microservices, the prevailing architecture style of choice these days. As with SOA, the microservices umbrella includes not just technical design best practices but also prescribes best practices across the development lifecycle. To a first approximation, the organizational and technical practices microservices champions are a wiser version of SOA — Microservices = REST + Wise SOA.

Microservices offers many important improvements to the way software development is done. Read The Microservices Scorecard for an introduction and framework for planning evolution to this style. It is important to point out that microservices do not help solve the previously identified limitations introduced by SOA, specifically that one-size-fits-all service contracts are generalized not optimized for specific clients, clients are burdened with implementing orchestrations, and chatty interactions cause poor performing client experiences.

A final point worth mentioning is that to a second approximation, the scope of best practices microservices covers is not as comprehensive as SOA and there also appears to be a need to fine tune the formality of practices it recommends. The pendulum swing away from (1) heavyweight SOA governance to (2) no governance to (3) some governance leaves organizations with development controls that are still a bit lightweight.

It is anticipated that a future wave of technical evolution will introduce software engineering governance practices whose application is data-driven based on past team development performance.

To Cloud-Hosted Software

While REST and later microservices were gaining traction in industry, the movement to the cloud was also growing, albeit at a slower pace. What started out as Amazon offering basic compute and storage for lease to consumers has grown today to be a multi-vendor industry with a comprehensive set of services for building modern software applications. Large enterprises have gone all-in, getting out of the business of running their own data centers in favor of leasing network infrastructure, host hardware, middleware, runtime containers, development tools, and a variety of horizontal application functionality.

Migration to cloud-based solutions is a stunning example of successful reuse that is enabling enterprises to focus on delivering software for their core business and not on platform software, the substrate on which these applications depend. In the context of SOA and microservices this means organizations have largely been alleviated of the need to stand-up, operate, and maintain such software as service buses, containers, registries, portals, and more.

Moving to the cloud comes with some architecture implications: (1) There is a strong economic incentive to build software from technology stacks offered as first-class cloud-vendor offerings (managed services) and this will inevitably drive down technology stack diversity, (2) a non-trivial portion of responsibility to solve for high availability and disaster recovery has been offloaded to cloud-vendors, (3) the many software dependencies to third party applications previously managed down through abstracted interfaces will give way to hard-to-avoid dependencies on cloud-vendor provided services, (4) deployment architectures will be shaped by cloud-vendors that establish the available intra-service connectivity patterns that set how software components can be connected and the lingua franca (reference architecture) used to express this architecture view.

To Microservices + API Query Language (QL)

Concluding the story of architecting for reuse over the last decade, API query languages offer an important solution to several of the problems introduced originally by SOA and still suffered by REST APIs that are designed for reuse by multiple clients. Unlike SOA, REST, microservices, and the cloud, API query language industry adoption is not a major architectural advancement but instead a smaller scale improvement with more narrow applicability.

Providing client applications with a query language to retrieve information exposed via multiple APIs has been in industry practice for a minimum of almost a decade. GraphQL (2015) is the most recent and popular technology offering, its predecessors include ql.io driven by eBay (2011) and the Yahoo Query Language (2008). The primary capability these languages provide is the ability of client applications to have a convenient way to get the information they require to drive a single customer experience without having to make multiple API calls and without being sent information that is unneeded to render the experience.

Second, these query languages offer the benefit of utilizing the platform to process the information requested such that it is easily accessible and execution time efficient for clients to use.

Standard Elements of a Query Language Solution

Query language. Provides a language for the client to express their data needs, cross-resource, that is, across the full information space exposed via APIs. The language offers general purpose operators (<, >, = etc.).

Query API. At least one end point for the client to make the query request.

Query receiving platform. Responsible for processing queries, orchestrating the needed API calls, building the response, and sending that response back to clients. The platform typically sits close (low latency communication paths) to the APIs it needs to invoke and also has the ability to parallelize API calls.

Query platform configuration. The platform needs to be setup with the data elements exposed via the query language. Additionally, the configuration needs to map those data elements to the corresponding elements found in business APIs. Configurations typically provide some ability to process data in addition to facilitating one-to-one data mappings. The configuration also needs to be setup with all of the information required to make API calls.

Business APIs. The set of APIs that provide access to the underlying functionality and data offered by source systems.

Problems Solved

One Size Fits All Service Contracts are Generalized Not Optimized for Specific Clients. An API query language allows clients to express exactly what information to include in a response and the structure of that response.

Clients Burdened with Implementing Orchestrations. An API query language solution enables clients to send a single request that expresses their information needs for interpretation by a platform that makes all of the API calls needed to service the query. Clients no longer need to call the business APIs directly, nor even know about them if they make all their requests via queries.

Chatty Interactions Cause Poor Performing Client Experiences. The API query language platform isolates the client from chatty interactions with business APIs by making those calls for the client. Platform calls to business APIs have better execution time performance because these interactions happen closer to each other on the network than would otherwise be the case if clients called business APIs directly.