Building An “Outside-In” Approach To APIs
Some time ago, I met with an Apigee customer, a large company in the midst of a major transformation to a more customer-centric way of approaching its market. As you may expect, APIs played a large role in this transformation, comprising the vehicle by which they could re-work their interactions with customers, business partners, and even themselves.
The overriding goals were to improve customer satisfaction and — perhaps more importantly — maintain their position in the market by speeding up both internal and external innovation. But there were of course many challenges.
The company was fortunate: the initiative had a strong executive sponsor who had a clear understanding of the goal and the will — both personal and political — to make it happen. But the process still turned out to be daunting because the company had to implement changes to processes, systems, and mindsets that had become entrenched over years over success in the “olds ways.”
In particular, the company discovered that the inflexible, tightly-coupled, and highly contextual point-to-point approach that teams had developed for building integrations and exposing customer-facing experiences made change difficult, time-consuming, and very expensive — a recipe for disaster in a market that was moving fast and threatening to commoditize the company’s core business. REST-oriented APIs, as a flexible means of fostering innovation and providing new products, services and experiences to customers, formed the backbone to their transformation into a more digital, fast-moving and customer-oriented company.
Before APIs — of any type — can accelerate development, unlock new business opportunities, or otherwise fulfil their promise, some fundamental changes need to be made in how they are designed, implemented, deployed, and used. Some of these changes are technological and relatively easy to implement, but some require a mindset change, away from the “inside-out” approach sometimes present in non-digitally native legacy businesses and toward an “outside-in” approach that puts fundamental stakeholders — the people who create value for your company, and the people who consume that value — first.
In this article I’ll share some particular thoughts on how this shift in the way APIs are designed and implemented can pay big dividends for customers struggling to transform themselves into leaner, faster-moving organizations.
What exactly is an “outside-in” approach, and how does it apply to APIs?
“[L]ong-term innovation success doesn’t revolve around what innovations ‘do’, it centers on what they invite customers to become. Successful companies have a ‘vision of the customer future’ that matters every bit as much as their vision of their products.”
— Michael Schrage, MIT
“Who Do You Want Your Customers To Become?” in Harvard Business Review
In a technology context, “outside-in” is fundamentally about seeing the world through the eyes of several key stakeholders:
- Customers, the ultimate consumers of the value a company produces. Acquiring, retaining, and satisfying customers is an organization’s core mission, and customers typically gravitate toward companies that provide the most value, in the most consumable way, with the least fuss.
- Developers, the people who work with the systems that make a company run (systems of record), the systems that provide differentiation from competitors (systems of differentiation), and the people that are creating an organization’s next new product or service offering (systems of innovation). Developers are responsible for providing the capabilities behind all of the interaction channels companies use to connect with customers and provide value.
- Partners, the people and organizations with which an organization collaborates to bring an ecosystem of capabilities to market. This ecosystem enhances the value of all involved, improves value delivery, and repackages value with complementary offerings to make new revenue channels possible.
An “outside-in” approach fundamentally flips the API design perspective from a systems-centric, “inside-out” view that starts with what systems do now, to a holistic, “outside-in” view that focuses on fulfilling customer, partner and developer needs through the products and services of your organization.
Examples of a systems-centric, “inside-out” API design include:
- CRUD-oriented operations that are not easily understood by developers;
- business process invocations that require data from many different sources;
- APIs that push business process orchestration logic out into either the API-consuming application itself or into a separate orchestration layer that is channel-specific and focused on building exactly the payload that the particular target application needs;
- API design processes in which the people implementing the APIs are familiar with underlying systems and don’t think about the needs of people that don’t know (and mostly, don’t want to know) the underlying business domain or technology.
Examples of a customer-centric, “outside-in” design include:
- flexible designs with a focus on reuse. The idea is not to cover every single future case, as businesses should avoid over-designing services and APIs, but rather to build extensible APIs that can be leveraged for currently unknown opportunities;
- hiding implementation details by thinking in terms of bounded contexts, e.g., contexts in which functionality is exposed through a well-understood “contract” between systems, with the sausage-making necessary to fulfill that contract remaining hidden;
- “API-first” design that builds the contract with an understanding of the scope of what the system will do, but not how the API will be implemented. This encourages users to focus on experimentation instead of implementation details.
When we apply “outside-in” to APIs, the overarching goals are to design APIs that make it as easy as possible for developers to repeatedly leverage given digital assets, including providing functionality that helps customers and other API users meet their goals. That is, applying an “outside-in” approach to APIs involves thinking of developers as a type of customer and managing APIs as products for developers.
Done well, this approach forces API creators to think about consumption of services (the customer’s “outside” view) rather than exposure (the technology or “inside” view). When API creators have this perspective, their work can enable the developers who consume the APIs to uncover opportunities to reuse digital assets and produce better, more delightful customer experiences.
How can your organization implement “outside-in”?
Begin by assessing your company’s current use and management of APIs. The following questions — as well as Apigee’s free tool, Apigee Compass — can help you do so:
- Is there an “API Product Management” role that monitors and optimizes the value created through APIs? Can you measure that value?
- Do your internal and external developers believe that your API fosters or hinders innovation? If given a choice of multiple functionally equivalent APIs from different companies, would they choose yours?
- How well do your teams know the technologies used in your customer interactions? Can they build systems that work effectively with these new technologies and leverage the most current and relevant tools and techniques? Or does their experience in legacy technologies hinder their ability to understand how new applications (single-page apps, mobile apps, assistant apps, etc.) are created?
- To what extent do teams constrain their activities based on “what the system does now” versus “what the customer wants to do”? Is there an overriding mindset (e.g., project-based funding) that stifles broader, longer-term thinking?
- Do the individuals responsible for designing and implementing APIs have an understanding of aspects of the business beyond their own? Can they identify opportunities for reuse? Can they design APIs that are extensible to support those reuse scenarios?
- Is the team capable of truly delivering incremental value in an “agile” fashion?
As these questions may suggest, developing a successful API program is partly about appropriately attracting, cultivating, and managing talent. It is important to develop core business competencies related to key digital technologies — APIs, microservices, web technologies, digital interaction channels such as mobile, IoT or voice assistants, etc. — and to find ways to “cross-pollinate” so that teams can get a better understanding of requirements outside their own domain. Mixing teams in a structured way and fostering internal hackathons may be useful.
Challenge developers to think beyond what the current system does. Assume the role of someone who knows nothing about the system or the business, and think about how you can provide value to that user.
Consider the creation of a cross-functional API team that can provide a broader perspective to teams responsible for systems of record/innovation/differentiation. The team should include members responsible for representing customers or stakeholders, developers, business partners, executive, and software engineering leaders.
Other implementation considerations
Implementation of “outside-in” strategies may also require a range of adjustments and considerations throughout the organization.
Internal and External Evangelism
Build an understanding across the organization of the value that “outside-in” brings to developers. This is particularly important for business unit stakeholders who need to understand how an incremental investment now can yield rewards later.
To do so, it’s important to demonstrate and socialize the benefits. Look for ways to translate the investment made in APIs into measurable business value. Enterprises shouldn’t focus on how many APIs are produced but rather on how the APIs are consumed. Important questions may include: “How are developers using these APIs?” “How does this usage support apps and digital ecosystems and ultimately contribute to transactions, revenue, and growth?”
When opportunities for reuse arise, or when the “outside-in” approach brings benefits to partners, developers and customers, an enterprise should make sure stakeholders know that their investment has paid off, even in small ways.
Companies can find ways to interact with developers to understand how they want to consume services and the challenges they have in doing so; they can even enable “self-service” as a means of supporting developers as they learn about an API. Self-service capabilities can help developers:
- find APIs that suit their needs;
- learn and understand about what the APIs do, both through documentation and direct experimentation;
- form communities with other developers via forums and other social features;
- sign up for access, with simplified handling of legal and financial concerns.
“Outside-in” strategies — and the process of incremental improvement as more is known about developer needs — work well with modern agile application development strategies. Businesses should adopt agile processes and continuous integration/continuous delivery (CI/CD) throughout the organization. As the pace of business continues to increase, waterfall development approaches and other legacy techniques become less and less tenable. In particular, continuous deployment — perhaps complemented with techniques such as feature flags that can disable the functionality until an appropriate time — can make it easier to introduce new functionality. However, to have an effective CD approach, it is important to include comprehensive automated tests as part of the code management process.
Enterprises may want to implement a Behavior-Driven Development (BDD) or Behavior-Driven Testing (BDT) approach. Using behavior-driven techniques can help API development teams focus on the expected customer-perceived behavior of a process rather than the actual implementation. Cucumber and its BDT language, Gherkin, are useful for implementing behavior-driven approaches.
If existing systems of record are rigidly structured, IT teams can leverage microservices architectures to bridge the gap to a more flexible structure.
Move to an “API-First” API development approach. Before implementing APIs, it’s important to use mocks, prototypes or desktop experiments to explore how easily the proposed API can be consumed by developers, and how it supports the creation of robust customer experiences (including currently unknowable ones that may arise in the future).
Here are a few considerations when designing an API:
- What do developers really need to understand (or, perhaps, what do you want them to understand) about your system in order to use your API? How can you hide or manage the complexity of a given function?
- How can you structure the API such that the developer has to do the minimum amount of work in order to build their desired consumer experience or system?
- Can you leverage the distinction between “core” and “experience” APIs to not only produce reusable components but also create ways to optimize the user experience for a given interaction channel?
- How do you build APIs that make multi-step business processes easy and remove the need for detailed business process knowledge?
- How do you facilitate reuse of APIs so that innovation can occur?
- How do you structure APIs to support long-running processes and asynchronous operations?
- What’s the best way to facilitate discovery of useful services, and to avoid redundancy in development?
- Do you need to put any “guardrails” (e.g., request quotas/rate limits, or additional security constraints) in place to keep developers from abusing the API?
What challenges should I watch out for?
Sometimes teams in a hurry to release new business-driven functionality can inadvertently incur technical debt in their new services while integrating those services with legacy systems. In an environment in which competition is fierce and the pressure to move quickly is intense, the business mandate to deliver functionality is also very high. The best way to address this is to give development teams the space to implement a disciplined approach to API design and implementation and to invest in education of stakeholders, making them as comfortable as possible with the additional investment that may be required.
It can be tempting for teams that are used to working with current systems to fall back into a “this is the way the way the system works” approach to API design. It’s important to continuously drive home the value of the “outside-in” approach via exposure to the benefits.
Often, the success of “outside-in” API design and development is measured in the success of applications and partnerships that leverage the APIs produced. By exposing stakeholders and API development teams to the benefits that accrue from improved API reuse, accelerated development times and increased developer adoption, an organization can position itself to truly benefit from its “outside-in” perspective.