Build an Organisational Platform
The ability to fast deliver and release functions to customers has become more and more important to the success of a business. However, building software is complicated, especially in a large organisation. Teams are often distracted by many other things, such as how it is deployed, how it runs, how it is sold, etc. How wonderful if we could allow the product teams (stream-aligned teams) to only focus on customer values?
In the last article, I mentioned that there are four types of platform services apart from products, which helps product teams reduce the cognitive load and keep their focus, as well as scale. They are:
- Shared product features: provide common capabilities such as a currency exchange service
- Services that support sales of products: provide capabilities to sell the products (or services), such as a website
- Infrastructure: provide capabilities to support products to build and run, such as AWS. They usually serve non-functional requirements such as security, service availability, application changeability (CI/CD tools).
- A data platform: provide capabilities to support better decision making for product teams, such as Redshift
In this article, I’ll explore some principles about how and when to build a good platform for an organisation. By having a good platform, the organisation will be able to scale and accelerate fast.
Build the platform services as products
Applying the product thinking to the internal platform and engineering the platform teams have become the standard approaches in the industry. Like building a product, the first thing that needs to know is who are the customers.
The customers of the platform
Many people think the customers of an internal platform are also internal. They treat the internal needs as the first priority and ask internal teams for approval for releasing any changes. This is partially right. They are important stakeholders. But they are not first-class customers. The first-class customers are the real clients who use the products of the company. Platform teams should look for the best user experience for those customers, just like the product teams do. Actually, both sides of teams should share the same goal. Otherwise, there will be value conflicts between the internal users and the external customers.
One example is security. The product teams want to release a function but there is a security issue that hasn’t been solved. The security team should block the release until the issue gets fixed. Of course, this block should automatically happen, such as failing the pipeline.
Another example is about sales. People in a traditional sales team are willing to sell more products by phone because their commission depends on it. So they would unlikely agree on a self-service experience such as a shopping cart or self-cancellation. They would love more to display their phone numbers on the website or a cancelling page and have a management system just for them to do such things. But this management system is very expensive because it needs to support them to do whatever they want, such as giving a very cheap price to a product, which is usually much simplified in a self-service experience.
Therefore, as the platform teams, we should do user research on them, analyse their needs, and deliver features for them. This includes the infrastructure teams and the data teams, we should only choose the tools that help to serve customers better, instead of providing many options for the product teams for the same purpose. However, this doesn’t mean we should ignore the internal product teams. We still need to simplify the onboarding process for them so that our services can be easily integrated and increase the speed to market.
In summary, platform teams should provide functions for the interest of external customers as well as minimise the cost of integration with other products.
Engineering in platform teams
Platform teams require a better engineering capability than product teams so that they can choose the right tools and guide product teams to use them, or build services that are easy to be integrated. Otherwise, the product teams would have bad experiences with the platform services, which would lead to either good developers leaving or building on their own. The worse case is that the platform teams might become blockers of product teams.
Thus, good engineering in platform teams is very important. It defines the upper limit of the engineering capability of the organisation. It is critical to the performance of the entire organisation. But how can we know how good engineering is for each team? Luckily, the industry already gives the answer: use the four key metrics to track the performance of teams.
We should make sure that there are elite teams (multiple production deployments per day) in platform teams who could lead the engineering practices and spread them (with tools) across the organisation. These elite teams are normally formed in a long time with a good engineering leader. So it won’t work if you break an elite team, distribute the developers into other teams, and expect other teams to become elite. The better way is to keep all elite teams stable and to grow or hire more engineering leaders, leverage the leaders to grow other teams in the organisation. However, hiring good engineering leaders is very hard because they are rare and precious assets to all companies. Therefore, growing them internally is relatively easier. Thus, a learning culture is very important to the success of an organisation who wants to build a platform.
In summary, platform teams need to be very good in engineering practices. They should not only provide tools and services but also spread the best practices, which is usually missing in many organisations.
When do we build our own platform services?
Unlike the products that make money directly, the value of building a platform isn’t straightforward. It is not a wise choice to build it at the beginning with your first product, but building it too late also slows down the process of scaling. So when is the proper time? The short answer is when you feel the pain. Let’s explore more about it.
The desire for automation
Nowadays, there are so many external platforms that can be used by a startup, which allows them to focus on the new product. For example, AWS and Buildkite can easily support the products to build and run. An online CRM software can easily manage sales when the business is small. An online data platform such as Segment can also easily support data-driven decisions at the beginning of a company. So everyone will choose the external platforms when starting a new business.
However, on the one hand, these platforms are pretty much very standard. They target a large number of users to serve their common needs. So they provide a lot of standard functions but normally lack the capability of automation for complex business rules, which are more required for large organisations. ( eg. building a service mesh as the infrastructure in some organisations to take care of the automation of authentication, or building a billing system to take care of the automation of invoicing and payments, as well as financial reports) On the other hand, as the size of a company increases, the complexity of the business also increases. Running a business like that manually isn’t just expensive but also slow, thus the desire for automation will grow as the size of the organisation grows.
So as the business grows, the existing platforms will not be able to meet the requirements at some point. Those points are the opportunities to start building our own platform services.
Return on investment (ROI)
Another opportunity is when we feel the external platforms are too expensive. Most platforms are charged by usage. As the organisation grows, the cost will be significant. So sometimes, it makes sense to build and maintain internal platform services to replace the external ones. But how can we know the investment is worth it?
The key to answering the question is to understand the cost of building. The cost of saving is normally easy to find out because it is about the current state but the cost of building is uncertain. Fortunately, we can refer to the existing market (or other companies) to estimate the cost. Let’s have a look at how.
The reason the external platforms exist is that there are a large number of users in the market, and they have the same needs. But at the same time, there will be more competitors if they are cheap to build. So we can just estimate the cost by the number of similar platforms in the market.
So based on this, we wouldn’t choose to build an internal platform for a cloud infrastructure because there are few competitors (AWS, Azure, Google Cloud) and they are all large and specialised, which indicates it is very expensive to build.
The movements of platform services
So far, we haven’t talked about the shared features. It is because shared features are evolved from the features of products. The criteria to move them into the platform land is the degree of standardisation. The platform services are normally highly standardised because they serve more users than other products.
When the first product is created, there isn’t shared features since we only have one product, so all the features are part of the base product or add-ons. None of them is highly standardised at this moment.
After the second product starts to be built, the common features (such as sign up and sign in) are identified. We start to extract those features into the standardised integrated services and move them into our platform.
At the same time, as the organisation grows, we start to look for our special needs on the sales and reporting processes, especially to make them fully automated so that the organisation can scale in speed. However, most standardised software can’t support this level of automation. So we have to build them on our own.
As I mentioned earlier, some services such as infrastructure or a high-performance data platform are too expensive to be built by ourselves. Using others always has a better ROI than creating our own platform. So they should be left as external unless you are a cloud provider. In that case, the cloud is actually your product.
Summary
The platform services are evolved by the special needs and ROI. Large organisations need to build good engineering teams around those services so that they are easy to be consumed and well maintained. The quality of the platform services decides the speed of the organisation. It is the key to the success of scaling in a large organisation. There is no way to deliver a feature in one day if a deployment requires a week. So measure your team performance and make sure the high quality of your platform teams.
