Platform for Enterprise Transformation | 2- How we Built the Platform

Published in

Garanti BBVA Teknoloji

13 min readApr 20, 2022

In this article I would like to talk about how we built the platform here in Garanti BBVA. You may find the reasoning on why we built the platform in my first article “1 — Why we built the platform”.

Platform for Enterprise Transformation | 2- How we Built the Platform (Shutterstock)

I will be describing it in two main groups; Building Blocks and Success Factors.

Building Blocks — The platform build stage is a long and challenging journey where the environment is continuously changing. We needed these building blocks to ensure the platform teams are working towards the same common objective.

Success Factors — These are the list of decisions we have taken that have enabled a successful build.

Building Blocks

1. Discovery

This is the initial phase of the build. This is where we created the Blueprint of the platform that would define the first 18 months of the journey. The Blueprint was created together with the vendor in 8 weeks.

The Blueprint defined the;

Architecture Overview of the platform and its coexistence with existing systems.
Top level Architecture Principles and its Requirements.
For each layer; Architecture Principles and Requirements, Architecture Overview , List of Capabilities, and for each of the Capabilities; Description, Alternative Solutions, Priority, Readiness and Proof of Architecture, Status.
Roadmap ; Release plan.
Teams, and for each team; List of backlogs for each release, Team Structure, Roles and Responsibilities.

The blueprint contained 180 capabilities over the 8 layers (a.k.a disciplines) — the Channel (Front-end), iPaaS ( Integration), aPaaS ( Back-end), DaaS ( Data), Security, DevOps, Logging and Monitoring, IaaS ( Infrastructure) layers

Key Points:

The Blueprint has enabled us to ramp-up the teams in a short amount of time and enabled the teams to start working on their backlogs, knowing what to deliver and when.

2. Roadmap

The platform was built in 2 major phases;

Phase 1 — Platform as a Product
Phase 2 — Platform as a Set of Platform Services

a) Phase 1 — Platform as a Product

In this phase the platform is built as a single product and with all the capabilities of the platform defined in the Blueprint.

Phase 1 — Platform as a Product Release Plan

Discovery: The Blueprint is designed as described above
Rel 0.1: Pre-release for Developers — The developer ready version of the platform to develop an end-to-end application is built. The platform is ready to onboard its first business domain project.
Rel 1: Production MVP — The production ready version of the platform with complete application development life cycle automation, with minimal security and monitoring capabilities is built. The platform is ready to deliver its first project into production.
Rel 2: Full Features — The platform is built with the full blueprint capabilities. The platform is ready for all business domain projects.

Key Points:

Each sub-release is 3 months, dates are fixed and deliverables are published at the end of each sub-release.
Workshop at the end of the first 3 sub-releases was necessary to validate integration of all components.
Later a long-term Reference Application team was established to validate the deliverables (libraries, services and documents) at the end of each sub-release
The production delivery at the end of Rel 1 was delayed as the platform needed stabilization, so the key area of sub-release 2.1 was set to “Stabilization”.
All the platform teams took part in stabilization as we needed to spread knowledge of the platform across all the teams.
A different vendor was invited to perform a complete review of the platform as part of stabilization.
After stabilization, we concentrated on performance of the platform, so the key area of sub-release 2.2 was set to “Performance and Optimization”. We set up a short-term team to run extensive performance tests and defined a set of performance improvement requirements and guidelines.
Onboarded project early on at the end of Rel 0.1. It ensured that the platform deliverables are of high quality, provided valuable feedback to the platform from business domain developer teams and starting creating value to the organization.

b) Phase 2 — Platform as a Set of Platform Services

At the end of Phase 1, the platform had grown too large to add new features, design and deliver them. Therefore we split the system into individual platform services. There were 40+ services at the time of writing the article.

Phase 2 — Platform as a Set of Platform Service Release Plan

Each platform service (a.k.a. product) is defined with the following information group:

Title — contains service name ownership, and icons used by domain teams application architectures
Specification — domains architects are the aimed audience, and therefore contain benefits and use cases of the service, with tools provided to the business and related design patterns and reference architectures, SLAs and certification information
Developer Productivity — aimed to provides information to the developers contains self-services, catalogs, developer and diagnostic tools
Deliverables — contains runtime services and SDKs provided by the service
Architecture — aimed mainly to the platform teams provides architecture details of the service
Technology — again aimed at the platform teams provides 3rd party product information

Key Points:

The platform services enable teams to concentrate on the services independent of the rest of the platform
Each Release had a set of key areas defined as shown in the image above. The aim of the key areas is to set the objective of the release and for the platform teams to work towards a set of common objectives and inform and align with the organization expectations.

c) Sub Release

A sub-release is the shortest period in which a platform delivers its deliverables (where a deliverable can be a library, service, documentation etc. and sub-releases are 0.1, 1, 2.1, 2.2 … 4.3) Each sub release has 5 key activities;

Planning — teams create their plans and they are reviewed jointly with the aim to align all the platform teams. It contains the teams’ release objective, features, backlogs, dependencies, team structure members, architecture overview, capability map, milestone and work products.
Architecture Reviews — teams design their architectures and they are reviewed jointly. It contains their Architecture Overview, Architecture Decisions, Components and Products.
Development — teams develop their deliverables.
Closure — teams create their release notes based on their sub-release planning indicating completed and moved to next sub-release items. The closure reviews are done jointly by all the platform teams.
Sub Release Delivery — teams prepare their deliverables, libraries, services, documentation, delivery notes and deploys them into Non Prod and Prod environments.

Key Points:

In each sub-release we decided to concentrate on a specify key area, for example “Stability”.
Sub-release dates are fixed at the start of a release and published to organization and these dates never change.

2. Governance

We defined a governance structure in order to have effective communication between all the platform teams. The governance structure also evolved over time as the transformation rate increased. This is the core governance structure of the platform and it is based on the “Design Authority Board”.

Key responsibilities of the Design Authority Board are:

Review and approve Architectural Decisions
Review and approve Planning, Architecture and Release Closures
Resolve escalated issues
Review issues, risks and announcements

Key roles are:

Head of Architecture — is responsible for the overall strategy, architecture, delivery and supports the organization’s transformation on to the Platform.
Head of Product — after moving to “Phase 2 — Platform as a Set of Platform Services” defining the Platform Services has become important. The Head of Product is responsible for ensuring the Platform Services Development Lifecycle is performed by all the platform teams.
Head of Technology — is responsible for the overall technology direction of the platform and works closely with the platform teams and vendors to evaluate and incorporate into the platform.

Key Points:

DAB is key to effective communication within the platform teams.
Provides a defined method on how to make decisions, who approves and how they are published.
All planning, architecture reviews, release closures are done in the DAB.
Team escalations are resolved, issues and risks are reviewed and necessary actions are taken.
All information sharing and announcements are brought to DAB.
The platform only has a single Program Manager that manages vendor onboarding and invoicing, contracts, steering committees, gathering and reviewing team reports once per sub-release. The Program Manager has no intra-platform program management responsibilities.
All teams are encouraged to contribute to the DAB, silence is not welcome.
All DAB agenda and meeting notes are published.

3. Platform Operating Model

Governance defines how teams work together, whereas the Platform Operating Model defines how individual teams can work effectively and be productive, with minimum dependency on other teams and how transformation is enabled.

The teams are set up as:

Cross-functional — the teams must be able to design, develop, build, deploy, monitor and operate their own platform services.
Independent — the teams must minimize the dependencies to other teams to deliver and operate their platform services.
Dedicated — the team members must be dedicated to platform tasks.
Change enablers — the teams have change enablers that create the Developer Guides, Platform Onboarding Training content and Reference Applications. They prepare the platform for the Transformation.
Site Reliability Engineering (“SRE”) — teams are performing the majority of the SRE responsibilities of monitoring, responding to incidents, releasing their services, managing capacity, running performance tests and developing automations. This is a work in progress.

Key Points:

Independent teams have increased team productivity. Dependencies between teams have decreased over time. The platform teams still have heavy dependency to the DevOps team and these dependencies are determined at the planning phase. We are moving towards to teams developing their own automations with the principles and guides provided by the DevOps team.
Dedicated teams are more productive and achieve higher quality deliverables. There are teams that also have non platform responsibilities. Besides the extra effort required, it is causing “cognitive load” in the teams. We have made improvements throughout the releases and are continuing to improve to improve dedication.
We currently do not have SREs in the platform team, but have started an initiative to onboard them. The teams are having to put in extra effort for SRE responsibilities. Besides the extra effort required, the development of automations/self-services has not reached the expected level. The benefits of the SREs is not limited o the platform, but also the work carried out here will be used by the business project development team SREs.

4. Team Structure

The platform teams are Agile teams. We have extended these teams to have a Chief Architect lead the team. The Chief Architect may assign a separate Product Owner depending on the workload on the team.

The teams have a ;

Chief Architect — overall lead of the team.
Product Owner — takes on Agile method responsibilities, where the Chief Architect and Product Owner are usually the same person.
Scrum Master, Architects, Developers, Analysts, Specialists, Change Enablers are dedicated to the team where as SMEs are not dedicated and do not take on tasks.

Key Points:

The teams are formed according to the Platform Operating Model.
The platform requires a strong architecture therefore required the lead to be a Chief Architect.
They work with Agile method.
Chief Architects and Product Owners contribute to the DAB.

Success Factors

The build phase has had to handle changes throughout the journey. Some of these changes are;

External factor 1— technologies are evolving.
External factor 2— started working from office, moved to home office and now starting hybrid working.
Internal factor 1 — started onboarding projects and gradually ramped up the transformation.
Internal factor 2 — business domain development teams changing priorities.

These are the list of decisions we have taken that have enabled us to have a successful build.

Success Factor Topics; Teams, Organization, Platform, Architecture, Method, Measuring

1. Team

The platform objectives are always defined in each release and published. The teams are asked to define their objectives in each sub-release.
Teams own Platform Services.
Team boundary is defined in the Blueprint and the platform services and validated in architecture reviews.
Team roles and responsibilities are defined at the start of each sub-release and reviewed during sub-release planning.
Team sizes are defined by Agile method and do not exceed 10.
Disciplines (a.k.a platform layers) have Discipline Leaders. They ensure all teams in a discipline are aligned and take on the role of facilitator for these teams.
Teams are cross-functional, have high cohesion, minimum cross-dependency with other platform teams as defined in the Platform Operating Model.
Every individual is empowered and encouraged to contribute.
Teams accept requirements from domains in quarterly planning only.
Rarely and on critical key area needs, dynamically short-term teams are created.
Teams are created for the long-term, where they own their platform services.
A lesson learnt is not to merge teams when the team size shrunk due to departures. It took us time to recover.
Teams defined in the Blueprint are based on the platform architecture and not organization chart. This enabled teams to be long-term as they remained almost unchanged since the Discovery phase. In the rare case, the organization adapted to the platform team. In a way applied reverse-Conway.
The platform requires teams to understand their architecture concerns, therefore the Chief Architects took training on Architectural Thinking.

2. Organization

Platform worked in a Single-Team spirit with high collaboration, high trust, safe, visible and transparent environment.
Had the continuous support of the sponsors. Things have gone wrong at times, but the teams have raised the issues, kept the sponsors updated and the sponsors gave their full support and had the belief in the platform teams.
Single team success is more important than individual success and individuals are continuously reminded of the platform objectives instead of their own personal interests.
The platform is based on cloud technologies and requires it to be operated with an updated organization model. We have had multiple workshops with vendors and internal discussions to determine the best model. Operating the existing systems and the new platform is a work in progress.
Cross team communication is provided by the Design Authority Board.
Intra team escalations are resolved immediately in the DAB.
Cross team validations are initially done in Integration Workshops and later in Reference Applications.
A Platform Portal exists to share all information — news, delivery notes, platform service definitions, architectures, standards, policies, developer guides and more.

3. Platform

Platform contains a set of Platform Service and are clearly defined (See “Phase 2: Platform as a Set of Platform Services”) and has a lifecycle.

Platform Service Lifecycle

Platform service requirements are quarterly planned and gathered from business domain developer teams, Solution Architects and platform teams. These are published. New requirements during a sub-release are not accepted into the same sub-release to avoid platform team distraction.
Platform services are designed, developed and delivered in a sub-release.
Platform Services are validated in the platform by the Reference Applications.
Platform Services are also validated in the business domains developer teams by piloting.
A new “Head of Product” role was created to handle the switch from a Platform as a Product to a Platform as a Set of Platform Services and the Platform Service Lifecycle.

4. Architecture

Architecture Principles are defined at top level and discipline level. All architectures must comply with the principles and are reviewed during architecture reviews.
All platform service boundaries are defined in the Platform Service Definition and validated in the architecture reviews.
All platform services are asked to maintain their list of architectural decisions. These are reviewed during architecture reviews and DAB sessions and published. Architectural decisions are reviewed in each sub-release.
Architecture is reviewed in every sub-release.
Extensive sets of artifacts are created; key ones are Architectures, Standards, Policies.

5. Measuring

The progress of the platform was initially measured by the following metrics. The Platform Readiness metrics did not provide us value as we were able to validate the platform by Reference Applications. But the surveys gave us the feedback we required.

Platform Readiness — number of capabilities completed
Platform Readiness — number of backlog completed
Platform Quality — number of blocker/major issues
Platform Satisfaction — satisfaction survey metrics

We are planning to move to platform service based metrics, where we can ensure the platform is providing value to the organization and encourage platform teams to work closer to the business domain development teams and the services are delivered with high quality;

Platform Improvement — number of new services / features
Platform Utilization — number of business products using service / feature
Platform Quality — number of issues raised per service
Platform Performance — throughput, response times
Platform Satisfaction — satisfaction surveys metrics
Platform Delivery — velocity

6. Method

Teams select one of the Agile methods of Scrum and Kanban that is best suited to their work.

Conclusion

Building a platform is a long challenging journey. We are in Release 4 and still continuing. In our journey, some of the challenges we faces were, having a large number of platform teams, high organization expectations, large number of capabilities to build, evolving technologies, changing business domain priorities, vendor in different time-zone, changing working models, ramping transformation.

We found that setting up the building blocks and applying the “the success factors” and continuously adapting to the changes ensured that we were set up with established teamwork, empowered individuals and effective communication resulting in solid platform architecture.

Teamwork — Work as a Single-Team spirit with high collaboration, high trust and safe, visible and transparent environment.

Communication — Have a structure in place to make decisions and communicate strategy, plans, architectures, decisions, standards, policies, issues and risks.

“Requiring everyone to communicate with everyone else is a recipe for a mess” — Team Topologies by Matthew Skelton and Manuel Pais
“The main obstacles to improved business responsiveness are slow decision making, conflicting departmental goals and priorities, risk-averse cultures and silo-based information.” — Economist Intelligence Unit, “Organisational Agility: How Business Can Survive and Thrive in Turbulent Times”

Empower — Empower individuals to open communication and sharing information, owning their products, making their decisions and also facilitating them during the journey for them to become leaders.

Adapt to Changes — At the end of “Phase 1: Platform as a Product” we had achieved 70% of the capabilities defined in the Blueprint, dropped 5% and moved 25% into next release. The main reason is that we had to adapt to change in domain priorities and change in platform priorities to stabilize and improve performance. Since the decisions are taken in the DAB and communicated to the domains and sponsors, the platform continued to deliver value to the organization even if it is not what was initially planned.