Platform for Enterprise Transformation | 4- The Platform
In this article I will explain the Platform we built in Garanti BBVA starting with “What is the Platform”.
What is the Platform
The Platform is;
- multi-tenant,
- cloud ready,
- technical (non business),
- built on open source technologies,
- based on container architecture,
- by design scalable and highly available, deploying business applications into two regions in an active-active strategy,
- offering a wide range of services in the Channel (Front-end), iPaaS (Integration), aPaaS (Application), DaaS (Data), Security, DevOps, Logging & Monitoring and IaaS layers,
- enabling Domain Teams to develop a wide range of applications, secure and faster.
I will explain the Platform in the following 5 points.
- Architecture Principles
- Services
- Productivity Products
1. Platform Architecture Principles
The Platform is built on a set of Architecture Principles. These are the guiding principles where all the decisions are based on. There are a total of 63 Architecture Principles including all the Platform Layers. The following are the 8 top level Platform Architecture
- API Driven — API First approach is to enable Domain Team designers to think about the services their products will expose. The designers published these API definitions in the Platform’s API catalog for the Domain Teams discover and reuse these APIs. This enables well defined APIs and minimizes change requests to the APIs.
- Coexistence — is key to the success of the transformation onto the platform. Coexistence is supported at the;
- Channel Layer for business users and customers to seamlessly use both Platform and existing front-end applications.
- iPaaS Layer for applications on the Platform and the existing applications to use/call each other via REST services over API GW and Direct Calls to Mainframe.
- DaaS Layer for exchanging data between the Platform and existing informational systems in near-real-time data replication, event-based messaging, stream processing and ETL. - Event Driven — this is a key principle in the selected re-imagine transformation strategy (3- How we Enabled Transformation). The business processes need to be redesigned in order to benefit from the event driven architecture.
- Cloud Ready — the Platform is ready for the Public Cloud. Once the Platform is prepared on the Public Cloud, the only change required is to update the deployment location in the DevOps to the Public Cloud and the applications will run on the Public Cloud with no change.
- Scale and Elasticity — the Platform is based on container architecture, therefore the system will scale elastically based on the load requirements.
- Open Source — technology is rapidly changing and we use open source technologies and avoid lock-in and re-development of business domain team applications by encapsulating technologies. These open source technologies come with enterprise support.
- Automation — The platform has to be used by the architects to design, developers to develop and build and SREs to deploy and monitor and the Platform teams to provision new capacity. All these activities are automated.
- Deep Security — security is built into the Platform throughout all the layers and not left to the application developers. The security cannot be modified cannot be switched off.
2. Platform Services
The Platform has 40+ services in 8 layers and these services are growing in number and features.
What is a Platform Service
Platform Services are the Products of the Platform. All the services provide a unique set of elements. This ensures that all the services are production ready to be used by the Domain teams in their development. All the concerns like multi-tenancy, scalability, disaster recovery, standards, tools and more have been thought through. This leaves the developers to concentrate on their business product development.
It also simplifies the process when it comes to being compliant with the regulations. Instead of reviewing each individual application, the Platform Services are reviewed for regulation compliance.
The following are the set of elements provided by the Platform Services;
- Title — Service Name, Owner Team, Service and Resource Icons — aimed at the Domain Team Architects & Developers and SREs
- Specification — Benefits, Use Cases, Features, Design Pattern, Standards, SLA, Certifications — aimed mainly at the Domain Team Architects and Platform Teams
- Developer Productivity — Self Services, Catalogs, Developer Tools, Diagnose Tools, Developer Guides — aimed mainly at the Domain Team Developers
- Deliverables — Runtime Services, SDK — aimed mainly at the Domain Team Developers
- Architecture — Architecture Overview — aimed at Platform Teams
- Technology — 3rd Party Tools/Products — aimed at Platform Teams
The following sections explain the Platform Services. Instead of explaining each Platform Service in detail, the capabilities provided by the layers are described and the Platform Services providing these capabilities are associated.
2.1. Channel
The channel layer enables the developers to develop Single Page Applications (SPA) for Employee/Customer Facing Mobile and Web Applications as well as Web Site & Portals. It provides its capabilities in 8 category groups.
- Graphics Components — provides the responsive web components, form elements, styles, layouts, CSS preprocessors capabilities to develop SPA applications.
- Platform Services: Channel Service - Validations — provides form, data type and application validation capabilities.
- Platform Services: Channel Service - MVC Client Side- provides model, view and controller client side capabilities with client side accelerator capabilities.
- Platform Services: Channel Service - Progressive Web App — provides storage, websockets and service worker capabilities.
- Platform Services: Channel Service - Coexistence — provides interaction between existing and platform applications capability for employee and customer mobile applications.
- Platform Services: Device Manager Service - Common Services — provides configuration, error management, authentication, security, multi-language, api invoker, logging, CMS and analytics capabilities.
- Platform Services: Channel Service - Mobility- Provides container and native capabilities.
- Platform Services: Channel Service - Performance- provides server side rendering, web workers, state management capabilities.
- Platform Services: Channel Service
2.2. iPaaS (Integration)
The iPaaS layer provides Platform Services for 4 iPaaS Capability Groups.
- Gateway — provides external, open api and platform api gateway, security mediation and throttling capabilities. It provides synchronous ingress into the Platform via secure REST services.
- Platform Services: API GW Service - Integration — provides routing, error handling and template based development capabilities. It does not provide orchestration, message or protocol transformation to maintain a lightweight API GW.
- Platform Services: API GW Service - Management- provides API governance capability, including API lifecycle management.
- Platform Services: API GW Service - Mainframe Integration — provides two-way integration to Mainframe capability.
- Platform Services: Mainframe Direct Connect Service
Both the API GW Service and the Mainframe Direct Connect Services are using custom built generators therefore the developers do not have to do any development.
2.3. aPaaS (Application)
The aPaaS layer provides Platform Services for 3 aPaaS Capability Groups.
- Online — provides chain of responsibility, security, cross services, orchestration, communication and resilience capabilities. It provides a framework for developing and running container based application services. Java and Python are the current languages supported, where Python is aimed at using machine learning models. The platform is designed with technology agnostic architecture principle and therefore new languages can be easily introduced.
- Platform Services: Online Service, Online Python Service - Batch — provides batch processing capability to develop and execute Batch jobs in the container environment.
- Platform Services: Batch Service - Process Automation — provides process engines, visual components, communication, security capabilities to execute the business processes in the container environment.
- Platform Services: Process Automation Service
2.4. DaaS (Data)
The DaaS layer provides Platform Services for 5 DaaS Capability Groups.
- Data Persistence- provides data access capabilities to datasources by connectors.
- Platform Services: Relational DB Service, NoSQL Service, Search Service, Caching Service, Distributed File System Service - Stream Processing — provides stream processing engine capabilities, providing acquisition, events hub, processing layer, low-latency storage and access layers.
- Platform Services: Stream Processing Service - Messaging — provides queue and topic messaging capabilities.
- Platform Services: Messaging Service - Data Integration — provides ETL and near-real-time data replication capabilities.
- Platform Services: CDC Service, Data Integrator Service - Data Governance — provides data metadata and lineage capabilities.
- Platform Services: Metadata & Lineage Service
2.5. Security
The Security layer provides Platform Services for 4 Security Capability Groups.
- Identity Management — provides identity governance, access request and provisioning and privileged access management capabilities. The first two capabilities are provided by existing systems and only privileged access management capability is provided by the Platform.
- Platform Services: Secret Management Service - Application & Service Security — provides 3 sub group capabilities;
* Authentication — provides authenticators, session management, token management, central sign on (CSO), multi-factor authentication (MFA) capabilities for both customers and employees.
* Authorization — provides fine and coarse grained authorizations to validate the authenticity of tokens (OAuth2 and OpenID) and determine if the user has the privileges to run a service.
* Secure SDLC — provides Security Automation Testing integrated into the Jenkins pipelines and secure code training capabilities.
- Platform Services: Employee Authentication Service, Employee Authorization Service, Customer Sign On Service, Customer Password Service, Customer Preferences Service, Security Parameter Service, Customer Device Binding Service, Captcha Service, Security Picture Service, SMS OTP Service, Security Automation Testing Service - Data Security — provides 2 sub-groups of capabilities;
* Data Encryption — provides general purpose usage online cryptographic utilities and payment crypto functions for encryption of data in motion and Transparent Data Encryption (TDE) for data at rest on datasources.
* Database Audit — provides audit logs for user behavior on datasources.
- Platform Services: Cryptography Service, Relational DB Service, NoSQL Service, Distributed File System Service - Platform Security — provides 3 sub-groups of capabilities;
* Image Security — provides Image Vulnerability Management, Software Composition Analysis, Backdoor Analysis, Automation Integration capabilities.
* Network Security — provides Network Policies, Namespace Isolation, Traffic Encryption, Network Filtering capabilities.
* Runtime Security — provides intelligent Policies, Behavioral and Reactive Threat Protection , Resource Quotas capabilities.
- Platform Services: Vulnerability Management Service, Network Security Service, Runtime Security Service
2.6. DevOps
The DevOps layer provides Platform Services for 3 DevOps Capability Groups.
- Application Lifecycle — provides the application management processes needed for application from development to production.
- Platform Services: Application Lifecycle Service - Continuous Integration & Continuous Delivery — provides the pipelines for building, testing and deploying applications from Dev, Test, QA and to Production environments for;
* Channel Layer — frontend SPAs,
* iPaaS Layer — API GW Service and Mainframe Direct Connect Services,
* aPaaS Layer — Online, Batch and Business Process Services
* Daas Layer — Streams Services
- Platform Services: Continuous Integration & Continuous Delivery Service - Automation Testing Management — provides test automations and incorporates these into the Jenkins pipelines. These test automations are for Unit, Integration , Functional, Performance and Security.
- Platform Services: Automation Testing Service
2.7. Logging & Monitoring
The Logging & Monitoring layer provides Platform Services for Logging & Monitoring Capability Groups.
- Logging — provides centralized logging capability, aggregating and enabling traceability logs from all the Platform layers. Users can search and analyze the logs by using a single application. Application, operation and audit logs are supported. This layer also provides developers to create technical and business metrics.
- Platform Services: Logging Service - Monitoring — provides the capability to monitor the state of the systems and applications of the entire organization (i.e. the Platform and existing systems) via dashboards and alerts.
- Platform Services: Monitoring Service
2.8. IaaS
The IaaS layer provides Platform Services for 3 IaaS Capability Groups.
- Underlying Infrastructure — provides Compute, OS, Storage, Network, Regions and Zones capabilities.
- Platform Services: Container Platform Service, Load Balancer Service, Container Storage Service, Virtualization Platform Service - Container as a Platform — provides container platform, platform networking, platform storage, platform registry capabilities.
- Platform Services: Container Platform Service, Load Balancer Service, Container Storage Service, Virtualization Platform Service - Management — provides automation and orchestration capabilities.
- Platform Services: Container Platform Service, Load Balancer Service, Container Storage Service, Virtualization Platform Service
3. Platform Productivity Products
The platform provides a set of products to improve the productivity of the users of the platform; the architects, developers, analysts and SREs.
- Platform Portal — is a single point of contact for all information of the Platform — news, delivery notes, platform service definitions, architectures, standards, policies, developer guides and more.
- Platform Console — is a single point of contact for all the self-services provided for the developers, SREs and Platform teams.
- Catalogs — for channel, integration, messaging, contain the reusable elements created by the Platform and Domain Teams. They contain web components, API definitions, aPaaS Service GRPC Proto definitions, event definitions, business process definitions, Mainframe Direct Connect definitions.
- Architecture Icons — these are a set of icons for the Domain Team Architects to create architectures that will later be used as an input into the Platform Studio tool. They standardize architecture drawings, easy to understand covering all the required architecture concerns and provide an efficient way into development.
- Developer Studio — The Developer Studio was initially explained in 3- How we Enabled Transformation. It is one of the main tools to speed up development, ensuring the initial designed architecture is being developed.
Conclusion
The Platform was developed to handle the Organization Challenges and for its Benefits to the organization (see 1- Why we Built the Platform).
Organization Challenges:
- Business Challenge
- Talent Challenges
- Technology and Trend Challenges
- Architecture Challenges
- Operation Challenges
- Regulation Challenges
Benefits:
- Platform Services
- Multi Tenant
- Productivity
- Cloud Ready Architecture
- Open Source
- Full Automation
- Continuously Evolving Modern Platform
- Abstract Changes
- Deep Security
- Built in Standards and Policies
We are observing that the Platform has started to handle these Organization Challenges and getting the rewards of the Benefits. We are also observing further benefits. The main benefit is coming from the change to the culture of the organization.
What is Organizational Culture — “The way we do things around here.”
Culture Change
The changes comes in the way we innovate and how the teams are working individually and across the organization.
Innovation
The Platform has all the teams necessary to incorporate new technology, new products into the organization. There is a well defined way of working and what a Platform Service must provide. Therefore introducing new technology, new products are being developed and delivered efficiently. The Platform is also open to everyone in the organization to contribute. Therefore it has become a center for innovation.
Empowered Teams
This applies to both the Platform Teams and the Domain Teams. The important points are:
- Ownership — teams are owning and we are helping the teams own both the technical and the business side of their products. We have well defined Products for the Platform Teams (i.e. Platform Services) and Domain Teams (i.e. Business Products as defined by our Application Architecture Model — see 3- How we Enabled Transformation).
- Share Information — teams create, communicate and share information. All the information is made public to the organization.
- We are encouraging teams to create information and then communicate and publish this information. We have created structures to enable teams to communicate.
- We are discouraging individuals from providing information by email. We have a communication platform, the Platform Portal, we encourage them to publish in this Portal.
- We work on correcting individuals from talking to the people they know for information. We remind them of the portal and structures for communication and access of information. - Cross Functional Product Teams — we have built Platform teams to develop all their components with minimum dependency to other Platform teams and even non Platform teams. Teams have better ownership and better knowledge of all aspects of their Platform Services and work more productively.
- Decision Making — there are 2 points. The first one is who is making the decisions, the second is are we taking decisions. To resolve the first point, we are encouraging the Domain teams to take their own decisions and bring these decisions for review. For the second point, we review their architectures and ensure they are taking decisions and publishing them.
We have found these changes have improved the working environment. The teams that were working in Business Units are now connected across the whole organization. This has resulted in better communication and sharing of information.
The culture change is very fragile, it has a tendency to fall back into its previous state. Therefore it needs continuous support from everyone, especially the leadership to maintain and improve the culture.
Where are we now
We are 3 years into the Platform Evolution journey;
- We are working on the 4th release of the platform, with 40+ platform services and growing.
- The transformation is in its 8th wave and the Platform is handling 210 million API calls per day and also growing.
Where do we go from here
- The Platform evolution continues. We will continue to add new Platform Services and new features to these services. We can go as far as, all products including PaaS and SaaS purchases in the organization, that will be used by more than one business unit can be and should be defined as a Platform Service, to benefit from the values a Platform Service provides. Also, it will continue to provide the capabilities and the capacity required for the Transformation. And finally, since the Platform evolution continues, there will be no new future platform that will supersede this one.
- The Transformation journey continues. We will continue the Transformation journey and provide the business outcomes.
- The Culture change continues. We will continue to work on to improve our culture.
Articles
- Platform Overview
- Organization Challenges
- Benefits provided by the Platform
- How to avoid the downsides of a platform
- Building Blocks
- Success Factors
3. How we enabled Transformation
- Transformation Strategy
- Plan
- Execution
- Measure Progress
4. The Platform
- Architecture Principles
- Services
- Architecture Overview
- Technologies
- Productivity Products
