Integrating Platform Services with Pivotal Cloud Foundry
PCF Architecture and Four Levels of Service Integrations
Pivotal CloudFoundry (PCF) is a Platform as a Services (PaaS) solution originally developed by VMWare and later moved to Pivotal Software Inc, a joint venture by EMC, VMWare and General Electric. PCF is the commercial version of the open source Cloud Foundry solution which includes additional commercial features such as the operations manager, enterprise services, extensions, support, docs, etc. The following diagram illustrates its component architecture in detail:
In contrast to Kubernetes, OpenShift, DC/OS and Docker Swarm, which are considered as today’s most widely used open source container cluster management platforms (CCMP), PCF architecture is quite complex and considerably heavy to deploy. For an instance, a typical PCF production deployment would require nearly fifty virtual machines for installing its management components whereas Kubernetes would only require three instances. Moreover, PCF’s own infrastructure management component; BOSH, BOSH’s wrapper component; Operations Manager, the OS image and server runtime abstractions; Stemcells and Buildpacks, it’s own container runtime; Diego, PCF router, are some of the Pivotal specific components that users may need to learn when they start using PCF. Moreover, as I found the community support for these components are comparatively low and at the same time it may require a considerable amount of time and effort for troubleshooting deployment issues.
Nevertheless, if you are just getting started with PCF, PCF Dev can be used for setting up a lightweight PCF environment on a local machine using VirtualBox. This can be used for trying out the basic features of PCF including deploying applications using Docker and Buildpacks, configuring routing, integrating with services, etc. It is important to note that PCF Dev does not include BOSH and Operations Manager. If required BOSH Lite can be installed separately on VirtualBox and Operations Manager is only available on complete PCF installations on AWS, Azure, GCP, VMWare vSphere and OpenStack.
At WSO2 we did couple of evaluations on deploying WSO2 middleware on PCF (without using BOSH/Operations Manager) and found a collection of technical limitations in year 2016 related to exposing multiple ports, service discovery, container to container communication, TCP routing, separating out external and internal routing, etc. Very recently, we started doing another evaluation and found that PCF supports integrating such services iteratively at four different levels depending on how integrations need to be implemented, where service instances need to be deployed and how service instances need to be managed. This article explains details of those four levels and when to use each:
Level 1: User-Provided Service
This is the most simplest way of integrating platform services with PCF. Any software product can be integrated with PCF with this approach using its API without having to implement any extensions or resources to deploy the software on PCF itself. Once integrated, applications running on PCF will be able to bind to a service and programmatically read service information such as API URL and credentials via environment variables. Afterwards applications will be able to consume the service with the given configurations. For an example if a RDBMS is need for applications, it can be registered as an user provided service in PCF as follows:
cf create-user-provided-service SERVICE-INSTANCE -p "host, port, db-name, username, password"Afterwards, applications can bind to the above service as follows:
cf bind-service APP_NAME SERVICE-INSTANCEThis might be the best approach for getting started with a PCF service integration. It would be less time consuming, and may not require implementing any extensions. Nevertheless, it might be worth to note that the only advantage of using this approach would be the ability to inject service configurations to a collection of applications via a central PCF feature without having to do it by manually injecting environment variables to each application. Otherwise, the same can be achieved without using PCF services.
Level 2: Brokered Service
In level 2 with brokered service approach, an extension API needs to be implemented according to PCF Service Broker API for the integration. Unlike in level 1, this approach does not directly expose software product’s API with applications, rather the following service broker API resources would mapped to the services provided by the product:
// Service Broker API Resources:// Return service catalog
HTTP GET /catalog// Return the status of the last operation
HTTP GET /service_instances/{instanceId}/last_operation// Create service instance
HTTP PUT /service_instances/{instanceId}// Bind an application to a service instance
HTTP PUT /service_instances/{instanceId}/service_bindings/{bindingId}// Unbind an application from a service instance
HTTP DELETE /service_instances/{instanceId}/service_bindings/{bindingId}// Delete service instance
HTTP DELETE /service_instances/{instanceId}
For an example, if the platform service is a RDBMS; the create service instance API resource could create a new tenant in an existing database server, the bind an application to a service instance API resource could create a new database in the above server, the unbind an application from a service instance API resource could delete the created database and so forth. The service broker API can be implemented in any language and deployed on PCF as another application. The software product can run outside PCF while ensuring routing between the two environments.
For an example once a service broker API is implemented for a third party software it can be deployed on PCF using a Docker image:
cf push SERVICE-BROKER-API-NAME --docker-image SERVICE-BROKER-DOCKER-IMAGE-TAGAfterwards, the service broker can be registered in PCF by executing the following commands:
cf create-service-broker SERVICE-NAME SERVICE-BROKER-API-USERNAME SERVICE-BROKER-API-PASSWORD SERVICE-BROKER-API-URLcf create-service SERVICE-NAME PLAN SERVICE_INSTANCE [-c PARAMETERS_AS_JSON]
Finally an application can bind to the above service via the following command:
cf bind-service APP_NAME SERVICE_INSTANCE [-c PARAMETERS_AS_JSON]At WSO2 we implemented a Service Broker for WSO2 API Manager using this approach for providing API Management services for microservices deployed on PCF. The service broker API was implemented in Ballerinalang and it can be deployed on PCF using Docker. Refer the README.md and the source code in the above repository for more information. The main advantage of level 2 over level 1 would be the ability to automate the service functionality binding with the applications without having to implement logic in the applications for specifically invoking the product API.
Level 3: Managed Service
Level 3 is much similar to level 2 except that third party software is also deployed on PCF using a PCF Tile together with its brokered service API. A PCF Tile provides a packaging model and a deployment blueprint for executing software installations on PCF by creating virtual machines, containers, managing configurations, networking, routing, etc. It is designed to be deployed via the Operations Manager and once deployed it will generate a BOSH release and execute the deployment via BOSH. Pivotal provides a tool called PCF Tile generator for implementing tiles by generating the required folder structure and the tile definition.
Tiles allow software to be deployed on infrastructure platforms supported by BOSH either using virtual machines or Docker. It provides features for defining resources configurations (CPU, memory, disk), routing rules for load balancing, dependencies between tiles for installing dependent components, BOSH errands (scripts) for executing deployment commands including pre and post validations, etc. Since managed services only create one deployment of the given software at this level, this approach might only be suitable if multi-tenancy is supported by the third party software itself or if tenancy is not needed at the PCF context.
Level 4: On-Demand Service
The main feature provided at level 4 is the ability to create single tenant, dedicated deployments of third party software for each service binding. If PCF is used in multi-tenant mode and if the third party software does not support multi-tenancy, an integration at level 4 would be needed for creating separate deployments for each PCF organization or space. Unlike in level 2 and 3, in level 4 a service adapter will need to be implemented using the On-Demand Services SDK. As shown in figure 5, On-Demand Broker (ODB) handles all interactions between the Cloud Foundry and BOSH. ODB make use of the service adapter for handling service specific tasks.
Summary
PCF provides four different levels for integrating platform services with applications running on PCF. At the initial level, third party software products can be directly integrated with PCF using product API without having to implement any extensions. At level 2, a service broker API needs to be implemented for mapping product functionality to service binding workflow. At this level the given software can be run outside PCF and would not need any deployment automation specific to PCF. At level 3 both service broker API and the software product will be deployed on PCF using CF Tiles. Here the software product will only have one deployment for all service instances. At level 4, each service instance will get a dedicated deployment of the third party software providing multi-tenancy with isolated deployments.
References
- PCF Documentation: https://docs.pivotal.io/
- A guide to using BOSH: http://mariash.github.io/learn-bosh
- BOSH: http://bosh.io/
