How to Deploy Pivotal Cloud Foundry on AWS Flawlessly

Installing Pivotal Cloud Foundry on AWS using AWS Quick Start Reference Deployment

At WSO2 we have been developing BOSH releases, Service Brokers and PCF Tiles for WSO2 middleware for automating their deployments on @Pivotal Cloud Foundry (PCF) since September, 2017. On high level, BOSH releases are used for orchestrating the software components on the infrastructure, service brokers are used for integrating provided software services with PCF and PCF Tiles are used for packaging. According to our experience the best approach for implementing BOSH releases is to use BOSH Lite. If the resources required by the deployment cannot be accommodated on a local machine a BOSH director can be installed on an external infrastructure such as VMWare, OpenStack, AWS, Google Cloud, Azure, SoftLayer or RackHD. Service Brokers can also be developed the same way using PCF Dev. However, PCF Tiles can only be developed and verified using a fully fledged PCF environment since PCF Dev does not include Pivotal Operations Manager.

A standard Multi-AZ PCF deployment on AWS would require nearly fourty EC2 instances. This includes 1 NAT instance (t2.medium, user - configurable), 1 Ops Manager instance (m4.large), 1 Bootstrap instance (t2.micro), 1 BOSH instance (m4.large) for the Ops Manager installation and 27 t2.micro instances, 3 r4.xlarge instances, 4 m4.large instances, and 1 t2.small instance for the Cloud Foundry Elastic Runtime (ERT). Recently, Pivotal introduced a light weight version of this called Starter deployment by reducing instances required for the ERT which only require twenty two instances in total. In this article I will explain how a PCF Starter deployment can be created on AWS according to AWS Quick Start Reference Deployment.

Deployment Architecture

Reference: Pivotal Cloud Foundry Quick Start Reference Deployment for AWS [2]

This deployment would create a VPC with four private subnets and two public subnets. The NAT instance, Boostrap instance and the Ops Manager will be installed in one of the public subnets. The BOSH director and the ERT will be created in the first private subnet group and the second private subnet group will be used for creating the RDS instance. Three load balancers will be created for routing SSH, TCP and HTTP traffic. A root or sub domain name registered with a domain registrar will also be needed for accessing the Ops Manager, ERT and applications deployed on ERT. A self-signed SSL certificate can be used for routing SSL traffic for evaluation purposes.

Steps to Follow

1. Login to your AWS account, select US West (Oregon) region, navigate to EC2 section and create a key pair. According to the quick start guide, this may not work in all AWS regions.

2. Check the current service limits in this region and make sure it has the ability to create a minimum of 22 EC2 instances, 5 S3 buckets, 1 db.m4.xlarge RDS instance with 100 GiB of storage, and 3 load balancers for installing PCF components. A higher number in each category would be needed depending on the workloads that you are planning to deploy on PCF.

3. Register a new domain name or create a sub domain (let’s refer this as <domain>) from an existing domain name. When I tried this I created a sub domain name called “pcf” under my existing domain name as follows: “pcf.imesh.io”.

4. Create a public hosted zone in Route 53 using the above domain/sub domain name and update name servers in the domain name registrar dashboard with the values given by Route 53.

5. Generate a self-signed SSL certificate for the above domain using the below bash script:

wget https://raw.githubusercontent.com/aws-quickstart/quickstart-pivotal-cloudfoundry/master/scripts/gen_ssl_certs.sh
chmod +x gen_ssl_certs.sh
./gen_ssl_certs.sh pcf.<domain>

This script will generate two files with the names pcf.<domain>.crt and pcf.<domain>.key supporting following sub domains:

*.pcf.<domain>
*.sys.pcf.<domain>
*.apps.pcf.<domain>
*.login. sys.pcf.<domain>
*.uaa. sys.pcf.<domain>

6. Install AWS CLI by following its official installation guide:

# OSX using Homebrew
brew install awscli
# Linux
pip install awscli --upgrade --user

7. Configure the AWS CLI by providing the Access Key, Secret Access Key and the AWS region:

aws configure
AWS Access Key ID [None]: AccessKeyValue
AWS Secret Access Key [None]: SecretAccessKeyValue
Default region name [None]: us-west-2
Default output format [None]:

8. Upload the above SSL certificate to the AWS Certificate Manager using the AWS CLI:

aws iam upload-server-certificate --server-certificate-name pcf-<domain> --certificate-body file://pcf.<domain>.crt --private-key file://pcf.<domain>.key
# note the ARN in the response:
{
"ServerCertificateMetadata": {
"Path": "/",
"ServerCertificateName": "pcf-imesh-io",
"ServerCertificateId": "ASCAJLAGWERNEHVL5WBRG",
"Arn": "",
"UploadDate": "2018-03-14T08:45:48.014Z",
"Expiration": "2028-03-11T08:43:09Z"
}
}

9. Sign up and login to Pivotal Network at https://network.pivotal.io/ and find the API token by navigating to your profile page. This token will be used by the installation process for downloading the ERT distribution via the Pivotal Network:

10. Open a web browser and visit the below link. This will open up AWS Cloud Formation (CF) console and load the AWS Quick Start Reference Deployment CF template:

https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/new?stackName=Pivotal-Cloud-Foundry&templateURL=https://s3.amazonaws.com/quickstart-reference/pivotal/cloudfoundry/latest/templates/pivotal-cloudfoundry.template

11. Press the Next button on the above screen. At this point select the key pair name, enter the SSL certificate ARN received at step 8 and a prefix for the load balancer URLs:

12. Next, enter a CIDR range for controlling access to the Ops Manager and Bootstrap instances, then select the Route 53 hosted zone created in step 4, and enter the PCF domain name registered in step 3:

13. Next, select the size of the deployment as “Starter”, skip SSL validation as true, Pivotal network token obtained from the Pivotal Network, your email address and a password for the Ops Manager administrative login:

14. Next, enter a username and a password for the PCF RDS instance:

15. Next, accept the End User License Agreement and press the Next button:

16. On the next page add a tag to track the AWS resources created by this CF template if required:

17. Finally review the configuration, check the acknowledgement and press the Create button:

18. This may take nearly 2.5 hours to complete. Watch the status of the deployment on the Cloud Formation Stacks page:

19. Once the stack creation is completed, open a web browser, visit the Ops Manager URL: https://opsman.pcf.<domain> and login with the admin credentials given in step 13:

20. Find the ERT admin credentials by navigating to the Pivotal Elastic Runtime/Credentials page and searching for the text “Admin Credentials”. Then visit the Apps Manager URL: https://apps.sys.pcf.<domain> and login using the admin credentials:

21. Install Pivotal Cloud Foundry CLI by following the official installation guide.

22. Create a new space called “dev” via the Apps Manager UI and deploy a sample microservice:

# clone MSF4J git repository
git clone https://github.com/wso2/msf4j
# build helloworld sample
cd msf4j/samples/helloworld
mvn clean install
...
# push helloworld sample to pcf
cf push hello-service -p target/helloworld-2.5.3-SNAPSHOT.jar
Creating app hello-service in org system / space dev as admin...
OK
Creating route hello-service.apps.pcf.imesh.io...
OK
Binding hello-service.apps.pcf.imesh.io to hello-service...
OK
Uploading hello-service...
Uploading app files from: /var/folders/ly/c07j8bln25q3rpw22dbd_pgh0000gn/T/unzipped-app111196830
Uploading 6.4M, 4974 files
Done uploading
OK
...
1 of 1 instances running
App started
OK
...
Showing health and status for app hello-service in org system / space dev as admin...
OK
requested state: started
instances: 1/1
usage: 1G x 1 instances
urls: hello-service.apps.pcf.imesh.io
last uploaded: Thu Mar 15 09:40:24 UTC 2018
stack: cflinuxfs2
buildpack: client-certificate-mapper=1.2.0_RELEASE container-security-provider=1.8.0_RELEASE java-buildpack=v4.5-offline-https://github.com/cloudfoundry/java-buildpack.git#ffeefb9 java-main java-opts jvmkill-agent=1.10.0_RELEASE open-jdk-like-jre=1.8.0_1...
state     since                    cpu    memory       disk         details
#0 running 2018-03-15 03:12:28 PM 0.0% 3.8M of 1G 1.3M of 1G

23. Make a HTTP request and verify the sample microservice:

curl -v -k https://hello-service.apps.pcf.imesh.io/hello/imesh
...
> GET /hello/wso2 HTTP/1.1
> Host: hello-service.apps.pcf.imesh.io
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept: */*
< Content-Type: */*
< Date: Thu, 15 Mar 2018 10:07:55 GMT
< User-Agent: curl/7.54.0
...
Hello imesh

Conclusion

Pivotal Cloud Foundry is a resource heavy, complex Platform as a Service solution used by large enterprises. It’s complete deployment requires nearly 40 virtual machines just for isntalling its server components. Recently, Pivotal introduced a trimmed down version of it called “Starter” which only requires 22 virtual machines in total. The PCF documentation provides two options for installing PCF on AWS; Manual installation and Terraform based installion. Both options require a series of manual intervensions and a collection of configuration parameters. AWS PCF quick start reference deployment provides three nested Cloud Formation templates which automate the entire PCF deployment on AWS. It takes nearly 2.5 hours to complete and executes smoothly if required resource limits and AWS resource permissions have been granted. Once the installation is completed, additional platform services can be installed via the Ops Manager and applications can be deployed via the Apps Manager and PCF CLI.

References

[1] AWS Quick Starts: https://aws.amazon.com/quickstart/

[2] AWS Pivotal Cloud Foundry Quick Start Reference Deployment: https://s3.amazonaws.com/quickstart-reference/pivotal/cloudfoundry/latest/doc/pivotal-cloud-foundry-on-the-aws-cloud.pdf

[3] BOSH Documentation: https://bosh.io/docs

[4] Installing Pivotal Cloud Foundry on AWS: https://docs.pivotal.io/pivotalcf/2-0/customizing/aws.html