Deploying bosh-lite v2 and Cloud Foundry on AWS

Over the years, BOSH and Cloud Foundry has gone through numerous iterations as part of the product evolution. With that, there has also been some changes in how the software is installed. While there are instructions in the official docs and blog posts (some of them outdated) abound on how to install bosh and Cloud Foundry, getting Cloud Foundry up and running quickly can get somewhat challenging for someone new or the casual hobbyist trying to get their feet wet.

This tutorial walks through the steps to install bosh-lite director using bosh cli v2 and Cloud Foundry using cf-deployment on a single AWS EC2 instance. Once the platform deployment is successful, we will push a sample app using the Cloud Foundry cf cli.

Assuming you have an AWS account, follow the steps below to setup bosh-lite director and Cloud Foundry.

Pave your AWS

1. Obtain AWS credentials (access key and secret key). Where’s my Secret Access Key?
2. Create a VPC using the VPC wizard and choose VPC with Single Public Subnet Scenario. After creating the VPC, note the Availability Zone eg : “us-east-1a” and public subnet CIDR eg : 10.0.0.0/24
3. Create an elastic IP. This IP will be assigned to the bosh-lite director EC2 instance. Note this IP address.
4. Create and configure a security group with the correct inbound rules. Note the security group name.
5. In EC2, create a key-pair . Note the key name and download the key file

Fig. 1 Architectural topology of bosh-lite on AWS

Deploy a bosh-lite director

1. Install bosh cli v2.

2. Create workspace and clone repo.

# Create directory to keep state
$ mkdir bosh-1 && cd bosh-1
# Clone Director templates
$ git clone https://github.com/cloudfoundry/bosh-deployment

3. In a scratch pad, copy the following command options and fill in the appropriate variables specific to your installation and run the “bosh create-env” command.

$ bosh create-env bosh-deployment/bosh.yml \
 --state=state.json \
 --vars-store=creds.yml \
 -o bosh-deployment/aws/cpi.yml \
 -o bosh-deployment/bosh-lite.yml \
 -o bosh-deployment/bosh-lite-runc.yml \
 -o bosh-deployment/jumpbox-user.yml \ 
 -o bosh-deployment/external-ip-with-registry-not-recommended.yml \
 -v director_name=bosh-1 \
 -v internal_cidr=10.0.0.0/24 \
 -v internal_gw=10.0.0.1 \
 -v internal_ip=10.0.0.6 \
 -v access_key_id=AKI… \
 -v secret_access_key=wfh28… \
 -v region=us-east-1 \
 -v az=us-east-1a \
 -v default_key_name=bosh \
 -v default_security_groups=[bosh] \
 --var-file private_key=~/Downloads/bosh.pem \
 -v subnet_id=subnet-ait8g34t \
 -v external_ip=<elastic_ip>
As noted in "Pave your AWS" section above, the following values are used in variables:
internal_cidr: public subnet CIDR range from step (2)
region: AWS region eg: us-east-1
az: Availability Zone from step (2) eg: us-east-1a
subnet_id: public subnet name associated with the VPC
default_security_groups: security group name from step (4) eg: [bosh]
external_ip: elastic IP from step (3)
default_key_name: key name from step (5)
var-file private_key: location of private key file path from step (5)

The above command deploys the bosh-lite director on AWS as a EC2 instance. Check the AWS console for an EC2 instance created inside of the assigned VPC.

Connecting to the director

1. Set the bosh director env alias.

bosh alias-env bosh-lite -e <elastic_ip> --ca-cert <(bosh int ./creds.yml — path /director_ssl/ca)

2. Set the following bosh environment variables.

$ export BOSH_CLIENT=admin
$ export BOSH_CLIENT_SECRET=`bosh int ./creds.yml --path /admin_password`
$ export BOSH_ENVIRONMENT=<bosh-director ip>
$ export BOSH_CA_CERT=`bosh int ./creds.yml --path /director_ssl/ca`

3. Setup ssh tunnel so that bosh cli can access components inside of the 10.0.0.0/24 subnet.

$ bosh int ./creds.yml --path=/jumpbox_ssh/private_key  > jumpbox.key
$ chmod 600 jumpbox.key
$ ssh -4 -D 5000 -fNC jumpbox@<elastic-ip> -i jumpbox.key
$ export BOSH_ALL_PROXY=socks5://localhost:5000

The “jumpbox” user and “jumpbox.key” can now be used to ssh to the bosh-director if need be.

$ ssh -i jumpbox.key jumpbox@<elastic-ip>

4. Verify you can access bosh director.

$ bosh -e bosh-lite env

Deploying cloud foundry using cf-deployment

1. git clone the repo.

$ git clone https://github.com/cloudfoundry/cf-deployment.git

2. Upload the bosh cloud-config.

bosh -e bosh-lite update-cloud-config cf-deployment/iaas-support/bosh-lite/cloud-config.yml

3. Upload a stemcell.

bosh upload-stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent

4. Deploy Cloud Foundry (“cf”).

In order to configure a “system domain” , we will use sslip.io which will give a magic domain based on the public IP address assigned to the director. Substitute <elastic_ip> for the public IP in the command below :

$ bosh -e bosh-lite -d cf deploy cf-deployment/cf-deployment.yml -o cf-deployment/operations/bosh-lite.yml --vars-store deployment-vars.yml -v system_domain=<elastic_ip>.sslip.io

Total duration to complete the deployment on a m3.xlarge EC instance takes about an hour and 12 mins.

Started Mon Feb 5 03:24:31 UTC 2018
Finished Mon Feb 5 04:36:24 UTC 2018
Duration 01:11:53

5. Use various bosh commands to explore the “cf” deployment.

$ bosh -e bosh-lite -d cf vms
$ bosh -e bosh-lite -d cf ssh diego_cell/0

Connecting to cloud foundry using cf cli

1. Connect to cloud controller api

cf api api.<elastic_ip>.sslip.io --skip-ssl-validation

2. Extract cloud controller admin password.

bosh int cf-deployment/deployment-vars.yml --path /cf_admin_password

3. Login to cloud controller using using cf cli.

cf login -u admin -p <password from step 2>

4. Create an org called “foo”.

cf create-org foo
cf target -o foo

5. Create a space called “bar”.

cf create-space bar

Push a sample app to cloud foundry

1. Git clone a sample app.

git clone https://github.com/svennela/spring-music-only-war

2. Target the org and space and push the sample app.

cf target -o foo -s bar
cd spring-music-only-war
cf push

3. Once the app is pushed successfully, the url for accessing the app is now available using the “cf apps” command.

$ cf apps
Getting apps in org foo / space bar as admin…
OK
name requested state instances memory disk urls
spring-music started 1/1 1G 1G spring-music-sixfold-grayling.<domain_name>

For additional reading, check out the Cloud Foundry documentation

Disclaimer : Accessing the bosh director directly via the public IP is not recommended. This tutorial is only for instructional purposes and not to be used to deploy bosh director and cloud foundry for production workloads.

In the next installment, we shall dig a deeper into all of the bosh cli commands and sub-options we used here to gain a better understanding of bosh and Cloud Foundry.