Open in app

Sign in

Write

Sign in

Highly available applications using IBMs Cloud Foundry PaaS — Part 2: In practice

By Simon Moser, Richard Johnson and Fred Tucci

Simon Moser
Highly available applications using IBMs Cloud Foundry PaaS
10 min readAug 21, 2018

--

So … you read part 1 of this story and are curious enough to try it out.

Note: You will need your own domain in order to complete this tutorial, i.e. a domain you have control over such you can (temporarily) delegate DNS. If you already own a domain that you can delegate DNS, just proceed, otherwise go and get one at a service like godaddy.com.

First thing you need is an account on IBM Cloud so go to https://console.bluemix.net/ and sign up for a free account if you don’t already have one. Be sure to also obtain the ibmcloud command line tool from here

As we will be using US-East and EU-DE as our regions, make sure that you are connected to US-East after your login. So in a console window, type

ibmcloud login -a mccp.us-east.bluemix.net
ibmcloud target --cf

You should now see something similar to this

In case you don’t, you might have to create a space first (by executing ibmcloud cf create-space dev, in order to create a space called dev)

Let’s push an application!

By now we should be all ready to Rock’n’Roll! Next we need an application to push. For this example, we’ll use an asset from the open source Cloud Foundry acceptance test suite as our example. Open a terminal window, and execute

git clone https://github.com/cloudfoundry/cf-acceptance-tests.git

This will download the acceptance test suite onto your laptop, and then change directory into the cf-acceptance-tests folder and descend into assets/dora. “Dora the Explorer” is not only a popular TV show, but also the name of a Hello World style test application (that has a set of useful parameters, see here for a documentation of all the endpoints). Next, we’ll actually push the dora application. In your terminal, whilst located in the assets/dora directory, type

ibmcloud cf push dora_wdc_20180729sm -m 256M

Note that dora_wdc_20180729sm can be any name you choose — we just used a suffix with date and name initials. After a little while, you’ll see something like this on your screen:

As you can see, the app is now available under the URL dora-wdc-20180729sm.us-east.mybluemix.net. So type your own app’s URL into your browser, or in the terminal window type your equivalent of

curl dorawdc20180729sm.us-east.mybluemix.net

The app will respond with Hi, I'm dora.

Let’s scale it to three instances in that region!

Going back to part one of the series, you might remember that we have only 1 instance now, therefore we are located in one AZ only. In order to get better resiliency, let’s scale it to 3 instances. It’s literally as easy as typing

ibmcloud cf scale dora_wdc_20180729sm -i 3

Now, remember we told you that each instance has its own id, and that within a region you’d get a built in load balancer that sprays the requests across the instances. So if we could make dora reply with its instance ID, we could prove the load balancer is working. Luckily, we can :-) Just type your equivalent of

curl dorawdc20180729sm.us-east.mybluemix.net/id

a few times, and like in the screenshot below you’ll see it responding — our instances have the IDs 732a7e10-e4df-4913–6fa9–2a8f, 133bd88e-8ccc-448a-6c1d-881a and d0598b3a-3885–4255–55ea-106b, respectively.

Let’s push & scale the application again in another datacenter !

OK, now we are going to do the exact same steps in another region — EU-DE. For brevity, just follow the commands (again you can use your own app name):

So let’s summarise where we are: We have a dora app deployed with three instances in US-EAST, and it is reachable under something like dora-wdc-20180729sm.us-east.mybluemix.net, and we have three more instances in EU-DE, reachable under something like dora-fra-20180729sm.eu-de.mybluemix.net.

Let’s add a GLB using IBM Cloud Internet Services

So now we need a global load balancer to spray the requests over the two regions. In order to do so, go back to the IBM Cloud Console UI (https://console.bluemix.net/), and browse the catalog for a service called “Internet Services”.

Click on it, and give it a name (we chose “Simons GLB Service”, but anything will do). Make sure you pick the Free Trial plan.

Note: Once you have the instance created with a `free trial` plan, you will be stuck with it, as every user can only create one `free trial` instance ever, which unfortunately means if you delete it you will not be able to re-create one.

Now, click “create”.

After a few seconds, the GLB Service is provisioned and now we have to configure it.

Follow the five steps in the “Welcome to IBM Cloud Internet Services” section. After getting started, Step 2 will ask you for a domain where we can hook things up to — i.e. a domain that you own and that you want the applications to be running under. In this example, we went to GoDaddy and got ourselves a domain called “multizone-ibmcf.com” — but of course you can use any domain you have already. So in Step 2 of the Internet Service configuration panel, enter your domain and click “Connect and Continue”

The next step (Step 3- DNS) you can skip, but Step 4 Delegate Domain Mangement is very important: For the domain that you have chosen, you have to delegate the name server (NS) records to the Cloud Internet Services Service. In our case, we had to copy the two NS records from this page (e.g. ns011.name.cloud.ibm.com and ns014.name.cloud.ibm.com) and actually insert them into our domain registrar. In our case, we have to log in to godaddy.com and find a section titled “custom DNS” or “DNS server settings” (the section’s title varies from provider to provider). When there, copy both of the new NS records into your provider and save changes.

Returning to the IBM Cloud UI, click on “Check name servers”, as now that test should be working. Once you’ve confirmed it does, in the left navigation menu go to Reliability->Global Load Balancer. There, we create a GLB and named it “dora.multizone-ibmcf.com”, so do something equivalent for your domain. Be sure to turn the proxy button on too.

Next, under “default origin pools”, click “Add Pool” and set that up similar to what’s shown here:

The key here is the “origins” section — as you can see, we defined two origins and pointed them to our two regional URLs — dorawdc20180729sm.us-east.mybluemix.net and dorafra20180729sm.eu-de.mybluemix.net. Next, click “Add” to add the pool and then provision the GLB resource (Click on “Provision 1 Resource” in the main page).

After the GLB provisioning is done, in the main page under “Default Origin Pools”, click on “view details” and verify it looks equivalent to this:

Now, we should be able to curl our domain (dora.multizone-ibmcf.com in our case), and it should forward the requests to our two backends, right? Let’s try it:

curl dora.multizone-ibmcf.com

Sadly, not working. So what are we missing? Well, in order to make this work properly, we have to register custom domains and routes to Cloud Foundry as well because otherwise it won’t recognize what to do with traffic from the custom domain. So for EU-DE, we have to

and then repeat the same steps for US-East

In the end, our setup looks like this:

Basic testing

We now have configured https://dora.multizone-ibmcf.com/ as our GLB domain and pointed it to our two apps in US-East and EU-DE. If we now execute

curl https://dora.multizone-ibmcf.com/id

we expect that all 6 app IDs respond (in no particular order, as we don’t know which region the GLB will choose to routes the request too, and inside a region we don’t know which instance it will get routed to). So let’s test if our assumptions hold true…

First, we need to get the current IDs for Washington. Execute your equivalent tocurl https://dorawdc20180729sm.us-east.mybluemix.net/id few times until you get three IDs (3c38f8f1-e9c9–4959–5582–9462, fd81e61d-2de1–4bcc-671a-0235 and 4137f0dd-4545–4c6d-69e1-ce72 for our example)

Next, execute your equivalent tocurl https://dorafra20180729sm.eu-de.mybluemix.net/ida few times until you get three IDs there, too (0ecdd52b-afe9–4baf-78ee-dcd9,
42a14ade-15c1–4813–6ec9-c104 and 19bea1bd-2eb5–4d5e-4b96–341c for our example)

Then, finally execute curl https://dora.multizone-ibmcf.com/id and observe that now all 6 instances from US-East and EU-DE reply to the request happily.

Resiliency testing

Next, lets simulate to take down a region. We cannot really simulate taking down an “AZ”, as this means we’d have to stop a single app instance, which isn’t possible as Clouod Foundry will automatically restart it! So we’ll stop the entire app (all instances) in one region and then we can expect that curl https://dora.multizone-ibmcf.com/id will only reply with the IDs from the other region

We can verify that now our EU-DE instances are stopped, curl https://dora.multizone-ibmcf.com/id will only reply with 3c38f8f1-e9c9–4959–5582–9462, fd81e61d-2de1–4bcc-671a-0235 and 4137f0dd-4545–4c6d-69e1-ce72

The latin “quod erat demonstrandum” seems an appropriate phrase at this point in time :-)

But wasn’t there a database ? You tried to trick us again!

Remember, in the first part of the series, we said

assume you have an application that has a two-tier architecture, i.e. a web application that is backed by a database

but then we never used a DB in the example. And the reason is that our nice little Dora example above just doesn’t use a database. But we can easily demonstrate how this would work with a Cloudant database, and you could easily use it for your real-world use-case — here are the high-level steps how to get that attached:

  • Go to https://console.bluemix.net/catalog/services/cloudant
  • In the “Choose a region” dropdown select “US-East”
  • As a name, call it “MyCloudant_us_east”
  • Select the lite plan, click “create”
  • Repeat the above steps, exchanging “us_east” with “Germany” in both the drop down and the name

Now you have two instances of the database. For the next steps, we’ll just quote from an article available here

To synchronize the application data across regions, configure each Cloudant database to continuously replicate from the database in each of the other regions.

This can be accomplished manually from the Cloudant dashboard according to the steps below, or it can be automated through the use of this command-line utililty created by the IBM jStart team.

Note: Configuring continuous replication as described in this article will result in frequent API calls between the configured regions. With the default (“Shared”) plan on Bluemix, these calls will count toward the totals on your monthly bill. Consider setting Spending notifications to avoid unexpected charges. Alternatively, consider upgrading to an Enterprise plan that is better suited for the continuous replication feature.

From the Bluemix console, in each region, expand the Show Credentials section of your Cloudant NoSQL DB service tile, and note the usernames and passwords for your each Cloudant account.

After noting the password, click the tile itself and then LAUNCH the Cloudant Dashboard. Each region’s Cloudant instance should have a single database with the name my_sample_db.

In each Cloudant dashboard, open the Permissions view for the my_sample_db database. Grant Replicator permissions for the Cloudant user names in each of the other regions, or use the Generate API key button to create a key/pass combo for each of the other regions to use.

Next, open the Replication section for each region, and configure a continuous replication task from each of the other regions:

This should create a new database in each region with the name “_replicator” and a single document per replication task.

Lee’s article also has a section on testing the database replication that you might want to check out.

With that, we’re at the end of our article, and we’d like to finish up with a little outlook how this might evolve: Currently, the Cloud Foundry community is reworking their routing layer and basing it on top of ISTIO, and once that is available you would be able to create a cross-regional service mesh that would add a bunch of really nice features like A/B testing and circuit breaking patterns into the mix. So stay tuned and watch out for further updates on the subject.

Cloud Foundry
High Availability
IBM
Cloud Computing
Cloudflare

--

--

Simon Moser
Highly available applications using IBMs Cloud Foundry PaaS

Written by Simon Moser

62 Followers

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams