Automating Existing F5 Load Balancers with HashiCorp Consul
In today’s enterprise datacenters and cloud environments, may it be on premises or in a public cloud, agility within most compute and development infrastructures is a given. Many companies have already adopted DevOps practices including “Infrastructure as Code” to deploy, scale, and adjust their environments.
Unfortunately, the aforementioned is in many cases only true outside of the classical networking realm. Many networking operation teams still operate networking equipment which is hard to automate due to missing API interfaces or equivalent tooling, hence having the need to follow a ticket-based approach to adjust the networking infrastructure to what is requested by application developers or server infrastructure teams.
Recently HashiCorp and F5 gave a webinar which highlighted the F5 BIG-IP Integration with HashiCorp’s Service Discovery tool Consul to circumvent those manual configuration changes and to automate server pools based on what is registered in Consul’s Service Registry
This webinar focused on a greenfield approach, where basically everything an application needs to have configured within an F5 BIG-IP load balancer gets implemented by means of AS3 (F5 Application Services 3 Extension), including tenant, virtual server, etc.
This approach is helpful if one wants to deploy a new application from scratch. But what if there is already an existing configuration on an F5 load balancer, like a virtual server, iRules, etc. which was not created by means of AS3? Does this all need to be re-created through AS3 to get the benefits and deployment speed out of the F5-Consul integration?
As one can tell by the title of this blog post, the answer is:
“No, you do not need to re-create all of what is already configured in your F5 BIP-IP through an AS3 declaration to integrate with HashiCorp Consul.”
And as the first iteration of the joint webinar of F5 and HashiCorp only focused on the greenfield approach, I decided to explain how one can integrate and automate existing brownfield F5 load balancers with HashiCorp Consul, which might be a lower entrance hurdle into automating an F5 load balancer with Consul than re-defining all existing configurations as AS3 declarations.
Within an already existing F5 brownfield deployment one can assume, that at a minimum, there are already virtual servers configured with their respective backend pools. Those backend pools typically are maintained manually by the F5 load balancer administrator who gets information around updating them in case new backend servers are available or have been decommissioned by a ticket-based system or via email.
With these manual processes, it may only take seconds or minutes, depending on the underlying technology, to deploy or scale an application backend. But what is this speed of deployment good for, if it then takes another week or even longer until actual user traffic hits these newly deployed instances and they can start serving customer traffic?
To get rid of those error-prone manual tickets or email-based processes, HashiCorp and F5 offer an integration allowing F5 load balancers to automatically reconfigure their backend server pools based on what is registered in Consul’s Service Catalog.
The following steps will show how this integration can be introduced within an already existing environment.
Let’s assume an F5 load balancing setup with a virtual server (and possibly a bunch of associated profiles) connected to a manually created and configured server pool.
As reconfiguring all of those virtual servers, profiles, etc. as AS3 declarations can be pretty cumbersome, the goal is to keep everything of the current configuration as is except migrating to a server pool which is dynamically reconfigured by means of the F5-Consul integration.
One obvious prerequisite to leverage the F5-Consul integration is to have a running Consul environment which can be used for Service Discovery. As I assume readers of this blogpost have a good understanding of HashiCorp Consul, I will not go into the setup details here, but if one needs to learn how to set up Consul, all the required information can be found in the HashiCorp Learn Portal.
Another prerequisite to leverage this integration is to install the AS3 extension on the F5 load balancer itself, which can be found here.
As soon as this extension is installed on the F5 load balancer, one can go ahead and send AS3 declarations to the AS3 API of the load balancer e.g. via the Terraform BIG-IP provider or with tools like cURL or Postman.
To only create a server pool which gets automatically populated with members based on which endpoints are registered in Consul’s Service Catalog only a relatively simple AS3 declaration is needed:
{
"class": "ADC",
"schemaVersion": "3.7.0",
"id": "Consul-Pool",
"label": "Shared_pool",
"Common": {
"class": "Tenant",
"Shared": {
"class": "Application",
"template": "shared",
"webapp-pool-consul": {
"class": "Pool",
"monitors": [
"http"
],
"members": [
{
"servicePort": 80,
"addressDiscovery": "consul",
"updateInterval": 5,
"uri": "http://CONSUL-IP:8500/v1/catalog/service/nginx"
}
]
}
}
}
}This declaration will create a shared pool within the Common partition of the F5 load balancer which will be named webapp-pool-consul.
To upload this AS3 declaration called as3-declaration.json to a F5 load balancer with a simple cURL command one can use:
curl -s -X POST -H "Authorization: Basic $(echo -n username:password | base64)" -d @as3-declaration.json https://F5-IP:PORT/mgmt/shared/appsvcs/declareThe F5-Consul integration will query Consul’s Service Catalog API every 5 seconds to discover all registered service endpoints for service nginx (in this example) and will populate the created pool automatically with those endpoints.
The new server pool will appear in the F5 load balancer and will automatically have all endpoints as members which are currently registered in Consul.
Migrating to this new Consul automated pool can be done by simply changing the default pool within the virtual server’s resources to the newly created webapp-pool-consul pool:
The next time the application scales up or down and service endpoints register themselves within Consul, there is no need anymore for a ticket or an email to the respective F5 load balancer administrator, as the F5-Consul integration will take care of adjusting the respective server pool automatically within seconds instead of days or weeks.
You can find all required Terraform code to set up a HashiCorp Consul and F5 environment and test out all things described here in the corresponding HashiCorp GitHub repository which also includes a complete section around the brownfield migration approach discussed here.
