Centralized inbound gateway options for Azure Container Apps — ACA with AppGw (Terraform sample)
Azure Container Apps (ACA) has been seriously gaining traction and with my team, we are seeing a majority of our start-up customers choosing ACA to reduce operational complexity while still having a very capable and scalable platform for running their containerized applications.
Centralized inbound gateway options for ACA
A common architectural pattern, adopted in a number of production environments I have seen so far, is running ACA in a custom VNET with an internal accessibility level (private mode) fronted by a centralized inbound gateway.
There are multiple good products that you can choose to control the inbound traffic to your ACA environment. A good starting point to explore the available services in Azure is the Load-balancing options article in the Azure Architecture Center. In most cases, the selection of Azure services reduces to using either Application Gateway (AppGw) or Azure Front Door.
In this article, we will explore the integration between Application Gateway and Azure Container apps.
Configure Azure container Apps with Application Gateway
Before we start configuring the AppGw, we must be aware that there are two main options for doing this.
a) Default ACA domain
The first option is to use the default ACA domain and make sure that you override the host name in AppGw.
This option is simpler to configure but comes with a serious risk of your application having various problems since it will not see the original host name. The problems of not preserving the original HTTP host name between a reverse proxy and its back-end application are very well explained in this article in the Azure Architecture Center, please read it before you decide to do this.
To see a working example of this setup please check this git repository:
gjoshevski/aca-appgtw-default-domain (github.com)
b) Custom Domain
This configuration is recommended for production-grade scenarios and meets the practice of not changing the host name in the request flow. You are required to have a custom domain (and associated certificate) available to avoid having to rely on the default domain.
To see the entire source code of this setup please check this git repository:
gjoshevski/aca-appgtw-custom-domain (github.com)
Bellow, I’ll walk you through some of the aspects of the setup process:
1. Domain Name and SSL cert setup
To create our Azure Container Apps Environment with a custom domain, besides the domain we also need a valid certificate.
In this example, our DNS records are stored Azure DNS.
To simplify the setup and generate the required certificate let's use the ACME Certificate and Account Provider Terraform provider.
The snippet below will create and manage accounts on an ACME server as well as create the TLS certificate.
resource "tls_private_key" "private_key" {
algorithm = "RSA"
}
resource "acme_registration" "reg" {
account_key_pem = tls_private_key.private_key.private_key_pem
email_address = var.email_address
}
resource "acme_certificate" "certificate" {
account_key_pem = acme_registration.reg.account_key_pem
common_name = var.domain_name
certificate_p12_password = var.cert_password
dns_challenge {
provider = "azure"
}
}Bear in mind that this is an example, and we use a Let’s Encrypt’s staging server endpoint, which is not suitable for production use and will result in ERR_CERT_AUTHORITY_INVALID.
For production use, change the directory URLs to the production endpoints, which can be found here.
2. ACA Setup
The next step is to create our ACA environment and application. For this, we are going to use the azurerm provider which from v3.43.0 supports ACA.
In the snippet below you can see that we are providing the infrastructure_subnet_id and internal_load_balancer_enabled arguments, which will result in creating the ACA environment into our own VNET in internal mode.
You can also notice that we are providing the custom_domain block and referencing to the p12 certificate we created in the first step.
resource "azurerm_log_analytics_workspace" "main" {
name = format("%s-%s", var.prefix, "logs")
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
sku = "PerGB2018"
retention_in_days = 30
}
resource "azurerm_container_app_environment" "main" {
name = format("%s-%s", var.prefix, "env")
location = azurerm_resource_group.main.location
resource_group_name = azurerm_resource_group.main.name
log_analytics_workspace_id = azurerm_log_analytics_workspace.main.id
infrastructure_subnet_id = azurerm_subnet.aca.id
internal_load_balancer_enabled = true
}
resource "azurerm_container_app" "main" {
name = "example-app"
container_app_environment_id = azurerm_container_app_environment.main.id
resource_group_name = azurerm_resource_group.main.name
revision_mode = "Single"
template {
container {
name = "examplecontainerapp"
image = "mcr.microsoft.com/azuredocs/containerapps-helloworld:latest"
cpu = 0.25
memory = "0.5Gi"
}
}
ingress {
allow_insecure_connections = true
external_enabled = true
target_port = 80
traffic_weight {
percentage = 100
latest_revision = true
}
custom_domain {
name = "example-app.${var.domain_name}"
certificate_id = azurerm_container_app_environment_certificate.main.id
}
}
}
resource "azurerm_container_app_environment_certificate" "main" {
name = "mycertificate"
container_app_environment_id = azurerm_container_app_environment.main.id
certificate_blob_base64 = acme_certificate.certificate.certificate_p12
certificate_password = acme_certificate.certificate.certificate_p12_password
}4. AppGtw Setup
Now we can move to create the AppGtw. In order to handle the requests coming to the AppGtw we will configure the http_listener block to use Https and set the host_name argument.
Notice that here we use Standard_v2 Application Gateway which does not support WAF. In a production environment where you want to create and associate a WAF Policy to the Application Gateway,
you should rather use WAF_v2.
Here again, we will use the SSL Cert we created in the first step.
resource "azurerm_application_gateway" "appgtw" {
name = "${var.prefix}-appgtw"
resource_group_name = azurerm_resource_group.main.name
location = azurerm_resource_group.main.location
sku {
name = "Standard_v2"
tier = "Standard_v2"
capacity = 1
}
gateway_ip_configuration {
name = "my-gateway-ip-configuration"
subnet_id = azurerm_subnet.appgtw.id
}
frontend_port {
name = local.frontend_port_name
port = 443
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.pip.id
}
backend_address_pool {
name = local.backend_address_pool_name
fqdns = [var.domain_name]
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
path = "/"
port = 80
protocol = "Http"
probe_name = local.probe_name
pick_host_name_from_backend_address = true
}
ssl_certificate {
name = var.domain_name
data = acme_certificate.certificate.certificate_p12
password = acme_certificate.certificate.certificate_p12_password
}
http_listener {
name = local.listener_name
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Https"
host_name = var.domain_name
ssl_certificate_name = var.domain_name
}
request_routing_rule {
name = local.request_routing_rule_name
rule_type = "Basic"
http_listener_name = local.listener_name
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
priority = 1
}
probe {
name = local.probe_name
protocol = "Http"
path = "/"
interval = 30
timeout = 30
unhealthy_threshold = 3
pick_host_name_from_backend_http_settings = true
}
}To keep the setup as simple as possible, in this particular case, we decided to terminate the SSL at the gateway, after which the traffic will flow unencrypted to the backend server (ACA).
To learn more about end-to-end TLS and TLS termination please check this article.
3. Private and Public DNS setup
The solution uses a Private DNS Zone linked to the virtual network hosting the Application Gateway and the infrastructure of the ACA environment and an A record to let the AppGw resolve the FQDN of the Container App custom domain to the private IP address of the internal load balancer. The name of the Private DNS Zone needs to be equal to the name of the custom domain used by the Container App and the listener of the Application Gateway. By doing this, we will ensure that every internal request (including those coming from the AppGw) targeting our domain (demo.example.com) is resolved to the static IP address of ACA.
resource "azurerm_private_dns_zone" "main" {
name = var.domain_name
resource_group_name = azurerm_resource_group.main.name
}
resource "azurerm_private_dns_zone_virtual_network_link" "main" {
name = "${azurerm_virtual_network.main.name}-link"
resource_group_name = azurerm_resource_group.main.name
private_dns_zone_name = azurerm_private_dns_zone.main.name
virtual_network_id = azurerm_virtual_network.main.id
}
resource "azurerm_private_dns_a_record" "main" {
name = "@"
zone_name = azurerm_private_dns_zone.main.name
resource_group_name = azurerm_resource_group.main.name
ttl = 300
records = [azurerm_container_app_environment.main.static_ip_address]
}Finally, we will make sure that the A record for our domain name (demo.example.com) points to the static IP address of the AppGw.
This step is not automated, you will have to go and update your DNS record manually. You can find the static IP in the terraform output or in the Azure portal.
Depending on how you manage your DNS records, this step can be automated, for example when using Azure DNS this step can be automated using Terraform.
Once you have updated your DNS records you should be able to access the ACA app from the internet.
Summa Summarum
- Consider whether to use the default ACA domain or a custom domain when configuring your ACA environment with Application Gateway. The latter is recommended for production-grade scenarios.
- Check the example code in the GitHub repositories for using the default ACA domain (gjoshevski/aca-appgtw-default-domain) or a custom domain (gjoshevski/aca-appgtw-custom-domain).
- Test your setup and monitor your ACA environment for any issues. Remember to consider security, availability, and scalability when designing and deploying your ACA environment with Application Gateway.
- If you are new to ACA make sure you understand the Networking architecture in Azure Container Apps
- Check this blog post discussing the other services capable of running containers in Azure: Container-Based Applications on Azure — Simple and Kubernetesless
