Open in app

Sign in

Write

Sign in

Azure API Management Service — Integration with Nginx-Ingress on Kubernetes and Azure Application Gateway

Almog Golbar
6 min readDec 4, 2023

--

If you have Kubernetes applications exposed with the Nginx-Ingress controller and Azure Application Gateway and wish to use Azure API Management for one or more services, here is a step-by-step guide.

The primary goal is to expose a Kubernetes service through the API Management Service, maintaining all components internally and managing incoming requests through the Application Gateway WAF.

Prerequisites:

  • Running AKS with Nginx-Ingress controller. Deployment Guide
  • Azure Application Gateway. Learn more
  • Public IP for the API Management Service.
  • Subnet with “/28” prefix in the same AKS vnet.
  • DNS for the API Management.
  • Pod with Ingress pointing to the Application Gateway IP. Use the proper Ingress annotation: external-dns.alpha.kubernetes.io/target
    Example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/target: x.x.x.x
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/whitelist-source-range: x.x.x.x/x
  name: app
  namespace: somenamespace
spec:
  ingressClassName: ingressClassName
  rules:
  - host: app-example.com
    http:
      paths:
      - backend:
          service:
            name: app
            port:
              number: 8080
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - app-example.com
    secretName: secretName

In the Azure portal, go to API Management Service and create a new one. Follow Azure documentation for security group configuration. Set the virtual network type to virtual network internal mode. Choose the AKS vnet, subnet, and public IP from prerequisites.

It takes some time for the API Management to come up. Once up, configure a custom domain, mentioning the previously created DNS and its certificate. Apply changes, and after a few minutes, observe the gateway URL change.

Create an API to respond to health checks from the Application Gateway. Go to APIs, click HTTP, fill in the name and display name (e.g., “healthz”).

Add a GET operation with the /healthz URL and a fixed “200 OK” response.

Click on All operations, add a policy, choose mock-response, and select a “200 OK” response.

In the Application Gateway, create:

  • Backend pool
  • Backend settings
  • Listener
  • Rule
  • Health probe
  • WAF rule

Most configurations are intuitive, but consider the following:

Backend pool: Target the API Management gateway DNS, associated with the rule.

Backend settings: Override the hostname with the API Management DNS.

Listener: We expect that the kubernetes app will reach the Application Gateway first, so point to the app DNS. Use Multiple/Wildcard host type with the app DNS. and select proper certificate.

Rule: Create a rule associated with the listener and backend. Pay attention to the priority.

Health probe: The health probe should point to the Api Management dns and “/healthz” path, associate it with the proper backend settings.

WAF rule: Create a new custom rule of string type that matches the request header Host.

Back in API Management, create a new API pointing to the ingress controller. In the “Web service URL” box, put the IP/DNS of your load balancer (associated with your ingress controller) (e.g., https://x.x.x.x). From the new API, add an operation with the desired path (e.g., "/isAlive"). In the API policy, add the following and replace {LB IP} with the IP of the load balancer or DNS of the ingress controller:

<policies>
    <inbound>
        <base />
        <set-backend-service base-url="https://{LB IP}" />
        <set-header name="Host" exists-action="override">
            <value>app-example.com</value>
        </set-header>
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

The Application gateway sets the Host header to the Api Management dns, the ingress inspect the header and see an address that he is not familiar with. Overriding the Host header with app-example.com which is the pod ingress dns(points to the Application Gateway) here is essential.

This override is crucial because the original request comes with the ingress DNS, which directs to the Application Gateway. To ensure the requests proper flow, the API Management sets the Host header again to the pod’s ingress DNS. This step is taken to align the request sent to the LB with the relevant DNS, facilitating a seamless communication flow between the components involved.

You’re all set. If issues persist, troubleshoot with API Management logs/Application Gateway logs to catch and identify stuck requests.

For automation, use the provided Terraform code for the API Management Service:

provider "azurerm" {
  features {}
}
data "azurerm_client_config" "current" {}

provider "aws" {}

data "azurerm_key_vault" "this" {
  name                = var.key_vault_name
  resource_group_name = var.subnet_resource_group_name
}

resource "azurerm_key_vault_access_policy" "example-principal" {
  key_vault_id = data.azurerm_key_vault.this.id
  tenant_id    = data.azurerm_client_config.current.tenant_id
  object_id    = azurerm_api_management.this.identity[0].principal_id
  certificate_permissions = ["Get", "List"]
  secret_permissions = ["Get", "List"]
}

resource "azurerm_subnet" "this" {
  address_prefixes     = var.subnet_address_prefixes
  name                 = "subnet-${var.api_management_name}"
  resource_group_name  = var.subnet_resource_group_name
  virtual_network_name = var.vnet_name
}

resource "azurerm_network_security_group" "this" {
  location            = var.location
  name                = "sg-${var.api_management_name}"
  resource_group_name = var.subnet_resource_group_name
  security_rule {
    name                       = "inbound_3443"
    description                = "api management to virtual network"
    priority                   = 100
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_address_prefix      = "ApiManagement"
    destination_address_prefix = "VirtualNetwork"
    source_port_range = "*"
    destination_port_ranges    = ["3443"]
  }
  security_rule {
    name                       = "inbound_6390"
    description                = "azure lb to virtual network"
    priority                   = 110
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_address_prefix      = "AzureLoadBalancer"
    destination_address_prefix = azurerm_subnet.this.address_prefixes[0]
    source_port_range = "*"
    destination_port_ranges    = ["6390"]
  }
  security_rule {
    name                       = "inbound_443"
    description                = "application gateway to api management"
    priority                   = 120
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_address_prefix      = var.application_gateway_subnet
    destination_address_prefix = azurerm_subnet.this.address_prefixes[0]
    source_port_range = "*"
    destination_port_ranges    = ["443"]
  }
  security_rule {
    name                       = "outbound_443"
    description                = "api management outbound to keyvault"
    priority                   = 130
    direction                  = "Outbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_address_prefix      = azurerm_subnet.this.address_prefixes[0]
    source_port_range = "*"
    destination_address_prefix = "AzureKeyVault"
    destination_port_ranges    = ["443"]
  }
  tags = var.tags
}

resource "azurerm_subnet_network_security_group_association" "this" {
  network_security_group_id = azurerm_network_security_group.this.id
  subnet_id                 = azurerm_subnet.this.id
}

resource "azurerm_public_ip" "api_management" {
  allocation_method   = "Static"
  location            = var.location
  name                = var.api_management_name
  resource_group_name = var.api_management_resource_group
  sku = "Standard"
  tags = var.tags
  domain_name_label = var.api_management_name
}

resource "azurerm_api_management" "this" {
  depends_on = [azurerm_subnet_network_security_group_association.this]
  name                 = var.api_management_name
  location             = var.location
  resource_group_name  = var.api_management_resource_group
  publisher_name       = var.publisher_name
  publisher_email      = var.publisher_email
  sku_name             = var.sku_name
  virtual_network_type = var.api_management_vnet_type
  identity {
    type = "SystemAssigned"
  }
  public_ip_address_id = azurerm_public_ip.api_management.id
  virtual_network_configuration {
    subnet_id = azurerm_subnet.this.id
  }
  tags = var.tags
}
resource "azurerm_api_management_api" "healthz" {
  api_management_name = var.api_management_name
  name                = "healthz"
  resource_group_name = var.api_management_resource_group
  revision            = "1"
  path = ""
  protocols = ["https"]
  display_name = "healthz"
  subscription_required = false
}
resource "azurerm_api_management_api_operation" "gethealthz" {
  api_management_name = var.api_management_name
  api_name            = azurerm_api_management_api.healthz.name
  display_name        = "get-healthz"
  method              = "GET"
  operation_id        = "get-healthz"
  resource_group_name = var.api_management_resource_group
  url_template        = "/healthz"
}
resource "azurerm_api_management_api_policy" "healthz" {
  api_management_name = var.api_management_name
  api_name            = azurerm_api_management_api.healthz.name
  resource_group_name = var.api_management_resource_group
  xml_content = <<XML
<policies>
    <inbound>
        <base />
        <mock-response status-code="200" content-type="application/json" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>
XML
}
resource "azurerm_api_management_custom_domain" "this" {
  api_management_id = azurerm_api_management.this.id
  gateway {
    host_name = "${var.api_management_name}.${var.domain}"
    key_vault_id = var.key_vault_id
  }
}
resource "aws_route53_record" "this" {
  zone_id = var.zone_id
  name    = "${var.api_management_name}.${var.domain}"
  type    = "A"
  ttl     = 300
  records = [azurerm_public_ip.api_management.ip_address]
}

Let’s understand what happened here:

The client connects to “app-example.com” and reaches the APPGW.
On the APPGW, WAF validation is enforced.
The APPGW then sets the Host header to <APIM DNS> and forwards the request to the APIM.
The APIM, via the configured API, sets the “Host” header to the APPGW’s DNS and forwards the request to the ingress controller.
The ingress controller forwards the request to the app.

Azure Api Management
Kubernetes
Azure Application Gateway
Nginx Ingress Controller

--

--

Almog Golbar

Written by Almog Golbar

3 Followers

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams