Shubhojit Dasgupta — Independent API Architect

Part-3 of this blog series established the declarative onboarding workflow, for managing backend API services within the Kong Konnect Routing Control Plane using automation-driven configuration management. We onboarded the Kong Echo API and HTTPBin API backend services through automated Bash scripting and multiple decK reconciliation operations, converted the Echo API OpenAPI specification into Kong declarative configuration, and patched the generated configuration to automatically transform traditional routes while declaratively injecting the Request-Termination plugin for handling unnecessary browser /favicon.ico requests. Finally, we refreshed the backup.yaml declarative configuration state and validated zero-drift synchronization to confirm that the backup configuration remains the single source of truth for the live Routing Control Plane.

In the last part of this blog series, we will validate Expressions-based routing policies in Kong Gateway, to filter inbound API requests by destination port number, verify request flow behavior across multiple Gateway listener ports, and analyze how Kong Expressions Routing improves route matching flexibility and operational control within the Kong Konnect-managed self-managed Gateway environment. Additionally, we will implement an automated backup and controlled reconciliation workflow to safely synchronize the Kong Konnect Routing Control Plane with the local backup.yaml declarative configuration state, ensuring that the declarative configuration remains the single source of truth for the live Routing Control Plane.

Validating API Requests to echo API Backend Service

The response headers depict a successful browser request sent to https://localhost/echo on Kong Gateway port 443. The browser issues an HTTP GET request to the /echo path, and because the request matches the Expressions route configured with http.method == “GET”, http.path ^= “/echo”, and net.dst.port == 443, Kong Gateway forwards the API traffic successfully to the local Kong echo API backend service. A GET request from the Browser does not carry a request body, so the upstream echo service responds only with an HTTP 200 OK status and no message payload in the response body. The response headers visible in the browser confirm that the request was processed through Kong Gateway. In particular, the Server and Via headers expose the Kong Gateway version that proxied the request to the echo API backend and relayed the successful response back to the browser client.

An automatic browser request to /favicon.ico is intercepted and handled directly by Kong Gateway instead of being forwarded to the upstream echo API backend service. When the browser requests the page on port 443, it also issues a background GET request for the/favicon.ico endpoint of echo-api-service. That request matches the echo-api-favicon Expressions route because the path begins with /favicon.ico, the method is GET, and the destination port is 443. Instead of proxying this unnecessary browser-generated request to the echo API backend, the attached request-termination plugin responds immediately at the Kong Gateway layer with an HTTP 204 No Content response. This avoids needless upstream traffic, keeps the echo service focused on real API calls, and demonstrates how Kong Gateway can efficiently absorb non-business browser requests without involving the backend service.

A browser request to https://localhost:8443/echo returns 404 Not Found because the Kong echo API backend service route is intentionally restricted to Kong Gateway port 443. Although the request path begins with /echo and the method is GET, the route expression also requires net.dst.port == 443. Since this request arrives on port 8443, the full expression evaluates to false, no Kong Route is matched, and Kong Gateway returns a 404 Not Found response immediately. This confirms that traffic filtering is being enforced not only by path and method, but also by destination port, and that requests sent to the wrong Gateway port are rejected before they ever reach the upstream echo API backend service.

An automatic browser request to /favicon.ico on Kong Gateway port 8443 is intercepted by the example-favicon-route instead of being forwarded to the HTTPBin upstream service. The Expressions route matches because the request path begins with/favicon.ico and the destination port satisfies net.dst.port == 8443. Once matched, the attached request-termination plugin responds directly at the Kong Gateway layer with the configured response, preventing an unnecessary call to the upstream HTTPBin API. This keeps background browser noise out of the backend service, reduces needless proxy traffic, and confirms that even non-business requests on port 8443 can be cleanly handled at the Gateway itself.

The browser request to /favicon.ico on Kong Gateway port 8443 is matched by example-favicon-route, which is intentionally dedicated to absorbing this non-business browser traffic before it reaches the HTTPBin upstream service. Once the route expression evaluates successfully, the attached request-termination plugin returns the configured 200 response directly from Kong Gateway along with the echoed request details, because the plugin is configured with echo: true. This confirms that favicon requests are handled entirely at the Kong Gateway layer, reducing unnecessary backend calls while still giving a clear visible response in the browser.

Validating API Requests to HTTPBin API Backend Service

A browser request sent to https://localhost:8443/anything is successfully matched by the example-status-route because the request path begins with /anything and the destination port satisfies net.dst.port == 8443. Once the expression evaluates to true, Kong Gateway forwards the request to the HTTPBin upstream service configured under example-service. The /anything endpoint is designed to echo request details back to the client, so the browser receives an HTTP 200 OK response containing headers and other request metadata returned by HTTPBin through Kong Gateway. At the same time, any automatic browser request to /favicon.ico on port 8443 is matched by example-favicon-route and handled entirely inside Kong Gateway by the attached request-termination plugin, which returns the configured 200 OK response directly from the gateway without forwarding that background browser request to the HTTPBin API backend service.

A browser request to https://localhost:8443/status/200 is successfully matched by example-status-route because the request path satisfies the regular expression for the /status/<status_code> endpoint and the destination port matches 8443. Kong Gateway therefore forwards the request to the HTTPBin upstream service, which responds with the expected HTTP 200 OK status and sends that successful response back to the browser through the Gateway. At the same time, the browser also issues its usual background request to /favicon.ico on the same port. That request does not reach HTTPBin, because it is matched separately by example-favicon-route, where the attached request-termination plugin handles it directly inside Kong Gateway and returns the configured response with the echoed message “Request terminated by plugin.”. This confirms that business API traffic is forwarded upstream as intended, while unnecessary browser-generated requests are absorbed and responded to within the Kong Gateway itself.

A browser request to https://localhost/anything returns 404 Not Found because the HTTPBin API route is intentionally restricted to Kong Gateway port 8443. Although the request path begins with /anything, the Expressions route also requires net.dst.port == 8443. Since this request arrives on port 443, the full route expression evaluates to false, no Kong Route is matched, and Kong Gateway immediately returns a 404 Not Found response. This confirms that traffic for the HTTPBin API is filtered not only by path, but also by destination port, ensuring that requests sent to the wrong gateway listener are rejected before they ever reach the upstream HTTPBin service.

A browser request to https://localhost/status/200 returns 404 Not Found because the HTTPBin API route for the /status/<status_code> endpoint is intentionally restricted to Kong Gateway port 8443. Although the request path matches the regular expression used by example-status-route, the Expressions route also requires net.dst.port == 8443. Since this request arrives on port 443, the full expression evaluates to false, no Kong Route is matched, and Kong Gateway immediately returns a 404 Not Found response. This confirms that the HTTPBin status endpoint is protected by the same port-aware routing policy, ensuring that requests sent to the wrong Kong Gateway listener port are rejected before they ever reach the upstream HTTPBin service.

A browser request to /favicon.ico on Kong Gateway port 443 is matched by the echo-api-favicon Expressions route, which is dedicated to handling this automatic browser traffic before it can reach the upstream Kong echo API service. Because the route is attached to the request-termination plugin, Kong Gateway immediately returns the configured 204 No Content response directly to the browser. This confirms that unnecessary favicon requests received on the Kong Gateway listener port 443 are absorbed entirely at the Gateway layer, reducing backend noise and ensuring that only meaningful API traffic is forwarded to the upstream service.

These validation steps confirm the core outcome of this implementation: Kong Gateway Expressions routing can reliably enforce API traffic segregation based on destination port while still evaluating request paths and methods. The Kong echo API is reachable only through port 443, the HTTPBin API is reachable only through port 8443, and requests sent to the wrong port are rejected with 404 Not Found before they ever reach the upstream service. At the same time, automatic browser requests to /favicon.ico are intercepted and terminated directly at the Kong Gateway layer, preventing unnecessary upstream calls for both services. Together, these results demonstrate a clean, secure, and high-performance approach to port-aware API traffic control in Kong Gateway using the Expressions router.

Kong Konnect Routing Control Plane Backup

Remember that while onboarding both the Kong echo API and the HTTPBin API into the Kong Konnect Routing Control Plane, a backup.yaml file was created in the /scripts/sync directory of the project. This backup file acts as the single source of truth for the current live state of the Routing Control Plane because it is generated directly from Kong Konnect using deck gateway dump after each successful apply operation. As a result, it captures the full Control Plane configuration in one place, including the self-signed certificate and SNI configured earlier through the Kong Konnect UI, the echo-api service with its Expressions-based routes and request-termination plugin, and the example-service for HTTPBin API with its combined Expressions routes and favicon-handling plugin. Because the file reflects the exact live runtime configuration, it becomes the most reliable artifact for validation, drift detection, auditability, and recovery. A subsequent deck gateway diff against backup.yaml should always report zero entities to create, update, or delete whenever the local backup remains aligned with the live Control Plane. In practice, this means the backup.yaml file should be preserved, version-controlled, and refreshed after every intentional configuration change so that the live Kong Konnect Routing Control Plane can always be reconstructed or verified from a single declarative artifact.

Automated Bash Script to Synchronize Kong Konnect State with Backup

Create a bash file sync.sh under the sync directory of the project. The bash script defined in sync.sh file detects any changes in the Kong Konnect Routing Control Plane Configuration by comparing the live Kong Konnect state with the local backup.yamlfile and synchronizes the Routing Control Plane only when drift in Kong configuration is detected.

File sync.sh:

#! /usr/bin/env bash

# Directory in which the deck sync script runs
cd kong/scripts/sync

# Importing configuration variables
TOKEN_FILE=$(yq e '.token_file' ./config.yaml)
CONTROL_PLANE=$(yq e '.control_plane' ./config.yaml)
CONTROL_PLANE_ADDR=$(yq e '.control_plane_addr' ./config.yaml)

# Backup file to sync (backup.yaml should be in this directory)
BACKUP_FILE="backup.yaml"

# Check if backup.yaml exists
if [ ! -f "$BACKUP_FILE" ]; then
    echo "Error: $BACKUP_FILE not found in the sync directory"
    exit 1
fi

# Check for changes before synchronizing
echo "Checking for changes in ${BACKUP_FILE}..."
DIFF_OUTPUT=$(deck gateway diff \
  --konnect-token-file ${TOKEN_FILE} \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  ${BACKUP_FILE})

# Check if diff output shows no changes (all counts are 0)
if echo "$DIFF_OUTPUT" | grep -q "Created: 0" && echo "$DIFF_OUTPUT" | grep -q "Updated: 0" && echo "$DIFF_OUTPUT" | grep -q "Deleted: 0"; then
    echo "No changes detected. Skipping sync."
else
    echo "Changes detected. Synchronizing Kong Gateway configuration with Control Plane..."
    deck gateway sync \
      --konnect-token-file ${TOKEN_FILE} \
      --konnect-control-plane-name ${CONTROL_PLANE} \
      --konnect-addr ${CONTROL_PLANE_ADDR} \
      ${BACKUP_FILE}
fi

# Root directory of POC
cd ../../../

The sync.sh script automates safe reconciliation between the local backup.yaml file and the live Kong Konnect Routing Control Plane. It first changes into the kong/scripts/sync directory and uses yq to read the token file path, Control Plane name, and Konnect API address from config.yaml, so the script remains reusable without hardcoding environment-specific values. It then verifies that backup.yaml exists and runs deck gateway diff to compare that local declarative state with the current live state of the Control Plane, storing the command output in the DIFF_OUTPUT shell variable. The conditional check uses grep -q, where grep searches text for a matching pattern and the -q flag means quiet mode, returning only a success or failure exit code without printing matching lines. In this script, grep -q checks whether the diff summary contains Created: 0, Updated: 0, and Deleted: 0. If all three checks succeed, the script concludes that no drift exists between backup.yaml and the live Routing Control Plane and safely skips synchronization. If any one of those values is non-zero, the bash script in sync.sh detects configuration drift and runs deck gateway sync with backup.yaml as the declarative source, bringing the Kong Konnect Routing Control Plane back into alignment with the local backup file. This makes sync.sh a practical automation step for drift control, controlled reconciliation, and maintaining backup.yaml as the single source of truth.

Updated project folder structure after adding sync.sh to reconcile backup.yaml with the live configuration state of the Kong Konnect Control Plane:

KongRouting/
├── .env
├── docker-compose.yaml
├── start.sh
└── kong/
    ├── echo/
    │   ├── config.yaml
    │   ├── echo.sh
    │   ├── echo.yaml
    │   ├── patches.yaml
    │   ├── request-termination.yaml
    │   └── spec.yaml
    ├── httpbin/
    │   ├── config.yaml
    │   ├── example.sh
    │   └── example.yaml
    ├── scripts/
    │   └── sync/
    │       ├── backup.yaml
    │       ├── config.yaml
    │       ├── kong_connect.sh
    │       ├── routing.deck.yaml
    │       └── sync.sh
    └── shared/
        ├── self_signed_certificate.sh
        ├── logs/
        │   ├── proxy_access.log
        │   ├── proxy_error.log
        │   ├── status_access.log
        │   └── status_error.log
        └── ssl/
            ├── cluster.crt
            ├── cluster.key
            ├── server.crt
            └── server.key

Synchronizing Backup

Execute the sync.sh bash automation script from the root project directory KongRouting to synchronize the backup.yaml Kong declarative configuration with the Kong Konnect Routing Control Plane. Ensuring that the terminal session is positioned in the root project folder allows the script to correctly resolve all relative file paths and configuration references used during the synchronization workflow.

In this execution flow, the script first performs a configuration comparison using the deck gateway diff deck command instead of directly synchronizing the configuration. This helps prevent unnecessary synchronization operations when no configuration drift exists between the local backup artifact and the current state of the Control Plane.

The local backup.yaml declarative configuration is already fully synchronized with the Konnect Routing Control Plane. The script safely skips executing deck gateway sync deck command because no configuration differences were detected.

Let us try deleting the echo-api-message route of echo-api service from Kong Konnect UI in Routing Control Plane. Login to Kong Konnect Platform and from the API Gateway → Gateways navigation item on the left navigation pane navigate to the Gateway manager page in Kong Konnect.

Select the Routing Control Plane to list out navigation options on left navigation pane of the Kong Konnect UI for API Gateway Routing Control Plane.

Navigate to the Gateway services page from the Gateway Services navigation item on the left navigation pane of Kong Konnect Routing Control Plane, select the echo-api Gateway service. In the Routes tab section of echo-api gateway service configuration page, click on the three dots on the right side of echo-api-message route opening a context-menu containing route management actions such as Edit and Delete. Select the Delete action from the context-menu.

Selecting the Delete option from the route context-menu opens the Delete a route confirmation dialog, prompting the user to type the route name echo-api-message before permanently removing the Expressions route and its associated plugins if any from the Kong Konnect Routing Control Plane. Enter the route name echo-api-message and select the “Yes, delete” button.

A notification message stating Successfully deleted route appears at the bottom-right corner of the page, confirming that the echo-api-message route associated with the echo-api Gateway service has been permanently deleted from the Kong Konnect Routing Control Plane.

Since the deletion operation was performed directly through the Kong Konnect UI instead of through the declarative configuration workflow managed by decK, a configuration drift now exists between the live state of the Routing Control Plane and the locally maintained backup.yaml Kong declarative configuration file. The local backup.yaml artifact still contains the deleted Expressions route definition, while the current Routing Control Plane configuration no longer contains that route entity.

After the configuration drift is introduced into the live Kong Konnect Routing Control Plane, execute the sync.sh Bash automation script again from the root project directory KongRouting to reconcile the current Control Plane state with the declarative configuration defined in backup.yaml.

Running the sync.sh script again in the KongRouting directory shows that the deck tool detected configuration changes and re-applied them into the Kong Konnect Routing Control Plane. The output logs from the deck operations indicate that it is creating route echo-api-message, and the summary shows Created: 1, Updated: 0, Deleted: 0. This means exactly one new route (the echo-api-message route) was added, and none were changed or removed. In effect, the previously deleted route under the echo-apiGateway service has now been restored in Kong Konnect Routing Control Plane, bringing the live configuration back in line with the backup.yaml Kong declarative file.

Synchronizing the backup eliminates any drift between the live Kong Konnect Control Plane state and the backup file backup.yaml. After this step, the live configuration matches exactly with the Kong declarative configurations defined in backup.yaml. The deck gateway sync deck command configures the Kong Konnect Routing Control Plane to mirror the provided declarative configuration as defined in the backup.

This synchronization approach demonstrates how Kong declarative configuration management with decK can be used to continuously enforce configuration consistency across Kong Konnect Control Plane environments. The backup.yaml file effectively becomes the single source of truth for the Routing Control Plane configuration, ensuring that the live Control Plane state can always be restored, reproduced, version-controlled, and audited through automated infrastructure workflows.

Conclusion

In this blog, we implemented an end-to-end Expressions Routing solution using Kong Gateway and Kong Konnect to filter and process incoming API requests based on the destination Gateway port numbers. By enabling the Expressions router through KONG_ROUTER_FLAVOR=expressions environment setting, the Routing Control Plane was able to leverage the powerful Expressions ATC router engine that matches requests using Kong’s Expressions language instead of the legacy traditional router logic, to create highly flexible and precise traffic matching rules using conditions such as net.dst.port == 443.

The blog demonstrated how declarative API management using decK can automate the complete lifecycle of API onboarding, configuration validation, synchronization, backup management, and route restoration within the Kong Konnect Routing Control Plane. Through automated Bash scripting workflows, OpenAPI-driven onboarding, declarative patch transformations, and plugin injection, the entire routing solution was provisioned in a repeatable and GitOps-friendly manner.

The implementation also showed how multiple HTTPS-enabled Kong Gateway ports can coexist within the same Kong Gateway Data Plane node while allowing Expressions routes to selectively process API traffic based on the destination port receiving the client request. This provides a powerful mechanism for implementing advanced routing strategies, API segregation patterns, environment isolation models, and API Gateway-level traffic control policies without relying solely on traditional path-based routing.

Additionally, the backup synchronization workflow highlighted how Kong declarative configuration artifacts such as the backup.yaml file can be used to continuously reconcile and restore the desired state of the Kong Konnect Routing Control Plane, ensuring configuration consistency and preventing drift caused by manual administrative changes.

By combining Expressions Routing, Konnect-managed Control Plane, declarative configuration management, Docker Compose automation, TLS-secured Kong Gateway endpoints, and bash automated synchronization workflows, Kong Konnect provides a robust SaaS based API platform for building scalable, secure, and operationally consistent API Gateway architectures for modern cloud-native environments.

Shubhojit Dasgupta — Independent API Architect

Part-2 of this blog series established the operational foundation for integrating a self-managed Kong Gateway Data Plane with the Kong Konnect Control Plane. We provisioned the Kong Konnect Control Plane, installed and configured the self-managed Kong Gateway runtime, generated and uploaded a self-signed TLS certificate into the Kong Konnect Platform, and validated the operational behavior of the Kong Gateway Proxy and Status API endpoints.

In this part of the blog series, we will onboard the Kong echo API and HTTPBin API backend services into the Kong Konnect Routing Control Plane using automated Bash scripting and multiple decK operations. We will convert the echo API OpenAPI specification into Kong declarative configuration, patch the generated configuration using automation to onboard services, transform traditional routes to expressions based routes, and inject the Request-Termination plugin declaratively, and repeat the same automated onboarding workflow for the HTTPBin API service. Finally, we will refresh the backup.yaml configuration to confirm zero-drift synchronization, ensuring the Kong Konnect configuration state defined in backup.yaml file remains the single source of truth for the live Routing Control Plane.

Onboarding echo API Service

Create a folder named echo within the kong directory of the project. This folder contains all configuration assets required to provision, configure, and onboard the Echo API service into the Kong Konnect Control Plane.

The Echo API is exposed to client applications securely through the Kong Gateway using the HTTPS listener on port 443. To access the service, client applications must send HTTPS API requests to Kong Gateway endpoint on port 443, where the request is evaluated by the Expressions router before being forwarded to the upstream Echo API service.

To demonstrate Expressions-based routing in Kong Gateway, a lightweight Echo API service is deployed as a backend upstream application. The Echo service is useful for testing because it returns request details such as headers, protocol information, and request metadata, making it easier to verify how incoming traffic is processed and routed through the gateway.

The Echo API container is deployed alongside the Kong Gateway Data Plane within the same Docker network, enabling seamless internal communication between the Kong Gateway and the upstream service.

Echo API Docker Compose Service Definition

kongecho:
  networks:
    - ${NETWORK}
  image: kong/go-echo:latest
  container_name: kongecho
  hostname: konglocalecho
  restart: on-failure
  ports:
    - '${TCP_PORT}:1025/tcp'
    - '${UDP_PORT}:1026/udp'
    - '${HTTP_PORT}:1027'

Service Configuration Overview

● Docker Network Integration: The Echo API service is attached to the same Docker bridge network as the Kong Gateway Data Plane, allowing the gateway to route requests internally using container DNS resolution.

● Official Kong Echo Image: The service uses the official kong/go-echo:latest container image, which provides a lightweight API endpoint for testing HTTP, TCP, and UDP traffic flows.

●Container Identity: The container_name & hostname values establish a predictable network identity for service discovery within the Docker network.

● Automatic Restart Policy: The on-failure restart policy ensures that the container is automatically restarted if the service exits unexpectedly.

● Multi-Protocol Port Exposure: The Echo API exposes:

○ TCP traffic on port 1025

○ UDP traffic on port 1026

○ HTTP traffic on port 1027

These ports are mapped dynamically using environment variables defined in the .env file for echo API service, enabling flexible local development and testing.

Add the Docker Compose Service definition of echo API into docker-compose.yaml file, below the service definition for kong-dp. The environment variables used in echo API Docker Compose service definition is added to the .env file.

# Kong echo API configurations
TCP_PORT=1024
UDP_PORT=1026
HTTP_PORT=1027

Installing Kong echo service

In the root KongRouting folder of the project execute the start.sh bash file from the terminal. Ensure that the root folder of the project is the current working directory in the terminal while executing the command to run the bash script.

Once the container is up, it becomes the local upstream service that Kong Gateway will proxy to when the /echo route matches on the correct destination port. Bringing this service online is a prerequisite for validating end-to-end request flow through Kong (client → gateway → upstream) and confirming that Expressions routing is working as expected.

Control Plane Configuration

The config.yaml file created in the echo directory of the project, defines the connection settings required for interacting with the Kong Konnect Control Plane using decK commands. This configuration externalizes Control Plane details from scripts and decK commands, making the onboarding workflow reusable, portable, and easier to maintain across environments.

File config.yaml:

# Kong Konnect Routing Control Plane
konnect_addr: https://in.api.konghq.com
control_plane: Routing 
token_file: ../scripts/sync/routing.deck.yaml

Configuration Parameters:

● konnect_addr: Specifies the regional Konnect API endpoint used by decK to communicate with the Kong Konnect Control Plane. The konnect_addr: https://in.api.konghq.com provides India-region Konnect endpoint used for Control Plane operations.

● The control_plane: Defines the name of the target Control Plane where API entities such as Services, Routes, Plugins, and Certificates will be provisioned. The value control_plane: Routing must match the Control Plane Name created earlier in the Konnect UI.

● The token_file: References the file containing the Konnect Personal Access Token used for authentication. The mapping token_file: ../scripts/sync/routing.deck.yaml references routing.deck.yaml file. This file stores the token securely and is consumed by decK commands during synchronization and validation operations.

This approach promotes a cleaner separation between operational scripts and environment-specific configuration.

OpenAPI Specification

The OpenAPI YAML specification file spec.yaml, created in the echo directory of the project, defines the contract for the echo API service that will be onboarded into Kong Konnect using decK. The spec.yaml file uses OpenAPI 3.1.0 specification format, enabling standard tooling support for docs and automation (as indicated by openapi: 3.1.0).

File spec.yaml:

openapi: 3.1.0
info:
  title: Kong Echo API
  description: API specification for the Kong Echo service.
  version: 1.0.0
servers:
  - url: https://localhost
    description: Local internal server for Kong Echo
paths:
  /favicon.ico:
    get:
      summary: Get favicon with termination plugin
      description: Returns the favicon image for the service, with a kong request termination plugin returning 204 No Content.
      operationId: favicon
      tags:
        - Assets
      responses:
        '204':
          description: No Content. Request terminated by plugin.
        '200':
          description: Favicon image returned successfully.
          content:
            image/x-icon:
              schema:
                type: string
                format: binary
            image/png:
              schema:
                type: string
                format: binary
            image/jpeg:
              schema:
                type: string
                format: binary
            image/gif:
              schema:
                type: string
                format: binary
            image/svg+xml:
              schema:
                type: string
                format: binary
  /echo: 
    get:
      summary: Echo a message
      description: Accepts a JSON payload with a message and echoes it back.
      operationId: echoMessage
      tags:
        - Echo
      requestBody:
        required: false
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/EchoRequest"
            example:
              message: "Hello, Echo!"
      responses:
        '200':
          description: Message echoed successfully.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/EchoResponse"
              example:
                message: "Hello, Echo!"
components:
  schemas:
    EchoRequest:
      type: object
      properties:
        message:
          type: string
          description: A message to be echoed back.
          example: "Hello, Echo!"
    EchoResponse:
      type: object
      properties:
        message:
          type: string
          description: The echoed message.
          example: "Hello, Echo!"
tags:
  - name: Assets
    description: Operations related to static assets like favicons.
  - name: Echo
    description: Operations related to echoing messages.

The OpenAPI specification for Kong Echo API defines two primary GET endpoints:

1. The /favicon.ico path, for handling browser favicon requests efficiently

2. The /echo path, for accepting JSON REST API requests via the Kong Gateway and echoing back to the client

This specification acts as the foundational API definition from which Kong entities such as Services, Routes, and associated policies can be generated and managed declaratively via decK. The spec.yaml file serves as the primary input artifact for the onboarding workflow. It is processed through multiple decK operations defined in the echo.sh bash file.

Architecture Overview

Automated Bash Script

Create echo.sh in the echo directory. This script runs the decK commands to onboard echo API into Kong Konnect Routing Control Plane, using the OpenAPI specification of echo API in spec.yaml file as the input and generating the Kong declarative configuration file echo.yaml in the same directory.

This script orchestrates the complete onboarding workflow using decK commands, transforming the OpenAPI specification of echo API into a valid Kong Declarative Configuration, applying Expressions-based routing policies, validating the generated configuration, synchronizing it with the Routing Control Plane, and creating a backup artifact for recovery purposes.

File echo.sh:

#! /usr/bin/env bash

# Directory in which the deck connect script runs
cd kong/echo

# Importing configuration variables
CONTROL_PLANE=$(yq e '.control_plane' ./config.yaml)
CONTROL_PLANE_ADDR=$(yq e '.konnect_addr' ./config.yaml)
TOKEN_FILE=$(yq e '.token_file' ./config.yaml)

# Convert openapi to decK configuration
deck file openapi2kong \
  --konnect-token-file ${TOKEN_FILE} \
  -s spec.yaml \
  -o echo.yaml

# Applying patches to the generated Kong Declarative Config for Echo API
deck file patch \
  --konnect-token-file ${TOKEN_FILE} \
  -s echo.yaml -o echo.yaml \
  patches.yaml

# Add request-termination plugin
deck file add-plugins \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  --konnect-token-file ${TOKEN_FILE} \
  -s echo.yaml -o echo.yaml \
  request-termination.yaml

# Preview changes made by echo API before applying
deck gateway diff \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  --konnect-token-file ${TOKEN_FILE} \
  echo.yaml

# Validate the echo API configuration before applying
deck gateway validate \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  --konnect-token-file ${TOKEN_FILE} \
  echo.yaml

# Apply echo API configuration in Kong Konnect
deck gateway apply \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  --konnect-token-file ${TOKEN_FILE} \
  echo.yaml

# Backup echo API configuration in Kong Konnect
deck gateway dump \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  --konnect-token-file ${TOKEN_FILE} \
  -o ../scripts/sync/backup.yaml

# Validate the backup configuration
deck gateway validate \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  --konnect-token-file ${TOKEN_FILE} \
  ../scripts/sync/backup.yaml

# Preview changes made by backup configuration before syncing
deck gateway diff \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR} \
  --konnect-token-file ${TOKEN_FILE} \
  ../scripts/sync/backup.yaml

echo "Applied Kong Konnect echo API configuration into ${CONTROL_PLANE} Control Plane."
# Root directory
cd ../..

The automation eliminates manual configuration steps and enables repeatable onboarding workflows that integrate well with GitOps and APIOps delivery pipelines. The bash script in echo.sh file reads the variables from config.yaml file in the echo directory fetching the Control Plane name, API address, and the path of the Kong Konnect Personal access token file routing.deck.yaml in the /scripts/sync directory of the project.

The onboarding workflow begins with the deck file openapi2kong command converting the OpenAPI specification of echo API in spec.yaml file into a Kong Declarative Configuration file echo.yaml in the same echo directory.

During the conversion process, decK automatically generates predictable names for Services and Routes. The Kong Service name for echo API is derived from the info.title section of the OpenAPI spec (Kong Echo API) in spec.yaml file. The decK command normalizes it (lowercase + hyphens), so Kong Echo API becomes kong-echo-api in echo.yaml.

In the Echo API OpenAPI spec (spec.yaml), each path + operation is converted by decK into a traditional Kong Route. decK constructs each Route name by concatenating the generated Kong Service name from echo.yaml with the operation’s operationId from the paths section of spec.yaml, using an underscore (_) as the separator.

Example: /favicon.ico

operationId: favicon
Route name: kong-echo-api_favicon

Example: /echo

operationId: echoMessage
Route name: kong-echo-api_echomessage

These deterministic Route names are later referenced by the patching step to selectively transform the generated traditional Routes into Expressions Routes. In the echo directory create the file patches.yaml.

Patching openapi2kong Output for Expressions Routing

The openapi2kong step initially produces traditional Kong Routes based on paths and methods. This blog demonstrates Expressions routing with port-based filtering (net.dst.port), hence we use deck file patch with patches.yaml to transform the generated config echo.yaml in-place.

● The selectors section use JSONPath syntax to find the exact objects to modify (for example, routes named kong-echo-api_favicon and kong-echo-api_echomessage). The predictable names generated by openapi2kong make these targets reliable.

● The values section overwrites fields to create Expressions routes. It renames the routes, adds an expression that matches http.method, prefix-based http.path, and the destination listener port (net.dst.port == 443), and applies a common echo tag. Tags help grouping/filtering in Konnect and in dumps.

● The remove section deletes paths and methods from the traditional routes so the Expressions router evaluates only the expression condition.

● The service selector retargets the generated upstream to the local Docker service by setting protocol, host(kongecho), and port (1027), and removes any default path. Because Kong and Echo run on the same Docker network, kongecho resolves automatically via Docker DNS.

File patches.yaml:

patches:
  - selectors:
      - $..routes[?(@.name=='kong-echo-api_favicon')]
    values:
      name: echo-api-favicon
      expression: >-
        http.method == "GET" && 
        http.path ^= "/favicon.ico" && 
        net.dst.port == 443
      strip_path: false
      tags:
        - echo
    remove:
      - paths
      - methods
  - selectors:
      - $..routes[?(@.name=='kong-echo-api_echomessage')]
    values:
      name: echo-api-message
      expression: >-
        http.method == "GET" && 
        http.path ^= "/echo" && 
        net.dst.port == 443
      strip_path: false
      tags:
        - echo
    remove:
      - paths
      - methods
  - selectors:
      - $..services[?(@.name=='kong-echo-api')]
    values:
      name: echo-api
      protocol: http
      port: 1027
      host: kongecho
      tags:
        - echo
    remove:
      - path

The recursive descent operator (..) is important because the generated Kong declarative configuration may contain nested structures. Using .. ensures that the selector can locate matching entities regardless of where they appear in the document hierarchy. For example in the patches.yaml file - $..routes[?(@.name==’kong-echo-api_favicon’)] searches for a route named kong-echo-api_favicon anywhere in the generated echo.yaml, regardless of whether routes appear at the top level or are nested under sections like _format_version or services. After patching, the transformed echo.yaml will contain the renamed Routes configured for Expressions routing (with expression conditions such as net.dst.port == 443), ready to be synchronized to Konnect.

Injecting Request-Termination Plugin

After transforming the traditional routes into Expressions-based routes, a request-termination.yaml file is created inside the kong/echo directory to declaratively inject the request-termination plugin into the favicon route. This plugin is attached specifically to the Expressions route matching the name echo-api-favicon using a JSONPath selector $..routes[?(@.name==’echo-api-favicon’)]. The selector recursively searches echo.yaml decK file and locates the target route by name. The request-termination plugin configuration returns HTTP 204 No Content for API requests to/favicon.ico on destination port 443,based on the expression: http.method == “GET” && http.path ^= “/favicon.ico” && net.dst.port== 443 for echo-api-favicon route. The request is never forwarded to the upstream echo API service.

This helps avoid unnecessary upstream calls via Kong generated automatically by browsers requesting favicon assets.

File request-termination.yaml:

# request-termination.yaml
add-plugins:
  - selectors:
      - $..routes[?(@.name=='echo-api-favicon')]
    overwrite: false
    plugins:
      - name: request-termination
        config:
          status_code: 204

Because the request-termination plugin returns a 204 (No Content), the response contains no body and the request is handled entirely at the gateway without calling the upstream Echo service. The deck file add-plugins command injects the plugin defined in request-termination.yaml file into the echo.yaml declarative config, updating the file in place.

Updated Project Folder Structure after adding files for onboarding echo API service into Kong Konnect Control Plane:

KongRouting/
├── .env
├── docker-compose.yaml
├── start.sh
└── kong/
    ├── echo/
    │   ├── config.yaml
    │   ├── spec.yaml
    │   ├── patches.yaml
    │   ├── request-termination.yaml
    │   ├── echo.yaml
    │   └── echo.sh
    │
    ├── scripts/
    │   └── sync/
    │       ├── config.yaml
    │       ├── kong_connect.sh
    │       ├── routing.deck.yaml
    │       └── backup.yaml
    │
    └── shared/
        ├── self_signed_certificate.sh
        │
        ├── logs/
        │   ├── proxy_access.log
        │   ├── proxy_error.log
        │   ├── status_access.log
        │   └── status_error.log
        │
        └── ssl/
            ├── server.crt
            ├── server.key
            ├── cluster.crt
            └── cluster.key

Executing the Bash Script

Executing the bash script defined in echo.sh file will synchronize the generated Kong Declarative Configuration file echo.yaml with the Kong Konnect Routing Control Plane. Once executed successfully, the echo API configuration is synchronised into the Routing Control Plane and automatically propagated to the connected Kong Gateway Data Plane.

The Gateway services page in Kong Konnect is empty, as no APIs have been onboarded into Kong Konnect Routing Control Plane. In the terminal, with KongRouting project root folder as the current working directory, run echo.sh file.

The output from the execution of the bash script in echo.sh file is sent to the stdout of the terminal. The output from the deck operations show that it’s creating the service echo-api with two routes along with a request-termination plugin for echo-api-favicon route but the output also show that it is deleting sni localhost and certificate added earlier in the blog from the Kong Konnect UI in Certificates page.

Since the echo.yaml declarative configuration does not include the certificate and SNI created earlier in Kong Konnect, decK detects them as differences between the local file and the current state of the Routing Control Plane. This is why the deck gateway diff output shows those resources as candidates for deletion. The deck gateway diff command is only a dry run, so it does not apply any changes. Instead, it previews what would happen if echo.yaml were synchronized with the Kong Konnect Routing Control Plane.

To safely synchronize only the Echo API entities generated in echo.yaml, the workflow uses deck gateway apply rather than a full destructive sync. This is important because the goal at this stage is to add the new echo-api service, its two routes, and the request-termination plugin to the current Kong Konnect Routing Control Plane without removing the certificate and localhost SNI that were previously configured from the Kong Konnect UI. After those changes are applied, a backup of the live Control Plane state is immediately created and stored as backup.yaml in the /scripts/sync directory of the project using the deck gateway dump command. This generated backup.yaml file contains both sets of live configuration: the entities added by onboarding the Echo API and the certificate and SNI that were configured earlier from the Certificates page in Kong Konnect UI. When deck gateway diff is then executed against backup.yaml, the output summary shows 0 Kong entities created, updated, and deleted, confirming that the local backup file exactly matches the current live state of the Kong Konnect Routing Control Plane. This makes backup.yaml in the sync directory the single source of truth for the current runtime configuration of the live Kong Konnect Routing Control Plane.

The routes shown above are onboarded as Expressions-based routes rather than traditional path-only routes. This means each route evaluates not just the request path and HTTP method, but also the gateway destination port before traffic is forwarded upstream. In this setup, the echo-api-message route matches GET requests whose path starts with /echo and that arrive on port 443, while the echo-api-favicon route matches browser requests to /favicon.ico on the same port. By encoding these conditions directly in the route expressions, Kong Gateway can accurately filter incoming API traffic and send only valid matching requests to the echo-api service upstream.

With this, the onboarding of the echo API into the Kong Konnect Routing Control Plane is complete. The service, its Expressions-based routes, and the request-termination plugin are now declaratively managed, synchronized to the live Control Plane, and propagated to the connected Kong Gateway Data Plane. At this point, the Echo API is no longer just defined on paper it is live, governed, and ready to prove how Expressions routing in Kong Gateway can enforce precise, port-aware traffic control in a real deployment.

Onboarding HTTPBin API Service

Create a folder httpbin under the kong directory of the project. The folder includes all the files related to the onboarding of HTTPBin API service into Kong Konnect Routing Control Plane as an example-service in Kong Konnect.

The HTTPBin API service is used in this routing demonstration as a lightweight upstream testing service behind the Kong Gateway.

The endpoint /status/(?P<status_code>[1–5]\d{2}) dynamically returns HTTP response codes ranging from 100 to 599, making it useful for validating Expressions-based routing behavior, error handling, response propagation, and Gateway policy execution. The /anything endpoint accepts requests using any HTTP method and returns complete request metadata including headers, query parameters, payloads, and request details. This makes it ideal for verifying how client API requests are received, processed, and forwarded through the Kong Gateway Data Plane when filtering traffic based on destination gateway port numbers.

Control Plane Configuration

The config.yaml file created in the httpbin folder externalizes the Kong Konnect connection details required by decK. Keeping these values in a small configuration file avoids hardcoding environment-specific parameters inside the automation script and makes the onboarding flow easier to reuse across environments. In this file, konnect_addr points decK to the India-region Konnect API endpoint, control_plane identifies the target Control Plane as Routing, and token_file references the Personal Access Token file stored under ../scripts/sync. The file content is shown below.

File config.yaml:

# Kong Konnect Routing Control Plane
konnect_addr: https://in.api.konghq.com
control_plane: Routing
token_file: ../scripts/sync/routing.deck.yaml

Initial Kong Declarative Configuration

The next artifact created in the httpbin folder is example.yaml, the initial Kong declarative configuration that will be synchronized into the Control Plane. Unlike the Echo API flow, this onboarding starts directly from a hand-authored decK state file rather than generating configuration from an OpenAPI specification. The file defines a single upstream service named example-service that proxies to https://httpbin.konghq.com, along with two Expressions-based routes. One route handles business traffic for /status/<code> and /anything only when requests arrive on destination port 8443, while the second route catches /favicon.ico requests and terminates them at the gateway using the request-termination plugin. This example.yaml file becomes the initial input to the automation script example.sh.

File example.yaml:

_format_version: "3.0"
services:
- name: example-service
tags:
- example
url: https://httpbin.konghq.com
routes:
- name: example-status-route
expression: (http.path ~ r#"^/status/(?P<status_code>[1-5]\d{2})$"# || http.path ^= "/anything") && net.dst.port == 8443
strip_path: false
tags:
- example
- name: example-favicon-route
expression: http.path ^= "/favicon.ico" && net.dst.port == 8443
strip_path: false
tags:
- example
plugins:
- name: request-termination
config:
status_code: 200
message: "Request terminated by plugin."
echo: true
tags:
- example

Architecture Overview

Deploying HTTPBin API using Automated Bash Script

To automate the full onboarding flow, create a bash script named example.sh in the same httpbin folder. This script begins by switching into the kong/httpbin directory and then uses yq to read the values of control_plane, konnect_addr, and token_file from config.yaml. Those values are assigned to shell variables and reused across all decK commands, which keeps the script concise and prevents duplication of configuration values. The declarative file example.yaml is then used as the source of truth for the initial apply operation.

File example.sh:

#! /usr/bin/env bash
 
# Directory in which the deck connect script runs
cd kong/httpbin
 
# Importing configuration variables
CONTROL_PLANE=$(yq e '.control_plane' ./config.yaml)
CONTROL_PLANE_ADDR=$(yq e '.konnect_addr' ./config.yaml)
TOKEN_FILE=$(yq e '.token_file' ./config.yaml)
 
# Preview changes made by example configuration before applying
deck gateway diff \
--konnect-control-plane-name ${CONTROL_PLANE} \
--konnect-addr ${CONTROL_PLANE_ADDR} \
--konnect-token-file ${TOKEN_FILE} \
example.yaml
 
# Validate the example configuration before applying
deck gateway validate \
--konnect-control-plane-name ${CONTROL_PLANE} \
--konnect-addr ${CONTROL_PLANE_ADDR} \
--konnect-token-file ${TOKEN_FILE} \
example.yaml
 
# Apply example configuration in Kong Konnect
deck gateway apply \
--konnect-control-plane-name ${CONTROL_PLANE} \
--konnect-addr ${CONTROL_PLANE_ADDR} \
--konnect-token-file ${TOKEN_FILE} \
example.yaml
 
# Backup configuration in Kong Konnect
deck gateway dump \
--konnect-control-plane-name ${CONTROL_PLANE} \
--konnect-addr ${CONTROL_PLANE_ADDR} \
--konnect-token-file ${TOKEN_FILE} \
-o ../scripts/sync/backup.yaml
 
# Validate the backup configuration
deck gateway validate \
--konnect-control-plane-name ${CONTROL_PLANE} \
--konnect-addr ${CONTROL_PLANE_ADDR} \
--konnect-token-file ${TOKEN_FILE} \
../scripts/sync/backup.yaml
 
# Preview changes made by backup configuration before syncing
deck gateway diff \
--konnect-control-plane-name ${CONTROL_PLANE} \
--konnect-addr ${CONTROL_PLANE_ADDR} \
--konnect-token-file ${TOKEN_FILE} \
../scripts/sync/backup.yaml
 
echo "Applied Kong Konnect example configuration into Konnect ${CONTROL_PLANE} Control Plane."
# Root directory
cd ../..

Injecting Request-Termination Plugin

With the example-favicon-route in place to catch all /favicon.ico requests arriving on port 8443, the next step is to decide what Kong Gateway should actually do with them and that’s where the Request-Termination plugin comes in.

Rather than forwarding these browser-generated noise requests to your upstream service, the plugin intercepts them and immediately returns a response — no upstream call made, no backend resources consumed.

In this configuration, the plugin is set to respond with a 200 OK status code and a message of Request terminated by plugin.. This keeps browsers happy without triggering any errors. What makes this setup particularly interesting is the echo: true flag. When enabled, the plugin mirrors a copy of the incoming request back to the client, making it an excellent tool for debugging live traffic without disturbing real upstream HTTPBin API service.

It’s worth noting that the Request Termination plugin runs at priority 2, the lowest of all plugins (except Post-Function) — meaning all other plugins will execute first before the request is terminated.

plugins:
  - name: request-termination
    config:
      status_code: 200
      message: "Request terminated by plugin."
      echo: true
    tags:
      - example

Updated project working directory after adding files for HTTPBin API service into Kong Konnect Control Plane:

KongRouting/
├── .env
├── docker-compose.yaml
├── start.sh
└── kong/
    ├── echo/
    │   ├── config.yaml
    │   ├── spec.yaml
    │   ├── patches.yaml
    │   ├── request-termination.yaml
    │   ├── echo.yaml
    │   └── echo.sh
    │
    ├── httpbin/
    │   ├── config.yaml
    │   ├── example.yaml
    │   └── example.sh
    │
    ├── scripts/
    │   └── sync/
    │       ├── config.yaml
    │       ├── kong_connect.sh
    │       ├── routing.deck.yaml
    │       └── backup.yaml
    │
    └── shared/
        ├── self_signed_certificate.sh
        ├── logs/
        │   ├── proxy_access.log
        │   ├── proxy_error.log
        │   ├── status_access.log
        │   └── status_error.log
        │
        └── ssl/
            ├── server.crt
            ├── server.key
            ├── cluster.crt
            └── cluster.key

Executing the Bash Script

The decK workflow in example.sh follows a safe and repeatable sequence. First, deck gateway diff performs a dry run and shows what would change if example.yaml were applied to the current state of the Kong Konnect Routing Control Plane. Next, deck gateway validate checks that the declarative file is syntactically and structurally correct before any change is made. The actual onboarding is then performed by deck gateway apply, which adds the example-service, its Expressions-based routes, and the request-termination plugin into the live Control Plane. Immediately after the apply step, deck gateway dump exports the full live state of the Control Plane into ../scripts/sync/backup.yaml. This backup file captures not only the entities introduced by onboarding the HTTPBin API service, but also any other existing live entities already present in the Control Plane. The script then validates backup.yaml and runs a final deck gateway diff against it. When the summary reports zero entities created, updated, and deleted, it confirms that the local backup.yaml file exactly matches the current live Kong Konnect Routing Control Plane state. That is why the backup.yaml file in the /scripts/sync directory becomes the single source of truth for the live current configuration state of the Kong Konnect Routing Control Plane.

From the KongRouting project root, execute the example.sh bash script with KongRouting set as the current working directory in your terminal.

The terminal output reflects each decK operation, executed by example.sh, and follows the same safe workflow used earlier for the Kong echo API onboarding. The first deck gateway diff compares the local state file example.yaml with the current live state of the Kong Konnect Routing Control Plane. Because example.yaml contains only the HTTPBin API configuration and does not include the certificate, SNI, or the previously onboarded Kong echo API entities already present in the Control Plane, decK reports both creates and deletes in the diff output. The create side corresponds to the four Kong entities required for HTTPBin API onboarding a Service (example-service), two Routes (example-status-route and example-favicon-route), and one route-level request-termination plugin. The delete side appears only because those already-existing resources are absent from the local example.yaml file during the dry-run comparison. However, when the script proceeds to deck gateway apply, decK performs a non-destructive apply operation rather than a full deck gateway sync, so it creates only the four HTTPBin Kong entities and does not remove any existing certificate, SNI, or Kong echo API objects from the live Control Plane. After the apply step, deck gateway dump exports the complete current Control Plane state into /scripts/sync/backup.yaml. Since that backup file already exists from the earlier Kong echo API onboarding, decK prompts for confirmation before overwriting it, which is expected and desirable because the backup must now be refreshed to include both the previously existing live configuration and the newly added HTTPBin API entities. Finally, when deck gateway diff is run against the updated backup.yaml, the summary shows 0 Kong entities to create, update, or delete because the dumped backup file now exactly matches the live state of the Kong Konnect Routing Control Plane. That zero-drift result confirms that backup.yaml has been successfully refreshed and remains the single source of truth for the current live configuration state of the Routing Control Plane.

The routes are configured as Expressions-based routes, allowing Kong Gateway to evaluate multiple request conditions within a single Route object instead of creating a separate Route for every individual path. In this onboarding, the example-status-route combines the/status/<status_code> and /anything HTTPBin API endpoints into one Kong Route object by using the logical OR operator within a single route expression. API requests either matching the regular expression for valid /status response codes or starting with /anything are routed to the same upstream service, provided they arrive on destination port 8443. This is a practical advantage of the Kong Expressions router where multiple route paths belonging to the same service and sharing similar routing conditions can be consolidated into a single Kong Route object, reducing configuration sprawl, simplifying operational management, and improving route evaluation efficiency. In large-scale Kong Konnect deployments, fewer Route objects mean lower configuration complexity and better overall routing performance across the Kong Konnect API platform.

The request-termination plugin attached to example-favicon-route ensures that automatic browser requests to/favicon.ico on port 8443 are handled directly at the Kong Gateway layer instead of being forwarded to the HTTPBin upstream service. This prevents unnecessary backend calls, reduces noise in upstream traffic, and keeps the service focused only on meaningful API requests such as /status/<code> and /anything. With this, the onboarding of the HTTPBin API service into the Kong Konnect Routing Control Plane is complete. The example-service, its Expressions-based routes, and the request-termination plugin are now live, declaratively managed, and propagated to the connected Kong Gateway Data Plane, ready to demonstrate precise destination-port-based traffic filtering in the validation steps that follow.

Next, in Part-4 of this blog series, we will validate Expressions-based routing policies in Kong Gateway to filter inbound API requests by destination port number, verify request flow behavior across multiple Gateway listener ports, and analyze how Kong Expressions Routing improves route matching flexibility and operational control within the Kong Konnect-managed self-managed Gateway environment. Additionally, we will implement an automated backup and controlled reconciliation workflow to safely synchronize the Kong Konnect Routing Control Plane with the local backup.yaml declarative configuration state, ensuring that the declarative configuration remains the single source of truth for the live Routing Control Plane.

Blog Series — Part-4:

Validating API Traffic Flow Across Multiple Kong Gateway Destination Ports with Backup Reconciliation Workflows

Use of Expressions Routing in Kong Gateway, to Filter Inbound API Requests by Destination Port…

Shubhojit Dasgupta — Independent API Architect

Part-1 of this blog series established the foundational architecture and configuration required for implementing expressions-based routing in Kong Gateway. We explored the inbound API traffic flow, reviewed the fundamentals of Kong Expressions syntax, examined the overall solution architecture, and configured the Kong Konnect API Platform along with the Personal Access Token required for secure Control Plane management.

In this part of the blog, we will provision the Kong Konnect Control Plane with a self-managed Kong Gateway Data Plane, install and configure the self-managed Kong Gateway runtime, generate a self-signed TLS certificate and upload it into the Kong Konnect Platform, and finally validate the operational behavior of the Kong Gateway Proxy and Status API endpoints.

Provision Kong Konnect Control Plane with Self-Managed Kong Gateway

To create a new Control Plane in Kong Konnect along with an associated Kong Gateway instance, navigate to the Gateway Manager by selecting API Gateway → Gateways from the left-hand navigation pane.

Click on the “New gateway” button to initiate the setup. Configure Konnect Control Plane with the name “Routing” and assign a label using the key-value pair:

created-for: expressions-routing

Provide a meaningful description for the Kong Konnect Control Plane, such as “Implementing Expressions Router Solution in Kong Gateway.”

Clicking the “New gateway” button launches the Create API gateway modal dialog, enabling one to configure the Control Plane and determine the provisioning model for the associated Kong Gateway instance in Kong Konnect.

In the “Create API gateway” modal dialog the Gateway name and Gateway description are Control Plane name and Control Plane description respectively. In this blog Kong is installed as a self-managed gateway running in a docker container. So, for the setup “Where do you want to run your gateway?” choose “Self-managed”. This will allow the Kong Gateway runtime to be installed anywhere even in a local computer connected to Kong Konnect Control Plane hosted in India (Mumbai / ap-south-1) region of AWS cloud, as selected during Kong Konnect Organization creation.

Install Self-Managed Kong Gateway Data Plane Node

Select “Create” button to provision the configured Konnect Control Plane setup. The “Create a data plane node” modal dialog opens where one selects the version of the Kong Gateway as “Self-Managed Gateway 3.14” and for the platform where Kong Gateway shall be deployed select the drop down option “Mac (docker)”. In this blog Kong gateway is installed running as a docker container in Mac. One can select the operating system of their choice from the dropdown menu to implement the routing solution described in this blog. However, ensure that Docker Desktop is installed and properly configured on the machine before proceeding.

Select the “Generate certificate and script” option to automatically generate the required certificates and Docker run commands based on the official Kong Gateway Docker image. These commands can be used to provision the Data Plane node and establish connectivity with Kong Konnect.

Alternatively, one can also choose the “Set up later” option to create only the Konnect Control Plane and defer the Data Plane provisioning to a later stage.

Copy the environment variable values from the generated Docker run script and store them in the .env file, mapping each value to its corresponding environment variable key. This approach helps centralize configuration and simplifies container deployment.

Adding the Kong Gateway Script and Environment Settings

Copy the value for environment KONG_CLUSTER_CERT starting from — — -BEGIN CERTIFICATE — — - till the — — -END CERTIFICATE — — - from the script generated above into the cluster.crt file created earlier under /ssl folder.

Copy the value for environment KONG_CLUSTER_CERT_KEY starting from — — BEGIN PRIVATE KEY — — - till the — — -END PRIVATE KEY — — - from the script into the cluster.key file created earlier under /ssl folder.

The cluster.crt file will contain the following content:

-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIBATAKBggqhkjOPQQDBDA0MTIwCQYDVQQGEwJJTjAlBgNV
BAMeHgBrAG8AbgBuAGUAYwB0AC0AUgBvAHUAdABpAG4AZzAeFw0yNjA1MDUwNzAz
MzNaFw0zNjA1MDUwNzAzMzNaMDQxMjAJBgNVBAYTAklOMCUGA1UEAx4eAGsAbwBu
AG4AZQBjAHQALQBSAG8AdQB0AGkAbgBnMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcD
QgAEl5FP6/tB2//ZxdLgDNDN2kGfjSN2geQ8ARVD9jLfoDZUAJ1zMPYv6+//ibCk
6PIH5GZVJKT/cZ4tb8SZ7e18ZqOBrjCBqzAMBgNVHRMBAf8EAjAAMAsGA1UdDwQE
AwIABjAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwFwYJKwYBBAGCNxQC
BAoMCGNlcnRUeXBlMCMGCSsGAQQBgjcVAgQWBBQBAQEBAQEBAQEBAQEBAQEBAQEB
ATAcBgkrBgEEAYI3FQcEDzANBgUpAQEBAQIBCgIBFDATBgkrBgEEAYI3FQEEBgIE
ABQACjAKBggqhkjOPQQDBANIADBFAiBGqEzVV8ZBZ/pCN4WYDdaoVlAXz9BizaJg
0LEvOG27AAIhALOhzxUegCC0zMYDj6clC2bV/Zve9Kh9i5qQQT6m6r5J
-----END CERTIFICATE-----

The cluster.key file will contain the following content:

-----BEGIN PRIVATE KEY-----
MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQgX3U2ZmOiETBJZaEx
ZaqVYWp/7iCfnjXHGPLv+NCLMpKgCgYIKoZIzj0DAQehRANCAASXkU/r+0Hb/9nF
0uAM0M3aQZ+NI3aB5DwBFUP2Mt+gNlQAnXMw9i/r7/+JsKTo8gfkZlUkpP9xni1v
xJnt7Xxm
-----END PRIVATE KEY-----

Copy the values for the following environment variables from the script generated:

· KONG_CLUSTER_CONTROL_PLANE

· KONG_CLUSTER_SERVER_NAME

· KONG_CLUSTER_TELEMETRY_ENDPOINT

· KONG_CLUSTER_TELEMETRY_SERVER_NAME

The environment variable and the certificates need to be updated every time a new Konnect Control Plane is created.

Add the above environment variable values from the script into the .env file under the root project KongRouting folder, mapped to the corresponding environment keys:

· KONG_CP_ENDPOINT

· KONG_CP_SERVER_NAME

· KONG_TELEMETRY_ENDPOINT

· KONG_TELEMETRY_SERVER_NAME

To operationalize Expressions Router solution, a self-managed Data Plane node of Kong Gateway must be provisioned and securely connected to the Kong Konnect Control Plane. In this setup, Docker Compose is used as the orchestration layer to declaratively define and run the Kong container with all required runtime configurations.

This approach provides:

· Environment consistency through declarative configuration

· Portability across local development environments

· Separation of concerns between configuration (.env) and deployment (docker-compose.yaml)

The setup is broken down into three key steps:

1. Defining environment variables using a .env file

2. Referencing these variables within the Docker Compose specification

3. Bootstrapping and validating the Data Plane connection to Konnect

Adding the .env File

The .env file serves as the central configuration artifact for the Docker Compose-based deployment. It defines all runtime parameters required to initialize and run the Kong Gateway container in Data Plane mode.

Each entry in the .env file follows a key-value pair structure, where:

· The key represents the configuration parameter expected by Kong or Docker Compose

· The value is derived from:

→ The generated script in Kong Konnect (for Control Plane connectivity)

→ Local environment decisions (e.g., ports, volumes, container naming)

→ Security artifacts (e.g., certificate paths)

Key Configuration Settings:

· Container & Runtime Settings — Define container identity and lifecycle behavior:

→ KONG_CONTAINER_NAME, KONG_HOST_NAME, KONG_RESTART

· Networking & Ports — Control how traffic is exposed:

→ KONG_PROXY_PORT, KONG_PROXY_SECURED_PORT1, KONG_STATUS_PORT

· Konnect Control Plane Connectivity — These values are directly sourced from the generated script:

→ KONG_CP_ENDPOINT, KONG_CP_SERVER_NAME

→ KONG_TELEMETRY_ENDPOINT, KONG_TELEMETRY_SERVER_NAME

· Security & mTLS Configuration — Required for secure clustering between Data Plane and Control Plane:

→ KONG_CLUSTER_MTLS=pki

→ Certificate paths mapped via mounted volumes

· Routing Behavior — Enables Expressions-based routing:

→ KONG_ROUTER_FLAVOR=expressions

· Mode Configuration — Defines Data Plane operation:

→ KONG_ROLE=data_plane

→ KONG_DATABASE=off

→ KONG_KONNECT_MODE=on

· Observability & Logging — Configures log persistence:

→ Volume mappings (KONG_LOGS_VOLUME, etc.)

→ Log file paths inside the container

This design ensures that all configuration is externalized, making the deployment reproducible and easier to manage across environments. The .env file will contain the following environment variable key/value pairs:

# Kong configurations
KONG_VERSION=3.14.0.1
NETWORK=kong-edu-net
KONG_CONTAINER_NAME=kong-routing
KONG_HOST_NAME=kong-routing
KONG_RESTART=on-failure
KONG_PROXY_PORT=8000
KONG_PROXY_SECURED_PORT1=8443
KONG_PROXY_SECURED_PORT2=443
KONG_STATUS_PORT=8101
KONG_LOGS_VOLUME=./kong/shared/logs
KONG_DOCKER_LOGS_VOLUME=/srv/shared/logs
KONG_CERT_VOLUME=./kong/shared/ssl
KONG_DOCKER_CERT_VOLUME=/srv/shared/ssl
KONG_ROLE=data_plane
KONG_DATABASE=off
KONG_VITALS=off
KONG_UPSTREAM_HOST=host.docker.internal
KONG_CP_ENDPOINT=ec8ad6cb89.in.cp.konghq.com:443
KONG_CP_SERVER_NAME= ec8ad6cb89.in.cp.konghq.com
KONG_TELEMETRY_ENDPOINT= ec8ad6cb89.in.tp.konghq.com:443
KONG_TELEMETRY_SERVER_NAME= ec8ad6cb89.in.tp.konghq.com
KONG_CLUSTER_MTLS=pki
KONG_LUA_SSL=system
KONG_UNTRUSTED_LUA=sandbox
KONG_PROXY_LISTEN=0.0.0.0:8000, 0.0.0.0:8443 http2 ssl, 0.0.0.0:443 http2 ssl
KONG_STATUS_LISTEN=0.0.0.0:8101 http2 ssl
KONG_NGINX_WORKER_PROCESSES=1
KONG_KONNECT_MODE=on
KONG_DP_LABLES=created-for:routing,type:docker-MacOSArm
KONG_ROUTER_FLAVOR=expressions
KONG_DEBUG_HEADER=on
KONG_INTERVAL=30s
KONG_TIMEOUT=30s
KONG_RETRIES=3

The value for the environment Kong_DP_LABELS should match the label added in the “Create a data plane node” modal dialog under the “Add Labels” section. It expects a key: value pair. These labels provide metadata to identify Kong Data Plane nodes running, connected and synchronized with the Kong Konnect Control Plane.

Configuring Kong Gateway Data Plane Node

In this blog we are using Kong Gateway Docker Image with gateway version 3.14.0.1 as specified in the .env file by the environment key/value pair:

KONG_VERSION=3.14.0.1

With the .env file in place, the docker-compose.yaml file defines the container topology and runtime behavior of the Kong Data Plane node.

The key concept here is environment variable interpolation. Docker Compose uses the ${VARIABLE_NAME} syntax to dynamically substitute values from the .env file into the service definition.

Network Definition

The networks section defines a custom bridge network used for container communication. In this setup, a network named kong-edu-net is declared, with its actual name dynamically resolved using the ${NETWORK} variable from the .env file.

networks:
  kong-edu-net:
    name: ${NETWORK}
    driver: bridge

The bridge driver enables isolated, internal networking between containers running on the same host. By externalizing the network name through the .env file, the configuration remains flexible and environment-agnostic, allowing the same Docker Compose definition to be reused across different setups without modification.

This network is then referenced by the kong-dp service to ensure that the Kong Gateway Data Plane container can communicate reliably within the defined container ecosystem.

kong-dp:
    networks:
    - ${NETWORK}
   container_name: "${KONG_CONTAINER_NAME}"
   hostname: "${KONG_HOST_NAME}"

The kong-dp service is explicitly attached to the custom bridge network defined earlier using the ${NETWORK} variable, ensuring that the container participates in the same isolated network namespace. The networks attribute associates the service with the dynamically resolved network from the .env file, enabling consistent inter-container communication.

The container_name parameter assigns a deterministic name to the container, simplifying container management and debugging operations (e.g., docker logs, docker exec).

The hostname defines the internal DNS name of the container within the network, which can be used for service-to-service communication if additional containers are introduced into the same network.

Together, these configurations ensure that the Kong Gateway Data Plane container is properly networked, identifiable, and accessible within the Docker environment.

Volume Mounting & Kong Gateway Container Health Check

The volumes configuration enables persistent storage by mapping host directories to paths inside the container. In this setup, log files and TLS certificates required by the Kong Gateway Data Plane node are externalized to the host system:

volumes:
  - ${KONG_LOGS_VOLUME}:${KONG_DOCKER_LOGS_VOLUME}
  - ${KONG_CERT_VOLUME}:${KONG_DOCKER_CERT_VOLUME}

The first mapping ensures that Kong access and error logs are persisted outside the container, making them available for debugging and observability even if the container is restarted or recreated. The second mapping provides the container with access to SSL/TLS certificates required for secure communication, including mTLS with the Control Plane.

The healthcheck directive defines a container-level health probe:

healthcheck:
   test: ["CMD", "kong", "health"]
   interval: ${KONG_INTERVAL}
   timeout: ${KONG_TIMEOUT}
   retries: ${KONG_RETRIES}

This periodically executes the kong health command inside the container to verify that the runtime is operational. The interval, timeout, and retry values are dynamically injected from the .env file, allowing fine-grained control over health monitoring behavior.

Finally, the restart policy ensures resilience:

restart: ${KONG_RESTART}

With on-failure (as defined in the .env), the container is automatically restarted if it exits due to an error, improving overall system reliability. Together, these configurations provide persistent storage, secure certificate management, runtime health validation, and fault tolerance for the Kong Gateway Data Plane container.

Port Mapping

The ports section defines how container ports are exposed to the host system, enabling external access to the Kong Gateway Data Plane services.

ports:
 - "${KONG_PROXY_PORT}:8000/tcp"
 - "${KONG_STATUS_PORT}:8101/tcp"
 - "${KONG_PROXY_SECURED_PORT1}:8443/tcp"
 - "${KONG_PROXY_SECURED_PORT2}:443/tcp"

Each entry follows the format: <host_port>:<container_port>/<protocol>

Environment values are dynamically injected from the .env file using variable interpolation.

Port Breakdown:

· Proxy (HTTP — Default Port) — ${KONG_PROXY_PORT}:8000

Exposes the primary HTTP proxy endpoint used for routing incoming client requests.

· Status API — ${KONG_STATUS_PORT}:8101

Exposes the status endpoint for health checks and runtime monitoring of the Kong instance.

· Proxy (HTTPS — Default Port) — ${KONG_PROXY_SECURED_PORT1}:8443

Enables secure HTTPS traffic over a configurable port (commonly used for development or non-standard setups).

· Proxy (HTTPS — Standard Port) — ${KONG_PROXY_SECURED_PORT2}:443

Maps the container’s HTTPS endpoint to the standard TLS port, allowing clients to access services without specifying a custom port.

By externalizing these port values in the .env file, the configuration remains flexible and can be adapted to different environments without modifying the Docker Compose definition. In this setup, the Kong Gateway Data Plane is configured with two TLS-enabled proxy listeners, allowing it to accept HTTPS requests on both the standard port (443) and a Kong default secure port (8443). This dual-port configuration is essential for implementing Expressions-based routing, where incoming requests can be filtered and routed based on the destination port number. This setup ensures that the Kong Gateway Data Plane node is accessible over both HTTP and HTTPS, while also exposing operational endpoints for observability and health monitoring.

Environment Section with Important Configuration Properties

The environment section defines the runtime configuration of the Kong Gateway Data Plane container. These variables are injected at startup via Docker Compose using values sourced from the .env file.

environment:
  UPSTREAM_HOST: ${KONG_UPSTREAM_HOST}
  KONG_CLUSTER_CONTROL_PLANE: "${KONG_CP_ENDPOINT}"
  KONG_CLUSTER_SERVER_NAME: "${KONG_CP_SERVER_NAME}"
  KONG_CLUSTER_TELEMETRY_ENDPOINT: "${KONG_TELEMETRY_ENDPOINT}"
  KONG_CLUSTER_TELEMETRY_SERVER_NAME: "${KONG_TELEMETRY_SERVER_NAME}"
  KONG_CLUSTER_MTLS: "${KONG_CLUSTER_MTLS}"
  KONG_CLUSTER_CERT: "${KONG_DOCKER_CERT_VOLUME}/cluster.crt"
  KONG_CLUSTER_CERT_KEY: "${KONG_DOCKER_CERT_VOLUME}/cluster.key"
  KONG_PROXY_LISTEN: "${KONG_PROXY_LISTEN}"
  KONG_STATUS_LISTEN: "${KONG_STATUS_LISTEN}"
  KONG_SSL_CERT_KEY: "${KONG_DOCKER_CERT_VOLUME}/server.key"
  KONG_SSL_CERT: "${KONG_DOCKER_CERT_VOLUME}/server.crt"
  KONG_STATUS_SSL_CERT_KEY: "${KONG_DOCKER_CERT_VOLUME}/server.key"
  KONG_STATUS_SSL_CERT: "${KONG_DOCKER_CERT_VOLUME}/server.crt"
  KONG_ROUTER_FLAVOR: "${KONG_ROUTER_FLAVOR}"
  KONG_PROXY_ACCESS_LOG: "${KONG_DOCKER_LOGS_VOLUME}/proxy_access.log"
  KONG_PROXY_ERROR_LOG: "${KONG_DOCKER_LOGS_VOLUME}/proxy_error.log"
  KONG_STATUS_ACCESS_LOG: "${KONG_DOCKER_LOGS_VOLUME}/status_access.log"
  KONG_STATUS_ERROR_LOG: "${KONG_DOCKER_LOGS_VOLUME}/status_error.log"

Key Responsibilities of this Configuration:

· Control Plane Connectivity: Parameters such as KONG_CLUSTER_CONTROL_PLANE, KONG_CLUSTER_SERVER_NAME, and telemetry endpoints establish secure communication with Kong Konnect.

· mTLS Security Setup: KONG_CLUSTER_CERT and KONG_CLUSTER_CERT_KEY define the client certificate pair used for mutual TLS authentication between the Data Plane and Control Plane.

· TLS Listener Configuration: KONG_PROXY_LISTEN and KONG_STATUS_LISTEN configure HTTPS-enabled listeners for both proxy traffic and the status endpoint.

· Expressions Routing Enablement: KONG_ROUTER_FLAVOR=expressions activates the Expressions router, which is central to implementing port-based API request filtering.

· Logging Configuration: Log file paths are explicitly defined to ensure structured logging for both proxy and status endpoints.

· Proxy TLS Certificate (Private Key): KONG_SSL_CERT_KEY points Kong’s HTTPS proxy listeners (ports 443/8443) to server private key file (server.key) mounted inside the container at ${KONG_DOCKER_CERT_VOLUME}, enabling TLS termination for inbound API traffic.

· Proxy TLS Certificate (Public Cert): KONG_SSL_CERT specifies the server certificate file (server.crt) presented to clients connecting to the HTTPS proxy listeners; together with KONG_SSL_CERT_KEY, it completes the cert/key pair used to establish encrypted client connections.

· Status API TLS Certificate (Private Key): KONG_STATUS_SSL_CERT_KEY points the TLS-enabled Status API listener (for example, port 8101 when configured with ssl) to the same private key (server.key) so health and readiness endpoints can be served over HTTPS.

· Status API TLS Certificate (Public Cert): KONG_STATUS_SSL_CERT provides the certificate (server.crt) that the Status API presents during the TLS handshake; using the same cert/key pair as the proxy keeps localhost HTTPS behavior consistent across both proxy and status endpoints.

Note: The proxy and status endpoints can share the same certificate/key pair because they terminate TLS on the same Kong Gateway container for the same hostname (e.g., localhost), just on different listener ports.

Log and Certificate Propagation to Host:

Logs and certificates generated or consumed by Kong Gateway container are persisted on the host machine through Docker volume mappings defined earlier:

· Logs are written to: ./kong/shared/logs

· Certificates are read from and maintained in: ./kong/shared/ssl

These host paths are mounted into the container as:

· ${KONG_DOCKER_LOGS_VOLUME} → /srv/shared/logs

· ${KONG_DOCKER_CERT_VOLUME} → /srv/shared/ssl

Because of this mapping:

· Any logs generated inside the container (e.g., proxy_access.log, proxy_error.log) are automatically written to the host filesystem

· TLS certificates (cluster and server certificates) stored in the host /ssl directory are directly consumed by the container at runtime

This ensures:

• Persistence beyond container lifecycle
• Traceability for debugging and monitoring
• Seamless certificate management without rebuilding the container

The container does not need explicit awareness of host paths. Docker handles the synchronization transparently through the defined volume bindings.

The docker-compose.yaml file in this project defines the complete containerized setup for running the Kong Gateway Data Plane node. It declaratively specifies the network configuration, container properties, volume mappings, port exposure, and environment variables required to establish secure communication with Kong Konnect Control Plane.

The configuration also leverages environment variable interpolation using the .env file, ensuring that all runtime parameters are externalized and can be easily modified without altering the core Docker Compose definition.

The file content is as follows:

networks:
  kong-edu-net:
    name: ${NETWORK}
    driver: bridge

services:
  ######################################################
  ########  Kong Data Plane Self-Managed  ##############
  ######################################################
  kong-dp:
    networks:
    - ${NETWORK}
    image: kong/kong-gateway:${KONG_VERSION}
    container_name: "${KONG_CONTAINER_NAME}"
    hostname: "${KONG_HOST_NAME}"
    volumes:
    - ${KONG_LOGS_VOLUME}:${KONG_DOCKER_LOGS_VOLUME}
    - ${KONG_CERT_VOLUME}:${KONG_DOCKER_CERT_VOLUME}
    healthcheck:
      test: ["CMD", "kong", "health"]
      interval: ${KONG_INTERVAL}
      timeout: ${KONG_TIMEOUT}
      retries: ${KONG_RETRIES}
    restart: ${KONG_RESTART}
    ports:
    - "${KONG_PROXY_PORT}:8000/tcp"
    - "${KONG_STATUS_PORT}:8101/tcp"
    - "${KONG_PROXY_SECURED_PORT1}:8443/tcp"
    - "${KONG_PROXY_SECURED_PORT2}:443/tcp"
    environment:
      KONG_ROLE: "${KONG_ROLE}"
      KONG_DATABASE: "${KONG_DATABASE}"
      KONG_VITALS: "${KONG_VITALS}"
      UPSTREAM_HOST: ${KONG_UPSTREAM_HOST}
      KONG_CLUSTER_CONTROL_PLANE: "${KONG_CP_ENDPOINT}"
      KONG_CLUSTER_SERVER_NAME: "${KONG_CP_SERVER_NAME}"
      KONG_CLUSTER_TELEMETRY_ENDPOINT: "${KONG_TELEMETRY_ENDPOINT}"
      KONG_CLUSTER_TELEMETRY_SERVER_NAME: "${KONG_TELEMETRY_SERVER_NAME}"
      KONG_CLUSTER_MTLS: "${KONG_CLUSTER_MTLS}"
      KONG_CLUSTER_CERT: "${KONG_DOCKER_CERT_VOLUME}/cluster.crt"
      KONG_CLUSTER_CERT_KEY: "${KONG_DOCKER_CERT_VOLUME}/cluster.key"
      KONG_LUA_SSL_TRUSTED_CERTIFICATE: "${KONG_LUA_SSL}"
      KONG_UNTRUSTED_LUA: "${KONG_UNTRUSTED_LUA}"
      KONG_PROXY_LISTEN: "${KONG_PROXY_LISTEN}"
      KONG_STATUS_LISTEN: "${KONG_STATUS_LISTEN}"
      KONG_SSL_CERT_KEY: "${KONG_DOCKER_CERT_VOLUME}/server.key"
      KONG_SSL_CERT: "${KONG_DOCKER_CERT_VOLUME}/server.crt"
      KONG_STATUS_SSL_CERT_KEY: "${KONG_DOCKER_CERT_VOLUME}/server.key"
      KONG_STATUS_SSL_CERT: "${KONG_DOCKER_CERT_VOLUME}/server.crt"
      KONG_NGINX_WORKER_PROCESSES: ${KONG_NGINX_WORKER_PROCESSES}
      KONG_KONNECT_MODE: "${KONG_KONNECT_MODE}"
      KONG_CLUSTER_DP_LABELS: "${KONG_DP_LABLES}"
      KONG_ROUTER_FLAVOR: "${KONG_ROUTER_FLAVOR}"
      KONG_PROXY_ACCESS_LOG: "${KONG_DOCKER_LOGS_VOLUME}/proxy_access.log"
      KONG_PROXY_ERROR_LOG: "${KONG_DOCKER_LOGS_VOLUME}/proxy_error.log"
      KONG_STATUS_ACCESS_LOG: "${KONG_DOCKER_LOGS_VOLUME}/status_access.log"
      KONG_STATUS_ERROR_LOG: "${KONG_DOCKER_LOGS_VOLUME}/status_error.log"
      KONG_ALLOW_DEBUG_HEADER: "${KONG_DEBUG_HEADER}"

The docker compose services and environment configuration settings will be executed via start.sh file, using bash script.

Verify decK Communication with Konnect Control Plane

Before starting the Data Plane container, it is important to validate connectivity with the Kong Konnect Control Plane. This ensures that authentication, endpoint configuration, and network access are correctly set up.

In this step, decK is used to perform a lightweight connectivity check using the deck gateway ping command. This confirms that the local environment can successfully communicate with the Control Plane before deploying the Kong Gateway Data Plane node.

The validation is driven by two artifacts:

· A configuration file (config.yaml) that defines Control Plane parameters

· A helper script (kong_connect.sh) that executes the connectivity check

This separation keeps configuration declarative and execution reusable. The configuration file config.yaml and bash script file kong_connect.sh required for this validation are organized under the sync folder.

The kong/scripts/sync/ folder contains:

· The routing.deck.yaml → Stores the Konnect Personal Access Token

· The config.yaml → Defines Control Plane connection parameters

· The kong_connect.sh → Executes the decK connectivity check

This structure ensures that authentication, configuration, and execution logic are co-located, making the setup modular and easier to manage.

The updated project folder structure with additional files will look as below:

KongRouting/
├── .env
├── docker-compose.yaml
├── start.sh
└── kong/
    ├── scripts/
    │   └── sync/
    │       ├── config.yaml
    │       ├── kong_connect.sh
    │       └── routing.deck.yaml
    └── shared/
        ├── logs/
        │   ├── proxy_access.log
        │   ├── proxy_error.log
        │   ├── status_access.log
        │   └── status_error.log
        └── ssl/
            ├── server.crt
            ├── server.key
            ├── cluster.crt
            └── cluster.key

Configuration & Execution Flow

· Read Control Plane configuration dynamically using yq from config.yaml file. This config.yaml file references the file routing.deck.yaml containing Konnect personal access token used by decK for authentication and defines

• Control Plane Name
• Konnect Control Plane API endpoint

· Use the bash script in kong_connect.sh to read values from config.yaml & execute deck gateway ping operation to validate Control Plane reachability

File config.yaml:

# Control Plane Routing
token_file: routing.deck.yaml
control_plane: Routing
control_plane_addr: https://in.api.konghq.com

File kong_connect.sh:

#! /usr/bin/env bash

# Directory in which the deck connect script runs
cd kong/scripts/sync

# Importing configuration variables
TOKEN_FILE=$(yq e '.token_file' ./config.yaml)
CONTROL_PLANE=$(yq e '.control_plane' ./config.yaml)
CONTROL_PLANE_ADDR=$(yq e '.control_plane_addr' ./config.yaml)

# Connecting Kong Gateway runtime to Control Plane Kong_Accelerator
deck gateway ping \
  --konnect-token-file ${TOKEN_FILE} \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR}

# Root directory of POC
cd ../../../

This design ensures all Konnect-related configuration is centralized within a single directory, avoiding fragmentation across the project. This step eliminates misconfiguration early and ensures that the Data Plane node will successfully register with Konnect once started.

Automated Deployment of Kong Gateway Data Plane Node

Once connectivity with the Konnect Control Plane is verified, the next step is to deploy the Kong Gateway Data Plane node using Docker Compose. This is handled through a bootstrap script (start.sh) located at the root of the project. The script ensures a clean deployment, starts the containerized service, and validates connectivity post-deployment.

To streamline the provisioning process, the Kong Gateway Data Plane node is deployed using an automated bootstrap bash script (start.sh). This script encapsulates the full deployment lifecycle ensuring a consistent and repeatable setup.

Rather than manually executing multiple commands, the script orchestrates the entire workflow using configurations defined in the files within sync folder to verify connectivity after deployment.

· The .env file → Environment variables for container runtime

· The docker-compose.yaml file → Service Containers and Network definition

· The kong/scripts/sync/config.yaml → Control Plane configuration

Execution Flow

1. Clean Existing Containers using docker compose down — remove-orphans command

→ Removes stale containers

→ Ensures a clean deployment state

2. Start Kong Data Plane Service using docker compose up -d command

→ Launches the kong-dp container in detached mode

→ Applies all configurations from.env and docker-compose.yaml

3. Load Control Plane Configuration via CONTROL_PLANE=$(yq e ‘.control_plane’ ./kong/scripts/sync/config.yaml) command

→ Dynamically reads Control Plane name and API endpoint from config.yaml

4. Validate Connectivity Post Deployment using source kong/scripts/sync/kong_connect.sh command

→ Executes decK validation script

→ Confirms runtime connectivity between Data Plane and Control Plane

This guarantees that the deployment is not just successful at the container level, but also successfully connected and synchronized with the Kong Konnect Control Plane.

File start.sh:

#! /usr/bin/env bash

docker compose down --remove-orphans
docker compose up -d

# Source the configuration file to import environment variables
CONTROL_PLANE=$(yq e '.control_plane' ./kong/scripts/sync/config.yaml)

# Checking connection between Kong Gateway runtime and Konnect Control Plane
printf '\n'
echo 'Establishing connection between Kong Gateway runtime and' ${CONTROL_PLANE} 'Control Plane in Konnect'

sleep 2

source kong/scripts/sync/kong_connect.sh

Using a Bash-based bootstrap script makes setup highly compatible with GitOps and APIOps pipelines. The same script can be integrated into CI/CD workflows to:

· Automate provisioning of the Konnect Control Plane

· Deploy and configure the Data Plane node

· Validate connectivity as part of pipeline execution

This approach ensures that infrastructure and API gateway configuration remain version-controlled, repeatable, and environment-agnostic.

Execute bash bootstrap script (start.sh) via terminal with project root folder KongRouting as current working directory.

Once the Kong Gateway Data Plane node is successfully deployed and connected, it establishes a secure connection with the Kong Konnect Control Plane hosted in the AWS cloud India region.

Upon successful synchronization:

· The Data Plane node registers with the Control Plane

· Configuration state is synced automatically

· The connection status becomes visible in the Konnect UI

As part of the setup flow, a “View your data plane node” button becomes available within the Create a data plane node modal dialog. This indicates that the Data Plane has been successfully connected and is ready to handle traffic.

Clicking the “View your data plane node” button navigates to the Data plane nodes page under the API Gatewaysection in the left navigation pane of Kong Konnect. In Data plane nodes page Kong Gateway running instance can be monitored and managed from the Information and Analytics tab.

As observed on the Data plane nodes page in the Gateway Manager section of Kong Konnect, the Host and Versionfields are displayed for each registered node. The Host value corresponds to the hostname configured for the container in the Docker Compose service definition, while Version reflects Kong Gateway image version specified using KONG_VERSION parameter as defined in the .env environment file.

These values provide quick visibility into the runtime identity and software version of the connected Data Plane node, helping verify that the correct configuration has been applied during deployment. Kong Data Plane node is successfully installed and connected, also as seen, it is in sync with the Konnect Control Plane hosted in AWS cloud India region.

Generating Self-Signed Certificate

By default, Kong Gateway can process HTTP (unencrypted) traffic without requiring certificates. However, enabling HTTPS endpoints requires explicit configuration of SSL/TLS certificates. While the In Sync status confirms that the Data Plane and Konnect Control Plane are properly connected, it does not automatically provision certificates for secure client communication.

Why Certificates Are Required

Without configuring certificates in Konnect:

· API endpoints are exposed only over HTTP, leaving traffic unencrypted

· HTTPS requests to the gateway will fail due to TLS handshake errors

· Browsers and clients will reject secure connections

· Security and compliance requirements for production environments cannot be satisfied

Creating Self-Signed localhost Certificate

For development, testing, or internal environments, self-signed certificates provide a practical and efficient solution. They provide the following advantages:

· Enable HTTPS endpoints on the Kong Gateway

· Test SSL/TLS behavior without relying on external Certificate Authorities

· Secure local or non-production environments

· Validate certificate configuration and gateway behavior before moving to production-grade certificates

In the following steps, a self-signed certificate for localhost will be generated and configured within the Certificates navigation item from the left navigation pane of Kong Konnect. This will enable Kong gateway to accept and process HTTPS-based API requests, allowing one to fully test secure routing scenarios, including port-based filtering using the Expressions router.

The certificate generation script bash file self_signed_certificate.sh is created in the /shared folder of the project, while the generated certificate and key are stored inside the ssl subfolder.

Project Working Directory after adding self_signed_certificate.sh bash file:
KongRouting/
├── .env
├── docker-compose.yaml
├── start.sh
└── kong/
    ├── scripts/
    │   └── sync/
    │       ├── config.yaml
    │       ├── kong_connect.sh
    │       └── routing.deck.yaml
    └── shared/
        ├── self_signed_certificate.sh
        ├── logs/
        │   ├── proxy_access.log
        │   ├── proxy_error.log
        │   ├── status_access.log
        │   └── status_error.log
        └── ssl/
            ├── server.crt
            ├── server.key
            ├── cluster.crt
            └── cluster.key

File self_signed_certificate.sh:

#! /usr/bin/env bash

#cd kong/shared/ssl - For Windows
cd ssl # For unix system

openssl req -x509 -days 365 \
  -out server.crt -keyout server.key \
  -newkey rsa:2048 -nodes -sha256 \
  -subj '/CN=localhost' -extensions EXT -config <( \
   printf "[dn]\nCN=localhost\n[req]\ndistinguished_name= dn\n[EXT]\nsubjectAltName=DNS:localhost,IP:127.0.0.1,IP:::1\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")

The script navigates to the ssl directory and generates a self-signed certificate using OpenSSL. The script is executed by running the source self_signed_certificate.sh command in the terminal from the /shared directory but explicitly switches to the ssl folder using cd ssl command. It generates the following files:

· The server.crt (certificate)

· The server.key (private key)

These files are stored in the directory path kong/shared/ssl/ inside the ssl folder of root project directory KongRouting. This directory path kong/shared/ssl/ is already mounted into the container (/srv/shared/ssl) via Docker volumes.

As a result, the Kong Gateway runtime can directly access these certificates for:

· HTTPS proxy endpoints (ports 443 and 8443)

· Status API TLS endpoint

This ensures that once the script is executed, the gateway is immediately capable of serving secure HTTPS traffic using the generated self-signed certificate. Add the self-signed certificate server.crt file in the kong/shared/ssl/ directory as a trusted certificate in the system trust store of the Operating System and marked as trusted for all users. The browser should be able to verify the self-signed localhost certificate, so that the API request https://localhost:8443/ to the default HTTPS proxy port 8443 of Kong Gateway will be considered as secured by the browser. Enable this in security settings of the browser, for instance in Google Chrome by allowing imported local certificates from Operating System to be used as a trusted certificate.

Uploading Certificate in Kong Konnect and Configuring SNI

Navigate to Certificates page in Kong Konnect UI from the API Gateway section under Certificates navigation menu item in left navigation pane.

The Certificates page is where one manages SSL/TLS certificates associated with Kong Konnect Control Plane.

1. Click “New certificate” button

2. Provide the following details:

▪ Certificate → Paste the contents of server.crt under Cert input field

▪ Private Key → Paste the contents of server.key under Key input field

▪ SNIs → Add localhost to match the Common Name value in self-signed certificate issued from Client Browser

3. Click Save

This registers the certificate with the Kong Konnect Control Plane.

Click the “Save” button to add the new certificate with the configured certificate, private key and SNIs into Konnect Control Plane. The user then navigates to the Certificate Detail page in Kong Konnect displaying the configuration details of the new Certificate that has been registered with the Kong Konnect Control Plane.

A message Successfully created certificate pops up in Konnect UI from the bottom once the new certificate has been saved and registered with the Konnect Control Plane which is Routing in our case. An ID for the Certificate is also created by Kong Konnect Platform. This ID will be later used to map the certificate with SNI added into Kong Konnect.

After uploading the certificate, configure the SNI to ensure that incoming HTTPS requests are correctly matched to the certificate. Select the Certificates navigation item again from left navigation pane in Kong Konnect UI. Navigate back to the Certificates page in Kong Konnect UI which shows the self-signed-local certificate under Certificates tab.

Configuring SNI

Select the SNIs tab in the Certificates page. Notice that the SNI added while creating the new certificate is never added to the SNIs section of the Certificates page in Kong Konnect UI. Click on New SNI button to navigate to the New SNI page in Kong Konnect UI.

Provide the following details for the fields in New SNI page:

1. Name → Add localhost, the Common Name of Self-Signed Certificate added into the Certificates page

2. Tags → Add self-signed-local text for this field.

3. SSL Certificate ID → Select the ID of the newly added certificate from the dropdown option

4. Click Save

SNIs (Server Name Indications) allow the Kong Gateway to determine which SSL/TLS certificate to use based on the hostname in the client request. Here, the client is the Browser that sends GET API request via Kong Gateway. It allows to use multiple domain names to securely serve requests through the same Kong Gateway. Saving this SNI maps the localhost domain in Client’s request to the newly added self-signed certificate in the Kong Konnect Certificates page.

A message Successfully created SNI pops up in Konnect UI from the bottom once a new SNI has been created and added to the Kong Konnect Control Plane, mapping the self-signed certificate to a hostname provide in the Name field.

Validating Kong Gateway Proxy & Status API

Now, one can make a request https://localhost:8443 from the browser to the Kong Gateway on HTTPS enabled secured default port 8443.

Browser initiates HTTPS connection to localhost:8443 the default HTTPS port of Kong Gateway:

· Kong presents the newly added self-signed certificate during TLS handshake with the client browser

· Browser encrypts the connection using the certificate sent by Kong Gateway

· Kong decrypts during TLS termination and doesn’t find any matching routes

· Response is encrypted back to the browser with 404 response status stating no Route matched with those values

The Browser considers the HTTPS connection to localhost:8443 secure and verified by localhost.

To check the health of Kong Gateway and verify if API requests can be served via the Kong Gateway send a request to the Kong Status API endpoint https://localhost:8101/status/ready from the Browser.

In Kong Docker Compose setup, the Status API listener is configured through the container environment section using KONG_STATUS_LISTEN=0.0.0.0:8101 http2 ssl. The ssl flag instructs Kong to enable TLS on the Status API port 8101, so the gateway expects a TLS handshake when a client connects to 8101. Therefore, health checks must be sent as HTTPSrequests (for example, https://localhost:8101/status/ready); a plain HTTP call to http://localhost:8101 would fail because the connection is not negotiated over TLS.

Since the Status API listener is TLS-enabled, an HTTP request to port 8101 fails because Kong expects an HTTPS (TLS handshake) connection.

When clients make HTTPS requests via Kong Gateway with the hostname localhost (or when the TLS handshake includes localhost as the Server Name Indication), Kong Gateway will:

1. Select the self-signed certificate during the TLS handshake based on the SNI matching

2. Kong Gateway will Present that self-signed certificate to the Client

3. Terminate SSL/TLS using the private key associated with the certificate

4. Decrypt the request and forward it to upstream services

5. Response is encrypted back to the Client

A 200 Ok response status code from Status API with message as “ready” indicates that Kong Gateway is setup properly to serve requests from Clients by forwarding them to upstream services receiving response and sending it back to those Clients that issued these requests, in a seamless manner.

This completes the certificate setup required to securely expose APIs over HTTPS and enables further testing of port-based Expressions routing on the Kong Gateway.

Next, in Part-3 of this blog series, the Kong echo API and HTTPBin API backend services will be onboarded into the Kong Konnect Control Plane using automated Bash scripts with multiple decK operations. We will also take a backup of the Kong Konnect Control Plane configuration state as the single source of truth, and patch the Kong declarative configuration file to automatically transform traditional route paths into Expressions Routes while injecting the Request-Termination plugin to handle unnecessary browser /favicon.ico requests.

Blog Series — Part-3

Automated Onboarding of Backend API Services into Kong Konnect Using decK and Declarative Configuration with Backup Synchronization

Use of Expressions Routing in Kong Gateway, to Filter Inbound API Requests by Destination Port…

Shubhojit Dasgupta — Independent API Architect

This blog series on Kong Gateway Expressions Router flows across four connected parts, with each part building progressively on the previous one to provide a complete understanding of expressions-based API traffic routing in Kong Gateway.

In this part of the blog series, we begin by exploring the API traffic flow architecture, gaining a brief understanding of Kong Expressions syntax, its advantages, and reviewing the overall solution overview of the routing implementation covered throughout the series. This part also walks through setting up the Kong Konnect API Platform and generating the Kong Konnect Personal Access Token required for securely managing the Kong Gateway Data Plane from the Konnect Control Plane.

Default Gateway Listening Ports

· 8000: HTTP (unencrypted) proxy port for inbound API requests

· 8443: HTTPS (TLS-encrypted) proxy port for inbound API requests

Kong Gateway sits in front of the upstream service. Client applications send requests to Kong, and Kong forwards matching requests to the appropriate upstream API (for example, a “get customer details” capability).

How an Upstream API Is Onboarded

The upstream provider’s API is onboarded into Kong through the Kong Konnect Control Plane. Configuration — such as the API definition, routes, and policies — is synced from Konnect to a self-managed Kong Data Plane (runtime gateway).

The client application uses the Kong Gateway proxy URL as the API’s server URL and sends requests to the route path configured in Kong.

Request/Response Flow and Traffic Visibility

1. The client sends a request to Kong (HTTP on 8000 or HTTPS on 8443).

2. Kong matches the request to a route and selects the associated service.

3. Kong forwards the request to the configured upstream backend service.

4. The upstream returns a response, which Kong relays back to the client.

API traffic (request/response payloads and runtime details) remains within the self-managed data plane environment; it is not exposed to the Konnect Control Plane.

In a traditional Kong Gateway configuration, route selection is typically based on request attributes such as the URL path, host, method, and headers. In that model as seen above the Kong Gateway runtime selects the Route whose configured path matches the incoming request. However, Kong Gateway can listen on multiple HTTPS ports exposed by the runtime for secure client communication, and in some use cases, the destination port is just as important as the request path. If one wants to allow a client application to access an API only when it sends the request to a specific HTTPS port exposed by Kong Gateway, path-based routing alone is not sufficient. In such cases, Kong must be configured to use the Expressions Router, which enables route matching based on both the request path and the destination port, allowing more precise control over how client requests are filtered and routed.

Takeaway Skills

This blog will help to understand and implement in an hands-on manner installing & configuring a self-managed Kong Gateway Data Plane running in your computer, synced with the Kong Konnect Control Plane, and ready for Expression routing. The blog covers the following with complete code, implementation and simplifications so that one can try out a successful implementation of the solution discussed in this blog for a specific use case in their computer. Kong Konnect free subscription for 30 days can also be used to implement the solution in this blog.

1. Kong Gateway Expressions Router, its use case and its advantages, and Solution overview.

2. Login to Kong Konnect Plus Account and setup Personal Access Token. This token shall be stored in a special file to be used with decK for securely and declaratively onboarding APIs, plugins and configurations into Kong Konnect API platform.

3. Provisioning Kong Konnect Control Plane and a Self-Managed Kong Gateway Data Plane.

4. Generating self-signed localhost certificate and key. This will be used to allow secure HTTP/2 connections to Kong’s proxy and health status endpoints. So a request https://localhost:8101/status/ready will be considered secured by your browser locally with Kong Gateway responding with health check status.

5. Installation of Kong Gateway version 3.14.0.1 with Expressions router enabled.

6. Configuring self-signed localhost certificate & key enabling TLS on Kong Gateway proxy and status ports with two secured ports listening for incoming API requests. This allows client Apps to make Https based API request to Upstream backend service using localhost via the Kong Gateway.

7. Onboarding APIs and plugins declaratively using deck into Kong Konnect Control Plane.

8. Adding Kong request-termination plugin to prevent unnecessary requests to /favicon.ico path by browsers.

9. Bash .sh script file to automate the execution of multiple decK commands and installation of Kong Gateway.

10. Testing and Validating if API requests are getting filtered by destination port numbers in Kong Gateway.

11. Automated Backup and Reconciliation of the Kong Konnect Control Plane Configuration State.

Pre-requisites

1. Good laptop, internet connection and Kong Konnect SaaS API platform account.

2. Docker desktop and Docker compose installed.

3. CLI decK version 1.59.x to declaratively configure Kong Konnect Platform.

4. OpenSSL version 3.6.x.x to generate self-signed certificate and key for the Kong Gateway.

To learn how to install decK visit dynamic-routing-part-6 blog and refer site to install OpenSSL. Installation of decK and OpenSSL will not be covered in this blog. The decK commands used, shall be explained with full code description.

Kong Gateway Expressions Router

The routing engine in Kong Gateway (available from version 3.0+) for both traditional and expressions router is ATC (Abstract tree classifier) built using Rust language for high security with performance. Expressions router provides a Domain Specific Language (DSL) for defining complex routing rules. Instead of configuring routes with JSON criteria, that one does using the traditional route mode, one can write logical predicates and expressions that evaluate inbound API requests. Expressions router ensures good runtime matching performance in the gateway by providing expression like syntax to match specific route comparisons using predicates containing non-regex equality checks that are not available in traditional router.

A predicate has three parts:

http.path ^= "/echo"

· Field (http.path): Value extracted from the incoming API request issued by the Client Application

· Operator (^=): The comparison to perform. It checks if the given request path starts with specified value

· Constant value (“/echo”): What the field is compared against

Predicates can be combined using logical operators (&&, ||, !) to form complex expressions.

Expressions Router Performance Benchmark

The Expressions router in Kong Gateway is generally preferred because it provides a more efficient and flexible routing model: it uses a DSL-based approach that enables optimized comparisons (e.g., non-regex equality checks), short-circuit evaluation, and better handling of complex routing logic, which translates to improved runtime matching performance and scalability — especially when dealing with large numbers of routes or regex-heavy configurations. Kong itself recommends using the expressions router for new deployments because it is more powerful and expressive than the legacy JSON-based routing model.

Delay using Expressions routing in scenarios where backward compatibility or simplicity is critical. The traditional JSON router is still relevant when migrating legacy configurations, when teams prefer a simpler declarative structure without introducing a DSL, or when existing tooling depends on the older routing format. Kong recommends using JSON format traditional router when manipulating routes in APIOps pipeline via decK file patch commands. However, in the blog it will be shown how deck file patch commands in bash script elegantly convert existing traditional routes automatically by replacing paths: [“/api/v1”] with expression:http.path == “/api/v1” && http.method == “GET” without downtime or full config rewrites.

Advantages

1. Expressions Router is designed for better runtime matching performance by supporting efficient comparisons such as exact and prefix-based matching, instead of relying heavily on regex. This helps reduce routing overhead and improves request processing speed, especially in high-traffic API environments.

2. With its domain-specific language (DSL), the Expressions Router allows to define more advanced routing logic than traditional routing. One can match requests based on paths, headers, query parameters, source IPs, destination ports, and SNI, making it easier to implement precise and sophisticated traffic control.

3. In Konnect environments with many services and routes, the Expressions Router helps improve scalability by allowing teams to combine multiple routing rules into a single expression. Fewer route objects mean cleaner configuration, simpler management, and more efficient route evaluation as deployments grow.

4. The Expressions Router gives developers more control over how routes are evaluated through route priorities and logical operators such as && and ||. This optimises routing behavior by placing more likely matched routes before (as in, higher priority) less frequently matched routes., reduce unnecessary regex usage, and maintain cleaner, more readable configurations over time.

API Traffic Filtering Use Case

Consider a Kong Gateway deployment that exposes multiple upstream services through different gateway listener ports, while still sharing a single logical hostname. This pattern is common in platform engineering when isolating traffic flows without introducing additional domains or complex routing layers.

In this scenario, Kong is configured with two HTTPS listener ports:

• Port 443 → serves a local echo service
• Port 8443 → serves the public HttpBin service over the internet

The key requirement is strict traffic segregation: a request should only match a route if both the HTTP attributes (path/method) and the destination port are correct.

Routes and Expressions

Two routes are defined, each tightly scoped using expressions:

Echo service (port 443)

• Path: /echo
• Method: GET
• Expression: http.method == "GET" && http.path ^= "/echo" && net.dst.port == 443

This ensures the route only matches:

• HTTP GET requests
• Paths starting with /echo
• Requests arriving specifically on port 443

HttpBin service (port 8443)

• Paths: /anything and /status/<code>
• Combined using logical OR (||) operator
• Expression: (http.path ~ r#"^/status/(?P<status_code>[1-5]\d{2})$"# || http.path ^= "/anything") && net.dst.port== 8443

This route matches:

• /status/<code> where <code> is a valid HTTP status (100–599)
• Any path starting with /anything
• Only when the request arrives on port 8443

The critical detail here is that port matching is part of the routing condition, not an external configuration. This gives deterministic control over how traffic is segmented at the gateway level.

Expected request/response behavior (Success vs 404 cases)

With port-aware expressions in place, routing becomes unambiguous and predictable.

Successful requests

• GET https://<gateway-host>:443/echo
• Matches echo route
• Proxied to echo upstream
• Response returned via Kong
• GET https://<gateway-host>:8443/status/200
• Matches HttpBin /status/<status-code> route path
• Proxied to HttpBin
• Returns HTTP 200 response
• GET https://<gateway-host>:8443/anything
• Matches HttpBin /anything route path
• Proxied successfully to the /anything endpoint of HttpBin upstream service
• HttpBin service returns the request headers as response with HTTP 200 Ok response status back to client via Kong Gateway

Rejected requests (port mismatch → 404)

• GET https://<gateway-host>:8443/echo
• Path matches /echo, but port is incorrect
• No route match
• Kong returns: 404 No matching routes found
• GET https://<gateway-host>:443/status/200
• Path matches HttpBin pattern, but port is incorrect
• No route match
• Kong returns: 404 No matching routes found
• GET https://<gateway-host>:443/anything
• Same outcome: 404 due to port mismatch

API request Filtering is performed on the gateway’s destination port (net.dst.port), not based on upstream service configuration. The upstream is never reached unless the full expression evaluates to true. The “gateway-host” is the Kong Gateway proxy URL which is localhost for this blog secured by a self-signed certificate. Self-signed certificate and key are stored in separate files with absolute path of those files added to environment properties in Kong Gateway. Load the self‑signed cert/key into Kong so it can present localhost over HTTPS and decrypt incoming traffic before routing upstream.

Handling /favicon.ico with request termination

Browsers automatically issue background requests to /favicon.ico, which can introduce unnecessary noise and unintended upstream calls.

To prevent this, a request-termination plugin is configured to intercept /favicon.ico requests on both ports (443 and 8443) and return an immediate response. This ensures such requests are handled at the Kong gateway layer and never forwarded to either the echo or HttpBin upstreams. Throughout this blog series, we will verify the operational behavior of the request-termination plugin within Kong Gateway.

Solution Overview

Kong Gateway enables expressions routing by setting property `KONG_ROUTER_FLAVOR` to `expressions` in the environment configuration section of Kong Gateway installation in its docker compose service container. By setting KONG_ROUTER_FLAVOR=expressions in Kong container’s environment (Docker Compose), Kong allows routes to be defined using expression like syntax using fields such as net.dst.port, http.path, and http.method. This solution uses Kong Gateway with expressions router to enforce port-based API traffic isolation and handle unnecessary /favicon.ico browser requests efficiently. Kong Gateway listens on two secured ports that is tls enable for receiving HTTPS based requests one on port 443 for the Echo service and other on port 8443 for the httpbin service and evaluates incoming requests against route expressions that include equality checks on net.dst.port field.

Requests matching expression and port are forwarded to the appropriate upstream, service (green arrow), and responses return to the client (blue dashed arrow). Browsers sending requests for /favicon.ico on either port are intercepted by request-termination plugin, which returns a response directly from Kong Gateway without calling any upstream (orange arrow). If a request arrives on the wrong port but the path matches for the intended route (e.g., /echo on port 8443 or /anything on port 443), Kong returns 404 No matching route found response immediately (red dashed arrow from right to left), and the request is not forwarded to upstream. This ensures strict port-based routing, clean separation of API traffic and reduced noise from unnecessary browser requests (/favicon.ico).

Kong Konnect Login

Login to Kong Konnect using the link https://cloud.konghq.com/login and Sign In with your email and credentials.

Setting Up Kong Konnect Platform

Create an Organization and give it a proper name. Enable MFA enrolment as it is strongly recommended by checking the box. When enabled, users who sign in with a username and password must use multi-factor authentication (MFA).

This can be updated in security settings under Organization section of left-navigation pane in Kong Konnect Platform.

Disable multi-factor authentication if required but ensure Personal access tokens is enabled as decK CLI (Command Line Interface) tool authenticates to Kong Konnect using Personal Access Token (PAT) and interacts with the Konnect Control Plane APIs to declaratively manage and sync gateway configuration. The decK CLI tool pushes configuration (services, routes, plugins, etc.) to the Kong Konnect Control Plane, which then propagates it to connected Data Planes.

After the organization name has been created, select a Konnect region to determine where the Kong Konnect Control Plane, fully-managed by Kong team in AWS cloud is hosted and operated. Choosing the appropriate geographic region is essential for meeting data residency, privacy, and regulatory compliance requirements. Regional deployment also helps keep Konnect closer to end users and supporting infrastructure, which can improve operational alignment and performance. Some resources such as Consumers & Gateway Services are geo-specific and exist only within the region where they are created, while authentication, billing, and usage remain shared across Konnect regions. In this blog India is selected as the region, so the Kong Konnect Control Plane gets provisioned in AWS cloud and hosted in India, Mumbai (ap-south-1) region being fully-managed by Kong Konnect team following the best practices in AWS cloud.

Any available Konnect region can be selected, ideally the geographic location closest to the team & core infrastructure and the port-based routing solution in this blog remains unchanged. The only adjustment is ensuring that tooling targets the region-specific Konnect Control Plane API endpoints. When using decK to synchronize configuration, the Control Plane address must point to the selected region (for example, via — konnect-addr), so all reads and writes occur against the correct regional Control Plane.

After selecting India region for Konnect organization, ensure that automation tools point to the India (Mumbai / ap-south-1) Control Plane API endpoint. For decK, this is done by setting the --konnect-addr flag to the India regional API base URL:

deck gateway ping \
  --konnect-token-file ${TOKEN_FILE} \
  --konnect-control-plane-name ${CONTROL_PLANE} \
  --konnect-addr ${CONTROL_PLANE_ADDR}

The value for --konnect-addr flag is provided through a placeholder that is populated from the CONTROL_PLANE_ADDR environment variable. This keeps the command portable across Konnect regions by changing only the environment variable value.

In this blog, India region is used, so https://in.api.konghq.com is configured as the value for --konnect-addr flag to allow decK to connect & synchronize configuration with Kong Konnect Control Planes hosted in India, Mumbai (ap-south-1). These Geo specific Kong Konnect Control Plane APIs are documented in decK Konnect configuration guide.

The default API decK uses is https://us.api.konghq.com. The default region selected, if not specified, is the US region.

After creating the Kong Konnect Organization, assigning an appropriate name, and selecting the region, the next step is to generate a Kong Konnect Personal Access Token (PAT). This token is used to authenticate API calls and allows tools such as decK to connect to Konnect and manage configuration in the selected Control Plane.

Before generating the Kong Konnect Personal Access Token, create the local project structure used to store scripts and decK configuration. Create a project root folder named KongRouting, then create a kong folder inside it. Under kong, create scripts/sync folder, and add a decK state file named routing.deck.yaml in the sync folder. The file name can be anything. A recommended naming convention is <Control-Plane-Name>.deck.yaml.

Project Folder Structure:
KongRouting/
└── kong/
    └── scripts/
        └── sync/
            └── routing.deck.yaml

Generate Kong Konnect Personal Access Token

Kong Personal Access Tokens (PATs) are essential for authenticating with Kong Konnect APIs and CLI tools. They provide secure, programmatic access to your Konnect resources and are required for automation, CI/CD pipelines, and various Kong tools. A Personal Access Token is a secure credential that grants API access to Kong Konnect with the same permissions as your user account. PATs are used by:

• Kong CLI tools (kongctl, decK)
• CI/CD pipelines for automated deployments
• Custom applications integrating with Konnect APIs
• Infrastructure as Code tools like Terraform

To generate a Personal Access Token in Kong Konnect, navigate to your user profile by clicking the profile icon in the top-right corner of the Konnect UI. From the dropdown menu, select “Personal Access Tokens” to access the Kong Konnect Personal Access Token management page, where one can create and manage tokens for authenticating with Konnect APIs and tools such as decK.

Generate a New Token

1. Click the “Generate Token” button
2. Provide a descriptive name for token (e.g., “CI/CD Pipeline”, “Local Development”, “Terraform Automation”)
3. Optionally set an expiration date for enhanced security
4. Click “Generate”

Copy and Store the Token Securely

The Kong Konnect Personal Access Token is displayed only once at the time of creation. Ensure to copy and securely store it in the routing.deck.yaml file within the sync directory for use with decK.

Within the root project directory KongRouting, create a .env file to centralize all environment variables required for the Kong deployment and its dependent services. Alongside this, include a docker-compose.yaml file to define the containerized topology and a start.sh script to bootstrap Kong Gateway with configurations dynamically interpolated from the .env file.

Additionally, create a shared/ directory under the kong/ subdirectory in KongRouting to manage persistent and security-related artifacts. Inside this directory, define two subfolders:

· logs/ — used to store proxy and status logs generated by the Kong Gateway container

· ssl/ — used to store TLS assets, including cluster and server certificates along with their corresponding private keys

Updated Project Working Directory:
KongRouting/
├── .env
├── docker-compose.yaml
├── start.sh
└── kong/
    ├── scripts/
    │   └── sync/
    │       └── routing.deck.yaml
    └── shared/
        ├── logs/
        │   ├── proxy_access.log
        │   ├── proxy_error.log
        │   ├── status_access.log
        │   └── status_error.log
        └── ssl/
            ├── server.crt
            ├── server.key
            ├── cluster.crt
            └── cluster.key

Next, in Part-2, we will provision the Kong Konnect Control Plane with a Self-Managed Kong Gateway Data Plane, install & and configure the Self-Managed Kong Gateway runtime, generate a self-signed TLS certificate, then upload it into the Kong Konnect Platform, and finally validate the operational behavior of the Kong Gateway Proxy and Status API endpoints.

Blog Series — Part-2:

Provisioning Kong Konnect Control Plane with Self-Managed Kong Gateway

Use of Expressions Routing in Kong Gateway, to Filter Inbound API Requests by Destination Port…

By teaming up WunderGraph with DreamFactory, the Organisation can create a super cool unified API management and integration layer! This layer will bring together authentication, access control, and observability from all sorts of different systems. WunderGraph’s API Gateway is awesome, and when you mix it with DreamFactory’s data source connectors and automation tools, one gets a workflow orchestration that’s smooth as butter with consistent policy enforcement, and a reliable way to abstract the backend’s complexity.

This integrated approach delivers a scalable, secure, and future-ready API platform that supports omnichannel service delivery, accelerates time-to-market for digital initiatives, and maintains robust governance over critical business workflows and data assets.

Benefits for AI/ML Integration

The Unified API Layer, powered by DreamFactory, WunderGraph and enhanced by the Kong AI Gateway, enables seamless integration and management of AI and ML capabilities across the organisation.

DreamFactory and WunderGraph provide a unified access point to diverse data sources and services, while Kong AI Gateway governs & secures the AI prompt layer to ensure security, compliance, and consistent performance.

Together, these technologies create a scalable and efficient foundation for developing, deploying, and leveraging AI-driven solutions that accelerate innovation and business value.

How WunderGraph Helps

GraphQL Resolvers can be used to integrate AI/ML endpoints into the unified API. For example, a GraphQL query can be designed to fetch real-time traffic predictions:

query {
  predictTraffic(location: "XYZ") {
    trafficCongestionLevel
    accidentRisk
  }
}

WunderGraph allows you to expose these AI model predictions in a consistent API structure, ensuring that other services and apps can easily access and act on these predictions providing the following benefits.

● Data Accessibility: The Unified API Layer ensures that AI/ML models can easily access data from multiple sources, whether legacy systems, cloud databases, or real-time data streams.

● Real-Time AI Inference: By exposing AI/ML models through the Unified API, frontend applications can make real-time predictions and decisions based on AI models.

● Simplified AI Model Deployment: The Unified API abstracts the complexity of deploying and integrating AI models across different systems, making them accessible via a single API.

● Performance & Scalability: Caching and API orchestration provided by WunderGraph ensure that AI models can scale efficiently, even when dealing with large data volumes and frequent requests.

● Unified Data Pipeline: The API layer can act as a unified data pipeline, ensuring that data from multiple sources is pre-processed and delivered in the right format to the AI/ML models.

Simplified Deployment of AI/ML Models

The Unified API Layer can act as a bridge between AI/ML model hosting platforms and operational systems. Whether your AI/ML models are deployed in the cloud (AWS Sagemaker, Google Cloud AI) or on-prem, the Unified API Layer abstracts the complexity and provides a single access point for all applications to interact with the models.

Example: License Plate Recognition

For instance, an AI-based license plate recognition (LPR) model hosted on a cloud service. The Unified API Layer can:

● Fetch license plate data from surveillance systems or vehicle registration databases.

● Send the data to the AI/ML model for recognition.

● Return the recognised license plate and associated vehicle information via the same unified API to frontend applications (such as law enforcement apps or toll management systems).

This abstracts away the underlying complexity of integrating the AI model, allowing the frontend systems to easily access AI-powered services.

Unified Data Pipeline for AI/ML

AI/ML models often need to access data in a pipeline format, where data from multiple sources is cleaned, processed, and prepared before being used for inference or analysis. The Unified API Layer can orchestrate this data flow, allowing the data to be sent to the AI/ML models in the required format.

Example: Driver Behavior Analytics for Insurance

An Organisation might want to work with insurance companies to provide driver behavior analytics for risk assessment.

Unified API Layer:

● Aggregates data from driving history (from the vehicle database), accident history, and external factors (e.g., traffic, weather).

● Sends the aggregated data to an AI model for risk analysis.

● The AI model predicts the driver’s risk score (e.g., low-risk, high-risk).

● The score is returned to various stakeholders (insurance companies, law enforcement, etc.) through the unified API.

This ensures that the AI model gets consistent, relevant data and that the frontend systems can access predictions in real time, allowing insurers or authorities to make informed and timely decisions.

Infrastructure & Networking High-Level Overview

Benefits of GraphQL Based Unified API Layer

WunderGraph sits on top of DreamFactory and composes these REST APIs into a Unified GraphQL schema. It allows you to merge multiple REST APIs, third-party services, or any other APIs into a single GraphQL endpoint. It abstracts the complexity of querying multiple services by letting clients use GraphQL to fetch data in a structured, efficient way with following benefits.

Efficient Data Fetching

• With GraphQL, clients can specify exactly what data they need, reducing over-fetching or under-fetching of data.
• For example, instead of making multiple REST API calls to DreamFactory to fetch user data, order history, and shipping details, you can fetch all the necessary data in a single GraphQL query.

Flexible Queries

Aggregate data from different sources (databases, external APIs, etc.) into one unified response. GraphQL’s query language is powerful and flexible, allowing clients to query exactly what they need from the unified API layer.

Real-time Support

WunderGraph supports real-time subscriptions, allowing you to turn REST APIs (which are inherently not real-time) into something you can subscribe to, providing real-time updates for clients.

Example Workflow:

1. DreamFactory generates REST APIs for different data sources (e.g., a database with user data, a payment service, etc.).
2. WunderGraph composes these REST APIs and transforms them into a single GraphQL API.
3. Your application interacts with the GraphQL API, sending queries that internally get mapped to the corresponding REST APIs via WunderGraph.

GraphQL Query Example (Unified API Layer):

query {
  user(userId: 123) {
    name
    email
  }
  orderHistory(userId: 123) {
    orderId
    total
    items {
      name
      price
    }
  }
  processPayment(orderId: "abc123", amount: 50.00) {
    status
    transactionId
  }
}

This query fetches data from multiple sources (e.g., user information, order history, and processes a payment), but it appears as a single API endpoint thanks to the unified GraphQL layer created by WunderGraph. Behind the scenes, each part of the query might be hitting different REST endpoints generated by DreamFactory.

This approach will give you the flexibility, efficiency, and real-time capabilities of GraphQL while still leveraging DreamFactory’s ability to expose backend services as REST APIs.

Integrated Observability & Monitoring

WunderGraph supports OpenTelemetry out of the box. OpenTelemetry is a collection of tools, APIs, and SDKs used to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) for analysis to understand your APIs performance and behavior. With the help of OpenTelemetry, you can monitor your WunderGraph nodes and get a better understanding of your application’s performance. It instruments all outgoing HTTP requests and inner service calls with OpenTelemetry.

WunderGraph supports OpenTelemetry backend that supports the OTLP http protocol. One can also use the OpenTelemetry Collector to export traces to any other backend that is supported by the collector.

Grafana Loki

Grafana Loki collects logs independently of the built-in Prometheus exporter. Here’s how the integration typically works:

Log Shipping:

• WunderGraph itself doesn’t directly send logs to Loki. Instead, you need to set up log shipping in your application.
• Common log shippers include tools like Fluentd, Fluent Bit, Logstash, or Promtail (which is specifically designed to work with Loki).

Using Promtail:

• If you choose to use Promtail, you can configure it to watch log files generated by your application (including those from WunderGraph).
• Promtail reads the logs and sends them to your Loki instance. You can specify various configurations, such as how to parse the logs and which labels to apply.

Direct Integration:

• You can also directly instrument your application to send logs to Loki using the Loki client libraries available in various programming languages.
• This method allows for more customised log handling and flexibility.

Grafana Setup:

• After logs are collected and sent to Loki, you can then configure Grafana Alloy to use Loki as a data source.
• You can create dashboards and queries in Grafana to visualize the logs alongside metrics from Prometheus and traces from Jaeger.

Grafana Loki collects logs from your applications through log shippers or client libraries, not directly through the Prometheus exporter. The logs can then be visualized in Grafana, providing a comprehensive observability solution alongside metrics and traces.

Grafana Alloy

Grafana Alloy is designed to provide an integrated observability experience, it pulls the telemetry data from Prometheus Server, Grafana Loki and OpenTelemetry collectors like Jaeger and sends them to Grafana user interface (UI) for visualisation and dashboard creation.

Grafana Alloy acts as a bridge, allowing you to configure data sources that Grafana can query. You can set up Grafana to pull data from various telemetry sources, including:

• Prometheus: For metrics collection.
• Jaeger: For distributed tracing data.
• Zipkin: For tracing data.
• Loki: For logs.

Grafana Alloy aims to unify observability across your applications, aggregating metrics, logs, and traces from various sources (like Prometheus, Loki, and Jaeger) in one platform. It uses the Grafana UI, which is a powerful tool for creating custom dashboards, charts, and graphs based on the collected telemetry data. While it streamlines data collection and organisation, users still need to interact with the Grafana UI to create visualisations, alerts, and dashboards.

Grafana

Grafana, allows you to create dashboards that visualise the telemetry data. This includes:

• Graphs and charts for metrics like response times, request counts, and error rates.
• Trace views to visualise the flow of requests and understand performance bottlenecks.
• Logs for real-time monitoring and troubleshooting.

With Grafana set up alerts based on the telemetry data to notify your Teams of potential issues, such as high error rates or latency spikes.

API Led AI Experience

The mainstream availability of generative AI (genAI) technologies, large language models (LLM), and vector databases (DBs) has created an unprecedented race for AI adoption. AI’s rapid evolution and heavy reliance on data introduces new complexities that must be managed as increasing pressure mounts to leverage AI.

Kong AI Gateway facilitates an Organisation to integrate AI into their Applications with greater Ease, Security, and Standardisation. This simplifies the process of Connecting, Managing, and Securing access to multiple LLMs without needing to write any code.

No AI Without API

Kong AI Gateway

Kong AI Gateway serves as a centralised control layer connecting enterprise applications with multiple large language models (LLMs).

As an intelligent intermediary, Kong AI Gateway enables applications to consume AI capabilities securely and efficiently through built-in features such as AI security, AI prompt firewalling, context management, AI model token-based rate limiting, and multi-LLM endpoint orchestration.

It abstracts the complexity of prompt management, credentials, logging, and metrics, allowing developers to focus on business logic while ensuring governance, reliability, and scalability in AI-driven workflows. This unified approach helps organisations standardise & optimise their AI interactions across diverse AI-driven models and environments.

Multi LLM Strategy

AI LLM Models are trained via APIs managed and governed via Kong AI Gateway, facilitating in developing a Multi LLM strategy for Organisations with Compliance and AI Governance.

Kong AI Gateway’s multi-LLM strategy enables organisations to connect their applications to multiple large language models simultaneously, choosing the best model for each task based on factors like speed, cost, reliability, or topic expertise. Through advanced features such as semantic routing, prompt caching, and load balancing, the AI Gateway automatically directs AI requests to the most suitable LLM, ensuring optimal performance and redundancy if one model becomes unavailable. This unified approach not only increases resiliency and flexibility but also allows seamless scaling and governance across complex AI-driven workflows.

Conclusion

This is a two part blog, where in this part we covered the following;

● AI/ML integration with Unified API Layer generated using WunderGraph and DreamFactory significantly improving the operations of AI models.

● Exposing AI model predictions in a consistent API structure, using WunderGraph with Real-Time AI Inference based on AI Models.

● Deployment of AI/ML Model with performance and scalability simplified using Unified Data Pipeline orchestrated through the Unified API Layer.

● High-Level Overview of Networking and Infrastructure Architecture.

● Benefits of GraphQL Based Unified API Layer.

● Integrated Observability & Monitoring using Grafana Tech Stack.

● API led AI experiences via Kong AI Gateway facilitating an Organisation to integrate AI into their Applications with greater Ease, Security, and Standardisation and Compliance with Multi-LLM strategy.

Visit the previous blog to understand the solution for a robust, fully-integrated Unified API Layer that acts as an abstraction between multiple APIs, omni-channel systems, modern/legacy systems, databases, and applications.

In the previous blog we explored different platforms and High-Level Architecture combining strengths of DreamFactory & WunderGraph supporting for an omnichannel experience by delivering seamless, integrated service across multiple channels such as mobile apps, web platforms, kiosks, and call centres.

Summary

AI Driven Unified API architecture serves various business needs for an Organisation, providing unified API management Platform to interact with both legacy systems (via REST APIs) and modern services (via Unified API Layer), while maintaining performance and security.

The Architecture outlines a Scalable, Modular Architecture designed to optimise API consumption, Orchestration, Aggregation, Federation and REST API generation while maintaining clear separation between unified API Orchestration and Integration Layer (WunderGraph) and REST APIs generation layer (DreamFactory).

AI and APIs are revolutionising industries. While challenges exist, the potential for innovation and discovery is limitless. Kong’s AI capabilities, along with WunderGraph and DreamFactory integration, offer solutions for organisations that adapt, harness AI’s creative potential, and leverage APIs to explore new possibilities.

Solution Overview

A robust fully integrated unified API layer serves as an abstraction that connects multiple APIs, omnichannel systems, modern and legacy systems, on-premises and cloud databases, and the applications that directly consume them. This layer provides a single point of access and a standardised interface for various services. The objective is to streamline and simplify developer interactions with multiple APIs, irrespective of their complexity or implementation differences.

Proposed Platforms & Architecture

The solution will feature a Vendor-Agnostic Unified API Layer that combines the strengths of DreamFactory and WunderGraph. These Low-Code Open-Source Platforms facilitate seamless API generation, loosely coupled orchestration and minimal code integration. The solution will enable efficient API creation from multiple data sources and advanced API orchestration for Microservices, Legacy Systems, Databases, IoT devices and external RESTful APIs.

Once API Orchestration/Integration and REST API layers, are respectively generated and stable, then Kong AI Gateway integrates in-between these layers effectively managing and governing AI prompts from Unified API System powered by WunderGraph through API Orchestration/Integration Layer, with the REST API Layer generated using DreamFactory.

A Robust and Powerful Low-Code Open-Source solution which streamlines backend API generation and orchestration, providing a scalable, flexible, and secured standardised interface for developers and systems to interact directly with diverse Data Sources, whether on-prem or hosted on Cloud.

What is DreamFactory?

DreamFactory is an Open-Source REST APIs generation solution, best known for its ability to automatically generate, secure and documented APIs for databases like Microsoft SQL Server, MySQL, Snowflake, PostgreSQL, Oracle, MongoDB, Time-Series Databases and more. It is built on top of the Laravel framework, and includes a convenient web-based administration client. DreamFactory does the following;

• Generate powerful, reusable, documented REST APIs for SQL and NoSQL databases, files, email, push notifications and more in seconds.
• It uses the PHP, Python, and NodeJS scripting languages to easily customize REST API behavior at any endpoint, for both API requests and API responses.
• It secures every API endpoint with user management, Token-based authentication, role-based access control, OAuth 2.0, OpenID Connect and API Keys with basic Rate Limiting features. For advanced security features one would have to upgrade it later to an Enterprise version.

DreamFactory High-Level Architecture

DreamFactory Components

The DreamFactory application can be divided into several operational components. While these components are logical and do not necessarily represent the code structure itself, they are useful in discussing the subsystems and functionality of the application, as well as the anatomy of the API call later.

What is WunderGraph?

WunderGraph is a Backend for Frontend (BFF) Framework designed to optimize Developer Workflows through API Composition. At its core, WunderGraph combines two patterns, API Gateway and Backends for Frontends providing Single-purpose Edge Services for UIs and external parties with the concept of a package manager, making API composition simple.

API Composition

API Composition is a new pattern that allows one to interact with a heterogeneous set of APIs as if they were a single unified API. This not only eliminates a lot of glue code, but also allows you to reason about the API Dependencies of an application. Do you actually know what APIs and Services your application depends on? WunderGraph can easily answer this question for you, and even gives you analytics and observability into Organisation’s APIs and Endpoints that are consumed by all its applications and backend-services and also the quality of service all API dependencies provide.

WunderGraph High-Level Architecture

WunderGraph Benefits

● WunderGraph is an API composition platform focused on GraphQL. It enables developers to compose and unify multiple APIs (REST, GraphQL, databases, etc.) into a single unified GraphQL API.

● It abstracts complexity by providing a single interface for developers to query multiple services, which are composed behind the scenes.

● Offers real-time data, caching, and built-in security features.

WunderGraph and DreamFactory Working Together

1. DreamFactory quickly generates and exposes APIs (REST, SOAP, or GraphQL) from a variety of backends, like databases or external services as REST APIs with well defined OpenAPI specifications.
2. WunderGraph Orchestrates and Integrates on top of DreamFactory to compose these APIs (as well as other external APIs) into a single GraphQL endpoint. This GraphQL layer acts as a unified API layer, providing access to all the underlying APIs exposed by DreamFactory or any other external and backend services.

This gives the REST APIs generation power of DreamFactory and the API composition, aggregation and federation capabilities of WunderGraph.

Unified API System with WunderGraph

Virtual Graph

The Virtual Graph is a concept in WunderGraph. It’s the idea of combining multiple GraphQL Schemas into a single, unified, schema. As we’re using a JSON RPC compiler instead of directly exposing the GraphQL, we’re calling this concept the “Virtual Graph”.

The Virtual Graph is only really available during development. But when deploying to production, the GraphQL to JSON RPC compiler will automatically turn all defined GraphQL Operations into JSON RPC Endpoints. That’s why this Graph really only exists virtually, hence the name.

What’s so powerful about having a concept like the Virtual Graph is that it enables you to think explicitly about APIs as dependencies. The Virtual Graph is your gateway to the other APIs you want to use. With this idea in mind, you can understand GraphQL as the language to “program” with APIs.

Add API dependencies to your WunderGraph application and use GraphQL to define your API surface. The Virtual Graph allows you to meta-program your APIs with GraphQL. The artifact will be an API Gateway and one or more generated clients, speaking JSON RPC over HTTP between the them.

WunderGraph Server

WunderGraph Server is the heart of the WunderGraph framework. It’s the API Gateway of the stack and written in Golang / Go. The Gateway is highly optimized for performance and scalability. When starting the WunderGraph server, it will use our GraphQL to JSON RPC compiler and turn all the GraphQL Operations you’ve defined into JSON RPC calls. All of this happens at deployment time, so we’re actually removing GraphQL from the runtime at all.

In production environment, the Gateway is handling requests by calling the pre-compiled, pre-optimized JSON RPC calls. All aspects such as lexing, parsing and validating GraphQL Operations is handled during the deployment, not during the request execution.

Architecting WunderGraph this way makes it very efficient and secure. It’s possible because GraphQL Operations usually don’t change at runtime. So, if one is not changing Operations at runtime, one can turn GraphQL into a compile-time problem, removing it from the runtime entirely.

High-Level Unified API Layer Architecture

WunderGraph Client

WunderGraph is not just a framework to generate an API Gateway/Backend for Frontend (BFF). It also generate clients from the same configuration, giving organisations a full end-to-end solution, with no additional tools required. All configurations are in wundergraph.config.ts file. Alongside the backend configuration, one can define what templates to use to generate the client.

One good example of how a client could look like is TypeScript and Next.js. As Wundergraph’s monorepo is open source, all templates are available for use, and one can easily create their own.

The generated client is based on the Wundergraph RPC Protocol. This protocol defines how WunderGraph server and the generated client communicate. The protocol works over HTTP/1.1 and HTTP/2, supports Queries, Mutations, Subscriptions, Live-Queries, authentication and file uploads.

Client and server speak JSON RPC over HTTP, defined by the WunderGraph RPC Protocol. If a client wants to execute an Operation, it calls an Endpoint on the API Gateway, which will then execute the pre-compiled Operation against the Virtual Graph. Both request and response can be manipulated using hooks.

GraphQL Proxy Layer in WunderGraph

The built-in GraphQL Proxy Layer in WunderGraph provides a unique mechanism to handle both REST API requests and GraphQL requests from clients, offering a unified API platform.

Clients Sending REST API Requests: WunderGraph can act as a REST-to-GraphQL proxy, allowing REST API clients to interact with backend GraphQL or REST APIs.

● The client sends a REST API request to the WunderGraph server. The WunderGraph server receives the REST request, determines the associated operation, and forwards the request to the relevant backend service. This service could be a GraphQL API, REST API, or another type of data source.

● The GraphQL Proxy Layer in WunderGraph can transform the REST request into a GraphQL query if necessary, interacting with backend GraphQL services.

● Example: A REST request to /users/123 could be transformed into a GraphQL query like { user(id: “123”) { name, email } }.

● WunderGraph aggregates the response (either from a GraphQL or REST backend) and formats it back into a REST response format.

The client receives the REST API response as if they were calling a native REST endpoint, even if the backend services were GraphQL.

REST API Workflow with WunderGraph Key Features

● REST-to-GraphQL transformation: Clients can send REST requests, and WunderGraph handles the transformation into GraphQL queries when needed.

● Aggregation: WunderGraph can aggregate data from multiple sources (both REST and GraphQL) and present a single, unified REST response.

● Authentication: WunderGraph handles any required authentication (e.g., OAuth tokens) before forwarding the requests to backend services.

Clients Sending GraphQL Requests: clients that prefer to use GraphQL, WunderGraph serves as a GraphQL proxy, enabling the client to query and interact with multiple backend services through a unified GraphQL interface.

● Client sends a GraphQL query to the WunderGraph server.

● The GraphQL Proxy Layer processes the query and maps it to the appropriate backend APIs (REST or GraphQL).

● If the backend service is a REST API, the GraphQL Proxy Layer converts the GraphQL query into a REST request.

• Example: A GraphQL query like { user(id: “123”) { name, email } } could be transformed into REST request like /users/123.
• WunderGraph forwards the request to the backend service, collects the response, and formats it according to the GraphQL specification.

● Client receives the GraphQL response in the standard format, even if some data originated from REST APIs.

GraphQL Workflow with WunderGraph Key Features

● Unified Data Access: WunderGraph can interact with multiple backend services (both REST and GraphQL) and expose them through a single GraphQL schema.

● Schema Stitching: WunderGraph can stitch multiple GraphQL schemas together, allowing clients to query across various services in one request.

● Caching and Optimisations: WunderGraph can cache results for GraphQL queries, improving performance for repeated requests.

REST clients can interact with GraphQL backends through WunderGraph’s GraphQL Proxy Layer, which transforms REST requests into GraphQL queries.

GraphQL clients can interact with REST backends by having their GraphQL queries transformed into REST API calls.

WunderGraph seamlessly handles the transformations, aggregations, and response formatting, making it an ideal solution for environments where both REST and GraphQL are used. This approach enables flexibility for clients using different API paradigms, while still benefiting from the powerful features of WunderGraph’s Unified API Layer.

High Availability of REST APIs Generation Layer

Unified Caching Mechanism

GraphQL Proxy in WunderGraph is designed primarily to handle GraphQL API requests, it can also cache REST API requests. Since the proxy layer is part of the Unified API Layer, it has access to all incoming requests — whether REST APIs or GraphQL.

When a REST API request is received, the GraphQL Proxy Layer first checks if a cached response for that specific request exists. If the response is cached, it is served directly from the caching service without hitting the backend APIs and DreamFactory’s REST APIs Generation Layer. If not, the proxy forwards the request to the backend REST service, retrieves the response, and caches it for future requests.

The caching service in the Unified API Layer, allows to handle both GraphQL and REST requests efficiently. The WunderGraph Server which is also a Gateway Caches all API responses automatically and apply the necessary headers, including ETags for efficient content revalidation. The server will cache the response in memory and compute an ETag for the response, which will be sent to the client along with the response. On subsequent requests, the client will automatically attach the ETag to the request, which is the default behaviour of the browser. If the client cache is stale, but the ETag still matches, the server will respond with a 304 Not Modified, sending as little data as possible.

Process Flow Between WunderGraph & Middleware Systems

Applications consuming and interacting with APIs exposed from the Unified API layer don’t do so directly instead they communicate bidirectionally with the WunderGraph Client without being concerned about protocols used to orchestrate the Unified API Layer with all complexities abstracted by WunderGraph Server and the WunderGraph’s generated client.

Data Source Configuration

Let us add two APIs to our Application.

// wundergraph.config.ts
const weather = introspect.graphql({
    apiNamespace: 'weather',
    url: 'https://weather-api.wundergraph.com/',
  });
  
  const countries = introspect.graphql({
    apiNamespace: 'countries',
    url: 'https://countries.trevorblades.com/',
  });
  
  configureWunderGraphApplication({
    apis: [weather, countries],
  });

Operation Definition

Next, we’ll define a GraphQL Operation that uses the Cross API JOIN feature of WunderGraph. It takes away a lot of complexity, which is also quite handy to illustrate the different capabilities of WunderGraph.

# operations/Weather.graphql
query (
  $continent: String!
  # the @internal directive removes the $capital variable from the public API
  # this means, the user can't set it manually
  # this variable is our JOIN key
  $capital: String! @internal
) {
  countries: countries_countries(filter: { continent: { eq: $continent } }) {
    code
    name
    # using the @export directive, we can export the value of the field `capital` into the JOIN key ($capital)
    capital @export(as: "capital")
    # the _join field returns the type Query!
    # it exists on every object type so you can everywhere in your Query documents
    # with the @transform directive, we can transform the response
    weather: _join @transform(get: "weather_getCityByName.weather") {
      # once we're inside the _join field, we can use the $capital variable to join the weather API
      weather_getCityByName(name: $capital) {
        weather {
          temperature {
            max
          }
          summary {
            title
            description
          }
        }
      }
    }
  }
}

WunderGraph Example Sequence Diagram

Scalable and Secure System

When defining GraphQL Operations using WunderGraph, it automatically generates a JSON REST API for you. Additionally, it generates a 100% typesafe client. This means, you’re getting the same developer experience as if you were using a standard GraphQL API, but without the tradeoffs in terms of security and scalability.

If your DreamFactory instance is deployed on-prem but your databases are in cloud, you can securely configure connections between them using VPNs, VPC Peering, or SSL/TLS encryption to ensure secure communication between your on-prem DreamFactory and cloud-hosted resources. This enables you to manage both on-premise and cloud-based systems under a unified API framework without having to manually develop REST APIs for each data source. Open-Source DreamFactory provides the following security features for your REST APIs Generation Layer.

● Role-Based Access Control (RBAC) with fine-grained permissions.

● API Key Authentication for controlling access to APIs.

● User Authentication via local credentials, OAuth providers & JWT-based sessions.

● CORS Support to prevent unauthorized domains from accessing APIs.

● SSL/TLS Encryption to secure API communications.

● Basic Rate Limiting for managing API request loads.

● Custom Scripting for additional security logic.

Unified API/Orchestration Layer & REST API Generation Layer Decoupling

Decoupling the Unified API & Orchestration Layer (built using WunderGraph Open Source) and the REST API Generation Layer (from DreamFactory Open Source) involves creating clear boundaries between the two layers in terms of responsibility, communication, and management. Here’s how this can be achieved:

● Unified API & Orchestration Layer (WunderGraph):

• Primary Role: Orchestrates and unifies access to various APIs (REST, GraphQL, databases) by acting as a single point of entry for clients. It transforms, proxies, and handles both REST and GraphQL requests, including aggregation, caching, and security.
• REST-to-GraphQL Transformation: WunderGraph transforms REST APIs into GraphQL queries if needed and can call multiple services within a single query.
• Orchestration: Coordinates calls to multiple APIs and handles complex workflows (e.g., aggregating data from different sources).

● REST API Generation Layer (DreamFactory):

• Primary Role: Exposes backend services, databases, or other systems as REST APIs. It generates REST endpoints for various data sources (e.g., SQL databases, NoSQL databases, file storage).
• Focus on API Generation: DreamFactory’s focus is on creating REST APIs automatically from a variety of sources, and it also manages API documentation and access control for those APIs.

● Loose Coupling through API Contracts:

• The communication between the Unified API Layer (WunderGraph) and the REST API Generation Layer (DreamFactory) happens over well-defined REST API contracts. WunderGraph can consume REST APIs generated by DreamFactory as if they were any external service.
• Contract-based Communication: The two systems are decoupled through API contracts (OpenAPI specs or similar). DreamFactory exposes APIs, and WunderGraph consumes them based on the provided contract, with no direct dependencies on the internal workings of DreamFactory.

● Communication Over HTTP(S)

Both layers communicate over standard HTTP(S) protocols, typically via REST APIs. This allows for:

• Clear Separation: Each layer operates independently, and they only communicate through well-defined API calls.
• Standard Interfaces: Since the REST APIs follow common protocols and standards, there’s no need for deep integration, and either layer can be modified or replaced without impacting the other.

● Autonomous Lifecycles

• Independent Deployment: WunderGraph and DreamFactory can be deployed separately, each with its own lifecycle. You can deploy and scale each independently, manage their performance, and update them without affecting the other.
• Version Control: Each layer can be updated independently (e.g., updating the DreamFactory REST APIs without affecting the WunderGraph orchestration layer).

● Decoupled Authentication and Authorization

• Both WunderGraph and DreamFactory handle their own authentication and authorization, but they can be integrated loosely via token-based mechanisms.
• For instance, DreamFactory might handle API key, OAuth, or JWT-based authentication for the REST APIs it generates, while WunderGraph manages authentication for its unified layer, using those tokens when communicating with DreamFactory.

● Abstracting Business Logic in WunderGraph

• Business Logic in WunderGraph: WunderGraph handles the orchestration logic. It consumes DreamFactory’s REST APIs but can also call other services, aggregate data, and apply business rules without requiring changes in DreamFactory.
• DreamFactory Focus: DreamFactory remains focused on generating APIs and connecting to backend data sources. It doesn’t need to be aware of how WunderGraph orchestrates multiple APIs or combines data from other sources.

● Flexible, Extensible Architecture

• Replaceability: Since WunderGraph and DreamFactory are decoupled via standard APIs, you could theoretically replace DreamFactory with another API generation tool (e.g., a custom-built solution or another REST generator) without heavily impacting the WunderGraph orchestration.
• Extensibility: You could add other systems to either layer without needing to modify the core logic of the other. For instance, you can introduce additional REST or GraphQL services into the Unified API Layer without altering DreamFactory’s REST generation.

Error Handling and Caching at the Unified API Layer

WunderGraph can handle error management, retry policies, and caching for the APIs generated by DreamFactory. This ensures the DreamFactory APIs are decoupled from the performance or error-handling logic, and those aspects are managed by WunderGraph’s orchestration capabilities.

Decoupled Architecture

In the architecture diagram:

• The Unified API & Orchestration Layer (WunderGraph) would communicate with the REST API Generation Layer (DreamFactory) through a double arrow indicating API communication.
• The arrows represent API requests and responses over HTTP(S) and show the clear separation between the two layers.
• Each layer has its own box with a clear boundary, indicating they are separate systems, but connected via standard REST API calls.

Example Architecture Flow:

1. DreamFactory generates REST APIs from a database (e.g., GET /users).
2. WunderGraph calls this REST API to retrieve the user data and may aggregate it with data from other sources (e.g., a GraphQL API for profile data).
3. WunderGraph exposes the aggregated data as either a GraphQL API or a REST API to the client.
4. Both systems are decoupled: Changes in one layer do not directly impact the other.

The Unified API Layer (WunderGraph) and the REST API Generation Layer (DreamFactory) are decoupled by:

• Separating their roles and responsibilities.
• Communicating via standard REST APIs (using HTTP(S) and API contracts).
• Being deployed independently and managing their own lifecycles.
• Handling authentication and security independently but integrating token-based communication.
• Allowing flexible integration and extension without tightly coupling both systems.

This decoupling makes the Architecture Scalable, Maintainable, and Adaptable to changes in either layer.

Omnichannel Experience & API Modernisation

Designing the Unified API Layer using DreamFactory and WunderGraph for large Organisations offers significant benefits for both omnichannel experience and API modernisation.

Omnichannel Experience

An omnichannel experience means delivering a seamless, integrated service across multiple channels, such as mobile apps, web platforms, kiosks, and call centers. The Unified API Layer acts as a central hub that allows all these channels to interact with the same data and services, ensuring consistency and accessibility.

Example: Vehicle Registration Service Across Multiple Channels

A Vehicle Registration service where citizens can register their vehicles via

• Mobile App
• Web Portal
• Kiosk at Transport Offices
• Customer Support Center

How the Unified API Layer Supports Omnichannel:

• DreamFactory connects to multiple backend databases (like MySQL or PostgreSQL for vehicle records) and systems (like legacy systems for vehicle history).
• WunderGraph provides a unified API layer (REST and GraphQL) that aggregates data from DreamFactory and other services like third-party insurance verification systems.
• Each channel (mobile app, web portal, kiosk) interacts with the same APIs exposed by WunderGraph, which means:
• The mobile app and web portal get data from the same backend through standardised GraphQL queries.
• The kiosks use REST APIs generated by WunderGraph from the same data layer.
• Call centers access the same APIs to provide consistent information to citizens over the phone.

Business Value

This ensures that the Vehicle Registration Service delivers a consistent experience across all touchpoints. Whether a user accesses the service via their mobile phone or interacts with a call center agent, they see the same vehicle data and can perform the same registration tasks seamlessly.

API Modernisation

API modernisation involves transforming legacy systems and exposing them through modern API protocols (REST/GraphQL), making it easier to integrate with new systems, services, and channels. This enhances scalability, performance, and interoperability.

Example: Modernising Legacy Driver’s License Management System

Let us assume that RTA has a legacy driver’s license management system (e.g., a SOAP-based service or a custom-built monolithic system running on outdated hardware).

How the Unified API Layer Modernises the System:

• DreamFactory can take the legacy database or SOAP service and automatically generate REST APIs to expose the driver’s license data and services (such as applying for a license, checking renewal status).
• WunderGraph takes these REST APIs from DreamFactory and exposes them as GraphQL APIs, making them more modern, flexible, and efficient.
• Frontend applications (such as a new web-based Driver’s License Portal) use Unified GraphQL API to request only the data they need (e.g., license number, expiry date), improving performance compared to calling entire REST API endpoints.
• If your frontend can only work with REST APIs, WunderGraph allows you to expose REST endpoints that internally translate the requests into GraphQL queries to the Unified API Layer.
• How it works: WunderGraph can expose a RESTful endpoint, for example:

POST /api/drivers/licenses/12345

This REST request will be translated by WunderGraph into a GraphQL query behind the scenes. WunderGraph will then execute the GraphQL query against the Unified API Layer, gather the data, and return it in a REST-compliant format to the frontend.

• Example: A frontend system sends a REST request to a WunderGraph-generated endpoint:

GET /api/drivers/licenses/12345

WunderGraph transforms this request into a GraphQL query internally:

# GraphQL code
query {
  getDriverLicense(id: "12345") {
    name
    expiryDate
    licenseStatus
  }
}

It fetches the necessary data, then formats it into a JSON response typical of REST APIs and sends it back to the frontend.

Benefit: This allows legacy frontends or systems restricted to REST to still consume the modern GraphQL-powered backend without needing significant changes to the frontend code.

WunderGraph can also cache commonly accessed data (such as license verification requests) for improved performance, reducing load on legacy systems.

Business Value:

This modernisation allows Organisations to expose their legacy driver’s license system via modern APIs that are easy to consume by new web portals, mobile apps, and third-party integrators (e.g., insurance companies checking driver records). The system becomes scalable, performant, and much easier to integrate into future applications.

● Omnichannel Experience: The unified API layer ensures that all citizen-facing platforms (mobile, web, kiosks) can interact with the same services and data in real time, providing a seamless and consistent experience across all channels.

● API Modernisation: By leveraging DreamFactory to generate APIs from legacy systems and WunderGraph to expose them as modern, flexible APIs (GraphQL or REST), Organisations can modernise legacy systems without rebuilding them, making them scalable and accessible for future digital transformations.

Conclusion

These two Open-Source Low-Code Platforms working together streamline integration, improve user experience, and prepare Organisations for transitioning into future technological advancements, ensuring the infrastructure is both modern and omnichannel-ready.

By embracing this synergy, businesses not only reduce complexity and operational costs but also foster innovation through faster deployment cycles, stronger interoperability, and resilient architectures. Ultimately, this collaboration empowers organisations to remain agile, competitive, and equipped to continuously adapt in an evolving digital landscape.

What’s Next

This is a two part blog, where in this part we covered the following;

● Unified API Layer: This blog introduces a robust, fully-integrated Unified API Layer that acts as an abstraction between multiple APIs, omni-channel systems, modern/legacy systems, databases, and applications. This layer provides a single point of access and a standardised interface for different services, simplifying how developers interact with multiple APIs.

● Proposed Platform & Architecture: We proposed the platform and High-Level Architecture combining strengths of DreamFactory & WunderGraph, leveraging low-code open-source platforms for seamless API generation, orchestration & integration with minimal code. This enables efficient API creation from multiple data sources and advanced API orchestration for Microservices, legacy systems, databases, IoT devices, and external RESTful APIs.

● DreamFactory: DreamFactory is highlighted as an open-source REST API generation solution that automatically generates secure and documented APIs for various databases. It includes features like user management, token-based authentication, role-based access control, OAuth 2.0, OpenID Connect, and API keys with basic rate limiting.

● WunderGraph: WunderGraph is described as a Backend for Frontend (BFF) framework designed to optimise developer workflows through API composition. It combines API Gateway and Backends for Frontends patterns, providing single-purpose edge services for UIs and external parties. WunderGraph enables developers to compose and unify multiple APIs into a single unified GraphQL API Schema.

● Working Together: The Blog explains how DreamFactory & WunderGraph work together. DreamFactory generates and exposes APIs from various backends/Databases, while WunderGraph orchestrates and integrates these APIs into a single GraphQL endpoint. This combination provides a unified API layer, offering access to all underlying APIs exposed by DreamFactory or other services.

● Decoupled and Extendible Architecture: The Blog describes how the Unified API & Orchestration Layer (built using WunderGraph) and the REST API Generation Layer (from DreamFactory) are decoupled to ensure clear boundaries in terms of responsibility, communication, and management. This separation allows for independent deployment, version control, and flexible integration of additional systems without impacting the core logic of either layer. The architecture is designed to be scalable, maintainable, and adaptable to changes in either layer.

● Omnichannel Experience: The unified API layer supports an omnichannel experience by delivering a seamless, integrated service across multiple channels such as mobile apps, web platforms, kiosks, and call centres. This ensures consistency and accessibility, allowing all channels to interact with the same data and services.

● API Modernisation: Discussed API modernisation, which involves transforming legacy systems and exposing them through modern API protocols (REST/GraphQL). This enhances scalability, performance, and interoperability, making it easier to integrate with new systems, services, and channels.

In the next part of the Blog ,we shall explore how to leverage the Modern Unified API experience with AI/ML integrations and pipeline. We shall also cover at a high-level the overall Infrastructure & Networking Architecture along with a Robust and Integrated Observability Stack. The next part of the Blog, also covers, API led AI experiences with the Kong AI Gateway.

Let’s dive into the second part of the Blog: AI Driven Unified API Layer.

Testing Dynamic Routing & Transformation from the Browser

Testing Banking Service API For Savings Account Customer

The savings account user is redirected to the /savings-account route path in the banking service.

Testing Banking Service API For Current Account Customer

The current account user is redirected to the /current-account route path in the banking service.

Troubleshooting & Debugging Common Issues

In the proxy_error.log file, Kong generates and stores all the error details included in the kong/shared/logs host folder while running and executing API requests from clients. The path for proxy port request error logs is configured in the KONG_PROXY_ERROR_LOG environment variable of the kong-dp service definition. This path maps to the host application folder path specified in the volumes section of the kong-dp service container definition.

Not able to apply recent configuration changes to the Kong Gateway while testing the APIs. This happens when the Gateway is not in sync with the Konnect Control Plane. This can be observed from the following error message in the proxy_error.log log file generated from Kong.

2024/07/15 14:29:56 [error] 2406#0: *8 [lua] data_plane.lua:385: communicate(): [clustering] did not receive pong frame from control plane within 45 seconds [d90c1bbe77.us.cp0.konghq.com:443], context: ngx.timer
2024/07/15 16:43:44 [error] 2406#0: *10311 [lua] data_plane.lua:385: communicate(): [clustering] did not receive pong frame from control plane within 45 seconds [d90c1bbe77.us.cp0.konghq.com:443], context: ngx.timer
2024/07/15 17:49:16 [error] 2406#0: *14966 [lua] data_plane.lua:385: communicate(): [clustering] did not receive pong frame from control plane within 45 seconds [d90c1bbe77.us.cp0.konghq.com:443], context: ngx.timer

Wait until Kong Data Plane node is in sync with the Konnect Control Plane. This can be verified from the Konnect UI. Go to the Gateway Manager section from the left-side menu. Select the Control Plane. In the Gateway Manager’s Control Plane section, select the Data Plane Nodes section. You will be redirected to the Data Plane Nodes information page where one can see the kong-dp Data Plane node running with Sync Status as In Sync and Compatible.

Notice the “Labels” column in the Data Plane Nodes information page. It designates the Kong Gateway runtime running in the Kong Konnect SaaS Platform. It is the same label specified in the KONG_CLUSTER_DP_LABELS configuration parameter of the kong-dp service container definition and interpolated by the KONG_DP_LABLES environment key as imported from the “.env” file in the “docker-compose.yaml” Docker Compose specification file.

For any compatible issues or errors in the Kong Gateway click on the host kong-dp to review the configuration details and error message in the “Data Plane Node Details” page from the UI.

The Kong Gateway runtime has no compatible issues as seen from the info displayed at the top of the Konnect UI and from the compatibility status in the “Data Plane Node Details” Page.

The “Errors” list is empty indicating there are no errors in the Kong Gateway configuration. API requests via Kong Gateway can be tested to verify that all features are working based on the latest configuration applied using decK or Konnect API into the Kong Gateway Data Plane node.

If you are trying to access the Kong proxy gateway or Keycloak server hosted locally from a secured URL using the HTTPS protocol, you may run into issues with SSL certificate verification. This problem occurs when the self-signed localhost certificate expires after a month.

To resolve this issue in Kong, you will need to generate a new SSL server certificate and key. Then, update the configuration parameters in the “docker-compose.yaml” file for the kong-dp service container definition. Add the path of the certificate key to the KONG_SSL_CERT_KEY configuration parameter and the server certificate path to the KONG_SSL_CERT configuration parameter. For the keycloak service container definition, add the path of localhost server certificate to KC_HTTPS_CERTIFICATE_FILE configuration parameter and the certificate key path to the environment variable KC_HTTPS_CERTIFICATE_KEY_FILE configuration parameter.

When making configuration changes such as onboarding new APIs and security plugins using decK commands or Konnect API requests, you may encounter 401 unauthorised responses. This occurs if the Konnect Personal Access Token has expired. To resolve this, generate a new Personal Access Token, securely store it in the “novigo_accelerators.deck.yaml” file, and include it with every decK command or Kong API request. This will authorise the applied changes and fix the issue.

Conclusion

In part-16 of this Blog , we covered the following;

• Testing the Banking service API for Savings & Current customer from the browser, redirecting the user to /savings-account and /current-account route paths in the banking service, respectively and dynamically via the Kong Gateway.

• Testing that the transformations are applied correctly via the Kong Gateway from the browser for Savings & Current customer respectively.

• Troubleshooting and debugging common issues within Kong Gateway.

• Validating Kong Data Plane nodes compatibility and errors in the Gateway from the Konnect UI.

Blogs from part-1 to part-16 covers dynamic routing via JWT token claim parameters to determine route paths at runtime, with url transformations. It explains exhaustively how to utilise the claim parameters in the JWT token to dynamically direct the incoming requests to the appropriate route paths based on specific user or client attributes. Additionally, it covers the process of applying transformations to the incoming data based on the JWT token claim parameters, allowing for personalized responses tailored to the user’s attributes or access rights. Overall, it provides an in-depth guide on leveraging JWT token claim parameters for dynamic routing and transformations, enhancing the flexibility and customisation of the application’s routing and data processing.

Visit part-15 of the Blog, where the Banking service API was tested for both Savings and Current customers using the Insomnia client in combination with Kong Gateway. Request chaining and pre-request scripts were implemented to test if Savings and Current customer users are dynamically redirected to the /savings-account and /current-account routes. In part-15 of the Blog, we also explored JavaScript-based pre-request scripts in Insomnia to update environment variables, manage bearer tokens, and set authorisation header before sending API requests. For Savings customers, tokens were generated and applied to set an environment variable in authorisation header, while for Current customers the Auth header was updated either directly with a valid JWT or by generating a new token when required, validated first through Keycloak’s introspection endpoint. In all scenarios, valid tokens were appended to authorisation headers to securely access the Banking service API through Kong Gateway.

The entire implementation depicted in these blogs, shows how one can securely onboard a Banking API service via decK & bash scripts, ensuring that the API is managed via Kong Gateway securely and reliably. It covers associated technologies like Docker Compose, Python Application Development & Containerisation, and Keycloak IdP, right from installation and setting up security measures to testing and review using API design and testing tool like Insomnia with the latest features built by Kong.

Summary of Dynamic Routing & Transformations via Kong Gateway managed by Kong Konnect Platform

One can see how easy it is to leverage Kong API Gateway to create complex dynamic routing based on specific authentication attributes (in our case, JWT token claims).

All users have the same Login process, and according to their JWT claim attributes these customers are dynamically routed to the relevant application/path for one’s banking application service. This makes the maintenance of the deployed application much easier and cleaner.

In this solution, we used Kong’s Konnect (SaaS) platform which makes deployment and maintenance much easier. We can deploy Kong in many architectures (on-prem, cloud, hybrid, docker, k8s, etc…), and can reproduce these capabilities across environments no matter which form of deployment you choose.

By leveraging dynamic routing, developers can focus on building and iterating on their APIs while their API platform infrastructure takes care of routing requests efficiently and reliably.

Customers now expect outstanding experiences — and these experiences require more from applications and developers. For this, an API gateway like Kong is a crucial component. The right API management solution can make the difference between an experience that drives revenue and one that drives customers to the competition.

Banking Service API Testing Using Insomnia

The Banking service requested via the Kong Gateway through Insomnia requires tokens that can be generated automatically with request chaining or the pre-request script using Insomnia.

Savings Customer API Call Using Request Chaining

Get the token from the Keycloak Token API response for the saving customer before sending the request to the Banking service from Insomnia. In the Auth section of Insomnia select Bearer Token as the authorisation type. In the Token section click on the Response label.

In the Request dropdown select the Keycloak Token API for Savings Account User. Select the “When Expired” option from the Trigger Behavior dropdown list so that a request to generate the token is only made when the existing token has expired. Filter the response by entering $.access_token. The value will be dependent on the token API response. If you have set it up correctly, you should see a token in Live Preview.

The savings account user’s API request redirects the user to the /savings-account route path in the banking service application through the Kong API Gateway based on the token claim.

Savings Customer API Call Using Pre-Request Script

The pre-request script updates the saving_account_access_token environment variable with a valid token. It verifies the existing token value assigned to the saving_account_access_token environment variable and if the access token is valid it uses the token to authorise the user.

The pre-request script uses the introspection API endpoint in Keycloak to validate the token. If the response declares the token as inactive then generate a fresh token and assign this token to saving_account_access_token environment variable.

// Keycloak token generation endpoint and client credentials
const tokenUrl = insomnia.environment.get("token_url");
const clientId = insomnia.environment.get("client_id");
const clientSecret = insomnia.environment.get("client_secret");
const scope = insomnia.environment.get("scope");
const username = insomnia.environment.get("saving_account_username");
const password = insomnia.environment.get("saving_account_password");
const grantType = insomnia.environment.get("grant_type");

// Keycloak introspection endpoint
const introspectionUrl = insomnia.environment.get("introspection_url");

// Active request authorization token
const accessToken = insomnia.environment.get("saving_account_access_token");

/*Payload for introspection endpoint in Keycloak IdP
with urlencoded content in the body of the request*/
const verifyTokenReq = {
  method: 'POST',
  url: introspectionUrl,
  header: {
    'Content-Type': 'application/x-www-form-urlencoded',
  },
  body: {
    mode: 'urlencoded',
    urlencoded: [
      { key: 'token', value: accessToken, },
      { key: 'client_id', value: clientId, },
      { key: 'client_secret', value: clientSecret, },
    ]
  }
};

/*Send a request to Keycloak IdP introspection endpoint
to verify if an access token in active request is valid*/
const introspectionResp = await new Promise((resolve, reject) => {
  insomnia.sendRequest(
    verifyTokenReq,
    (err, resp) => err != null ? reject(err) : resolve(resp)
  );
});

/*Verify the validity of access token on successful response 
from Keycloak IdP introspection endpoint request for a client*/
if (introspectionResp.code == 200 && introspectionResp.json().active) {
  // JSON response
  let jsonResp = introspectionResp.json();
  console.log("Access token active: " + jsonResp.active);
  
} // Generate new token for client authorization when the token in active request is invalid
else {
  console.log("Access token is invalid:", introspectionResp.json());
  
  /*Payload for sending a new token generation request with 
  urlencoded content to Keycloak token generation endpoint*/
  const getTokenReq = {
    method: 'POST',
    url: tokenUrl,
    header: {
      'Content-Type': 'application/x-www-form-urlencoded',
    },
    body: {
      mode: 'urlencoded',
      urlencoded: [
        { key: 'client_id', value: clientId, },
        { key: 'client_secret', value: clientSecret, },
        { key: 'scope', value: scope, },
        { key: 'username', value: username, },
        { key: 'password', value: password, },
        { key: 'grant_type', value: grantType, },
      ]
    }
  };
  
  console.log("Generating new access token");
  
  /*Sending a request to Keycloak IdP token generation endpoint
  and generating a new access token for authorizing the client*/
  const resp = await new Promise((resolve, reject) => {
    insomnia.sendRequest(
      getTokenReq,
      (err, resp) => err != null ? reject(err) : resolve(resp)
    );
  });
  
  /*On successfully generating the token automatically from Keycloak 
  update the active request authorization type with new access token*/
  if (resp.code == 200) {
    // JSON response
    const jsonResp = resp.json();
    
    // New access token from Keycloak's token generation endpoint response
    const newAccessToken = jsonResp.access_token;
    console.log("New access token: " + newAccessToken);
    
    // Set access token variable with the new access token
    insomnia.environment.set("saving_account_access_token", newAccessToken);
    console.log("Access token variable updated in " + insomnia.environment._name + " environment");
  } else {
    // On failure to fetch a new access token from Keycloak's token generation endpoint
    console.error("Unable to fetch token, response status: " + resp.code);
  }
}

Current Account User API Call Using Pre-Request Script

The Pre-Request script for the current account user to access the banking service does not use any environment variable for storing the authorisation token. The script directly updates the active API request authorisation type and token and adds it as the authorisation header for the active request before sending the API call to the banking service.

// Keycloak token generation endpoint and client credentials
const tokenUrl = insomnia.environment.get("token_url");
const clientId = insomnia.environment.get("client_id");
const clientSecret = insomnia.environment.get("client_secret");
const scope = insomnia.environment.get("scope");
const username = insomnia.environment.get("current_account_username");
const password = insomnia.environment.get("current_account_password");
const grantType = insomnia.environment.get("grant_type");

// Keycloak introspection endpoint
const introspectionUrl = insomnia.environment.get("introspection_url");

// Active request authorization token
const accessToken = insomnia.environment.get("current_account_access_token");

/*Payload for introspection endpoint in Keycloak IdP
with urlencoded content in body of the request*/
const verifyTokenReq = {
  method: 'POST',
  url: introspectionUrl,
  header: {
    'Content-Type': 'application/x-www-form-urlencoded',
  },
  body: {
    mode: 'urlencoded',
    urlencoded: [
      { key: 'token', value: accessToken, },
      { key: 'client_id', value: clientId, },
      { key: 'client_secret', value: clientSecret, },
    ]
  }
};

/*Send a request to Keycloak IdP introspection endpoint to 
verify if existing access token for the active request is valid*/
const introspectionResp = await new Promise((resolve, reject) => {
  insomnia.sendRequest(
    verifyTokenReq,
    (err, resp) => err != null ? reject(err) : resolve(resp)
  );
});

/*Function to update active request authorization type

type(string): Authorization type
token(string): Authorization token
*/
let updateAuthActiveReq = (type, token) => {
    /*Set the auth type in active request with 
    an access token generated automatically*/
    insomnia.request.auth.update({
      type: type,
      bearer: [
        { key: 'token', value: token, },
      ],
    }, type);
    console.log("Auth token updated in the active request");
};

/*Verify the validity of access token on successful response 
from Keycloak IdP introspection endpoint request for a client*/
if (introspectionResp.code == 200 && introspectionResp.json().active) {
  // JSON response
  let jsonResp = introspectionResp.json();
  console.log("Access token active:", jsonResp.active);
  
  // Update active request bearer auth with the existing access token variable
  updateAuthActiveReq('bearer', accessToken);
  
} // Generate new token for client authorization when the token in active request is invalid
else {
  console.log("Access token is invalid:", introspectionResp.json());
  
  /*Payload for sending a new token generation request with 
  urlencoded content to Keycloak token generation endpoint*/
  const getTokenReq = {
    method: 'POST',
    url: tokenUrl,
    header: {
      'Content-Type': 'application/x-www-form-urlencoded',
    },
    body: {
      mode: 'urlencoded',
      urlencoded: [
        { key: 'client_id', value: clientId, },
        { key: 'client_secret', value: clientSecret, },
        { key: 'scope', value: scope, },
        { key: 'username', value: username, },
        { key: 'password', value: password, },
        { key: 'grant_type', value: grantType, },
      ]
    }
  };
  
  console.log("Generating new access token");
  
  /*Sending a request to Keycloak IdP token generation endpoint
  and generating a new access token for authorizing the client*/
  const resp = await new Promise((resolve, reject) => {
    insomnia.sendRequest(
      getTokenReq,
      (err, resp) => err != null ? reject(err) : resolve(resp)
    );
  });
  
  /*On successfully generating the token automatically from Keycloak 
  update the active request authorization type with the new access token*/
  if (resp.code == 200) {
    // JSON response
    const jsonResp = resp.json();
    
    // New access token from Keycloak's token generation endpoint response
    const newAccessToken = jsonResp.access_token;
    console.log("New access token: " + newAccessToken);
    
    // Update active request bearer auth with the new access token
    updateAuthActiveReq('bearer', newAccessToken);
    
    // Set access token variable with the new access token
    insomnia.environment.set("current_account_access_token", newAccessToken);
    console.log("Access token variable updated in " + insomnia.environment._name + " environment");

  } else {
    // On failure to fetch a new access token from Keycloak's token generation endpoint
    console.error("Unable to fetch token, response status: " + resp.code);
  }
}

If the token in the active request is invalid the pre-request script generates a new token and updates the active request bearer auth with the new token. The valid token is added to the authorisation header to test the API call for the current account customer through Insomnia.

API request from the current account user redirects the user to the /current-account route path in the banking service application through the Kong API Gateway based on the token claim.

What’s Next

In part-15 of this Blog , we covered the following;

• Testing the Banking service API as a Savings customer from Insomnia Client using request chaining feature and redirecting the user to /savings-account route path via the Kong Gateway.

• Testing the Banking service API as a Savings customer from Insomnia Client using pre-request scripts and redirecting the user to /savings-account route path via the Kong Gateway.

• Hands-on scripting in JavaScript using Insomnia’s pre-request script feature.

• Redirecting the Savings customer to /savings-account route path Dynamically via the Kong Gateway using Insomnia Client and authorising/authenticating the user by updating environment variable for bearer token in Auth header via the pre-request script executed in Insomnia before sending the API request to the Banking service API via the Kong Gateway.

• Testing the Banking service API as a Current customer from Insomnia Client using pre-request script feature and redirecting the user Dynamically to the /current-account route path in the banking service API via the Kong Gateway.

• Updating the Auth header of the Banking API request while testing for the Current customer directly, without using any environment variable, with a valid JWT token, while running the pre-request script in Insomnia, before sending the request to the Banking service API via the Kong Gateway.

• Checking the validity of the token using the introspection API endpoint in Keycloak, before the pre-request script in Insomnia generates a new token, if token is invalid.

• Adding a valid token as authorisation header to test the Banking service API call for the current account customer through Insomnia via Kong Gateway.

Visit part-14 of the Blog, to test the token generation API of Keycloak using Insomnia client tool. Here, we configure Kong Insomnia with environment variables for the request body and authentication details, to send API calls to the token generation endpoint of Keycloak, then verify the token API for both saving and current account customers. We then test the token API by sending POST requests to generate tokens for banking customers.

Next, in part-16 of the Blog we will test the Banking service API calls for Savings and Current customers via browser to validate Dynamic routing & transformations . In part-16 of the Blog we will troubleshoot banking service API requests via Kong Gateway, debug common issues by inspecting Kong proxy error logs and the Kong Konnect UI for any errors, also we debug issues with SSL certificate verification while accessing the Kong proxy gateway or Keycloak server locally. Finally, we conclude the Blog in part-16.

Let’s dive into Part-16 of the Blog: https://medium.com/@shubhojit.dasgupta/dynamic-routing-transformations-part-16-33893b1453c9.

Keycloak Token API Testing Using Insomnia

The Keycloak server accepts data in Form URL Encoded format. API request to generate the token sends the payload data with Content-Type set to application/x-www-form-urlencoded. The Keycloak server accepts data sent as form-encoded parameters in the body of the API request and responds with token info in JSON format.

The scope parameters, openid and bank are sent as part of the form-encoded parameters in the body of the API request, separated by whitespace. The authorisation type for the API request is specified as Basic in the Auth tab section of Insomnia. The Username is set to client_id and the Password is set to client_secret, as configured for the kong client in Keycloak Kong Realm.

The token API request is a POST request to generate tokens for Banking Customers.

The Token generation endpoint responds with a 200 OK status, including token information in the response body for both savings and current account customers.

What’s Next

In part-14 of this Blog , we covered the following;

• Testing the token generation API of Keycloak via Insomnia client tool.

• Configuring Kong Insomnia tool with environment variables for request body and auth details to send an API call to the token generation API endpoint of Keycloak and test the token API for both savings & current account customers.

• Testing the token API request by sending a POST request to generate tokens for Banking Customers.

Visit part-13 of the Blog, to use the decK tool to implement APIOps, check for drift in Kong configuration changes & execute Bash scripts to automate the deployment of Banking service into Kong Konnect Control Plane and then into Kong Data Plane with OpenID connect and request-transformer-advanced plugins applied on the Banking service.

Next, in part-15 of the Blog we shall test Banking service API onboarded to Kong Gateway in Konnect using Kong Insomnia tool. In part-15 of the Blog we will also test the Bank API using Kong Insomnia tool both for savings & current account customers via Kong Gateway, and also test the Banking API using request chaining & pre-request scripting features of Kong Insomnia tool.

Let’s dive into Part-15 of the Blog: https://medium.com/@shubhojit.dasgupta/dynamic-routing-transformations-part-15-d9b519b3a295.