Bridging the Gap: Converting FastAPI OpenAPI to Swagger 2.0 for GCP API Gateway compatibility

FastAPI, a Python framework for backend development, generates OpenAPI 3.0 docs, but GCP API Gateway only supports Swagger 2.0, creating deployment challenges. This guide details converting FastAPI’s OpenAPI v3.0 to v2.0 for seamless integration with GCP API Gateway. It covers the fundamental concepts of FastAPI and GCP API Gateway, key OpenAPI version differences, and step-by-step instructions for conversion and API Gateway compatibility. Whether you’re an experienced developer or new to APIs, this guide empowers you to bridge the compatibility gap, ensuring confidence in deploying FastAPI APIs on GCP API Gateway. 🚀 😎

This article (and the work behind it) have been conducted under the supervision of Davide Tresoldi (Cloud Solution Architect @ BIP xTech).

Image generated with Bing AI

Table of Contents

· ⚡️ FastAPI
∘ 🔑 Key Features
· 🐝 GCP API Gateway
∘ 🔑 Key Features
· Why Use Google Cloud API Gateway for FastAPI Projects?
· How to deploy a FastAPI Project with GCP API Gateway
· The Problem
· ⭐️ The Solution
∘ Discarded solutions
∘ A Starting Point: FastAPISwagger2 package
∘ My Integrations
· Conclusions
· References

⚡️ FastAPI

FastAPI¹ is a modern, high-performance web framework for building APIs with Python 3.8+. It is based on standard Python type hints and provides a rich set of features for building APIs quickly and easily.

🔑 Key Features

• Fast: FastAPI is one of the fastest Python web frameworks available, compared with NodeJS and Go. This is due to its use of Starlette, a high-performance web framework itself, and Pydantic, a powerful data validation library.
• Easy to use: FastAPI is designed to be easy to use for developers of all skill levels. It has a simple and intuitive API, and its type hints help to ensure that your code is correct and easy to maintain.
• Robust: FastAPI is a robust and well-tested framework. It has been used in production by many companies, and it has a large and active community.
• Easy to deploy: FastAPI is easy to deploy to a variety of platforms, including Docker, Cloud Run, AWS Lambda.
• RESTful APIs: FastAPI is a great choice for building RESTful APIs. It provides a simple and intuitive API for defining routes and handling requests.

🐝 GCP API Gateway

Google Cloud API Gateway² is a fully managed service that allows you to create, secure, and manage APIs for your Google Cloud serverless backends. It is built on Envoy, a high-performance open-source proxy, and provides a number of benefits.

🔑 Key Features

• High performance: API Gateway can handle high volumes of traffic with low latency. This is because it is built on Envoy, which is a very efficient proxy.
• Scalability: API Gateway can automatically scale your APIs up or down to meet demand. This means that you don’t have to worry about provisioning or managing servers.
• Security: API Gateway includes a number of security features, such as authentication and authorization, to help protect your APIs from unauthorized access.
• Visibility: API Gateway provides you with a number of tools to monitor and troubleshoot your APIs. This includes logs, metrics, and traces.

Why Use Google Cloud API Gateway for FastAPI Projects?

Several compelling reasons make Google Cloud API Gateway an ideal choice for deploying FastAPI projects:

1. Managed API Management: API Gateway simplifies API management by handling tasks like authentication, authorization, monitoring, and traffic management. This reduces the burden on developers, allowing them to focus on their core business logic.
2. OpenAPI³ Support: API Gateway seamlessly integrates with OpenAPI specifications, enabling developers to describe their APIs using a standard format. This facilitates easy API documentation and simplifies API consumption by clients.
3. Scalable Infrastructure: API Gateway leverages Google Cloud’s robust infrastructure to handle spikes in traffic and ensure seamless API performance.
4. Security and Compliance: API Gateway provides robust security measures, including authentication, authorization, and access control policies, to protect APIs from unauthorized access.
5. Cost-Effectiveness: API Gateway’s pay-as-you-go pricing model aligns with usage, ensuring efficient resource allocation.

How to deploy a FastAPI Project with GCP API Gateway

The deployment process involves several steps:

1. Containerize your FastAPI Application: Package your FastAPI application as a Docker image⁴ to ensure portability and deployment flexibility.
2. Push the Docker Image to Artifact Registry⁵: Store the built Docker image in a container registry like Google Cloud Artifact Registry for easy access.
3. Create a Cloud Run⁶ Service: Deploy your FastAPI application to Google Cloud Run, a fully managed serverless platform, using the container image from the registry.
4. Create an API Gateway: Set up API Gateway to serve as the front door for your deployed FastAPI application.
5. Create an API Config⁷: Define the API configuration using OpenAPI specifications, specifying endpoints, request/response models, security rules, and other necessary details.
6. Expose the API: Associate the API Gateway with the Cloud Run service, exposing your FastAPI application to external users through a public URL.

The Problem

The deployment process sounds amazing and the steps seems very easy! 😉

However, while I myself I was working on that, I got stuck in step 5 (Create an API Config) because of an incompatibility issue between the OpenAPI document supported by GCP API Gateway⁸, and the OpenAPI document automatically generated from FastAPI itself⁹.
Indeed, GCP API Gateway only supports the OpenAPI version 2.0 specification¹⁰.
On the other side, the OpenAPI doc produced using the .openapi() method, only supports version 3.0¹¹.

This mismatch necessitates a manual conversion of the OpenAPI document to version 2.0 before it can be utilized by API Gateway.
But since we are devoted in automation with CI/CD pipelines, we needed to find a way to automatize this process in some way! 🚀 💻

⭐️ The Solution

Discarded solutions

I started searching a (preferably ready) solution to use to perform the conversion. I found several StackOverflow¹² / GitHub issues¹³ in which several users were trying to address the same problem I encountered.

Despite attempting to override the default OpenAPI definition¹⁴ using a custom_openapi definition and explicitly specify the version, this approach has proven unsuccessful, resulting in an error message:

Unable to render this definition The provided definition does not specify a valid version field.

Another proposed solution was to use an api-spec-converter package¹⁵. The problem is that to use this package you need Node.js. But this solution wasn’t right for us either: using a Node.js package in a Python application deployed in a CI/CD pipeline can be problematic due to several reasons:

• Performance Overhead: Introducing a Node.js package into a Python application can introduce unnecessary overhead, potentially impacting overall performance.
• Size Concerns: Node.js packages are typically larger than their Python counterparts, due to the additional runtime environment and libraries they encompass. This can lead to larger Docker images, which can take longer to build and deploy, and consume more storage space. In a cloud-based environment, where resource optimization is crucial, using large packages can be detrimental to cost-efficiency.
• Security Considerations: Integrating Node.js packages into a Python application introduces additional security considerations. Node.js environments are more susceptible to vulnerabilities, and introducing Node.js packages can amplify the attack surface.

After discarding this solution too, I started writing myself a Python script to perform all the conversion steps. But after some hours of coding, I realized it would have been taken too long to complete it, even days. 😟

A Starting Point: FastAPISwagger2 package

Without giving up, I started again for a ready to use Python package to perform the conversion.
After some research I found a Python package in PyPi: fastapi_swagger2¹⁶. It promises to let you obtain the swagger 2.0 OpenApi using just the following lines of code:

from fastapi_swagger2 import FastAPISwagger2

app = FastAPI()
FastAPISwagger2(app)

I was very happy to have found a so quick and ready to use solution! 😍 However, when I opened the generated openapi.yaml, the result was not what I expected 😩 :

I would say there are far too many errors throughout the entire document, as you can particularly notice on the right side of the image! 😡

Delving deeper, I discovered the following not removed / not converted elements not supported in OAS 2.0:

1. content: application/json: {} in response nodes

2. anyOf + type: ‘null’ for optional parameters

3. lte and gte definitions for parameters (in v2.0 should be maximum and minimum)

4. examples property in definitions (in v2.0 should be example)

5. const for enums with a single item in definitions

6. security: - APIKeyHeader: [] in methods not supported in API Gateway

My Integrations

Despite the limitations just described, the fastapi_swagger2 package have been an optimal starting point and it saved me a lot of time. 😌

In the following I’m going to provide you the add-ons I’ve been working on:

• All the code snippets you can integrate in the openapi generation script to overcome the unsupported elements
• 💰 Bonus: Code snippets to add custom GCP entries to make your openapi.yaml file fully compatible with GCP API Gateway!

NB: In all the code sections in which you find env variables, you can set them in .env dedicated file, or, as in our case, providing them while your CI/CD pipeline is running, for example as substitutions in a Cloud Build Trigger, by explicitly declaring them or retrieving them from other Google Cloud Platform services using Google Cloud commands.

extract_openapi.py

I called extract_openapi.py the main file to execute to perform the conversion, which is located at the root level in the same FastAPI folder of your project, and which you can execute both locally or in a CI/CD pipeline step.

from v1.utils.openapi import add_custom_gcp_entries, redefine_definitions, redefine_paths

import yaml
from app.main import app

if __name__ == "__main__":

    openapi = app.swagger2()

    # Redefine paths
    openapi = redefine_paths(openapi)

    # Substitute definitions
    openapi = redefine_definitions(openapi)

    # Add custom gcp entries
    openapi = add_custom_gcp_entries(openapi)

    with open("openapi.yaml", "w") as f:
        yaml.safe_dump(openapi, f, sort_keys=False)

Importing the app, we use the swagger2() method to get the openapi generated as a starting point, and then we perform modifications with the redefine_paths, redefine_definitions, add_custom_gcp_entries methods imported from a specific utility created on purpose. Let’s analyze what’s inside! ✌️

v1.utils.openapi

1. substitute_at_position()

The purpose of this function is to be reused multiple times, because it allows dynamic modifications to the OpenAPI specification by replacing specific elements at specific positions, for integrating custom components and making adjustments.

def substitute_at_position(openapi: dict, index: str, substitution: List[str | dict] | None):
    if index in openapi.keys():
        pos = list(openapi.keys()).index(index)
        items = list(openapi.items())
        items.pop(pos)
        if substitution:
            items.insert(pos, (substitution[0], substitution[1]))
        openapi = dict(items)

    return openapi

2. redefine_paths()

The overall purpose of this function is to remove the unnecessary or unsupported elements, which are present in the “paths” section. In particular, it addresses the previously described 1, 2, 3, 6 points.

def redefine_paths(openapi: dict):
    METHODS = ['get', 'post', 'put', 'patch', 'delete']
    paths = openapi['paths']
    new_paths = {}
    for path in paths:
        new_path = path
        new_paths[new_path] = paths[path]
        for method in METHODS:
            if method in new_paths[new_path]:

                # remove security and APIKeyHeader in paths
                new_paths[new_path][method] = substitute_at_position(new_paths[new_path][method], "security", None)

                for parameter_index, parameter in enumerate(new_paths[new_path][method]["parameters"]):

                    # Convert gte with minimum
                    if "gte" in new_paths[new_path][method]["parameters"][parameter_index].keys():
                        new_paths[new_path][method]["parameters"][parameter_index]  = substitute_at_position(new_paths[new_path][method]["parameters"][parameter_index], "gte", ["minimum", new_paths[new_path][method]["parameters"][parameter_index]["gte"]])
                    
                    # Convert lte with maximum
                    if "lte" in new_paths[new_path][method]["parameters"][parameter_index].keys():
                        new_paths[new_path][method]["parameters"][parameter_index]  = substitute_at_position(new_paths[new_path][method]["parameters"][parameter_index], "lte", ["maximum", new_paths[new_path][method]["parameters"][parameter_index]["lte"]])
                    
                    # remove anyOf declarations
                    if parameter["required"] == False:
                        new_paths[new_path][method]["parameters"][parameter_index]  = substitute_at_position(new_paths[new_path][method]["parameters"][parameter_index], "anyOf", list(reduce(lambda x, y: x + y, new_paths[new_path][method]["parameters"][parameter_index]["anyOf"][0].items())))
                
                for response in new_paths[new_path][method]["responses"]:
                    # remove all content / application json {} (not supported)
                    new_paths[new_path][method]["responses"][response] = substitute_at_position(new_paths[new_path][method]["responses"][response], "content", None)
        
    openapi['paths'] = new_paths
    return openapi

3. redefine_definitions()

The purpose of this function is to refine the “definitions” section within the OpenAPI specification.

def redefine_definitions(openapi: dict):

    for definition in openapi["definitions"]:
        # Remove const
        if "const" in openapi["definitions"][definition].keys():
            enum = ["enum", [openapi["definitions"][definition]["const"]]]
            openapi["definitions"][definition] =  substitute_at_position(openapi["definitions"][definition], "const", enum)

        # Redefine examples
        if(openapi["definitions"][definition].get("examples")) is not None:
            new_example = ["example", openapi["definitions"][definition]["examples"][0]]
            openapi["definitions"][definition] = substitute_at_position(openapi["definitions"][definition], "examples", new_example)

    return openapi

4. 💰 Bonus: Add custom GCP Entries

By adding specific GCP Entries needed in the GCP API Gateway config, we can make the openapi.yaml file fully compatible with and immediately ready to be used within GCP API Gateway and enable secure authentication using Firebase.

def get_google_backend():
    return {"address": os.environ.get("X_GOOGLE_BACKEND")}

def get_google_endpoints():
    return [{
        "name": os.environ.get("X_GOOGLE_ENDPOINTS"),
        "allowCors": True
    }]

def get_security_definitions():
    authentication_project_id = os.environ.get("AUTHENTICATION_PROJECT_ID")
    return {
        "firebase": 
            {
                "authorizationUrl": "",
                "flow": "implicit",
                "type": "oauth2",
                "x-google-issuer": f"https://securetoken.google.com/{authentication_project_id}",
                "x-google-jwks_uri": "https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken@system.gserviceaccount.com",
                "x-google-audiences": authentication_project_id,
                "scopes": {"user-authentication": "firebase"}
            }
}

def add_custom_gcp_entries(openapi: dict):
    openapi["host"] = os.environ.get("SERVICE_URL")
    openapi["schemes"] = ["https"] 
    openapi["security"] = [{"firebase" : []}]
    openapi["x-google-backend"] = get_google_backend()
    openapi["x-google-endpoints"] = get_google_endpoints()
    openapi["securityDefinitions"] = get_security_definitions()

    return openapi

Conclusions

This article has provided a comprehensive guide on bridging the compatibility gap between FastAPI’s OpenAPI v3.0 and GCP API Gateway, which only supports Swagger 2.0. The provided code snippets and instructions can be used to effectively convert FastAPI OpenAPI to v2.0, enabling seamless integration with GCP API Gateway. Additionally, the bonus section demonstrates how to add custom GCP entries to the generated openapi.yaml file, making it fully compatible with GCP and enabling secure authentication using Firebase.

By following these steps, developers can effectively deploy their FastAPI APIs to GCP API Gateway, leveraging its robust features and benefits. This ensures that developers can focus on building innovative APIs without being hindered by compatibility issues. 🏆 🎯

Thank you for reading this article!
We hope you found it informative and helpful. We appreciate your feedback and suggestions. Please feel free to leave a comment below or reach out to us on social media.
We are always looking for ways to improve our content, and your input is valuable to us. 💙

See you next time! 😄

References

[1]: https://fastapi.tiangolo.com/
[2]: https://cloud.google.com/api-gateway
[3]: https://www.openapis.org/
[4]: https://docs.docker.com/build/
[5]: https://cloud.google.com/artifact-registry
[6]: https://cloud.google.com/run?hl=en
[7]: https://cloud.google.com/api-gateway/docs/creating-api-config
[8]: https://cloud.google.com/api-gateway/docs/openapi-overview
[9]: https://fastapi.tiangolo.com/how-to/extending-openapi/#the-normal-process
[10]: https://cloud.google.com/endpoints/docs/openapi/openapi-limitations
[11]: https://github.com/tiangolo/fastapi/issues/70
[12]: https://stackoverflow.com/questions/68184751/how-to-generate-swagger-2-0-documentation-for-fastapi
[13]: https://github.com/tiangolo/fastapi/issues/1665
[14]: https://fastapi.tiangolo.com/how-to/extending-openapi/#overriding-the-defaults
[15]: https://www.npmjs.com/package/api-spec-converter
[16]: https://pypi.org/project/fastapi_swagger2/