Monitoring and Debugging Python Applications in Production with Choreo
In the recent years of the programming ecosystem, Python has been gaining popularity as a programming language amongst beginner and enterprise programmers because of its elegant syntax and rich ecosystem. API and Web Service development using Python has been supported through popular frameworks and libraries such as Django, FastAPI, and Flask.
When you develop an API or a Service using Python, or any other language, the next step is deploying it and exposing it to the internet and monitoring the application through its lifecycle. If you decide to deploy the application by yourself, you have to figure out setting up a VPS, CI/CD pipelines, load-balancers, DNS, and all the other million things associated with creating and handling a platform by yourself. When it comes to monitoring and observing your application, you would need to set up separate tools. This is where Choreo comes in handy.
Choreo is an internal developer platform as a service, which facilitates building, deploying, and running your cloud-native applications in state-of-the-art Kubernetes based environments and provides you with a rich set of tools to monitor and observe your application. As a software developer, you can solely focus on developing and monitoring your application while Choreo as a platform manages all the underlying complexities.
In this article, we will explore how to develop a simple Python web application, deploy, monitor and debug it using Choreo. For this purpose, I will be using the web development framework FastAPI and PyCharm as the IDE.
Developing the PizzaShackAPI
Let’s create a new project using PyCharm. I’ll be using Python version 3.11 to develop my API.
Let’s create an application named PizzaShackAPI with 4 resources: Menu, Orders, Product, and Users. The models for the 4 resources are shown below.
from typing import List
from pydantic import BaseModel
class User(BaseModel):
id: int
name: str
email: str
users: List[User] = []
---
from pydantic import BaseModel
from typing import List
class Product(BaseModel):
id: int
name: str
price: float
stock: int
products: List[Product] = []
---
from pydantic import BaseModel
from typing import List
class Order(BaseModel):
id: int
user_id: int
product_ids: List[int]
total_price: float
status: str
orders: list[Order] = []
---
from pydantic import BaseModel
class MenuItem(BaseModel):
id: int = None
name: str
description: str
price: float
menu_items: list[MenuItem] = []Note that we’ve created lists of the models within the same files to eliminate the need for a database. These lists will serve as a temporary ‘database’ when performing operations on the resources.
After creating the models in the <DIR>/models directory, let’s proceed to create the APIs that will expose these models.
# API resources for menu
from fastapi import APIRouter, HTTPException
from models.menu import MenuItem, menu_items
import logging
router = APIRouter()
logger = logging.getLogger(__name__)
@router.get("/menu")
async def get_menu():
logger.info("Fetching menu items")
return menu_items
@router.get("/menu/{item_id}")
async def get_menu_item(item_id: int):
logger.info(f"Fetching menu item with id: {item_id}")
item = next((item for item in menu_items if item.id == item_id), None)
if item is None:
logger.warning(f"Menu item not found: {item_id}")
raise HTTPException(status_code=404, detail="Item not found")
return item
@router.post("/menu")
async def add_menu_item(item: MenuItem):
logger.info(f"Adding new menu item: {item.name}")
item.id = len(menu_items) + 1
menu_items.append(item)
return item
@router.put("/menu/{item_id}")
async def update_menu_item(item_id: int, updated_item: MenuItem):
for i, item in enumerate(menu_items):
if item.id == item_id:
updated_item.id = item_id
menu_items[i] = updated_item
return updated_item
raise HTTPException(status_code=404, detail="Item not found")
@router.delete("/menu/{item_id}")
async def delete_menu_item(item_id: int):
for i, item in enumerate(menu_items):
if item.id == item_id:
return menu_items.pop(i)
raise HTTPException(status_code=404, detail="Item not found")
---
# imports, router and logging
@router.get("/orders")
async def get_orders():
logger.info("Fetching all orders")
return orders
@router.post("/orders/", response_model=Order)
async def create_order(order: Order):
logger.info(f"Creating order for user: {order.user_id}")
if not any(u.id == order.user_id for u in users):
logger.error(f"User with id {order.user_id} not found")
raise HTTPException(status_code=404, detail="User not found")
total_price = 0
for product_id in order.product_ids:
product = next((p for p in products if p.id == product_id), None)
if not product:
logger.error(f"Product with id {product_id} not found")
raise HTTPException(status_code=404, detail="Product not found")
if product.stock <= 0:
logger.warning(f"Product {product.name} is out of stock")
raise HTTPException(status_code=400, detail=f"Product {product.name} is out of stock")
total_price += product.price
product.stock -= 1
order.total_price = total_price
order.status = "Pending"
orders.append(order)
logger.info(f"Order {order.id} created successfully")
return order
@router.get("/orders/{order_id}")
async def get_order(order_id: int):
logger.info(f"Fetching order with id: {order_id}")
order = next((order for order in orders if order.id == order_id), None)
if order is None:
logger.warning(f"Order not found: {order_id}")
raise HTTPException(status_code=404, detail="Order not found")
return order
@router.put("/orders/{order_id}")
async def update_order(order_id: int, updated_order: Order):
logger.info(f"Updating order with id: {order_id}")
for i, order in enumerate(orders):
if order.id == order_id:
updated_order.id = order_id
orders[i] = updated_order
return updated_order
logger.warning(f"Order not found for update: {order_id}")
raise HTTPException(status_code=404, detail="Order not found")
---
# imports, router and logging
@router.post("/products/", response_model=Product)
async def create_product(product: Product):
logger.info(f"Creating product: {product.name}")
if any(p.id == product.id for p in products):
logger.warning(f"Product with id {product.id} already exists")
raise HTTPException(status_code=400, detail="Product already exists")
products.append(product)
return product
@router.get("/products/{product_id}", response_model=Product)
async def get_product(product_id: int):
logger.info(f"Fetching product with id: {product_id}")
product = next((p for p in products if p.id == product_id), None)
if not product:
logger.error(f"Product with id {product_id} not found")
raise HTTPException(status_code=404, detail="Product not found")
return product
---
# imports, router and logging
@router.post("/users/", response_model=User)
async def create_user(user: User):
logger.info(f"Creating user: {user.name}")
if any(u.id == user.id for u in users):
logger.warning(f"User with id {user.id} already exists")
raise HTTPException(status_code=400, detail="User already exists")
users.append(user)
return userAfter creating the API resources, use main.py to set the API prefixes and assign the routes to the context.
from fastapi import FastAPI
from api import menu, orders, product
app = FastAPI(title="Pizzashak API")
app.include_router(menu.router, prefix="/api", tags=["menu"])
app.include_router(orders.router, prefix="/api", tags=["orders"])
app.include_router(product.router, prefix="/api", tags=["products"])
@app.get("/")
async def root():
return {"message": "Welcome to Pizzashak API"}Since we’ve included logging in the application, we’ll need to create a config.py file to configure the logging capability.
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
logger = logging.getLogger(__name__)Start the application locally by running the following command.
uvicorn main:app --reload --port 8080After testing the application, make sure to push your project to Github. The above code is available at https://github.com/rashm1n/choreo-python-example .
Now onto the fun stuff ! Let's see how we can deploy this application to Choreo!
Adding Choreo Specific Configurations
Before deploying our Python application to Choreo, we need to add a Choreo specific configuration file, named component-config.yaml. This configuration will essentially define your API endpoints, set contexts, set ports and link any existing OpenAPI definition files to our service. This file should be created in the .choreo directory of your repository.
apiVersion: core.choreo.dev/v1beta1
kind: ComponentConfig
spec:
inbound:
- name: Pizzashack API
port: 8080
type: REST
networkVisibility: Public
schemaFilePath: open-api.yamlChoreo utilizes cloud-native buildpacks to containerize your application, and in the case of Python applications, we need to add a Procfile to our repository, which will specify how our application will start. For example, python main.py, gunicorn main:app. This is the initial command used to start our web application.
In our case, this file would look like the following.
web: uvicorn main:app --host 0.0.0.0 --port 8080After all the changes, your application structure would look like this.
├── .choreo
│ ├── component-config.py
├── api
│ ├── __init__.py
│ ├── menu.py
│ ├── orders.py
│ ├── product.py
│ └── users.py
├── models
│ ├── __init__.py
│ ├── menu.py
│ ├── order.py
│ ├── product.py
│ └── user.py
├── config.py
├── main.py
├── Procfile
├── open-api.yaml
├── requirements.txtDeploying The Application to Choreo
- Sign up to Choreo and create an account.
- Provide a name and create an organization.
- Create a project by providing a name and description.
- Then, navigate to Components and Click + Create Component.
- From the Create a New Component menu, select Service and provide the following details.
Name - PizzashackAPI
Description - A simple pizza restaurant management API.
Repository URL - https://github.com/rashm1n/choreo-python-example (You can provide your own repository URL)
Branch - master (Select the appropriate branch)
Buildpack - Python
Language Version - 3.11.xAfter providing the repository, select Python from the buildpack section, this would use a standard Python buildpack to create a deployable image of your application.
6. After creating the service, navigate to the Build section in the left. sidepane and click Build Latest Commit, this would start building a container image using our provided code with a Python Buildpack.
7. After the build is successful, navigate to the Deploy section in the left pane, and click Configure & Build , this will deploy the built container to the Choreo Cloud Dataplane managed Kubernetes cluster. And just like that, within a few minutes you have a production grade Python application running on state of the art Kubernetes infrastructure! It’s that simple.
8. Once the application is deployed to the Development environment, navigate to the Test section from the side pane and invoke our API resources to test them out.
Choreo Observability and Insights
After the application is up and running in Choreo cloud, as a developer we need to constantly monitor and observe our application. This includes observing logs and inspecting insights about the API performance. Choreo provides you with a rich set of tools for all your monitoring and observability needs.
They are twofold,
- Observability
- Usage Insights
Observability
Choreo Observability provides a set of tools to monitor the internal state of a deployed application thoroughly. Runtime logs provide the user with all application and gateway level logs which are being recorded in real-time to get an idea of what’s happening in your application. Numerical measurements about your application, mainly the Kubernetes pod the application is deployed in, are visualized comprehensively via the Metrics section. With a combination of logs and metrics, a developer is able to get a complete picture of what’s going on inside the deployed application and utilize the provided data to debug and optimize the application.
Usage Insights
Usage insights provides you with a set of measurements which depicts various aspects of the API request flow in your application. When clients/users invoke the applications which are deployed on Choreo, a comprehensive set of measurements regarding those invocations are stored and displayed via the Usage Insights dashboard. Measurements like Request Latency, Error Rate, Overall Traffic and much more are available for the developer to analyze and utilize.
Utilizing Choreo Observability for Debugging
A trait of a good developer is putting Application logs in appropriate locations to track exceptions, possible errors and warnings which would help them to debug the application. Various logging levels such as Info, Warning and Error are available on most logging libraries to be used. In our Python application, we have put logs in appropriate locations to track exceptions.
Lets invoke some API resources and see the flow of logs using Choreo Observability Runtime Logs.
Lets invoke the /users endpoint in our application and add a new user to our system.
As the response, we have received a 404 Not Found Error response.
Let’s navigate to the observability view to see why this did not work. You can navigate to the Runtime Logs option in the Observability menu in the side pane.
Inside the Runtime Logs menu, you can see a set of filters like Log Type, Deployed Environment, Time Duration, etc, and a set of most recent logs according to the filters. You can set the time filter to the last 10 minutes to filter only the latest logs and make the log displaying window clutter-free.
In that window, you will see two logs printed. One application log and one gateway log. Here, application logs refer to the logs that we, as app developers, have implemented in our application. Gateway logs are generated when the user request hits the external gateway(where the user’s HTTP request initially hits, and then forwards it to your app) of Choreo.
A single log line is split into several sections, each with a different color. First, you could see the timestamp, environment, log type, and then the log message. At a glance, you can clearly identify the logs and it is easier to go through them than a typical single-colored, cluttered log line we get in a terminal.
In this situation, we can see that the application logs are indicating that /users was not found with a 404 error, and if we see the gateway logs, we could see a 404 Service Response. In the gateway logs RequestPath indicates that we have received a request to the given URL and the ServicePath indicates that it is mapped to the /users resource. We can deduce that the gateway has forwarded the request to the service but the resource /users was unavailable in our app.
Let’s track back to our application code and have a look at the location at which our resources are exposed, the main.py file.
app.include_router(menu.router)
app.include_router(orders.router)
app.include_router(product.router)And if we observe the exposed resources via app.include_router we can see that although we have created the user resource, we haven’t registered it. We can add the following code segment to expose our user resource.
app.include_router(users.router)You can now commit the change and build and deploy the application again following the steps mentioned above. When you try to add a user now, you will receive a successful response! Although this was a simple example, Runtime and Application logs can be utilized to debug and monitor any complex application you deploy on Choreo. Thoughtfully placed info, warn, and error logs, combined with Choreo's powerful runtime logs feature, creates a highly effective combination for monitoring and debugging applications.
Let’s say you’ve finished debugging your application and now need to monitor its resource usage to assess its performance. Choreo offers a comprehensive set of tools to help you do just that!
Once you navigate to the Metrics section of the Observability menu, you will see two sub-menus, Throughput and Latency and Diagnostics.
Let's observe the Throughput and Latency menu.
We are able to see two graphs depicting the Throughput and Latency of our app and see two spikes from when we invoked the application.
To get an idea about the resource usage of the application, let's navigate to the Diagnostics section.
Here you can see a set of graphs depicting CPU/Memory usage, latency, throughput, and error rate alongside the relevant timestamp and logs. We could see that at the instances we have invoked the application, CPU/Memory usage has risen to 1.26 millicores and 40 MBs which is a very low usage of memory, and from that we can deduce that our application is performing up to our standard. If there are certain resource-heavy sections in your code, you can put Application Logs in the locations and monitor the CPU/Memory usage during those to get an idea about the performance.
Deploying and monitoring Python applications on the cloud can be a challenging task in today’s technology landscape with all the complex tools available. Choreo provides the developer with a rich set of tools that abstracts the complex technology underneath and offers powerful monitoring and observability features that help to monitor, debug, and optimize your applications.
In this article, we talked about how to utilize Choreo to deploy a simple Python API and use its runtime logs and metrics features to monitor it. Hope you found the article useful and don't hesitate to ask any questions!
Resources
[1] — Choreo — https://choreo.dev/
[2] — FastAPI — https://fastapi.tiangolo.com/
[3] — Sample code — https://github.com/rashm1n/choreo-python-example
