Setting up Langtrace on Azure

Understanding the behavior and performance of Large Language Models (LLMs) APIs is paramount for organizations leveraging these powerful tools in their applications. For those fully immersed in Azure, this streamlined setup process ensures seamless integration, freeing up resources to focus on optimizing and refining LLMs.

Enter Langtrace — an open-source observability tool built on OpenTelemetry principles, designed to gather and analyze traces for enhancing LLM applications. By seamlessly integrating Langtrace with Azure, organizations gain effortless access to comprehensive monitoring and analysis capabilities tailored specifically for LLM apps deployed on Azure.

In this comprehensive guide, we’ll lead you through the step-by-step process of setting up Langtrace on Azure, empowering you to harness its full potential for maximizing the performance and efficiency of your LLM applications.

If you are interested in visual guide:

Prerequisites

• An Azure account with the necessary permissions to create and manage resources.
• A Clickhouse server (cloud or self-hosted)
• Azure developer cli on your system. (installation steps)

Resources Created

|    Resource Type    |         Name         |                          Comments                          |
|---------------------|----------------------|------------------------------------------------------------|
| Resource Group      | <project-name>-rg    | A resource group where all the resources will be deployed. |
| Postgres Server     | <project-name>-pg    | Azure Postgres flexible server with backup on.             |
| Postgres Firewall   | <project-name>-pg-fw | Firewall that is open to world.                            |
| App Service Plan    | <project-name>-sp    | App service plan to run Langtrace app.                     |
| App Service         | <project-name>-app   | App service to run Langtrace app.                          |

Steps

This setup is for development purpose only and serves as a reference for production usecases.

Step 0. Setup tools

• Install git, azd and terraform.
• Configure your Azure account by running `azd auth login`.

Step 1. Clone and update configuration

Clone the repo

git clone https://github.com/Scale3-Labs/langtrace-azure-setup.git
cd langtrace-azure-setup

Update default variables (recommended)

Update the region and username and passwords of the admin user and Postgres DB based on your preference in the file ‘main.tfvars.json’.
You can also modify variables.tf based on your setup requirements.

Step 2. Run the Deploy

Once you are in the cloned repo, run the following command to initiate the deploy process

azd up

Output of azd up command

Now you will be asked to input the following variables. Give a unique name for langtrace_project as it would be part of the self-hosted Langtrace app URL.

Terraform inputs for azd up

Once you hit enter, it will take around 15 minutes for Terraform to complete the provisioning. Once it’s provisioned, you will see the following output.

Outputs:

langtrace_url = "https://langtraceblog-app.azurewebsites.net"

Open the URL and log in using admin credentials.

Once you have created a project and generated a Langtrace API key, you can integrate it with your app as follows

langtrace.init(
    api_key="<API KEY>",
    write_to_langtrace_cloud=True,
    api_host="https://langtraceblog-app.azurewebsites.net/",
)

And your traces would be visible on the self-hosted Langtrace Dashboard.

If you want the starter code, check out the docs.

Step 3. Deprovision

If you wish to delete the Azure resources, the cleaner way to do it would be by running the following command.

azd down

While running this command, you will be asked for variables, you can enter anything or keep it blank.

Type ‘yes’ when asked for confirmation.

Conclusion

Deploying Langtrace on Azure empowers you with invaluable insights into the performance and behavior of your LLM apps, enabling you to optimize their usage and elevate overall application performance. With Azure’s robust security measures safeguarding your data, you can rest assured of your system’s integrity and compliance.

Exploring additional hosting possibilities? Take a look at our documentation for more options.

Join our Discord community for the latest updates and engage with fellow enthusiasts. Should you encounter any challenges or queries, don’t hesitate to reach out — we’re here to assist. Start your journey of enhanced observability and optimization with Langtrace today.

Happy langTracing!