Terraform basics: Deploy your own GPT-4 model with Azure OpenAI services

Published in

arconsis

6 min readMay 8, 2024

If you want to have your own OpenAI instance running in Azure, this article is perfect for you! We will dive into setting up an OpenAI GPT-4-turbo model in your own Azure account with the help of Terraform.

Terraform Basics

To set up your own GPT-4 model in Azure you could manually press the different buttons in the Azure portal or use the Command Line Interface (CLI) to set up all the services you want, but this takes quite some time and can be a hassle, because if you want to reproduce it, you have to repeat all the steps. To make infrastructure configurable and reproducible, there are tools like Terraform, that allow you to define the infrastructure you want.

For Terraform you create *.tf files to describe your infrastructure. There are different types of resources you can specify depending on the provider you pick (azurerm in our case) as well as variables you can define to make configuring your infrastructure easier and tailored to your needs.

The Terroform CLI expects all the files to be in the current working directory by default (you can specify another path via options). You could technically put all definitions into a single project.tf file, but this will quickly become confusing — remember that we are writing code not for the machine, but for humans (including yourself) to understand it.
So a best practice is to have separate files for the different types of definitions, but remember that the files have to be flat in the working directory — at least in our simple case. If you want to have more separation, you can create modules.

Providers

Terraform by itself, doesn’t know how to create resources in Azure. This is handled by the azurerm provider. There are different ones for the many cloud providers out there, but we will stick with Azure in this example.

To get started we will create a provider.tf file in your working directory and put this content into it.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.102.0"
    }
  }
}

provider "azurerm" {
  features {}

  client_secret   = var.az_client_secret
  client_id       = var.az_client_id
  tenant_id       = var.az_tenant_id
  subscription_id = var.az_subscription_id
}

In the upper part we defined the provider we want to use and in the lower part you might have noticed the references to a var object. In the next section we will take a look at this.

Variables

For our use case, we will take a look at input and output variables.

Inputs

Input variables define the variables that Terraform needs as an input from the user to fill in the gaps in the configuration or to change the configuration according to the user’s needs. They are comparable to function parameters in regular programming.

In our example, we will create two different files, variables.azurerm_access.tf

variable "az_client_id" {
  type = string
}

variable "az_client_secret" {
  type      = string
  sensitive = true
}

variable "az_subscription_id" {
  type = string
}

variable "az_tenant_id" {
  type = string
}

and variables.tf to have a better separation of concerns — more on this in the upcoming section.

variable "deployment_region" {
  type    = string
  default = "swedencentral"
}

If you were to run Terraform now, it would ask you for the missing variables (from variables.azurerm_access.tf). For the variable deployment_region there is a default, so this would be used.
Also note the sensitive option on the az_client_secret, this will hide the value in the command line outputs.

Secrets/Variable Sets

Since I don’t want to enter the four variables defined in variables.azurerm_access.tf every time I build up my infrastructure with Terraform, we can define a secrets.auto.tfvars file — the name secrets is how I named the file, you can choose any name you like.

Notice the auto part in the middle. This will tell Terraform to automatically import the variables defined in the file. Without auto in the name you would have to pass the *.tfvars files to Terraform via terraform <action> -var-file ./path/to/*.tfvars.

az_client_id       = "xxx"
az_client_secret   = "xxx"
az_subscription_id = "xxx"
az_tenant_id       = "xxx"

The benefit of having these secrets in a separate file, is also that you can add it to your .gitignore to make sure that no secrets are checked into git, while being able to share it with your team or whoever needs access to your subscription.

Resources

Now that we have configured our connection to Azure, it’s time to create some resources.

For our use case, we will need a resource group, an azure cognitive account and an azure cognitive deployment — so let’s go through the resource.tf file next.

resource "azurerm_resource_group" "gpt-group" {
  location = var.deployment_region
  name     = "MyOpenAIGPT"
}

resource "azurerm_cognitive_account" "open_ai" {
  name                = "OpenAIAccount"
  kind                = "OpenAI"
  location            = azurerm_resource_group.gpt-group.location
  resource_group_name = azurerm_resource_group.gpt-group.name
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "open_ai" {
  name                 = "OpenAIDeployment"
  cognitive_account_id = azurerm_cognitive_account.open_ai.id

  model {
    format  = "OpenAI"
    name    = "gpt-4"
    version = "turbo-2024-04-09"
  }

  scale {
    type = "Standard"
  }
}

After the keyword resource you put the kind of resource you want to define (see the azurerm documentation for possible options) and then give it a name. In the curly brace you have to set the required options for this type of resource and you can specify the optional ones.

Notice that in the second resource, we reference values from the resource group defined in the first resource. This is done by writing the resource type, the name and the property separated by dots, similar to how you would access properties of an object in other programming languages.

Please note, that the resources don’t have to be ordered. Terraform will figure out the dependencies between the resources and create them in the order that makes sense — or complain otherwise.

Outputs

Finally, I want to take a quick look at output variables. Since we are creating an OpenAI model, we need to know where and how to talk to it. To make our lives easier, we can define output variables which are then printed out in the command line and are available in the terraform.state for easy reference.

output "open_ai_endpoint" {
  value = "${azurerm_cognitive_account.open_ai.endpoint}openai/deployments/${azurerm_cognitive_deployment.open_ai.name}/chat/completions?api-version=2024-02-15-preview"
}

output "open_ai_access_key" {
  value     = azurerm_cognitive_account.open_ai.primary_access_key
  sensitive = true
}

As you can see, you can inline variables to form the actual value you need — in our case, for using the API endpoint.
Aagain, you can specify outputs as sensitive to hide them in the command line output.

Terraform Commands

With all our values in place, it’s time to run Terraform. There are two main commands that we will need.

terraform plan

Plan is useful to make sure, that the script has no obvious syntax issues and that everything is defined correctly. Furthermore, terraform plan also compares the current state with the new state and lists the proposed changes to the infrastructure. This way, you can double check if you really want to apply these changes.
So before applying the changes, make sure to test it first with terraform plan.

terraform apply

If everything looks good, you can run terraform apply to create the resources defined before. This will take a few moments, but after that you should be able to see your newly created resource group, OpenAI service account and model in Azure.

Chatting with our OpenAI model

You will see the endpoint in the output after running terraform apply. In addition to that, you will also need the primary access key, which we configured into the open_ai_access_key output variable. To get the value of it open terraform.tfstate, there you should find it somewhere at the top.

The Azure OpenAI API documentation is quite extensive, so here is a link to the details: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference

To submit a request you can use your favorite API tool and put the API key in the header like this: api-key: <open_ai_access_key>

You can use this body and make a POST request to your own OpenAI ChatGPT instance.

{
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "You are an expert in Terraform and love writing articles for medium."
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Write me an introductory article about setting up Azure OpenAI with Terraform."
        }
      ]
    }
  ],
  "temperature": 0.7,
  "top_p": 0.95,
  "max_tokens": 800
}

The answer would be:

If you want to run your own OpenAI instance on Azure, this article is perfect for you! We will dive into setting up an OpenAI GPT-4-turbo model in your own Azure account with the help of Terraform.
…

😉

Thanks for reading, I hope you found this article useful and learned something! Have fun forming your infrastructure with Terraform. If you want to read more about interesting software engineering topics, take a look at our arconsis medium page.