How to use the LigtSketch API to build a state-of-the-art AI headshot application.

In this tutorial, we will go through how to create a backend for a state-of-the-art AI headshot application, using the LightSketch API.

The LightSketch API empowers users to generate top-quality AI headshots via a single API call. LightSketch’s ‘headshot-optimizer’ and in-house checkpoint (LS-Headshots), available via the Dreambooth endpoint, have been carefully developed to turn raw selfies into the best AI-generated headshots. Using this feature will ensure that your application delivers stunning headshots to your users. (see here for more details on the headshot optimizer)

Results comparisons of training a Dreambooth model using SDXL 1.0, LS-Headshots and LS-Headshots combined with the headshot-optimizer.

The first step is to build a good selfie dataset. If you are unsure about how to do this, please consult this guide. Your dataset will be used to teach the AI what the subject looks like and picking the right training images is key.

Once you have made a dataset of 10 to 15 selfies, zip them up and host them at a publicly accessible URL on the internet (S3, Azure or any other CDN or hosting service will do). When you have a URL which hosts the zip file, you are ready to make your training request. The first thing you will need to do is set the payload with all of your training and generation parameters:

import json

payload = json.dumps({
    "base_model_id": "sdxlV8DwajSa",
    "model_name": "your_model_name",
    "headshot_optimizer": True, 
    "training_images_url": <URL link to your data>,
    "steps": 2000,
    "subject": "woman",
    "learning_rate": 0.00001,
    "instance_prompt": "photo of skw woman",
    "resolution": [
        1024,
        1024
    ],
    "sample_generation_job": {
        "prompt": "photo of skw woman, professional portrait, neutral background",
        "negative_prompt": "",
        "seed": -1,
        "sampler_name": "DPM++ 2M SDE Karras",
        "num_generations":5,
        "steps": 30,
        "cfg_scale": 7,
        "width": 1024,
        "height": 1024
    }
})

The first set of parameters are related to training and the second set, in the “sample_generation_job” dictionary, guides the image generation process. Let’s go through this list one by one, starting with the training parameters:

• Base_model_id: the model ID for the model that will be used as a base for training. In this example “sdxlV8DwajSa” corresponds to the LightSketch in-house headshot checkpoint (LS-headshot). I recommend using this one to get the best results possible, but more models are available if you want to experiment. To get the full list of model IDs, you can use the GET models endpoint.
• Model_name: this is the name that will be given to your model, it can be anything.
• headshot_optimizer: Set this to True to turbo-charge your training using LightSketch’s optimised training process.
• Training_images_url: the URL to the zip folder containing your training images.
• steps: the number of training steps. I usually set this to 160 times the number of images in the training data. So for example, if I have 10 images, I would use 1600 steps.
• Subject: the gender of the subject you are training. It can take the values “man” or “woman”.
• learning_rate: I suggest that you leave the learning rate as it is. If you are feeling adventurous, you can try values between 1e-7 and 1e-4.
• instance_prompt: This is the caption that will be used with your training images during the Dreambooth process. I recommend using “photo of skw” + the subject gender, so for example “photo of skw woman”
• resolution: The default 1024*1024 setting is what is usually recommended for SDXL. If you want to generate images in a different resolution, you can experiment with different settings. For example, in some cases, if you want to generate portrait images, setting this to 768*1024 can give better results.

The generation parameters are straightforward if you have any experience using Automatic1111’s webui, or another Stable Diffusion base image generator. But to go over them quickly:

• Prompt & negative_prompt: we are sure you are familiar with these by now. With SDXL, I find that leaving the negative_prompt empty usually gives good results.
• Seed: unless You can leave this as -1, which means random.
• Sampler_name: to get the full list of available samples, you can use the GET samplers endpoint. I personally like DPM++ 2M SDE Karras
• Num_generations: this is the number of headshots that will be generated.
• Steps: this is the number of steps that will be used to generate the images. With SDXL I usually use 25 to 30 steps. With SD1.5, 20 steps works well.
• Cfg_scale: this is a more advanced parameter and can be left at 7. The higher the number, the more the model will focus on the prompt when making the images, which usually comes at the expense of good image quality.
• Width & height: are the dimensions in pixels for your generated images. For best results, I would use your model’s training resolution (1024*1024 in this example).

Now that you’ve picked all the parameters, it’s time to look at the actual API call. First, you will need to post the Dreambooth job:

import requests

url = "https://api.lightsketch.ai/v1/training/dreambooth"

headers = {
  'Content-Type': 'application/json',
  'api_key': <URL API key>
}

training_response = requests.request("POST", url, headers=headers, data=payload)

In addition to the payload that we defined above, you will need to insert your LightSketch API key inside the header. You can create a key by going to your settings on the LightSketch website.

Once you have a job running, you can sit back and relax, it can take up to 2h for a job to complete. If you are running this API inside an app, you can add a webhook to the payload to check when the job is complete.

To retrieve your results, you will use the GET training/dreambooth endpoint:

id = training_response.json()["id"]

url = "https://api.lightsketch.ai/v1/training/dreambooth/" + id

response = requests.request("GET", url, headers=headers)
print(response.text)

The response will look like this:

{
    "base_model_id": "sdxlV8DwajSa",
    "model_name": "your_model_name",
    "timestamp_created": "2023-12-02T18:02:52.091824+00:00",
    "gpu_seconds": 5955.08845615387,
    "subject": "woman",
    "status": "success",
    "type": "dreambooth",
    "id": <your job id>,
    "tokens": 1500,
    "error": null,
    "webhook_url": null,
    "steps": 2000,
    "training_images_url": <URL link to your data>,
    "learning_rate": 1e-05,
    "resolution": [
        1024,
        1024
    ],
    "trained_models": [
        {
            "id": <your model id>,
            "type": "sdxl",
            "name": "your_model_name",
            "timestamp_created": "2023-12-02T22:24:09.163419+00:00",
            "origin": "user_trained",
            "url": <URL link to download your model>,
            "reg_data": null
        }
    ],
    "instance_prompt": "photo of skw woman",
    "sample_generation_job": {
        "id": <your job id>,
        "model_id": "",
        "timestamp_created": "2023-12-02T22:24:51.766129+00:00",
        "gpu_seconds": 271.21044850349426,
        "status": "success",
        "type": "text-to-image",
        "error": null,
        "tokens": 5,
        "webhook_url": null,
        "sample_images": [
          <URL links to your AI generated headshots>
        ],
        "prompt": "photo of skw woman, professional portrait, neutral background",
        "negative_prompt": "",
        "num_generations": 5,
        "width": 1024,
        "height": 1024,
        "steps": 30,
        "sampler_name": "DPM++ 2M SDE Karras",
        "seed": -1,
        "cfg_scale": 7.0
    }
}

Once the status of both the training and sample generation jobs are set to “success”, you can download the results from the URLs in the “sample_image” list. You can also download your trained model using the URL in the “trained_models” list.

Get started: https://www.lightsketch.ai/

API documentation: https://docs.lightsketch.ai/