Streamline Avatar Creation with Meshcapade Me API: From One Image to an Accurate Avatar in Seconds

Avatar from a single image

The Meshcapade Me API is a RESTful API that allows you to create and edit avatars from images, measurements, scans, and more. All API replies adhere to the JSON:API schema guidelines. You can find detailed documentation for each endpoint here.

Step by step we will create an Avatar from a single image

Meshcapade Me Account

First we create an account on me.meshcapade.com. To make the authentication token retrieval a bit easier, sign up with an email adress instead of the SSO options.

Sign up

Create a new account

New Avatar

Avatars are the central piece in the API. Almost every operation is focused on interacting with an avatar in one way or another. To create an avatar from an image, we first have to get an access token with our new account.

RESP=$(curl --location 'https://auth.meshcapade.com/realms/meshcapade-me/protocol/openid-connect/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'client_id=meshcapade-me-staging' \
--data-urlencode 'username=<email>' \
--data-urlencode 'password=<password>')

The reply looks like this:

{
    "access_token": "<token>",
    "expires_in": 36000,
    "refresh_expires_in": 36000,
    "refresh_token": "<refresh_token>",
    "token_type": "Bearer",
    "not-before-policy": 123123123,
    "session_state": "...",
    "scope": "email profile"
}

With jq we can read out the acces token like this:

TOKEN=$(echo $RESP | jq ".access_token" -r)

Next, we can initiate the avatar from images creation. This request will prepare an empty avatar which will wait for us to upload an image before processing it.

Tip: You can upload up to 4 images from different angles to improve accuracy of the body.

curl --location --request POST 'api.meshcapade.com/api/v1/avatars/create/from-images' \
--header "Authorization: Bearer $TOKEN"

The reply contains a single asset of the type AVATAR in JSON:API spec. An avatar is described by its id and a bunch of attributes. This empty avatar is not too interesting yet. Some API responses may contain a collection of assets in the data field.

{
    "data": {
        "type": "asset",
        "id": "e40ba1bd-2675-48ba-bda4-8b714cd3e4a7",
        "attributes": {
            "metadata": {
                "bodyShape": {
                    "shapeParameters": null,
                    "modelVersion": "SMPLX",
                }
            },
            "origin": "",
            "owner": "<your_user_id>",
            "source": "GENERATED",
            "source_build_id": "<some_id>",
            "state": "PENDING_AFI",
            "type": "AVATAR"
        },
        "links": {
            "image_upload": "https://api.meshcapade.com/api/v1/avatars/e40ba1bd-2675-48ba-bda4-8b714cd3e4a7/images",
            "self": "https://api.meshcapade.com/api/v1/avatars/e40ba1bd-2675-48ba-bda4-8b714cd3e4a7"
        }
    }
}

In the data.links.image_upload field, we already get an url pointing us to an endpoint outlining how to upload our image. Further described here.

Calling this endpoint informs the API that we have an intent to upload a file, and returns a presigned PUT url where we can upload the file to.

curl --location --request POST 'api.meshcapade.com/api/v1/avatars/e40ba1bd-2675-48ba-bda4-8b714cd3e4a7/images' \
--header "Authorization: Bearer $TOKEN"

The reply contains another asset, this time of type IMAGE and looks like this:

{
    "data": {
        "type": "asset",
        "id": "5ac5991f-dd79-46f0-8de6-235475361363",
        "attributes": {
            <attributes>
            "url": {
                "Link": "<presigned_PUT_url>",
                "Method": "PUT",
                "Internal": false
            }
        },
        "links": {
            "self": "https://api.meshcapade.com/api/v1/images/5ac5991f-dd79-46f0-8de6-235475361363",
            "upload": "<presigned_PUT_url>"
        }
    }
}

We can upload the image using the data.links.upload url. This will automatically link the image to the empty avatar.

The uploaded image should be a full body shot with as much contrast as possible between the body itself and the background.

Ashley ballerina

UPLOADURL=$(echo $RESP | jq ".data.links.upload" -r)

curl --upload-file "ballerina_ashley.png" \
      --header "Content-Type: image/png" \
      "${UPLOADURL}"

Start the creation process

After having uploaded the image, we can start the fitting process. We need a couple more parameters. While most are optional, the more we use, the higher the accuracy of the resulting avatar.

Request body can include these parameters:

{
    "avatarname": "Ashley", <- Optional
    "height":180, <- Optional
    "weight":65, <- Optional
    "gender":"female" <-  female/male/neutral,
    "imageMode": "BEDLAM_CLIFF" <- BEDLAM_CLIFF/AFI
}

About the imageMode

Instant mode (BEDLAM_CLIFF)

Use photos of people in dynamic poses and outfits. Results will focus more on poses than body shape.

Classic mode (AFI)

Ensure your input photo features tight-fitting clothing and a relaxed pose to achieve the best results.

What is the difference?

Running the request:

curl --location "api.meshcapade.com/api/v1/avatars/e40ba1bd-2675-48ba-bda4-8b714cd3e4a7/fit-to-images" \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/json" \
--data '{
    "avatarname": "Ashley",
    "height":180,
    "weight":65,
    "gender":"female"
}'

The reply will contain the same avatar asset as earlier, with the only difference being the state now in PROCESSING

{
    "data": {
        "type": "asset",
        "id": "e40ba1bd-2675-48ba-bda4-8b714cd3e4a7",
        "attributes": {
            "metadata": {
                "bodyShape": {
                    "shapeParameters": null,
                    "modelVersion": "SMPLX",
                }
            },
            "origin": "",
            "owner": "<your_user_id>",
            "source": "GENERATED",
            "source_build_id": "<some_id>",
            "state": "PROCESSING",
            "type": "AVATAR"
        },
        "links": {
            "image_upload": "https://api.meshcapade.com/api/v1/avatars/e40ba1bd-2675-48ba-bda4-8b714cd3e4a7/images",
            "self": "https://api.meshcapade.com/api/v1/avatars/e40ba1bd-2675-48ba-bda4-8b714cd3e4a7"
        }
    }
}

Now, we keep on requesting the same endpoint of data.links.self from the reply until the state is READY, which should roughly take a minute.
This should point at GET /assets/{assetID}. Then, you’ll also find big blob of floats in data.attributes.metadata.bodyShape.shapeParameters . Those are the shape parameters of the underlying SMPL model.

You can always take a look at your avatars in your vault

Or request all your owned avatars

Download your avatar!

Finally we can download your avatar. This requires a simple export job. We can choose between a couple of poses, namely T , A and the same pose as in the image SCAN. At this time only obj export is supported, but a fully rigged fbx download will soon be added as well!

Edit: fbx is now available too. Just change format:"obj" to format:"fbx" . Ceck out the documentation if you want to have an animation applied to your rigged avatar

Export request:

curl --location 'api.meshcapade.com/api/v1/avatars/e40ba1bd-2675-48ba-bda4-8b714cd3e4a7/export' \
--header "Authorization: Bearer $TOKEN" \
--header 'Content-Type: application/json' \
--data '{
    "pose":"scan",
    "format":"obj"
}'

Similar to how we started the fitting process from earlier, we can keep on requesting the same endpoint with the same data body, until the returned mesh asset has aREADY state. Then, the url attribute of the asset will contain a direct download link to the resulting obj 🎉

Avatar imported in blender

Congratulations, you have created your own avatar from a single image!

Follow Meshcapade on twitter, contact us here or write an email at support@meshcapade.com