Unit testing for AWS AppSync resolver mapping templates

Photo by Matt Artz on Unsplash

My team at Tech@ProSiebenSat.1 has been hosting our GraphQL APIs on AWS AppSync since early 2019. While we are reasonably happy with the service, one thing that really keeps bothering us, is the resolver mapping templates. Those mapping templates are used by GraphQL to implement the actual logic behind a query or mutation. In AppSync, the developer would for example use a mapping template to translate a GraphQL query to a DynamoDB query to return the requested object.

A GraphQL query like this:

{
    "query": "query video($id:String!){getVideo(id:$id){title}}"
    "variables": {"id": "foobar"}
}

could be translated to a DynamoDB query like this:

{
    "version": "2017-02-28",
    "operation": "Query",
    "query": {
        "expression": "#partition_id = :content_id AND #content_id = :content_id",
        "expressionNames": {
             "#partition_id": "partition_id",
             "#content_id": "content_id"
        },
        "expressionValues": {
          ":content_id": $util.dynamodb.toDynamoDBJson($context.arguments.id)
        }
    }
}

To do so, AppSync uses the Apache VTL (short for Velocity Template Language). Problem with that is: developer experience for VTL is rather poor in the IDEs we use (we use mostly PyCharm). The AppSync team is probably aware of that as they offer some kind of VTL IDE in the AppSync console that can be used for writing and testing the templates. But if we are honest, nobody likes switching tools while being in the tunnel. And copying code back and forth from browser to local IDE is tedious.

Our test setup until so far

The VTL templates are actually very predestined for TDD as you (at least us) usually know very well in advance, what the evaluated template result should look like. Also, we very know very well what the incoming data looks like. So writing a unit test with a given input and an expected output is rather easy. The tricky part in running those unit tests is getting VTL templates evaluated locally.

As mentioned above, we use PyCharm as our IDE. So you can guess we write our code in Python. VTL hardly exists for Python though. And to make things more complicated, the AppSync team has a broad range of VTL utils that are (or rather were) not available anywhere else but in the AppSync console.

So what we did was leveraging a package called airspeed which at least does support some of the VTL features and extended it with some AppSync specific features (we ended up calling it internally airspeed, unfortunately never made it open source though).

So to simple test for the mapping expected result mentioned above could look like so:

def test_get_video():
    # arrange
    content_id = str(uuid4())
    arguments = {"content_id": content_id}
    context = {"arguments": arguments
              "info": {"fieldName": "getVideo"}}
    namespace = {"context": context}

    expected_result = {
    "version": "2017-02-28",
    "operation": "Query",
    "query": {
        "expression": "#partition_id = :content_id AND #content_id = :content_id",
        "expressionNames": {
             "#partition_id": "partition_id",
             "#content_id": "content_id"
        },
        "expressionValues": {
          ":content_id": content_id
        }
        }
    }
    
    # act
    template_output = parse_appsync_template(GETVIDEO_TEMPLATE_PATH, namespace)

    # assert
    assert json.loads(template_output) == expected_result

This works reasonably well, until you either start using slightly elaborate VTL features (like maps) that are not yet supported by airspeed or features that AWS solely provides in AppSync (like all those nice $utilthings).

AWS SDK to the rescue

As we were about to start working on a new API, we were really relieved and thankful when AWS announced an SDK command to evaluate mapping templates from code with all batteries included.

Your preferred AWS SDK (ours would be boto3) now offers a method named
boto3.client("appsync").evaluate_mapping_template() that you can use to send your template and context data over to AWS, let them evaluate your template and eventually returned the mapping result.

Given a simple mapping template like

hello ${context.identity.username}

Your test would look like so now:

def test_hello_world():
    """ 
    when template is called with a name in args
    then mapping template greets user
    """
    # Arrange
    appsync_client = boto3.client("appsync")

    username = "fabian"

    with open(os.path.join("mapping_templates", "hello_world.vtl"), "r") as f:
        template = f.read()

    context = {
        "arguments": {},
        "source": {},
        "result": {},
        "identity": {
            "username": username,
        },
        "request": {}
    }

    # Act
    evaluated_template = appsync_client.evaluate_mapping_template(
        template=template,
        context=json.dumps(context)
    )

    # Assert
    assert evaluated_template["evaluationResult"] == f"hello {username}"

The good thing about this is, that you can now use all the fance utils like $util.Ksui()to automatically generate IDs or $util.error(String) to generate a completely different response:

def test_error():
    """ 
    when template raises an error, 
    then evaluation result should be none 
    and response contains an error message
    """
    error_message = "something went wrong"
    template = f'$util.error("{error_message}")'
    context = {}

    evaluated_template = appsync_client.evaluate_mapping_template(
        template=template,
        context=json.dumps(context)
    )

    assert evaluated_template == {
        "error": {"message": error_message}
    }

Caveat

While we will for sure use this a lot in our upcoming project, I see a major caveat in the solution that the AppSync team provided: it requires an active user session. Especially in the context of unit tests, we try to make tests as self-sufficient as we can. That means, our unit tests usually do not depend on external resources. We mock everything we can. Having to use an active session with AWS has a major downside: our CI pipeline job must be authorized with AWS to run those tests. Up until now we explicitly did NOT grant this CI job any permission on AWS to avoid accidental messing with actual data or infrastructure. A workaround we will probably implement is to run those particular tests in a dedicated job which has the necessary permissions while all the other unit tests still run in their unauthorized environments.