HTTP Direct with Datomic & Terraform

As a follow up to my “Datomic with Terraform” post and talk, I’d like to discuss how to use Datomic’s HTTP Direct feature with API Gateway. While the Cognitect tutorial can be helpful initially (I mean, hey, that’s where I started), there are a few things that weren’t explained as clearly as I would have liked. And, because the tutorial relies entirely on the AWS Console, I had to reverse-engineer some of the Console magic in order to figure out how to wire everything up using Terraform.

You can view the example code on GitHub. I’ve structured the project like my datomic-terraform-example, so if you've had a look at it, this should be familiar.

Unlike when using Lambda proxies, this is more like writing a traditional web backend. You’ll need to handle routing in your ion (rather than leaving it to API Gateway). This does mean you will lose some of the advantages of API Gateway, however, the improvement in latency is almost always worth it. I chose bidi for routing because I like that it just uses data for syntax.

(def ^:private routes
  {:get ["/"
         [["resource" resource/handler]
          [true not-found/handler]]]})

(defn handler
  [req]
  (let [{:keys [request-method uri]} req
        route (bidi/match-route (get routes request-method)
                                uri)
        handle (:handler route)]
    (handle req)))

datomic/ions/src/http_direct_example/ion/proxy.clj

Note that I’ve worked up my own quick-and-dirty handling for routing based off request method—YMMV.

(defn response
  [status body]
  (let [allow-origin (:allow-origin (ion/get-env))]
    {:status  status
     :headers {"content-type"                     "application/transit+json"
               "access-control-allow-origin"      allow-origin
               "access-control-allow-credentials" "true"}
     :body    body}))

datomic/ions/src/http_direct_example/ion/utilities.clj

Because apigw/ionize sorts out a lot, one thing that caught me off guard when transitioning from the Lambda proxy is that Datomic is rather specific with its response spec. Header keys need to be strings. I'm defaulting to lower-case because it seems most implementations run that way (the standard being case-insensitive).

Creating a VPC Link with Terraform is trivial:

resource "aws_api_gateway_vpc_link" "query_group" {
  name        = local.stack_name
  target_arns = [aws_cloudformation_stack.query_group.outputs["LoadBalancer"]]
}

datomic/main.tf

It took me a bit of experimentation to figure out how to properly proxy the path to the Datomic ion (thanks mostly to my lack of understanding of how API Gateway resource configuration works). With the API method, you have to first enable the proxy path to pass through to the integration:

resource "aws_api_gateway_method" "proxy" {
  authorization = "NONE"
  http_method   = "ANY"

  request_parameters = {
    "method.request.path.proxy" = true
  }

  resource_id = aws_api_gateway_resource.proxy.id
  rest_api_id = aws_api_gateway_rest_api.site.id
}

proxy.tf#L21

Then you have to map that path to something the integration can access:

resource "aws_api_gateway_integration" "proxy" {
  connection_id           = module.query_group.vpc_link_id
  connection_type         = "VPC_LINK"
  http_method             = "ANY"
  integration_http_method = "ANY"

  request_parameters = {
    "integration.request.path.proxy" = "method.request.path.proxy"
  }

  resource_id = aws_api_gateway_resource.proxy.id
  rest_api_id = aws_api_gateway_rest_api.site.id
  type        = "HTTP_PROXY"
  uri         = "${module.query_group.load_balancer_http_direct_endpoint}/{proxy}"
}

proxy.tf#L35

That’s pretty much it. The only other minor thing I would call attention to is that per the Datomic ion tutorial, you need to set a binary media type on API Gateway for Datomic integration (no matter whether you’re using the Lambda proxy or HTTP Direct):

resource "aws_api_gateway_rest_api" "site" {
  name               = "Site API${var.suffix == "" ? "" : " ${var.suffix}"}"
  binary_media_types = ["*/*"]
}

main.tf

In any case, I hope this will be helpful to future adventurers.

If you find my work interesting, sign up for my mailing list so you don’t miss a thing.

Originally published at https://theconsultingcto.com.