Securing front doors by signing standard requests with the AWS SDK v2
Security is not an optional feature when designing networked services, especially those accessible over the internet. It’s a given. For this reason secure services must be authenticated. There are an innumerable ways to solve this problem in the technology space. Fortunately for AWS customers, this is a wheel that needs no reinventing. There is a solution for this with AWS and it’s called sig v4 signing: a specification for authenticating a HTTP requests with IAM credentials in a way that AWS services can then validate them before providing access to your services capabilities.
Sig v4 is such a common a feature for securing your AWS infrastructure that in many cases it’s almost as easy as a one line change to your infrastructure definition to add it. Here’s an example of adding sig v4 auth protection to your AWS Lambda Function URLs in a sam template for illustration.
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Resources:
CoolApi:
Type: AWS::Serverless::Function
Properties:
# 👇 give your function a Http API URL
FunctionUrlConfig:
# 👇 yes, it's that easy
AuthType: AWS_IAM
# ... the restAs AWS customers, Meetup uses sig v4 auth to secure a number of our own services. This implies we need clients which are able to sign requests to make any use of them. In this post I’d like to share our recipe to doing so by using what you likely already have in your kitchen cabinets: the AWS SDK that’s likely already on your class path as well as the Java standard library HttpClient which requires no additional external dependencies.
Below is a full example of a utility, sans javadocs and wildcarded imports for brevity which fits neatly into a single file which allows you to securely sign std lib HTTP requests used with the std lib HttpClient using AWS’s Aws4Signer which uses it’s own representation of HTTP requests.
package com.meetup.sigv4.jdk;
import static java.net.http.HttpRequest.BodyPublishers;
import static java.util.concurrent.CompletableFuture.completedFuture;
import java.io.ByteArrayInputStream;
import java.net.http.HttpRequest;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.Flow;
import software.amazon.awssdk.auth.credentials.*;
import software.amazon.awssdk.auth.signer.Aws4Signer;
import software.amazon.awssdk.auth.signer.params.Aws4SignerParams;
import software.amazon.awssdk.http.*;
import software.amazon.awssdk.regions.Region;
public class Sigv4Signer {
private static final Set<String> RESTRICTED_HEADERS = Set.of("Content-Length", "Host", "Expect");
private final AwsCredentialsProvider credentialsProvider;
private final Aws4Signer signer;
private final String serviceName;
private final Region region;
public Sigv4Signer() {
this(DefaultCredentialsProvider.create());
}
public Sigv4Signer(AwsCredentialsProvider credentialsProvider) {
this(credentialsProvider, "execute-api", "us-east-1");
}
public Sigv4Signer(String serviceName, String region) {
this(DefaultCredentialsProvider.create(), serviceName, region);
}
// 👇 while it's common to sign requests for API Gateway in us-east-1 at Meetup,
// you can technically sign requets for any service in any region
public Sigv4Signer(
AwsCredentialsProvider credentialsProvider, String serviceName, String region) {
this.credentialsProvider = credentialsProvider;
this.signer = Aws4Signer.create();
this.serviceName = serviceName;
this.region = Region.of(region);
}
// 👇 given any std lib HttpRequest, return a sigv4 signed copy
public CompletableFuture<HttpRequest> signAsync(HttpRequest unsigned) {
return unsigned
.bodyPublisher()
.map(
publisher -> {
// 👇 collect the std lib request body into bytes
var collector = new FlowCollector<ByteBuffer>();
publisher.subscribe(collector);
return collector.items().thenApply(items -> items.get(0).array());
})
.orElseGet(() -> completedFuture(null))
.thenApply(
bytes -> {
// 👇 create an aws request based on std lib request
var scratch =
SdkHttpFullRequest.builder()
.method(SdkHttpMethod.valueOf(unsigned.method()))
.headers(unsigned.headers().map())
.uri(unsigned.uri());
var body =
bytes == null ? BodyPublishers.noBody() : BodyPublishers.ofByteArray(bytes);
if (bytes != null) {
scratch.contentStreamProvider(() -> new ByteArrayInputStream(bytes));
scratch.putHeader("Content-Length", String.valueOf(bytes.length));
}
// 👇 sign and extract signature headers
var signedHeaders =
signer
.sign(
scratch.build(),
Aws4SignerParams.builder()
.signingName(serviceName)
.signingRegion(region)
.awsCredentials(credentialsProvider.resolveCredentials())
.build())
.headers()
.entrySet()
.stream()
.filter(entry -> !RESTRICTED_HEADERS.contains(entry.getKey()))
.toList();
// 👇 return a copy of std lib request with signed headers
var builder =
HttpRequest.newBuilder(unsigned, (k, v) -> true).method(unsigned.method(), body);
signedHeaders.forEach(
entry -> builder.setHeader(entry.getKey(), String.join(",", entry.getValue())));
return builder.build();
});
}
// 👇 for clients that truly need to block
public HttpRequest sign(HttpRequest httpRequest) throws InterruptedException, ExecutionException {
return signAsync(httpRequest).get();
}
// 👇 how one goes about extracting the body of a std lib request
private class FlowCollector<T> implements Flow.Subscriber<T> {
private final List<T> buffer;
private final CompletableFuture<List<T>> future;
public FlowCollector() {
this.buffer = new ArrayList<>();
this.future = new CompletableFuture<>();
}
public CompletableFuture<List<T>> items() {
return future;
}
@Override
public void onComplete() {
future.complete(buffer);
}
@Override
public void onError(Throwable throwable) {
future.completeExceptionally(throwable);
}
@Override
public void onNext(T item) {
buffer.add(item);
}
@Override
public void onSubscribe(Flow.Subscription subscription) {
subscription.request(Long.MAX_VALUE);
}
}
}The minimum dependency you’ll actually need from the AWS SDK is
software.amazon.awssdk:aws-core:{current-version-here}
A few things to note about the code above are that that AWS SDK models HTTP requests in its own fashion and types, so most of the work involved is just adapting your HTTP requests, standard library HttpRequests, to into its types and then back again. Note that the signing method return’s a CompletableFuture . This has to do with the nature if extracting the bytes from a std library HTTP Request which may not be synchronously available.
Below is an example of how we use this utility.
// 👇 typically stored references as they can be reused
var signer = new Sigv4Signer();
var client = HttpClient.newHttpClient();
// 👇 compose a request
var unsigned = HttpRequest.newBuilder(URI.create("https://cool.service.com")).build();
// 👇 sign and send request
signer.signAsync(unsigned)
.thenCompose(signed -> client.sendAsync(signed, BodyHandlers.ofString())
.whenComplete((resp, err) -> ...);That’s it! We use this recipe pervasively through out our services and clients and it has worked extremely well for us by allowing us to fully leverage what is already available effectively all the while keeping our services safe and secure.
