Controlling and Manipulating AWS API Gateway Request Parameters
Extracting more from the integration between API Gateway and Lambda service
This blog post continues my previous one about building a .NET Core Lambda function. So, after building a Lambda function, which is invoked by API Gateway, let’s explore how to intervene in the parameters mechanism before and after connecting to a Lambda function.
Incentives
In short, this post focuses on the entry point of an API Gateway resource, mainly on the incoming parameters’ route.
You might be thinking what the benefit of this intervention is? Why do I need to intercept in this mechanism? Well, I can think of a few reasons:
- Filtering variables at the API Gateway level to avoid passing them to the Lambda function.
- Decoupling between the parameters that are sent to the API Gateway and the ones received by the Lambda function; changing the name of a parameter or its format allows creating a facade, which can differ from the calling service. The same applies to the response parameter.
- Injecting additional parameters into the Lambda function, such as stage variables or generic API Gateway variables.
- Validating parameters before invoking the Lambda function can prevent unnecessary calls, which alleviates the load and can reduce cost.
- Ensuring the parameters’ format: converting string to an integer, escaping string.
Intro: Reviewing the API Gateway steps
There are four steps in processing the API Gateway request and response. Two steps before calling the Lambda function and two steps afterwards. Prior to calling the Lambda function, we can intervene and alter the input.
Request Method Step
The Request Method step allows configuring security settings (Authorization, validation, API key), as well as query string parameters and request headers. Defining the query string and header parameters enables their validation and caching.
Integration Request Step
This is the steps in which things are happening. The integration definitions of the API Gateway with other AWS resources are defined here. Besides integrating with AWS Lambda, which is described in this blog, the API Gateway can integrate with HTTP URL or other AWS services.
In this step, you can define a mapping template, which is described later on in details.
Part 1: Input Manipulations
After integrating the API Gateway with a Lambda function, by default, the request is delivered as-is. If you want to intervene and manipulate the input, you need to change the configuration of the Integration Request and uncheck the option “Use Lambda Proxy integration”.
After unchecking this checkbox, the option to alter the input parameters appears below. Look for the Mapping Templates area. This is the place you can manipulate the parameters that flow into the Lambda functions.
If you keep the mapping template empty, then no parameter will flow into the Lambda function. Hence, it’s essential to fill it with valid mapping. Let’s see how to configure a simple mapping template, JSON based, that copies the query parameters and their values into the input for the Lambda function.
{
"queryStringParameters": {#foreach($key in $input.params().querystring.keySet())#if($foreach.index > 0),#end"$key":"$input.params().querystring.get($key)"#end}
}The above example is looping the query string parameters and build them into a valid JSON format. Eventually, it should look like that:
You can opt to manipulate the input and create another format or convention. This capability allows standardisation of the Lambda function’s input, regardless of the parameters that end-clients sends.
Let’s see another example. The script below takes the query string parameter named “param” and assigns it into a parameter called “var1”. By that, we can implement renaming of parameters.
{
"queryStringParameters": { "var1":"$input.params('param')" }
}# Equivalent to using the explicit call to querystring:{
"queryStringParameters": { "var1":"$input.params().querystring.get('param')" }
}
Note that if you do not surround the input with quotation marks, then the input will be interpreted as a number. This feature can be useful as a validation to the input.
This intermediate decoupling step gives a level of flexibility in designing a Lambda function and API Gateway. It also can be useful for ignoring parameters, renaming parameters, or omitting parameters and passing only on the relevant ones.
Using Stage Variables
Stage variables are defined on the Stage level, and they are not related to any input, unlike query string parameters, which are temporary for every query. These variables are set on the stage level and common to all APIs under the same stage.
You can access the Stage variables from the main stage editor:
These variables can be permeated into the Lambda function from within the Lambda function (in C#, access is via the IDictionary APIGatewayProxyRequest.StageVariables) or by passing them as parameters. In the example below, a stage variable is assigned to “var1”, which will be consumed as a query string parameter:
{
"queryStringParameters":
{"var1": "$stageVariables.get('TestStageVar1')"}
}Using Path Parameters
In short, path parameters are parameters that are embedded in the path itself, as opposed to query string parameters, which are not an inherent part of the path.
Path parameters are defined as a suffix for the path, or can be chained to prior path parameter, for example:
# A single path parameterhttps://xxx.execute-api.us-west-2.amazonaws.com/test/{path-parameter}# An example for chained path parametershttps://xxx.execute-api.us-west-2.amazonaws.com/test/{path-parameter-1}/{path-parameter-2}
You can create a path parameter as an API Gateway resource. A path parameter should be the last value on the path:
Path parameters can be converted into query string parameters, by a simple manipulation that fetches the Path parameter and plants it in the query string input:
{
"queryStringParameters":
{"var1": "$input.params().path.get('name')"}
}Things to Note 1: Deploying the changes
Any change in the request or response configuration must be deployed to be effective. Therefore, if you have done changes but they’re not working, try to remember when was the last time you have deployed the resource.
Things to Note 2: Input String Format
In some cases, you need to keep the query string holder to queryStringParameters, since your Lambda function expects this format. For example, if you’re working with the C# Lambda objects ( APIGatewayProxyRequest), you need to keep the expected JSON format for a request. Otherwise, the C# object will not be parsed and built correctly.
However, if you choose to work with a native stream as the input for the Lambda function, you can change the JSON structure as you wish since the stream will be parsed in the Lambda function, based on your desired structure. You can find more information about Lambda function’s input at the following article.
Part 2: Enforcing Validation
Using request validation is useful to offload your Lambda function. Invalid requests can be blocked at the API Gateway level before invoking the Lambda function, and thus can save cost since it prevents unnecessary calls.
The validation is on the method level (GET, PUT, POST, etc.). The primary purpose of validation is to ensure the expected parameters are passed to the API Gateway. The validation can include the header, the query string parameters or request body.
The decision of whether to enforce this validation or not is defined in the “Request Validator” field:
Body Validation
Body validation allows imposing the body format, parameters and their values. A simple example can be found here. In short, you can set mandatory parameters or minimum/maximum. For example, the model below is composed of 2 parameters: UserName and Age, while UserName is a mandatory string and Age is an integer that must be greater than 10. The JSON format to define this model looks like:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"properties": {
"UserName": {
"type": "string"
},
"Age": {
"description": "Age in years",
"type": "integer",
"minimum": 10
}
},
"required": ["UserName"]
}After determining the model (under Models menu-item), associating it with an existing resource is quite simple:
Not all methods accept Body, for example, the GET request doesn’t, and thus you should assign a Body validation at the relevant methods only.
In case the body format is not as expected, the result would be error 400 “Invalid request body”. The API Gateway provides a detailed message about the invalidity, for example: object has missing required propoerties ([UserName]) or numeric instance is lower then the required minimum .
You can find more details about Models creation and its format in the following link.
Header and Query String Validation
Validating the header or query string parameters is more straight forward than body format validation. The parameters can be mandatory or cached (a topic for another blog post).
Similarly to the Body validation, in case of invalid inputs, the result is an error 400 response that includes the details of the error:
Input Manipulation Using Body and Header Inputs
Before wrapping up, do you remember the Mapping Template? Well, after grasping the concept of Body input and Header parameters, let’s use the Mapping Template and fetch these values and assign them as we wish.
Fetching and assigning Body parameters based on the model that was defined above:
#set($inputRoot = $input.path('$'))
{
"queryStringParameters":{
"name": "$inputRoot.UserName" }
}Fetching and assigning Header parameters:
{
"queryStringParameters":{
"header-value":"$input.params().header.get('headerValue1')"
}
}Fetching and assigning Query String parameters:
{
"queryStringParameters":{
"ID":"$input.params().querystring.get('id')"
}
}Summarising the Mapping Templates Format and Variables
The mapping template allows a semi-coding environment, in which you can change the format, assign variables, use conditions (for, if, etc.).
The script below exemplifies these capabilities:
## See http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html
## This template will pass through all parameters including path, querystring, header, stage variables, and context through to the integration endpoint via the body/payload#set($allParams = $input.params())
{
“body-json” : $input.json(‘$’),
“params” : {
#foreach($type in $allParams.keySet())
#set($params = $allParams.get($type))
“$type” : {
#foreach($paramName in $params.keySet())
“$paramName” : “$util.escapeJavaScript($params.get($paramName))”
#if($foreach.hasNext),#end
#end
}
#if($foreach.hasNext),#end
#end
,
#if($input.params('operator')=='%2F')"/"#{else}"$input.params('operator')"#end
},“stage-variables” : {
#foreach($key in $stageVariables.keySet())
“$key” : “$util.escapeJavaScript($stageVariables.get($key))”
#if($foreach.hasNext),#end
#end
},“context” : {
“account-id” : “$context.identity.accountId”,
“api-id” : “$context.apiId”,
“api-key” : “$context.identity.apiKey”,
“authorizer-principal-id” : “$context.authorizer.principalId”,
“caller” : “$context.identity.caller”,
“cognito-authentication-provider” : “$context.identity.cognitoAuthenticationProvider”,
“cognito-authentication-type” : “$context.identity.cognitoAuthenticationType”,
“cognito-identity-id” : “$context.identity.cognitoIdentityId”,
“cognito-identity-pool-id” : “$context.identity.cognitoIdentityPoolId”,
“http-method” : “$context.httpMethod”,
“stage” : “$context.stage”,
“source-ip” : “$context.identity.sourceIp”,
“user” : “$context.identity.user”,
“user-agent” : “$context.identity.userAgent”,
“user-arn” : “$context.identity.userArn”,
“request-id” : “$context.requestId”,
“resource-id” : “$context.resourceId”,
“resource-path” : “$context.resourcePath”
}
}
Last words
We covered on the Request part of the API Gateway. However, the Response can be altered as well using the same concepts of Mapping Template. It can be useful to alter and standardise the response format.
API Gateway is a useful tool to receive and handle incoming requests and divert them to the relevant destination. Besides controlling the request and response format and validation, other features can be useful to manage the load and the quality of service, such as caching and throttling. I encourage you to explore its additional capabilities to improve your usability of this service.
I hope you enjoyed this post. Thanks for reading!
— Lior
