Things You Must Know When Configuring Lambda With VPC Resources Access
How to make your lambda talk to resources within VPC, AWS public cloud, and public internet
Lambda is arguably the backbone of the AWS serverless platform and has become an integral part of modern-day serverless applications in AWS.
With its event-driven nature, it provides seamless integration for a wide range of other AWS services. At the same time, lambda too wants to talk to various other services available within the AWS cloud and outside of it.
Therefore to facilitate this communication it’s worth knowing how to set up communication channels properly between lambda and all forms of other services.
Where does your lambda function run?
Lambda is indeed serverless for you. But it’s not serverless to AWS. AWS lambda creates lambda execution environments for your function with the configuration you declared (run time/memory etc.) and runs the function code inside them. Unless your lambda invocation is throttled due to concurrency limitations, AWS lambda will find an existing ready-to-use execution environment (instance of your function) or create a new one otherwise to serve your request.
Unlike EC2s, your lambda execution environments do not reside inside any of the VPCs owned by you. Instead they run inside a secured VPC called ‘Lambda Service VPC’ which is managed by AWS.
Networking capabilities inside the lambda service VPC (No-VPC mode)
In general for any AWS service to talk with another AWS service, it should satisfy the below two conditions.
- End to end network connectivity
- Sufficient IAM permissions
By default, your lambda function which is running inside the lambda service VPC has no network connectivity to your VPCs. Therefore no matter what IAM role you assigned to it, your lambda can not access any of the resources available only within your VPCs. (such as microservices running inside the VPC, RDS, Redshift, Elasticache cluster, etc.)
However, AWS lambda is making sure that the lambda service VPC has a route to the public internet. Therefore with correct permissions given it can access anything available to the public internet. It can be anything like public endpoints or services available in AWS public cloud. (like S3, DynamoDB, SNS, SQS, etc.)
Lambda’s this mode of running without attaching to any of the customer-VPC is called lambda ‘no-VPC mode’. However, certain use cases can not be served through this default lambda no-VPC mode, and in such cases, you have to attach your lambda to your VPC.
Attaching your lambda to your VPC
You can attach your lambda to your intended VPC in the lambda console by mentioning the VPC, subnets, and security groups you wish to have for it. But the important fact to remember here is that it does not relocate your lambda inside your VPC. Lambda function instances (execution environments) will continue to run inside the same lambda service VPC. But under the hood AWS lambda creates a VPC to VPC NAT (V2N) inside the lambda service VPC and connects it to an Elastic Network Interface (ENI) created inside the VPC / subnets you mentioned and apply the security groups you provided to that ENIs.
You have to explicitly grant necessary IAM permissions to lambda to manage (create/delete/autoscale etc.) these ENIs as they are created inside a VPC owned by you. That can be done by attaching the below permissions to your lambda function execution role.
- ec2:CreateNetworkInterface
- ec2:DescribeNetworkInterfaces
- ec2:DeleteNetworkInterface
For this you can use this AWS managed policy, AWSLambdaVPCAccessExecutionRole which contains all those necessary permissions.
This ENI in a given subnet will be shared across multiple lambda instances from multiple functions as long as their respective functions are configured to have the same set of security groups. However, if the network usage of the connected instances goes beyond a certain level, lambda will create additional such ENIs to support that increased traffic passes through them.
When to attach your lambda to your VPC
Use case 1: When lambda need access to resource available only within your VPC
If your lambda wants to access resources available only within the VPC, you must attach the lambda to that VPC (or to some other VPC peered with it).
As soon as you attach your lambda to a VPC subnet, AWS lambda will create an ENI inside the subnet, and that ENI will be assigned with a private IP address from the IP address range of that subnet. Thereafter through that ENI, your lambda can access anything available within that VPC (or in its peered VPCs).
Remember, no matter what the invocation source is (Lambda SDK, SQS, S3, Cognito, etc.) or what the execution model is (synchronous, asynchronous, or poll based), all the invocations (incoming traffic) take place through lambda service APIs and no one gets direct access to the Lambda execution environment.
Every invocation is protected with IAM at the lambda service API level. This means even if you place your lambda in a public subnet unlike EC2s, no one can access it unless that invocation comes with sufficient IAM permissions.
However, the traffic generated from your lambda takes a different path. It reaches the ENI through the V2N and after that as with any other traffic that originated from the same subnet, it will pick a route depending on the routes you have configured in your subnet route table and continue.
What if your lambda attached to the VPC needs public internet access too?
It’s important to know that as soon as you attach your lambda to your VPC it’s all up to you to provide the necessary network connectivity it requires. In other words the public internet access it had when it was running in no-VPC mode will no longer be available and you must provide it through your VPC network configuration.
Therefore the correct way of providing public internet access to your lambda attached to a VPC is attaching it to a private subnet and routing that private subnet to a NAT gateway / NAT instance running inside a public subnet.
A common mistake to avoid...
Attaching your lambda to a public subnet equipped with a NAT gateway would not provide it public internet access.
Three important points to remember on this;
- In a public subnet when you have configured both a NAT gateway and a route to the VPC internet gateway (IGW), then IGW gets precedence over the NAT gateway.
- To reach the internet through IGW you must possess a public IP
- Lambda ENI does not get a public IP. It just gets a private IP
Use case 2: When your lambda wants to securely access resources available in AWS public cloud
When your lambda wants to access AWS public cloud resources that reside in the same region as your VPC, you can attach your lambda to the VPC to make use of VPC endpoints. VPC endpoint has become popular mainly for two reasons:
- VPC endpoints are cheaper than NAT gateways
VPC endpoints are always cheaper than NAT gateways. Gateway-type VPC endpoints which are used to connect to S3 and DynamoDB incur no additional cost at all.
You can not use VPC endpoints when lambda is running in no-VPC mode.
2. Data transfer is secured through VPC endpoints
With VPC endpoints traffic between your VPC and AWS services is retained within the AWS private network itself without route through the public internet.
In such a case, you can simply create a VPC endpoint specific to your destination service and add it as a route in the route table of the private subnet where the lambda ENI exists.
Use case 3: When you want the traffic originating from your lambda to have a known static public IP
With no-VPC mode, you have no control over the public IP that gets assigned to your lambda when reaching the public internet. But there can be certain use cases where for security purposes third-party APIs expect them to be invoked only with pre-communicated known public IPs. You can not achieve that with no-VPC mode.
In such cases, you can attach lambda to your VPC private subnet and route it to the public internet through a NAT gateway which is configured to have an elastic IP.
When not to attach your lambda to your VPC
However, if all that your lambda does is some internal computation work or just accessing endpoints on the public internet, then leaving it in the default no-VPC mode is the best option rather than bringing it into the VPC and providing internet access through an expensive NAT gateway.
Conclusion
VPC is your private area within the AWS cloud which you have control of it to a great extent. Bringing your lambda functions into your VPC will give you some more control over them too. How you use VPC and lambda together in each use case is directly linked to your overall system performance, high availability, security, and cost of course
Thanks for reading !!
