Using AWS SAM to Develop Snowflake External Functions
Previously, I wrote a blog post on the workflow of deploying Snowflake external functions in AWS. Developing a cloud formation script could aid in the deployment process. Snowflake has provided cloud formation templates in this repo: sfguide-external-functions-examples. This template offers a good starting point. If you want to adopt this, I would recommend understanding it and making appropriate changes.
The AWS SAM Approach
The recommended AWS approach for developing serverless functionality is to use the AWS SAM/Serverless Application Model. The templated examples provided by Snowflake, however, would not work well if your team uses AWS SAM.
In order to provide the Snowflake community with another option, I am sharing an example SAM template codebase which will do the following:
- Create roles used by Lambda with the ‘AWSLambdaBasicExecutionRole’ privilege
- Create a Lambda function & API Gateway
- Configure the API Gateway with AWS_IAM as the authorizer
- Define resource policies
- Create a role, which will be assumed by Snowflake, with base trust defined
The aim is in no way to override Snowflake-provided templates. This is shared in the hope that it will reduce some work effort in figuring out how to develop the cloud formation for SAM.
Keep in Mind
- If you decide to use this template, please take the time to make appropriate changes to things like names as per your needs.
- It is assumed that you know and are experienced with CloudFormation templates and AWS SAM, etc.
- The deployment walk-thru demonstrated here, is not tested against Snowflake connected via private link or private VPC. So do expect some level of effort when using this example template.
Repo
The code sample is shared in the GitLab Repo: AWSSAM-based-SnowflakeExternalFn.
Head over to the documentation (Deploying Synchronous External Function using AWS SAM) for the deployment steps.
What has been done?
While the template above does not solve the interdependency steps needed between Snowflake and AWS, the main objective was to demonstrate how to adopt AWS SAM and reduce the overall effort involved.
Based on the SAM template, and with the appropriate AWS SAM permissions, I am able to perform the deployment in less than an hour.
What's next?
The current implementation was for synchronous external function invocation. In the future I plan to release an asynchronous external function implementation, using SAM following the Snowflake approach so stay tuned!
Need Help with Data Integration Into Snowflake?
Are you looking to gain a better understanding of what approaches, solutions, and tools are available in the data integration space and how to best address your specific integration requirements?
Hashmap’s Data Integration Workshop is an interactive, two-hour experience for you and your team where we will provide you with a high-value, vendor-neutral sounding board to help you accelerate your data integration decision-making process, and selection. Based on our experience, we’ll talk through best-fit options for both on-premise and cloud-based data sources and approaches to address a wide range of requirements. Sign up today for our complimentary workshop.
Other Tools and Content For You
Venkat Sekar is a Senior Architect at Hashmap, an NTT DATA Company, and provides Data, Cloud, IoT, and AI/ML solutions and expertise across industries with a group of innovative technologists and domain experts accelerating high-value business outcomes for our customers.
