Solutions to AWS Glue Errors
Errors while running Glue job, crawler, or connection
Glue is a managed and serverless ETL offering from AWS. Many a time while setting up Glue jobs, crawler, or connections you will encounter unknown errors that are hard to find on the internet. Here in this blog, I have captured 5 different error scenarios. The solutions to these errors are taken from the AWS support chat that I had used to solve these issues.
🔴Scenario 1: While running Glue Crawler
Error: Insufficient Lake Formation permission(s) on <database_name> (Database name: <poc>) (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: <request_id>). For more information, see Setting up IAM Permissions in the Developer Guide (http://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html ).
💚Solution:
Fix: In Data Lake formation under Data permission, granted super permission to the IAM role/IAM user attached with that crawler and database.
Lake Formation provides a fine-grained access model that augments IAM permissions. I can understand that sometimes it becomes a little tricky to manage. A better way to think about it is that with Lake Formation enabled, to get access to a resource we have to pass through…
