This post helps you in adding redshift connector to presto so that you can directly access redshift using Presto. Majority of the data resides on Redshift and our end users can only access Presto. So, tried on adding redshift connector to presto. By following below steps: 1. Add redshift properties under /etc/presto/conf/catalog/ (This will add redshift catalog) 2. Now, go to presto-CLI: to find redshift catalog as well

Follow below steps to move data from Hive to Dynamo DB. Source link: https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/EMRforDynamoDB.html Below is just a quick summary from the above documentation link. 1. Create a hive table. (Ignore this step if hive table is already created) 2. Now create another external hive table which just references to the hive table (but provides the mapping between Hive and DDB tables)

Scenario 1: To copy data from S3 to Aurora Postgres (greater than v9 or latest) How?: We can use aws_s3.table_import_from_s3 function (to migrate the data from S3 to Aurora Postgres). Steps: A sample file with columns — id, prefix, mstr_id is copied to S3. Create schema on Aurora Postgres (with the required columns). drop table core.mstr; CREATE TABLE core.mstr ( id varchar(300) NULL, prefix varchar(300)…

If we are using terraform, terraform state file is the heart of all the infrastructure that we spin up using terraform templates. There are several ways to deploy the infrastructure using terraform: 1. Using CLI (setup terraform and then run terraform commands) 2. Automated Build (terraform scripts integrated as part…

Here are the list of VS Code extensions that anyone working on cloud technologies can use to speed up their development. To download any extension, refer to the extension tab on your VS code window: As we will manage all our cloud resources using Terraform, we will start with Terraform…

Compiled instructions for beginners who wants to get started with AWS. Here are the getting started guides that I followed along to setup my new AWS account: https://aws.amazon.com/getting-started/?e=gs2020&p=console/#Get_to_Know_the_AWS_Cloud https://aws.amazon.com/getting-started/guides/setup-cdk/ https://aws.amazon.com/getting-started/?e=gs2020&p=console/#Launch_Your_First_Application Setting up AWS account:

From this post, you will learn how to use Glue to read the schema from the S3 file using its crawlers and build a common metadata store to other AWS services like Hive, Presto and Redshift Spectrum. Before Glue Data Catalog (GDC) feature, it is painful to create schemas for…