Become a member
Sign in
Get started
Anuvrat Singh
Anuvrat Singh

Anuvrat Singh

Follow

Editor of A Muggle’s Pensieve
40 Following
123 Followers
·
  • Profile
  • Claps
  • Highlights
  • Responses

Featured

Anuvrat Singh
Anuvrat Singh in A Muggle’s Pensieve
Apr 5, 2017 · 3 min read

Writing Into Dynamic Partitions Using Spark

Hive has this wonderful feature of partitioning — a way of dividing a table into related parts based on the values of certain columns. Using partitions it’s easy to query a portion of data. Hive optimizes the data load operations based on the partitions.

242
12 responses

Latest

Anuvrat Singh
Anuvrat Singh in A Muggle’s Pensieve
Jun 5, 2018 · 2 min read

Ira Spring Trail — Mason Lake

We reached the trail-head by 9:30 on a sunny Saturday morning early June to find the parking completely…

8
Anuvrat Singh
Anuvrat Singh in A Muggle’s Pensieve
Apr 29, 2017 · 2 min read

Reuse Hive Scripts

Amazon’s Elastic Data Pipeline does a fine job of scheduling data processing activities. It spawns a cluster and executes Hive script when the data becomes available. And after all the jobs have completed the pipeline shuts down the EMR resource and exits. Since the cluster is only created and…

2
Anuvrat Singh
Anuvrat Singh in A Muggle’s Pensieve
Apr 28, 2017 · 2 min read

Hive + DynamoDb: Always Specify Region

DynamoDb is a key-value storage store. One can query DynamoDb tables from Hive using the…

2
1 response