Hive has this wonderful feature of partitioning — a way of dividing a table into related parts based on the values of certain columns. Using partitions it’s easy to query a portion of data. Hive optimizes the data load operations based on the partitions.

Darkness rises, and light to meet it.

We reached the trail-head by 9:30 on a sunny Saturday morning early June to find the parking completely…

Amazon’s Elastic Data Pipeline does a fine job of scheduling data processing activities. It spawns a cluster and executes Hive script when the data becomes available. And after all the jobs have completed the pipeline shuts down the EMR resource and exits. Since the cluster is only created and…

DynamoDb is a key-value storage store. One can query DynamoDb tables from Hive using the…

Problem