Anuvrat Singh
Anuvrat Singh

Anuvrat Singh

Medium member since November 2019
·
Editor of A Muggle’s Pensieve
47 Following
159 Followers
·

FEATURED STORY

Writing Into Dynamic Partitions Using Spark

Hive has this wonderful feature of partitioning — a way of dividing a table into related parts based on the values of certain columns…

Apr 5, 2017 · 4 min read
Apr 5, 2017 · 4 min read

Rise of Skywalker: Balance Restored?

Darkness rises, and light to meet it.

Dec 25, 2019 · 2 min read

Ira Spring Trail — Mason Lake

We reached the trail-head by 9:30 on a sunny Saturday morning early June to find the parking completely filled up and had to park the car…

Jun 5, 2018 · 2 min read

Reuse Hive Scripts

Amazon’s Elastic Data Pipeline does a fine job of scheduling data processing activities. It spawns a cluster and executes Hive script when…

Apr 29, 2017 · 3 min read

Hive + DynamoDb: Always Specify Region

DynamoDb is a key-value storage store. One can query DynamoDb tables from Hive using the DynamoDBStorageHandler. It’s super easy to setup.

Apr 28, 2017 · 2 min read

SPOJ: NICEDAY — The Day of the Competitors

Problem

Apr 27, 2017 · 6 min read

Hive — Partition Column Name — CamelCase — Bad Idea

[ This was originally posted on my blog on July 3, 2015 ]

Apr 11, 2017 · 2 min read

Scrapy: Crawl WhoScored For Football Stats

I have written code earlier to crawl Google Play, iTunes AppStore and Goal.com websites. But every time I re-wrote the code to get content…

Apr 4, 2017 · 8 min read

Project Euler : The Millionth Lexicographic Permutation Of The Digits

The 24th problem of Project Euler wanted the one-millionth lexicographic permutation of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9. If all…

Apr 4, 2017 · 2 min read

Parse Json in Hive Using Hive JSON Serde

In an earlier post I wrote a custom UDF to read JSON into my table. Since then I have also learnt about and used the Hive-JSON-Serde. I…

Apr 4, 2017 · 2 min read

Writing UDF To Parse JSON In Hive

Sometimes we need to perform data transformation in ways too complicated for SQL (even with the Custom UDF’s provided by hive). Let’s take…

Apr 4, 2017 · 2 min read

Project Euler : Maximum Sum Traversing Top To Bottom In A Triangle

The 18th and the 67th problems in Project Euler are one and the same. The only difference is in the input test case. The problem 18 has a…

Apr 4, 2017 · 3 min read