Using Machine Learning to Auto Detect Column Types in Customer Files

Introduction

Tracking MapReduce job performance with counters

At LiveRamp, most of our heavy data processing is done by MapReduce jobs on our Hadoop YARN cluster. Since these jobs are critical to our data processing workflows, one of our top priorities is making sure they run quickly and reliably.

Joining Petabytes of Data Per Day: How LiveRamp Powers its Matching Product

Friday thoughts: fail, fast and furiously

tl;dr: When implementing a service or API, if you get a request you don’t quite understand, the kindest thing you can do is to return a noisy error.

Let’s consider an API like:

GET /mySum?num=3&num=42

Distributed Tracing at Massive Scale

When a developer thinks about monitoring and observability of their production application, two things generally come to mind: metrics and logs. While those are really useful for debugging and monitoring purposes, there is still a critical monitoring element that…

Migrating a Big Data Environment to the Cloud, Part 1

Migrating a Big Data Environment to the Cloud, Part 3

How do we get to the cloud?