Homepage
Open in app
Sign in
Get started
Oliver's Tech Blog
Trying to change the world one line of code at a time.
Follow
Transforming Spark Datasets using Scala transformation functions
Transforming Spark Datasets using Scala transformation functions
There are few times where I’ve chosen to use Spark as an ETL tool for it’s ease of use when it comes to reading/writing parquet, csv or…
Oliver Mascarenhas
Sep 25, 2019
Writing a Spark Dataframe to an Elasticsearch Index
Writing a Spark Dataframe to an Elasticsearch Index
In this post we will walk through the process of writing a Spark DataFrame to an Elasticsearch index.
Oliver Mascarenhas
Jan 6, 2019
Profiling with Java Agents — Part 2
Profiling with Java Agents — Part 2
In-case you haven’t read Part 1, you can find it here.
Oliver Mascarenhas
Dec 30, 2018
Profiling with Java Agents — Part 1: A Hello World Example
Profiling with Java Agents — Part 1: A Hello World Example
The Java Agent technology has been around since JDK 1.5. Here we will build a simple profiler which will instrument our Java program and…
Oliver Mascarenhas
Dec 30, 2018
Running Spark SQL applications with a remote Hive cluster
Running Spark SQL applications with a remote Hive cluster
The graphic above depicts a common workflow for running Spark SQL apps. The Hive metastore holds table schemas (this includes the location…
Oliver Mascarenhas
Dec 29, 2018
Detecting sensor failure using the MQTT protocol, an IoT Proof of Concept
Detecting sensor failure using the MQTT protocol, an IoT Proof of Concept
Motivation
Oliver Mascarenhas
Dec 29, 2018
Time-Series Scatter Plot of Server Requests using Python
Time-Series Scatter Plot of Server Requests using Python
In this post I will attempt to explain how I used Pandas and Matplotlib to quickly generate server requests reports on a daily basis.
Oliver Mascarenhas
Dec 29, 2018
About Oliver's Tech Blog
Latest Stories
Archive
About Medium
Terms
Privacy
Teams