Big Data

MongoDB and PySpark 2.1.0

The common problem with using the latest release of any framework is that there are no or very few adopters, docs are not updated or point to older versions. We encountered a similar problem while integrating MongoDB driver with Apache Spark 2.X. Majority of the library docs…

Spark 2.x External Packages

The bane of using bleeding edge technology is very less or hidden information of new features in the latest version. We at Unnati use bleeding edge releases of many data science tools for various research and production systems. In this post we explain how to add external jars to…

Music and Building Software Systems

Recently, I was at a data science conference, day 1 was concluded with wonderful djembe (an African percussion instrument) workshop.

Art is Science made clear — Wilson Mizner

Building Data Pipelines with Luigi

All data science solutions start off as a prototype. As we start proving our hypothesis, we build on top of existing prototypes, which soon bloats. The real challenge begins when we need to run this prototype in production which obviously has to be a scalable system…

Data Science and Beyond

Experts often possess more data than judgment

- Colin Powell

Big Data, Data Science, Machine Learning and Statistics are buzz words that you will keep hearing more and more in the time to come…