Are you finding it hard to synchronize processes that read and write data on a data lake?

Are your solutions overly complex?

If you answered yes to these questions, there is a simple pattern we use at dataxu that can help. It gets the work done with minimal overhead.

The Data Science Engineering team at dataxu spends most of its’ day designing, developing, and maintaining a wide range of data pipelines that support our AI-based bidding system.

A pipeline is a set of stages (or sub-processes) that process input data producing an output, in which the output of each stage…

At dataxu, we have been using Scala for a while now, and in our journey we have perfected our best practices. Among these best practices, we have refined our implementation of the Singleton design pattern in Scala. In implementing this pattern there are some aspects that require significant attention to detail to increase its ease of use, especially when testing.

The Singleton design pattern, according to Wikipedia, is a software design pattern that restricts the instantiation of a class to one object. This means that this pattern promises to be handy when dealing with a class that we want to…

Javier Buquet

Software Engineer. Technical Lead for Data Science @ dataxu / montevideo labs - Scala, Python, Spark, Big Data, Machine Learning and more

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store