Run any binary in a container like it exists on your computer

The second most important issue Docker has solved (the first one was a “works on my machine” problem) was the inability to run many binaries/libraries/packages of different versions side by side. Containers come with…

Error handling in AWS Lambda

AWS Lambda is a service that lets you run code without provisioning or managing servers. That presents a great step forward in evolution of applications. But it is still a very young technology with immature environment and you have to deal with a bunch of basic problems…

Using Amazon Redshift to Analyze S3 Access Logs

We have recently encountered a higher number of the following S3 errors when uploading small files (~10MB) to S3:

<?xml version=”1.0" encoding=”UTF-8"?>
  <Error>
  <Code>RequestTimeout</Code>
  <Message>Your socket…

Querying JSON Ad-hoc in Snowflake

Sometimes I end up with a serialized JSON stored in a relational database (or in a CSV file). It often contains data structures that are not easy to map to a relational/flat structure and their structure is not fixed by any means. Ad hoc querying of this sort of data is…

Life of a data extractor

No doubt, that extractors represent an important part in ETL. In Keboola Connection, the extractor is a Docker application, which helps the end-user to extract data of his choice (usually from database or API server). The developer of the extractor can use any programming language…

Parsing JSON in Papertrail

From a bookmarklet to Chrome Web Store

We love logging! It helps us locate bugs, debug unexpected behaviour, monitor hacking attempts. We do not log obsessively everything, but we're receiving the dreaded “Reached $20.00 of…

How Docker in Docker solved our concurrency problem in a second