The Road to an Agile Data Analytics Solution

Choosing the best agile solution

We decided to look for a different solution and defined the characteristics we were looking for:

  • Handles over 3 billion events per month
  • Lets us easily share data internally and with customers
  • Connects data from multiple sources
  • Stores data for up to one month and have previous data aggregated
  • Events available in under 5 minutes for querying (useful when monitoring gradual releases)
  • Events have standard enrichment (Geolocation and Device) and user session information
  • We own the data
  • Easy for our team to implement
  • Affordable pricing
  • Data warehouse solutions
  • Analysis & presentation solutions
  1. You can have different presentation tools for different roles in your organization that connect to the same data warehouse.

Data pipeline solutions

Data pipeline solutions move and process data from different sources to your data warehouse. Here are some of the solutions we looked at:

  • Based on specific DB schema but can be customized
  • Data stored in AWS S3 and can be replayed. No pause/play support
  • Code engine supports real-time calculations, alerts and data enrichment.
  • Available client SDKs
  • Realtime loading of data into Elasticsearch, 15–60 minute loading into Redshift
  • Available client SDKs
  • Data stored over Elasticsearch, Redshift and AWS S3
  • JS SDK or REST API endpoint for sending data with no data enrichment support

Data warehouse solutions

We were already using Redshift for our data warehouse solution. Another option was Google’s BigQuery, which might have lower maintenance requirements, but we decided to stick with RedShift to save money and resources.

Analysis & presentation solutions

We looked at the following solutions:

  • Easy to share data and supports embedded views
  • Price is calculated by the amount of connected data
  • Sharing views (not reports) requires the viewer to have an online user license
  • Priced per user (Desktop & Online)
  • Relatively high price tag
  • No embedded views

Decision

We decided to base our system on Alooma’s data pipeline, Redshift’s data warehouse, and Periscope Data’s presentation solution with the option of adding some Tableau seats for our data analysts.

Wibbitz Research & Development

Behind the Wibbitz video automation technology is a talented team of engineers focused on overcoming challenges and building a platform that is truly the first of its kind.

Uri Meirav

Written by

CTO @ Wibbitz

Wibbitz Research & Development

Behind the Wibbitz video automation technology is a talented team of engineers focused on overcoming challenges and building a platform that is truly the first of its kind.