Challenges on Build vs. Manage vs. Use

There are many technical and business challenges when building and managing a data warehouse. Below we address in a more conceptual way some of the challenges we usually see inside corporations (of all sizes).

At SlicingDice we divide our potential customers for the data warehouse and analytics database into four main market segments:

• Companies that prefer (or like) to build their own solutions;
• Companies that choose (or like) to manage 3rd party solutions;
• Companies that have to build their own solutions (core business);
• Companies that want to use 3rd party solutions that simply works;

Excluding companies that have their core business around data collection and processing, our daily interactions with the market indicates that the majority of the companies are still stuck on the first two segments, although the fourth group is growing really fast.

We noticed that most of the companies that are still on the first two segments are there because they were so focused on their day to day tasks that they never stopped to review this paradigm and to make this question: “do we really want to spend our company resources and our team’s time building, managing or tuning a data warehouse? Is this our core business?”

Once they ask this question, most of them realizes that, unless you are a very large corporation like Twitter or Google with a lot of resources or you have a very specific necessity/use case (core business), nowadays it doesn’t make so much sense to reinvent the wheel inside each company.

There are plenty of data warehouse as a service platforms like SlicingDice and Google BigQuery where you simply need to send your data and later make your queries, very fast and in a cost-effective way, so why build everything internally or even manage servers like Amazon Redshift?

What do you expect for your company / team future?

As the recent study from 451 Research on machine data analytics shows:

“When asked what challenges they have faced with achieving hoped-for results with analytics projects, respondents cited infrastructure requirements (cited by 36 percent), scaling challenges (cited by 33 percent), staffing requirements (cited by 33 percent), slow analytics (cited by 32 percent), expense (cited by 31 percent) and technical challenges (cited by 30 percent).”
Source: Information Management

On top that, the IT talent shortage is a global reality and will continue to be, so specialized professionals that can build or even manage a data warehouse will become increasingly difficult to find, hire and retain.

What are the companies from the fourth segment doing?

They are the ones that “want to use 3rd party solutions that simply works”, so they can focus on their core business.

Their movement are totally aligned to Gartner’s vision published on their latest Magic Quadrant for Data Management Solutions for Analytics (February 2017):

“More organizations are considering cloud IaaS or PaaS as the means of deploying their analytical environments. […] Cloud solutions are also challenging the traditional positioning of appliances, which is causing further disruption for traditional data warehouse vendors.”

What the future looks like for companies of the first two segments?

From a technical standpoint, we believe that platforms like SlicingDice and BigQuery will continue to become more flexible in terms of data types and analysis it can support, while still getting faster, easier and cheaper.

As stated before, considering that the IT talent shortage is a global reality and will continue to be, from a market and business standpoint we believe companies from all sizes will continue to “outsource” their data warehouse operation to specialized providers like SlicingDice. That way their teams can focus entirely on extracting value and insights out of the data, instead of spending time building or managing a data warehouse technology.

Not sure if you should Build, Manage or simply Use a data warehouse?

Click here and schedule a 15-minute talk with our developers, so we can evaluate your case together or let’s chat online now.

Why SlicingDice?

Developers want to store and query their data, but they hate having to manage complex data infrastructure, so we do it for them.
All they have to do is simply store and query billions of real-time and historical data, without managing any infrastructure.

Store your data and make fast queries using our simple API or any of our clients available in Python, Ruby, JavaScript, Go, Java, .NET/C#, PHP, and Arduino.

High availability?
You don’t need to worry. We always replicate and backup your data on three different datacenters, located in France, Canada and Germany.

Query speed?
If any query takes longer than 10 seconds to complete, no matter its complexity or data size, you receive a 10¢ discount on your account for each one. We believe ensuring fast queries is our responsibility, not yours.