Looking Forward at the Analytics Space in 2016
Yes, this is another set of predictions you probably didn’t ask for but hopefully it will shed some light in the ever evolving analytics space. I lead an analytics consulting practice for Slalom in San Francisco and I have the privilege of talking to many large & small companies. It’s my role to understand what is taking place in the analytics market but equally important is understanding where things are heading. The list below is what I expect to happen in the next year, but as most predictions go will likely be inaccurate so read at your own peril.
Data Viz & Enterprise “BI” will Quickly Follow the Web Tech & Digital Evolution
I’m using Web Tech & the Digital Design Evolution as a lens for where Data Visualization and Analytics Apps are heading. Too long have analytics applications lagged behind modern design & development paradigms; customers are now demanding their enterprise analytics apps work in a similar fashion as the websites & apps they use in their daily lives.
A lingering design principle in web based analytics solutions is designing for many user roles; this results in cumbersome user flows, and ultimately makes it challenging for users to find the most pertinent content for them. While it’s necessary to have multiple roles within an analytics application due to security implication and functionality needs, it’s more important to start simple and allow the user to discover more functionality. An example of a couple sites that do this well in my opinion are Pinterest and Tumblr. While there are social aspects to content discovery on those sites, I think analytics applications can learn from the practices they employ.
While more modern design practices are being employed at new analytics companies, many of these experiences are unfortunately similar. When you start looking at startups like DataHero (recently acquired), Chartio, or Argo in the Data Viz space you see don’t a ton of differentiation, which brings me to my next thought…
New Data Viz & Analytics Companies Need to and Will Differentiate Themselves more than “Can connect to your favorite cloud source”
This was the barometer for most new data viz & analytics startups from 2013–2015 (not including predictive, large data, or storage related startups): we are easy to use, can connect to that cloud source you like, and we don’t cost all that much. While there was and is a place in the market for these companies, this has been done at length now. Who is going to tackle the lack of collaboration in analytics today? ClearStory Data is trying, but do we yet understand how people want to collaborate with data? Who will make a mobile experience that people want to use? Vizable is interesting from Tableau, but it’s just the start.
New analytics companies started this year will hopefully move beyond the “simple interface, connect to cloud sources” paradigm of the last couple years and focus on unsolved issues in the enterprise.
Deploying Predictive Models will get Easier
A continued problem for companies that have data science teams building advanced models is the need to operationalize / deploy / productionalize. This isn’t a new problem but the proliferation of data science in the enterprise has magnified the challenge. Although we have standards like PMML and newer standards like PFA, they haven’t quite hit the mainstream.
The reality of the issue is that data scientists like to use different languages to build their models (R, Python, Java, etc) and oftentimes don’t have the desire or background to automate and deploy models, regardless of scale. A common paradigm I see at most places is that a data science team builds the models and then hands them off to a data engineering team to deploy & automate. There are few places I’ve seen this work well as the models can be complex, require validation and something get’s lost along the way in a highly iterative, discovery based environment. The Data Mining Group explains it well here. I expect a company or two to solve for this, likely leveraging these standards but lowering the barrier of learning / deploying the standards. Companies like Zementis are focused squarely in this area with a leaning towards large scale deployments and datasets.
Data Viz Programming Libraries will get Easier & Easier to Use
Libraries like D3.js continue to pick up momentum amongst analytics groups and I don’t foresee this declining. Given the prominence D3 has in the market, I expect it to get easier and easier to build leveraging these libraries. I’m also expecting more abstraction to lower the bar to building custom visualizations like C3.js or the fantastic work being done on Vega by Jeffrey Heer. (This type of web first based approach to data viz is the path forward.) It’s always exciting to see the continued evolution of the D3 library by Mike Bostock as well as projects like Block Builder by Ian Johnson.
As Analytics get “Easier to build”, Core Analyst Skills are More Paramount
It’s now easier than ever to integrate data sources, visually explore that data and run advanced analytical models on that data. With tools like Tableau, Qlik, Alteryx and Paxata the bar has been lowered. This has opened up the ability for folks to explore and analyze their data in ways never available to them. Finance, Marketing, HR analysts are empowered to do things only “technical” individuals could do in the past. It’s been incredible to see at clients I serve!
With this ease of use, comes the need to have foundational skills and understanding of data to avoid mistakes / wrong conclusions. The idea that you can drag a R model in Alteryx to a dataset to predict an outcome is amazing; if you don’t know how to interpret that model or understand what is taking place in that model it could lead to false conclusions. Many have written on this, but my hope with these tools is that a continued education on data & statistical analysis will happen in the workplace. Those core skills will continue to be paramount in drawing data driven conclusions.
Hadoop will continue to be deployed at a higher frequency but there will be innovation on solving computing at scale with common languages and skills
Computing at scale on commodity hardware has worked for a # of enterprises, although the silver bullet aspects of Hadoop seen through the last few years has been lessoned which is a good thing. Given the complexity of the Hadoop ecosystem and need for learning & supporting new languages, I expect some innovation on lowering that barrier. We will see more push towards common languages like SQL and standard DBA skills vs proliferation of languages like Hive, Pig, etc. The Cosmos platform from Microsoft is a step in the right direction, and I’m interested in what MapD will bring to the table as well.
Data Viz “Mashups” will see innovation in a world where “one tool across the enterprise” will struggle to work
Large enterprises oftentimes want to standardize on a single reporting, data visualization or analytics tool. While there are many pros to this argument, the reality is people want to use what they want to use…that could be because one tool is simple, another more powerful or they’ve used another for many years. Given that, I expect companies to start considering Mashups where they’ll allow their users to combine data visualizations from multiple tools that are supported within the organization.
The legacy word for this was a BI Portal, but those were mostly information in nature helping departments understand how to get a license, trained, etc on the tools available to them. The idea of a Data Viz Mashup is taking that a step further and allowing users to combine visualizations from all the tools they use to curate experiences & stories. Now doing this elegantly won’t be easy as every tool’s API is different and there isn’t a great solution today to bridge these APIs. Alongside the API bridging complexity, there is a an inherent tool knowledge gap and required investment in learning the ins and outs of these tools to be able to create a mashup.
All that complexity aside, I think there are numerous use cases to make this a compelling solution (imagine being able to update your Google Sheet data that feeds your Tableau visualization in real-time). Expect to see more of this in 2016 and the idea of “one tool to rule them all” to fade slowly.
The Next Tableau will be Born or Revealed
I’ve discussed with my colleagues at length over the last couple years on who the next Tableau would be in the marketplace. They’ve built an incredible following, an amazing product and have grown up as a company. This has quickly led to them becoming the enterprise standard in data visualization among many major companies. (Replacing traditional players like Oracle, IBM, MicroStrategy and Microsoft.) While they are the “it” toolset to use for analysts today, this can quickly change.
In 2016 I expect Tableau’s hold on the market to continue, with hopefully an ever evolving product, but I also expect a newcomer to make a lot of noise. Who will that be and will it be in 2016? Well, I expect a company or two to make traction this year at larger enterprises. Companies like Looker have a great chance as they truly differentiate themselves (with their powerful SQL abstraction and analyst collaboration) vs being another data visualization tool. I’ve always found ClearStory Data’s product interesting as they’ve focused on a few, key areas.
My hope is that someone, somewhere starts or evolves a company that can truly compete with Tableau. They are poised to grow even more across the enterprise. Do I expect someone to innovate to the degree that Tableau did in the same data viz space this year, no, but there are huge opportunities to address the areas they and others are not today. Areas like collaboration, mobile, customization and engineering friendly tools are all areas a company can gain traction around.
Analytics Companies you should be paying attention to in 2016
Looker: I’ve known the team for a couple years now, and their commitment to building an application for modern data companies is applaudable. Their SQL abstraction layer is powerful and they could very well be poised for making a big jump this year. They’ve focused on evolving an interface that allows data engineers & analysts to collaborate and build models in a single environment. Lately, however, they are in the same conversations at customers considering Tableau, Qlik, etc.
Alation: One comment I’ve heard over and over again across large Tech, Financial Services, Retail and Healthcare companies is “I don’t know what data is available to me or what reports have been built.” I’ve seen a # of IT departments aim to build solutions that expose this type of information, but oftentimes these solutions can’t be supported due to engineering cost, capacity, etc. Alation is tackling this challenging topic and it couples quite nicely as companies look to better govern their data. I expect Alation to make big moves in the market and/or more likely get acquired this year.
