Rapid Analytics Using Cloud Managed Services
5 Reasons to Buy Versus Build
Matt Lampe and Kashi Dev | April, 2017
Today’s complex business challenges require a modernized data architecture — one that is fast, scalable, and flexible enough to meet the business needs of today and tomorrow while enabling a new data-driven business culture. Cloud-based analytics products are at the forefront of this modern data architecture evolution. If you’ve only been thinking about Hadoop as your Big Data solution, read on to understand why Hadoop for analytics is no longer your only or best choice.
Why consider cloud-based analytics platforms?
There are many reasons to buy analytics products versus build. Some that continue to take priority:
(1) they’re faster to build and deliver value
(2) they’re less expensive both to build and operate
(3) they have scalable and elastic architectures
(4) your team of analysts and developers probably know how to use them
(5) the best-in-class vendors are stable businesses.
In looking at the data warehouse appliance market today, there are three main competitors: AWS Redshift, Azure SQL DW and Google BigQuery/Spanner. All of these products provide the same basic offerings: low start-up overhead (can get started in minutes); on-demand scaling up or down; ability to integrate with leading big data mechanisms like Hive, HBase, and Spark; and pay-as-you-go pricing models.
1) Faster to build
The major cloud vendors all provide an integrated suite of products that leverage a combination of proprietary technology along with full support for open source products and protocols. These products are available to be spun up in a matter of minutes via pre-packaged images and server configurations. The hardware configurations can be easily tweaked, configured and re-deployed by simple selections based on capacity and usage.
In addition to the software and hardware components that make the build process faster, each of the major cloud vendors provide migration services to quickly and cost effectively move existing data and services over to the cloud. This service is helpful for enterprises that have invested heavily in their on-premise infrastructure and are unsure of the approach to move to the cloud.
2) Cheaper to build and manage
In addition to the speed to deploy, the other main advantage of moving to the cloud is the cost savings that are realized based on only paying for what they need and use. The cloud providers are currently on a race to zero and the cost of running a full data workload is constantly getting cheaper. There is no longer a need to have servers running all the time. Each of the vendors provide the ability to run workloads based on pre-selected criteria. The server configurations can be scaled up or down based on demand.
The maintenance of hardware and software to run the data storage layer is handled by the provider which frees up valuable IT resources to support the business instead of managing infrastructure. The ability to run pre-built instances of all the major software (Hadoop, Traditional, No-SQL) makes the process to manage the data server instances significantly easier.
3) Ready to grow with your business — more scalable and elastic
· Analytics-centric — Multiple native connectors and a continually-growing base of business intelligence solution providers
· Hybrid architecture — While competitors offer independent infrastructure as a service (IaaS) products, SQL DW and the broader Azure product base can supplement existing on-premises resources
· Integrate with existing Microsoft products — If you’re already a Microsoft customer, adding SQL DW into your portfolio is seamless. In addition the SQL DW was built using Microsoft SQL Server, so the internals of the product, and thus our institutional knowledge of the product, is transferrable.
4) Easier to find and retain talent
In today’s competitive engineering job market, finding top talent with the latest open-source skills is difficult. They are out there but ask yourself this.
· What is your answer to the candidates question about job growth opportunity?
· Do you have a career path defined for data engineers and architects?
· How do you retain these talented individuals?
Most analytics products are based on well-known technology which limits how much investment is needed to re-train your existing staff to design, build and run them.
5) Proven, long-term technology partners
Don’t get fooled by a small vendor closing shop. While it’s tempting to jump on the latest tech bandwagon, many smaller technology vendors are exiting the market. Unless you’re staffed to take over supporting their products or enjoy giving your business partners whiplash as you swap out technologies, choose carefully. We’ve seen clients lose a lot of momentum and credibility when their cool new vendor pulls out. These niche software companies see the market changing and some will try to differentiate themselves and others will cut their losses.
For the big three players, this is a key business for them, not an experiment. Their products will be supported for the long-term even as new products come available. And your other partners will likely integrate with them, making it easier than ever to get the data you need.
Common Obstacles to Moving Into the Cloud
1) Security
Understand how security works with these cloud vendors. You may have highly sensitive information that cannot be hosted outside internal networks. Know how they protect your data at-rest, in-motion and in-use.
Counter point: Private Cloud connections and security audit processes can reduce this risk to acceptable levels.
2) Edge cases that public cloud products can’t deliver
This requires a strong engineering culture & bench. There are just a handful of companies with an exceptional engineering background that can build a better in-house cloud. If you’re one of them, going to one of the major cloud vendors shouldn’t be your first choice.
3) Vendor lock-in
Some products are generic enough they transport across platforms, especially if you’re building on an open-source stack. Keep in mind, bringing non-cloud vendor software to the cloud is risky. See #5 above
What’s next?
Companies’ ability to innovate and pivot is what keeps them in business — and cloud vendors are no different. When looking at the evolution of modern data architecture, there are two themes that come to mind:
1) enabling the fullness of the data warehouse dimensional model using best-in-class resources and
2) applying tested-and-true data warehousing methods to the cloud.
Tomorrow’s data warehouses will continue to integrate and expand upon more advanced analytics and collaboration offerings. This will be realized when the lines between data management, analytics, and visualization products no longer exist. Microsoft is already doing this today with the SQL DW and SQL Server products. By building into these products the ability to gain insights and share content collaboratively, we’re poised to broaden our understanding of the data around us.
References
AWS Database Migration Service https://aws.amazon.com/dms/
Race to Zero http://www.businessinsider.com/google-leads-cloud-storage-race-to-zero-2015-5
