Test the Big Data Waters
Big data success stories from across industries have induced substantial interest in executives and business managers. However, managers or executives, intrigued by the possibility of big data, can find it troublesome to get started. Blown away by new jargon every quarter describing some new big data technology with over-the-top promises, leading a big data initiative is more often than not confusing for managers. If you are intrigued by the possibility of big data but not sure about where to start, you have come to the right place.
Venus Informatics offers consulting services to organisations seeking the right strategy to initiate big data projects. Our ‘Test the Big Data Waters’ service is a six-to-eight week pilot project with clear deliverables. The objective of this project is to evaluate the big data readiness of all or pre-specified business units and identify the readiest business unit to be explored with big data technologies with minimum capital cost and visible ROI. Leveraging the deliverables of this project business can build a contemporary data analytics platform which forms the synthesis between traditional and big data analytics and enables the organisation to incrementally augment it with new and disparate data sources for more in-depth analysis. Deliverables of this project are:
- Identify the right target
- Define the project ownership
- Estimate the total cost of ownership
Identify the right target
Big data presents new possibilities for the organisation to derive ROI in three business areas; cost reductions, business process improvements and new products and services offerings. The organisation needs to select where they are going to apply big data and analytics within their business. It is vital decision with cascading effects on identifying the business unit that should lead the initiative and manage the project. Our consultants examine three factors in the identification of the right target where business can gain faster ROI on big data.
The first factor is the existing problems which can be fixed with big data. Instead of pushing new technology out of the box, fixing something slow or broken will get support from the executive managers. An organisation may be sitting on a goldmine of data unaware of its analytical value which could transform business strategies.The second factor is identifying such hidden structured or unstructured data sources which the business can apply in the decision making. The third factor is comparing existing massive data storage, and computational-heavy data transformations cost with the big data environment built on MPP (Massive Parallel Processing) architecture. Replacing the entire enterprise data warehouse with big data technologies is possible. But, the ideal incremental path would be to offload ETL process on cost-effective MPP architecture first. Augmenting existing data warehouse with MPP technology enables the organisation to expand big data capabilities gradually.
Define the project ownership
Existing analytics group or architectural groups within IT organisations are the most likely organisational structures to initiate or accommodate big data technologies. Ideally, a big data initiative should neither be an independent IT project nor an isolated effort from the analytical group. Once defined what the organisation wants from big data, the business unit benefiting most from the initiative should take over the big data project ownership.
Big data initiatives can cause change management issues with changes in roles, process design, and skills either during the data capturing stage or by the result of the analysis of new data sources. Therefore, the collaboration between business experts, system matter experts, process owners from the project owner business unit, IT team, and the BI team is essential for big data success. The deliverable of the pilot project defines a clear RACI (Responsible, Accountable, Consult and Inform) matrix with existing roles and new roles which may include new roles such as data scientist or big data engineer.
Estimate the total cost of ownership
The total cost of ownership is vital for new acquisitions. The cost of installing, deploying, using, upgrading, and maintaining the big data environment can vary significantly between self-managed big data environments and the Platform-as-a-Service (PaaS/Cloud) offerings. Organisations with rigid data security policies would prefer an in-house big data solution. However, if the chosen big data target requires external data or less sensitive sensor data, entire data processing workload can be configured on the cloud with embedding the final processed and cleaned data view in the existing analytical solution.
The cloud factor can drastically reduce capital expenditures, but the TCO analysis over an extended period is required to make the best decision. Another key factor is up-skilling existing staff for big data tools or hiring new data scientists/engineers. The cost of acquiring big data skills may outweigh the savings on the hardware and software. Therefore, the last and the most critical deliverable of this pilot project includes concise TCO analysis of next five years for all possible variants (including different MPP technologies) of the proposed big data solution including hardware, software, and necessary skills.
Contact US
Your primary interest in big data could be the cost reduction, improve decision-making, create new products and services or fix an existing business problem. Our ‘Test The Big Data Waters’ pilot study will provide you with a future-proofed strategy for technology architecture, implementation skills and management, and the ownership cost to develop the big data solutions to fit your current needs with the ability to extend it with more disparate data sources incrementally.
