Data Refining Simplified
Part 1: Visual Exploration and Analysis Development (Video)
IBM’s Data Refinery accelerates the end-to-end experience of refining data from development to production.
The daily activity of a data scientist is often to explore data and build analysis with a test data set, refine the workflow and then automate it for real-world data. This can be a time consuming, repetitive, and laborious process.
Data Refinery accelerates the effort to refine raw data into analysis-ready data. To expedite the process of data refining, visual snapshots, quality profiles and intelligent feedback at each step is provided by the tool.
Using Data Refinery, Data Scientists can transform, normalize and combine data from any domain using over 100 operations on numeric and non-numeric such as text, dates, etc. Over 23 charts are provided as a visualization aid. These include scatter- plots, word clouds, t-SNE, heat maps, 3D charts, and others to expedite visual understanding of the data.
New features in Data Refinery enable data scientists to proactively adjust their complex flows developed on test data to be ready for real-world data, before executing them. They can automate their analysis to run on an hourly, daily, weekly or monthly schedule. As an added benefit, they can personalize the execution profile to match real-world data volumes.
This video shows how Data Refinery expedites visually creating analysis on test data
- Visual exploration of data to get a sense of data distributions and quality
- Visualization guided data transformation to build the use-case specific analysis
- Validate analysis of test data
Use case
The Data Scientist’s task at hand is to find the most appealing companies offering flight tickets online, on a regular basis. She has access to test and real-world data from search engine result pages to perform her analysis. She has chosen IBM’s Data Refinery tool to perform this task.
IBM’s Data Refinery is available with Watson Studio, Watson Knowledge Catalog on public cloud, private cloud, and Watson Studio Desktop.
Get started for free at: https://www.ibm.com/cloud/data-refinery