Announcing CDAP 6.1.1 Release
On behalf of the Cask Data Application Platform (CDAP) community, it is my pleasure to announce the release of CDAP version 6.1.1. This release includes a new, intuitive UI for Field Level Lineage (FLL), and extends FLL support to more plugins. It also introduces a new failure validation framework for plugins for early detection of configuration errors in pipelines. Additionally, you can now connect to your data in Azure Data Lake Storage (ADLS) and transform it visually in Wrangler.
Field Level Lineage
The new field level lineage feature allows users to better visualize the journey of individual fields in datasets. The updated design also makes relationships between fields easier to see by connecting fields in the UI, and users can analyze relationships and transformations by selecting specific fields. For more information on Field Level Lineage, see this blog post series: part 1, part 2.
Pipeline Improvements
This release also introduces a new way for validating plugin configurations that allows us to detect and highlight errors within a pipeline, so that you can detect configuration errors sooner and build pipelines more efficiently. All the configuration fields with errors are highlighted once the user opens the plugin properties and a helpful error message is also displayed below the field . This greatly reduces the amount of time spent configuring and validating pipelines. For more information on the validation framework, please see this blog post.
This release also adds new widgets so users get better controls to configure their pipelines more intuitively. These include ConfigurationGroup — a wrapper component that can hold other widgets. It allows for easier visual grouping of related fields. Additionally, developers can now use dynamic filters to define custom conditions for when certain fields should be displayed/hidden depending on the contents of other fields within the ConfigurationGroup. For example, show the ZipCode field if the value of the Country field equals “USA”.
Platform Enhancements
This release contains performance improvements to several areas across the platform. The most notable improvements can be seen on the pipeline list page and from within the Studio. These improvements have significantly improved the performance of the UI under heavy load.
Download CDAP 6.1.1 today and take it for a spin! Also consider helping us develop the platform by reaching out to the community with any comments, feedback, suggestions, or improvements or by creating and following JIRA issues and submitting pull requests.
