🚀 Introducing UCX v0.9.0: Enhanced Assessment, Migration, and Error Handling for Unity Catalog Upgrades

We are thrilled to announce the release of UCX v0.9.0, a minor release that includes enhanced assessment, migration, and error handling. UCX is your trusted companion, guiding you through the account, groups, workspaces, and jobs upgrade process. Here’s a summary of the most important improvements in this release:

• A new assessment step has been added to estimate the size of DBFS root tables (#741). This enhancement helps you decide whether a table is a good candidate for cloning by providing you with the necessary information. This feature calculates the table size by executing spark._jsparkSession.table("db.table").queryExecution().analyzed().stats().sizeInBytes().
• Introducing the databricks labs ucx move command, which allows you to move tables and schemas between catalogs (#756). The databricks labs ucx move command supports moving tables from one schema to another within the same catalog, moving tables from one schema to another across catalogs, moving schemas from one catalog to another, and moving relevant permissions.
• The migration method is now determined based on whether the table is on the DBFS root (#759). This ensures optimal migration based on the table’s location, making the process more efficient. The migration functionality uses different approaches based on the table location, such as copying the table content, deep cloning, syncing it to an external UC table, and skipping the table if it points to a Databricks dataset.
• Error handling has been enhanced, including retry and rate limit adjustments for renamed workspace group operation and corrected rate limits for reflecting account groups to workspaces (#751). Improved error handling ensures better stability and user experience during the migration process.
• The UCX project now incorporates the databricks-labs-blueprint library for common utilities, ensuring compatibility and reusability across various projects (#758).
• Query execution exceptions from within Databricks Workflows have been converted to SDK exceptions, improving exception handling and stability (#769). Error message inference from failed workflow runs has been improved, addressing integration test flakiness and enhancing the user experience (#753). These changes ensure better error handling and consistent exception types for improved user experience.
• Addressing an issue where missing users and temp groups appeared after workspace-local groups migration and ensuring the crawling task skips a table if it no longer exists (#770). This improvement increases the stability of the assessment job and integration tests.
• The assessment dashboard has been reorganized for better readability and clarity (#738). The reorganized assessment report includes the quantity columns in all rows, making it more user-friendly and informative.
• A new feature ensures that the metastore exists in the current workspace, further optimizing the upgrade process (#735).

Experience the full potential of Unity Catalog with UCX v0.9.0 and unlock the benefits of these fantastic features! 🔓✨

Please share and subscribe to Updates from Databricks Labs newsletter to stay up-to-date with the latest releases from GitHub namespace. Subscribing ensures you’re the first to know about the latest enhancements, bug fixes, and exciting features that will take your Data Intelligence Platform experience to the next level. You‘re encouraged to follow the GitHub org as well ;)