Sivabalan NarayananStandalone HoodieCompactor UtilityWe have discussed about compaction for MOR tables in Apache Hudi here. This table service compacts the base and log files to form next…Aug 5Aug 5
Sivabalan NarayananFile Naming conventions in Apache HudiApache Hudi has two types of data files, base files and log files. Hudi names the file in a certain connotation to assist with regular…Aug 5Aug 5
Sivabalan NarayananHow to define your own merge logic with Apache HudiIn Hudi you can configure a payload class for a given Hudi table as per your choice. It gives users an opportunity to define the merge…Aug 1Aug 1
Sivabalan NarayananApache Hudi’s Ingenious way of handling partially failed commitsFailures and crashes are a norm in distributed systems and is no different in data lakes and lakehouse as well. Each system could handle it…Aug 22, 20231Aug 22, 20231
Sivabalan NarayananFetching completed commits for Incremental queryMany users have reached out to us asking how to fetch commit instants for a given hudi table so they can use it for incremental query…Aug 10, 2023Aug 10, 2023
Sivabalan NarayananThe memo we missed on a “table format war”……while we were busy making the data lake faster and easier.Jul 20, 20236Jul 20, 20236
Sivabalan NarayananApache Hudi Timeline: Foundational pillar for ACID transactionsHudi maintains a timeline of all actions performed on a given table to support efficient retrieval of data for read queries in an ACID…Jul 9, 2023Jul 9, 2023
Sivabalan NarayananMulti-writer support with Apache HudiApache Hudi has multi-writer support from 0.8.0. Essentially, two writers can concurrently write to hudi and successfully commit given they…Jun 25, 2023Jun 25, 2023