Interesting read. Looks like this is a problem across the industry. So far the closest open source project to this is Debezium. It supports wide range of databases MySQL, Cassandra, MongoDB, Oracle, PostgresSQL, SQL Server etc. It might be interesting to see the similarities and the differences with this framework.
Uber has something on similar lines as well.
Thanks for sharing these observations Rémy DAVID. I actually wanted to write on the comparison. Now I see this technique more valuable than before.
Adrian, you are right! I am actually in search for something more anti-fragile. Recently I came across one interesting library from Facebook. — https://code.fb.com/data-infrastructure/spiral-self-tuning-services-via-real-time-machine-learning/ This could be one real good example of antifragile system.
Hi Shawn Reisner , I believe on Data Minimization. Any point of time when are handling data we need to keep in mind to expose adequate, relevant and limited to what is necessary. This helps with GDPR as well. Hence I believe APIs should functionality specific.