Hacking Analytics’ Compendium of Data News — November 2020
With everyone trying to get everything out of the door before the Holiday season, November was a busy month for the data world, Airflow 2.0 moved to beta status, new tooling was released by Google to help with Machine Learning in the space of NLP and managing ML model bias, and Apple released some benchmark of the performance of their new M1 chip for ML workloads.
SQL and ETL
SQL got some attention this month; Google released an upgrade to their managed Postgres instance to the latest version, Postgres 13. Databricks released SQL Analytics providing a familiar SQL interface for querying delta lake tables. SQL Analytics provides both a workspace and connectors to popular BI solutions to facilitate the work of analysts. Amazon also played into the SQL game by introducing a SQL compatible query language for DynamoDB.
Amazon introduced a managed service for data workflows leveraging Apache Airflow. Talking about Airflow, Airflow 2.0 entered in BETA state this November and is due to be upgraded to release candidate status in December. Airflow also received a new Provider for Great Expectations.
Data Quality was a focus for more than Airflow this month. A case study for Great Expectation was released with Heineken. DBT got its’ own port of Great Expectations in the…
