3 Tools for More Productive Data Engineering
New integrations and features boost data engineering efficiency with Airbyte Cloud, Apache Airflow, and dbt.
As a data engineer working with Teradata, you’re used to managing large volumes of data. Managing your extract, load, and transform (ELT) pipelines is now easier with new features and integrations from Teradata. Let’s look at these exciting ways to increase your productivity.
You can now bring more data to Teradata Vantage™ without a need to build your own data movement infrastructure: Vantage is now a destination in Airbyte Cloud. Airbyte Cloud is a hosted solution that allows you to copy data from 300+ sources to Vantage (both on-premises Teradata VantageCore and cloud-based Teradata VantageCloud deployments are supported). It’s an excellent option to bring data from many SaaS applications, combine it into data products using dbt (data build tool), and leverage the power of ClearScape Analytics™ to analyze it.
We’ve also added support for dbt data contracts with the dbt-teradata 1.7.1 release. dbt is a data transformation tool that enables data to be transformed where it lives — in the data lake and warehouse.
Just like interfaces in programming languages, data contracts allow you to define how your data products are seen by other teams. You can evolve your dataset and honor the same data contract to ensure that downstream teams aren’t affected by your changes.
Finally, we recently released an Apache Airflow™ adapter for Vantage. Airflow is the go-to tool for automating data workflows. With Airflow, you can automate repetitive tasks and use the time you save on things that are important to your customers.
Learn more about these new features and integrations:
- Teradata Vantage™ destination now available on Airbyte Cloud
- Improved data quality in your dbt models with dbt-teradata 1.7.1
- A beginner’s guide to orchestrating pipelines with Apache Airflow and Teradata
That’s it, folks! I look forward to hearing how you intend to use these new features and integrations. Leave a comment below or connect with me on LinkedIn.
About the Author
Adam Tworkiewicz is director of product management at Teradata and leads the developer experience team. He has more than 20 years of experience in architecture, IT operations, product management, software engineering, and strategy.
