Using Portable to trigger ETL syncs across an end-to-end Pipeline
Cover the Long-tail of ETL with Portable and Orchestra
Introduction
Orchestra, the data industry’s first end-to-end Data Control Plane, announces today that it integrates with Portable, the Cloud ETL Provider with over 1000+ Mainstream and Long-tail Connectors.
If you want to get started, please see the docs here and portal here.
An alternative to Airbyte and Fivetran, Portable is a cloud-based ETL tool that focuses on delivering the long-tail of connectors. From commonly-used SAAS Products like Stripe and Hubspot to nicher systems in Finance, HR and literally any piece of software imaginable, Portable has you covered with over 1000 integrations.
Historically, making use of an application like Portable alongside internal data movement tools has not been possible. Therefore, we’re excited to bring the Data Industry’s first Portable plug-in that allows users to take advantage of Portable’s connector catalog and fixed pricing to finally get coverage over 100% of data sources.
What is Portable?
Portable.io is a cloud ETL (Extract, Transform, Load) tool that simplifies the creation of data pipelines. It enables users to quickly set up connections to over 1000 data sources, allowing for the centralization of source data from various business applications into cloud data warehouses like Snowflake, BigQuery, Redshift, and PostgreSQL.
This functionality supports a range of use cases including analytics, process automation, and the development of data products that drive revenue. Portable.io is designed for data management teams and aims to streamline workflows, reduce manual tasks, and enhance strategic decision-making through analytics tools such as Tableau or Looker.
Advantages of using Portable within Orchestra
Orchestra is like a rubber duck floating atop the sea — no matter how high the sea, the rubber duck remains on the top. In the same way, no matter how complicated your data stack, Orchestra is the single pane of glass into all of it.
A very common use-case we see is data engineers using Orchestra to trigger multiple Portable Syncs, alongside internal processes, and then orchestrating downstream workloads like dashboard refreshes or dbt model runs.
For example, you might leverage Portable to ingest data from a number of hard-to-reach APIs. In tandem, you may leverage an internal service like Azure Data Factory for computationally intensive copies of cloud resources.
After running these jobs, you might want to trigger and monitor a dbt Dag and subsequently refresh a dashboard in a tool like Power BI.
Orchestra makes this process a breeze by acting as the central point of governance and control for the entire process. This ensures that when failures occur, data teams are the first to know. It also ensures that end-to-end lineage is retained for governance and auditability purposes.
Over time, these flows can be monitored in Orchestra too, including important metrics like cost and data quality.
Conclusion
We hope that by offering a Portable integration within an Orchestration and Data Platform, data teams can continue modularising their data stacks and using the best piece of kit for the job.
Running jobs in Portable unlocks more options for data teams to move data from hard-to-reach APIs, as well as cost-effective alternatives for existing connectors.
If you’d like to learn more, you can read more in the docs here and if you’ve got a Portable account already you can get set up in minutes here. 🚀
Find out more about Orchestra
Orchestra is a platform for getting the most value out of your data as humanely possible. It’s also a feature-rich orchestration tool, and can solve for multiple use-cases and solutions. Our docs are here, but why not also check out our integrations — we manage these so you can get started with your pipelines instantly. We also have a blog, written by the Orchestra team + guest writers, and some whitepapers for more in-depth reads.
Also we did a load of cool semi AI-driven Guides that are actually pretty good!