Vertex AI Pipelines vs. Cloud Composer for ML Orchestration
A personal opinion based on workshops with many different customers
Back in the days, at a time when ML was not all over the place.
Companies already did a lot of data processing for other purposes. A large number of Cloud Composer workflows got created.
Nowadays companies often try to implement machine learning workflows into their existing Cloud Composer workflows. While this might work, you should at least consider using Vertex AI…