Great article Jessica, what was your approach towards Connection management?
Michael Smith
11

Hi Michael,

We actually haven’t been using Airflow Connections with this new approach! We do the following instead:
1. Create a service account in Google Cloud Platform with the appropriate permissions.
2. Create a Kubernetes Secret using that service account’s json keyfile.
3. Pass the Kubernetes Secret name to the Kubernetes Operator. Our Kubernetes Operator then uses the permissions/service account stored in the Secret to execute work.

We haven’t reused other Operator code in our containers. In the container code, we have been using Google APIs and libraries directly. For us, this is similar enough to our other coding environments to not hinder our developers.