Do not program in your CI!

I just realized the .yaml file for CI is the most hated area of writing. Why? Because you can’t test it. You’ll see if it works for the first time when you run it (in your CI!). There are no types. The CI and bash variables are intertwined. Some variables are set by CI, some come from secrets, some are created by CI script.

It’s hell to write and to support. Any ‘programmer’ tricks inside CI (code reuse, common blocks, etc) makes things worse.

The larger is CI file you touch, the more optimized it, the deeper your personal hell is.

Nope, nope, nope.

CI file looks like yamlsembler. An assembler in Yaml. Gross!

How programmers are able to create megabytes of gross machine code without ever reading it?

Compiling!

So, the idea: generating.

As soon as you start to see repetition in your yamls, instead of writing them as ‘fancy’ &anchors, or extends: or anything like that, just switch to code generation. Your yamls are linear and trivial to read. No optimizations, no compactification.

The actual CI configuration (those ‘repetitive pieces’) are generated. They are generated from templates and parameters. A template contain repetitive part, parameters are in separate file.

Then you generate your CI yaml and shovel them into git together with templates and parameters in a single commit. Yes, you store binary artefact (CI file) in a git, but there is special reason to it, your CI is executing it directly from there.

I’ve tried this approach and I realized I never want back to ‘clever CI tricks’.

Keep CI script long, dumb, and generate it. (And no, I don’t suggest any specific tool. Find or write it).