CD Pipelines vs CI Jobs

People who have done CI using “Jobs” take sometime to understand the value of doing CD using “Pipelines”. This post is to explain the real advantages of “CD Pipelines” over “CI Jobs”. There are other significant differences between CD & CI in general, but this post is limited to explaining modeling differences between the two.

To appreciate the difference I think we need a comparison, and a good example in my opinion is programming paradigms.

“CD Pipelines is to CI Jobs what Object oriented programming (OOP) is to Procedural programming.”

Why do i say that? Well, lets consider them in isolation.

OOP vs Procedural programming

  1. You can do everything in Procedural that you can do with OOP.
  2. There is no real difference when you are doing a tiny project. Infact, OOP looks like an over-kill to do something simple. The real differences emerge as a project grows.

Advantages of OOP

  1. Abstraction & Encapsulation - The teams start thinking & interacting at higher level of abstraction, leaving the details to implementer. You can set high level expectations off of a class / object. At any point of time the state of an object is considered as opposed to value of an integer or a boolean etc.
  2. Organization - Classes are usually organized into package & visibility to classes in other package is restricted to reduce cohesion.
  3. Dependency - Tracking dependency among classes/packages as opposed to functions is lot more easier & useful.
  4. Patterns start to emerge from experience which are more generic & useful than just “re-use” of code.

CD Pipelines vs CI Jobs

  1. You can do everything in a “Job” that you can do with a “Pipeline”.
  2. There is no real difference when you are doing build-test-release for a tiny project. Again, Pipeline structure looks like an over-kill to do something simple. The real differences emerge as a project or organization grows.

Advantages CD Pipelines

  1. Abstraction & Encapsulation - It allows teams to think & interact at a higher level, i.e. Pipelines (objects) as opposed to Jobs (functions). It allows teams to encapsulate Jobs into Pipeline that become coherent. i.e. all Jobs of a pipeline gets the same materials (source code, packages & pipeline dependency). The state (status, materials etc.) is for the whole Pipeline as opposed to single Job.
  2. Organization - Pipelines are organized into Groups & visibility to Pipelines can be restricted at Group level for security & cohesion reduction.
  3. Dependency - Tracking dependency among pipelines (Fan-in & Value Stream Map) is a lot more easier than tracking it among Jobs!
  4. The Pipeline-Stage-Job-Task model allows you to organize your whole infrastructure into Parallel-Serial-Parallel-Serial pattern. The Dependency management, Templates, Trigger types start forming patters that you can use in different scenarios.

Continuing on these lines…

You can do OOP with a language that was originally designed for Procedural programming, like say ‘C’. But its going to be very messy. Since the rules are not enforced by the language, its always going to be trust based implementation. Implementing OOP with a language designed for OOP like say ‘Java’ / ‘C#’ is always going to be much more cleaner & maintainable.

Similarly, implementing CD Pipelines in tools designed for CI is “possible” but if a tool is designed for CD Pipelines ground up the results will be far reaching!

Hopefully the comparison helped you put Pipelines & Jobs into perspective.