[OUTDATED POST] Announcing Popper 2.0 — Local Github Actions Executioner in Python

Update: Support for Github Actions syntax in Popper has been deprecated. Popper now uses its own YAML syntax for specifying workflows. Read more in the official documentation.

The goal of the Popper project is to aid users in the implementation of experimentation pipelines that follow a DevOps approach. Last year, we release version 1.0 of the Popper CLI tool, which allows users to specify pipelines in YAML syntax (see example below).

As we kept incorporating user feedback, the YAML-based pipeline definition syntax (shown above) kept evolving and, over time, started to look like a workflow specification format; we decided it was time to embrace workflows properly. Around this time, Github released Github Actions (referred from this point on as GHA), a workflow language and code execution platform. The GHA workflow syntax, being based on HCL, is one of the simplest workflow formats out there (see example below), which makes it a great way for implementing workflows in many distinct domains. In addition, the open actions specification allows anyone to implement distinct functionality, providing a great extensible framework that can address multiple needs.

Instead of creating yet another workflow language, we decided to leverage GHA’s syntax as the format for specifying workflows and make this the basis of Popper 2.0. Thus, Popper 2.0 is a workflow execution engine based on Github actions(GHA). Popper workflows are defined in HCL syntax and behave like GHA workflows. The main difference with respect to GHA workflows is that a Popper workflow can execute actions in other runtimes in addition to Docker. The workflow language is strictly a superset of GHA workflow language so Popper can run a GHA workflow locally as if it was being executed by the GHA platform.

Popper 2.0 is runs Github Actions workflows locally on your machine.

In addition to running a GHA workflow locally, the goal for the Popper 2.x series is to provide the following functionality:

• Runtimes. Actions can execute locally on the host where the popper command runs (i.e. "outside" a container, thus not strictly depending on having Docker installed). We are working in adding support for other runtimes are such asrkt, Vagrant, Singularity and others (see this project for more).
• Actions catalog. Provide users with a searchable catalog of available actions and workflows so that they can re-use as much as possible what others have already done.
• Continuous integration. Generate configuration files for distinct CI services, allowing users to execute Popper workflows on these services (Travis, Jenkins, Gitlab and Circle supported).
• Scaffolding. Aid in the implementation of new actions and workflows, as well as extending existing ones.
• Action library. As part of our efforts, we maintain a list of actions available at https://github.com/popperized.

For more about Popper, see our Github project or join our Gitter channel.