Meta: Executable processes for software development

Fighting entropy in software engineering processes, one automated task at a time

Omar
5 min readSep 2, 2018
The loom is one of the first automation systems in history. Meta aims to continue the tradition. Source: Lidya Nada on Unsplash

I am a software engineer at heart, with about 10 years of experience in various engineering and management roles.

During this time, I observed a singular pattern repeating, when it comes to software development be it for small or large teams.

  • A new software code base is created
  • A development process is elected (or designed)
  • During the “Honeymoon” phase the process is carefully followed by the developers
  • As reality kicks in, requirements, or the infrastructure changes, and the developers are slowly deviating from the process, because:
  1. The process became outdated and doesn’t correspond to reality anymore
  2. Short deadlines, human errors and whatnot causes developers to skip steps
  • The theoretically streamlined process becomes in practice a set of dark spells only fully understood by 1 or 2 developers, who tend to work over time to compensate for it.

At this stage, the team might be lucky, and whoever owns this process will get around to improve on it. However, more often than not, as reality (short deadlines!) made developers skip steps, it is equally likely that there is just no time to iterate on the process itself.

Even worse, if the team initially made the potentially costly investment to implement a CLI tool to speed up the process, entropy takes over even faster, and unless the team is very diligent and has enough resources, the CLI itself becomes a cost center that drags the team down more than it actually speeds it up.

Oh and what about that resource reallocation from HR, bringing new developers in the middle of the project, who also need to be trained on an unspoken process, of which only a historical view is documented somewhere, which increases the cost and time needed for the on-boarding?

The options become grim, ensues a quest to convince Management for the need of a refactoring budget, that will try to repay the technical, process and automation debts accumulated over time. Good luck!

Or wait?

What if there was a way to design and iterate on software development processes visually, easily and cheaply within the constraints of the real world?

What if this way allowed to map this designed process directly to a CLI tool that runs and automates the process steps?

What if this CLI tool could also automate any of the individual steps the developers type in their shell, and reduce human error to virtually nothing?

What if you could reuse this process on any new software project, keep the process itself versioned, and execute it 100% reliably?

Introducing Meta

Meta is a Software Process Management tool.

You can use it to define a process, and you will obtain as a result a CLI tool tailored to the exact commands you need.

Your CLI is kept in sync with your process definition through the use of a meta.json file, that you can commit to your code base repository, or share across many projects.

Each process is defined as a set of activities. For example, in a feature branch git workflow, an activity would be to start a new feature branch. Another activity would be to release a new version of the code base.

Each activity entails a set of steps needed to perform it. In the example case, it would be about creating a new branch for the feature.

The activities also correspond to commands in the CLI, where each command would execute in turn the software commands that correspond to the activity steps. Meta provides for that a large amount of functionality built-in to implement your commands, such as for Git and NPM. It is also fully compatible with any shell command.

What’s in a meta.json file?

This file contains the various commands of the CLI. All the workflows are described in this file, in a format mostly based on ASL.

It also specifies some configuration variables, that could be required by workflows.

Let’s assume a very simple process for a fictitious code base.

This process has a single activity, release. When releasing a new version of the code base, the developer should:

Visual representation of the process in MetaMaker

ƒ ƒ

  1. Check that the git repository is not dirty (no pending changes)
    ƒƒ ƒƒƒ ƒ
  2. Check out to the master branch
    ƒƒ ƒ
  3. Update the version of the package
    ƒƒ ƒ
  4. Push to origin master and the tags to the remote
    ƒƒƒƒ ƒ
    ƒƒ ƒ
    ƒƒ ƒ
    ƒƒ ƒ
    ƒƒ ƒƒ

The corresponding meta.json file contains the following:

{
"commands": {
"release": "release"
},
"config": {
"remote": "origin"
},
"workflows": {
"release": {
"name": "release",
"input": [
"releaseType"
],
"definition": {
"StartAt": "assertClean",
"Comment": "Commit a release",
"States": {
"assertClean": {
"Type": "Task",
"Resource": "git:assertClean",
"Next": "updateVersion"
},
"updateVersion": {
"Type": "Task",
"Resource": "npm:version",
"Next": "pushMaster"
},
"pushMaster": {
"Type": "Task",
"Resource": "git:push",
"Input": {
"branchName": "master"
},
"$Input": {
"remote": "config.remote"
},
"End": true
}
}
}
}
}
}

The generated CLI then can be invoked in the same folder that contains the meta.jsonfile:

~/fictitious-project> meta release minor

And it will automatically execute the steps defined above.

If the repository is dirty, it will throw and refuse proceeding.

MetaMaker

The screenshot you just saw above is directly taken from MetaMaker, the visual editor for meta.json files.

It helps designing the processes and iterating over, and generates a corresponding meta.json that can be directly dropped in the repository’s folder to empower it with meta.

Very Early Preview

I am releasing a Very Early Preview of meta, to a limited amount of people.

The software is as early stage as it gets, however it has been used for some time already to dog-food its own development to a great success (now we’re becoming really meta), and also for a few other projects.

Would you like to try it out? Shoot me an email at meta@omneity.xyz

I have a few slots available where I could personally help you integrate meta in your workflow, and find out how it can deliver maximum value to you and your team(s).

--

--