Blueprint Driven Cli Development
That I am at the moment deep in the dregs of Interplanetary File Systsems, Dharma Protocols and Cross-chain Token Burns is no excuse for some good old improving on coding skills and best practices.
Was brushing on up on the rusty Go programming skills when I chanced on a interesting problem with what I believe is an impactful solution — Blueprint Driven Cli Development.
Why Develop a Command Line Based Implementation?
For over a decade my go-to approach when starting coding on a large project has been to develop a Command Line ( CLI) application.
Taking a CLI first approach irrespective of the final project being for example a Sync(REST) or Async service helps focus thinking and allows one to model the problem space without the thrills and frills of deployments and complex call chains.
Further it gives one an opportunity to identify the program modules, distribute and consolidate implementation work across the team and as a BIG Bonus creates a test harness to be used down the road.
If you have not tried the approach I encourage you to do so. It is the closest to a blank canvas one can get in coding.
Why Create a Spec Driven CLI?
Unlike my other projects the BLKCTL (BLACKCHAIN Control) is a relatively large and not yet ‘well understood’ project. There will be some pretty complex code under the hood and it is comparable in extent to the well know Kubectl — the Kubernetes Cli.
When you work with REST Based Services in particular you learn to value the advantages of the Blueprint Driven Development approaches(with OpenApi in particular). Unfortunately Cli frameworks lacks a ‘blueprint driven development’ from which one could easily structure design discussions, generate code stubs and distribute among the development team to build.
Blueprint Based Cli Development could significantly increase productivity and also help plan and communicate within the project team much better.
On CLI Frameworks
In Python the wonderful Click and in Go the outstanding spf13/cobra frameworks have been my favourites Cli frameworks. Interestingly major contributors(Armin Ronacher & Steve Francia) of both frameworks are leading lights in the respective programming communities.
In Java I have only tried out the Spring Shell but on recent research the Picocli project looks particularly well designed.
For this project we will be using the Cobra Cli Framework.
The Cobra CLI framework is the de-facto standard for CLI frameworks in Go Lang. It has been used on many well known projects including Docker, Kubectl and Hugo.
Other than offering the required features for CLI frameworks such as commands, sub-commands(nested commands) argument passing and flags it offers a number of great features of a mature framework. such as
- Document Generation
- Auto Help
- Definition of Optional, Required and Combinational Parameters
- Generate shell auto-completion scripts
Structure of a CLI
Please note that the following structure analysis is taken from Create kubectl Like CLI With GO And Cobra authored by Mayank Bairagi. This is one of the best tutorials on getting started with the Cobra framework.
Let us first take a look at the syntax of a some popular command line applications
kubectl create deployment webapp ---port=80
aws s3 mb s3://mybucket --region=us-east-1
docker network create --subnet 203.0.113.0/24 iptasticThe structure could be abstracted as follows
Cobra is built on a structure of commands, arguments & flags. Commands represent actions, Args are things and Flags are modifiers for those actions. As Mayank states the ‘Resource’ needs to be replaced within Cobra with a Sub-command. I.e. A command which is a child of another command.
The Blueprint which is written in YAML replicates this Cli specific structure and can of course be extended with new commands as and when they are added just like we would with REST Service Blueprints.
The Blueprint
In the shoddy :-) YAML based blueprint below we see the top level group of commands with 2 commands. The second command in this case being a child of the first (please note that the definition is still bare-bones and can use significant improvement)
---
commands:
- command:
name: create
description: some descriptions
parent_comand: root
flags:
- flag:
name: some file
shorthand: f
default: na
usage: yaml file to use
- command:
name: card
description: some descriptions
parent_comand: create
flags:
- flag:
name: some file
shorthand: f
default: na
usage: yaml file to useThe Template
Not much more than the generic boiler plate command code.
package cmdimport (
"fmt"
"github.com/spf13/cobra"
)// {{.Command.Name }}Cmd represents the {{.Command.Name }} command
var {{.Command.Name }}Cmd = &cobra.Command{
Use: "{{.Command.Name }}",
Short: "{{.Command.Name }}",
Long: `{{.Command.Description }}`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("{{.Command.Name }} called")
},
}func init() {
/** Here we add the command to the parent command
avoiding some manual editing **/
{{ .Command.ParentComand }}Cmd.AddCommand({{ .Command.Name }}Cmd)
}The Main App
The Code Generation pipeline used is very minimal for now.
- Create a YAML based Blueprint Based on the above structure.
- Create a suitable code tamplate ( in this case only 1)
- The script combines Blueprint and Template to generate separate files for each command.
The main app predictably loads the Blueprint and generates code using the template after iterating over Objects defined in the spec. While there are other approaches which could avoid a specific application (such as gomplete) these are eschewed in favour of an application which also uses a defined structure for schema correctness as well as supporting other extensions in future.
On a sidenote the Type definition code was created using this online tool which autogenerates Go Type definitions based on the provided YAML: https://zhwt.github.io/yaml-to-go/ .
package main
import (
"log"
"gopkg.in/yaml.v3"
"html/template"
"os"
)
type CommandList struct {
Commands []struct {
Command struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
ParentComand string `yaml:"parent_comand"`
Flags []struct {
Flag struct {
Name string `yaml:"name"`
Shorthand string `yaml:"shorthand"`
Default string `yaml:"default"`
Usage string `yaml:"usage"`
} `yaml:"flag"`
} `yaml:"flags"`
} `yaml:"command"`
} `yaml:"commands"`
}
func check(e error) {
if e != nil {
log.Fatalf("error: %v", e)
panic(e)
}
}
func generate(commandList CommandList) {
for _,command := range commandList.Commands {
templateFile := "templates/cmd.tmpl"
tmpl, err0 := template.ParseFiles(templateFile)
check(err0)
f, erra := os.Create( "cmd/" + command.Command.Name + ".go")
check(erra)
err1 := tmpl.Execute(f, command)
check(err1)
f.Close()
}
}
func main() {
data, err0 := os.ReadFile("spec.yaml")
check(err0)
commandList := CommandList{}
err1 := yaml.Unmarshal([]byte(data), &commandList)
check(err1)
generate(commandList)
}Conclusion and Next Steps
The prototype helped me with understanding the benefits of a Blueprint based approach to Cli development as well as assess the effort required to convert the prototype into a full blown tool.
My conclusion was that we will proceed to build out the tools since it will definitely help us organise the development efforts around our Command Line Application BLKCTL a lot less manual and make us more productive. Further it will significantly help with improving application documentation. With the right type of ‘pipeline specific extensions’ we can also selectively replace commands and introduce command versioning.
Since this was a quick couple of hours exercise from start to finish I have not had the time to research if similar tools are used for ‘directly’ generating clients. You can derive clients from Open API specs but that seems too OpenAPI specific. Nonetheless a direction we will explore is applying OpenAPI specs to generate the clients.
To keep abreast of the progress follow me on Medium. A public Github repo will also be added soon.
