Creating a slack bot for your changelogs — PART I: parsing
This is a series of articles about creating an interactive slack integration: https://github.com/SoftupTechnologies/AngelBot
It started on a normal workday, when we (Kristi, Shpëtim and me) were discussing about how we would publish the changelogs of a software project we are working on. Of course, it’s not a secret, we programmers are known to be lazy when it comes to manual repeating tasks. Since we use slack a lot at our company, we thought to make use of it and post changelogs on a slack channel. But there was more potential I thought, what if we made some type of bot which automatically posts changelogs when we merge the pull requests?!
Gradually we extended this thought and considered adding interactive features like browsing a particular changelog version or a specific category e.g. breaking changes. Initially we did not know how the architecture would look like and Kristi proposed to try and use AWS Lambda Functions. We made a quick sketch of how it should work and came up with a primitive flow. Basically we would integrate something in our CI/CD pipeline which could send the changelog to a Lambda Function. The Lambda function would store it in a dynamoDB table and provide some way of interacting with slack. Sure enough, as it turns out later, there were some flaws which we would need to overcome.
The actual work starts
The first thing I did, was writing some kind of parser in JS, because we intended to extract information like category. Since I have some affection for an earley parser written in Javascript called nearley, I started straight up writing the parsing grammar. Nearley is a context free grammar (CFG) parser, which means it can parse context free languages. A context free language is a formal language where the deriving rules are applied independently of the context of a symbol. Maybe it was a little bit overkill to use nearley, but the simplicity of making a change afterwards in the process, added to the decision. The DSL of it is expressive and easy to understand. If the tokenisation is to complex for the built-in charsets, nearley makes a great pair with the moo-lexer. It’s also possible to have macros, and post-processors for each rule.
Writing a parser with nearley is not very hard, but parser writing decisions can be hard (e.g. grammar undecidable problems for the ones who want to go down the rabbit hole). Nonetheless, it took some amount of work to think about the production rules down to terminals, while also processing each rule to form a JSON object. The configuration from which the parser is created is in a EBNF-style format. The start symbol is listed first and is so to say the placeholder for every input which we throw at the parser.
Let’s create a simple example, which tells us if our assumptions for the food chain in nature are valid. The first symbol is START, which is everything we feed into the parser. The keyword “_” which stands for none ore many white spaces is the first token. After that we continue with CARNIVORE followed by whitespace and the keyword EATS, more whitespace and the parentheses, between which we expect to find HERBIVORES. HERBIVORES must have the production rule as defined in the 4th line, so e.g. SHEEP EATS (SUNFLOWER, GRASS), COW EATS (GRASS) .The production rule of MOER_HE is comma and then more HERBIVORES or (“|”) null if you want to terminate the chain. In this manner we can analyse the following lines.
Here is the link of the gist which unfortunately did not show up on medium, even if it was visible in the editing mode.
You can play around yourself with this snippet and other grammars on https://omrelli.ug/nearley-playground/.
The site also makes use of the un-parse functionality which spits out results which our grammar would evaluate as a valid food chain. Here are some examples:
Here is the actual grammar I used for changelogs, admittedly the RegEx part can be tweaked:
https://github.com/SoftupTechnologies/AngelBot/blob/master/dealer-write/grammar/grammar.ne
Stay tuned for the next part, where I explain how we packed all this in AWS lambdas and the requirements we had to fulfill to keep slack happy.
