Everyday Automation for Engineering Managers
Being an Engineering Manager, there are always parts of my job that are manual and repetitive. These include constant shuffling of rotas, preparing Trello boards for Scrum ceremonies, preparing slide decks for recurring meetings, and so on. They can distract mental focus and create a constant undercurrent of “to-do list unrest”. I’ve been experimenting with low-code platforms to automate a lot of my tasks. In this blog post, I will elaborate on some of my experiences and show you an example of setting up a low-code script.
What is a “Low-code” Platform?
Platforms like Zapier, Tray.io, and Automate.io have risen in recent years to help you connect your various work tools together without writing any code. For example, in about 30 minutes I can knock up a script that will create a Google Slide deck, post the link to a Slack channel, create a Trello card and add it as an attachment, and mail it out to a list of people. All of this without writing a single line of code. No need to understand how to authenticate with each provider’s API, or to read endless reference docs. Everything just becomes — smooth. Like a criminal in a Michael Jackson song.
Here at Bloom & Wild I’ve been using Zapier. I haven’t tried any of the others so I can’t say if they are any better — but this works well for me so far. The starter plan costs around £15 per month and, given the amount of time it saves, is worth every penny.
I’m not endorsed by Zapier in any way, so all opinions are my own.
The Benefits
Some benefits of automation are obvious. No one is stuck waiting on you to distribute a deck because you got distracted or pulled into a meeting. You can be on holiday and things will still get done. Same for weekends or early/late hours.
But there were some benefits that I didn’t expect:
- I underestimated the amount of “to-do list unrest” I go through. That nagging feeling that you need to get things done by a particular time — especially when you’re having a busy day. Over time I estimate I’ve freed up 5% headroom. Some of it is from doing the actual work, some from discussing with others what the process is meant to be, some from canvassing around looking for a volunteer to pick it up, and some is dread when faced with a long, boring, repetitive task.
- We have some scripts that are “reminder-bots”. The script will send periodic reminders to a person or group to do a particular action and will only stop when they have thumbs-upped the Slack message. With a Zapier job it will be a robot, not an EM, having to remember to ask “Annie, are you ok?”, followed 4 hours later by “Are you ok? Are you ok, Annie?”, followed by a final reminder 30 minutes before the action is due — “Annie are you ok? Will you tell us that you’re ok?”
- Once you’re familiar with Zapier, a script can be set up in 30 mins to an hour. It’s easy for me to find an hour in my week to focus on setting up a job, so the barrier to entry is low. Writing actual code for some of these would take days — which would be a deterrent to even attempting the automation.
Some Tasks We’ve Automated So Far
- Creating Trello boards for team retrospectives.
- Monitoring a particular mailbox for notice of outages to Royal Mail and creating a Pivotal card for the team to action. (Zapier is capable of parsing email content for specific data, which has been quite useful).
- A few hours before a particular meeting, Alfred sends a Slack message asking the team to leave a thumbs up if anyone has discussion points. If there are no thumbs by a certain time, he cancels the Google calendar event.
- Maintaining a rota for Scrum master and notifying the next person a few days in advance.
- Monitoring a spreadsheet used by the marketing team and alerting the tech team if there is likely to be a spike in user traffic.
- Creating a weekly Pivotal card for one of our teams to do regular deployments on one of our services. It also assigns the card to people on a round-robin rotation.
- Creating and distributing slide decks ahead of meetings
An Example
Every script (known as a “Zap”) starts with a “Trigger” to kick it off, and then goes over any number of actions. One real life example here at B&W is our bi-weekly “Iteration in Review”. The morning of the meeting, one of the Engineering Managers (on a rota which has been automated in Trello, but that’s a different story) has to create a slide deck from a fixed template, change the date on the first slide, make it shareable, distribute it out to the dev teams, and check they’ve filled out their sections in time.
Start with the trigger. Zapier checks my calendar for an event called “Tech Team Iteration in Review”. It knows to trigger this Zap 7 hours before the meeting start time.
Next, copy the standard deck template. Set up an “Action” to copy a Google Presentation file called “TEMPLATE Iteration in Review”. The name of the new file is “Iteration in review <today’s date>”. That date comes from the previous Google Calendar step.
Then, using the ID of the file you’ve just generated, invoke Google Drive’s permissions system to allow everyone to edit the file.
Drop a message in Slack. You can use variables from previous actions, so in this screenshot you can see I’m sharing the link to the presentation I generated earlier.
You can also choose to send the message as yourself or as a bot with a name and an emoji for an avatar.
Then set up a delay time. In this screenshot, I’ve set it to wait until 2 hours before the calendar event start time. Date/time manipulation in Zapier is a piece of cake. You don’t get bogged down by formatting, timezones, GMT offset, etc. It’s all text based and very intuitive.
Finally send out another Slack message with a reminder.
Would I Use it to Run Production Software?
Maybe. It depends. Probably not.
I believe Zapier does want tech companies to start using them as a production integration. There are Zaps to consume off an SQS queue or to write to Datadog. You can make webhook requests and even send an SMS. There’s a LOT of functionality here.
We have one Zap that interacts with a Lambda on production. It wasn’t our preferred solution and we eventually want to replace it with something more robust. However, in the spirit of Agile, you first build a skateboard, then a scooter, then a car. Zapier lets you build the skateboard quickly.
Zapier doesn’t have unit testing, so you have to test each action manually. This is ok for small jobs but might not be acceptable for production. Also, error handling and graceful retries aren’t easy to implement on the lower price-plans.
That said, there are some huge success stories for using Zapier in production. There is an example of a startup that relied heavily on Zapier during their early phase in lieu of a tech team. Doing this helped them grow to $10m revenue! It has potential.
Tips
Create shared accounts. Zapier needs to sign in to your Google account (for drive, gmail, etc), Trello, Slack, etc. I created a dummy Google account whose only purpose was to interact with Zapier. This way others in the company can log in using these credentials and share the account.
Think about data security. If Zapier has access to your Google Drive, then it has access to whatever data is in there. By creating a dummy account, I was able to restrict what the shared user had access to. I only give them permissions to files they need to perform the various automations. Zapier will let you connect with Zendesk and other CS management systems, so do think about what is accessible and where you might want to give a user read-only permissions. In the event of an account breach where you’ve been hit by, you’ve been struck by a smooth criminal, you want the data there to be as harmless as possible.
There’s no such thing as “I’ll knock up a script in 10 minutes”. By the time you think about how you need the script to run, make a few mistakes, experiment a bit etc, it can take 30 mins to an hour to write each script. That’s ok — it still pays out in bucket loads for the time and mental focus you save in the long run.
There’s a learning curve. Some of my early jobs took a whole day to figure out. Be kind to yourself if things aren’t moving as quickly as you hoped. It gets much easier with time. Start with a simple, 2–step Zap. As the phrase goes, you need to learn to walk before you can moonwalk.
Conclusion
Low code platforms offer a low barrier to entry to configure integrations across multiple tools. I’ve also found using them to be quite enjoyable. I can quickly get to the heart of a script (i.e. the functionality) without getting bogged down by authentication or reference API docs. I would strongly recommend getting a 2 week trial on one of these platforms and experimenting to see if it suits your needs.
To round out the Michael Jackson references, I’ll leave you with this video of Smooth Criminal. This is the Alien Ant Farm version from the 2001 that was much cooler. Check out that kid breakdancing at 1:00 and then again at 3:15. He’s got moves!
