Start functions with style using Cookiecutter

Whichever the cloud you use, injecting tooling in your functions make them better.

Last week, I had an interesting conversation with my teammate Quentin about the complexity of managing a lot of functions across multiple clouds and languages for many years.

Why are we doing this?

As an exemple, in my current team, it’s usual to deploy functions to process events, generate custom notifications, split jobs or create aggregated metrics from cloud providers quotas/metrics. Currently we have more than 50 functions, AppEnine applications and Cloud Run deployments and it grows every weeks.

In this context, you can imagine how it’s difficult to keep homogeneity between all functions.
Some of them are monitored, some aren’t, some Python code uses PyLint or maybe mypy, some are developed in Golang, but most of the time in Python. They are all deployed either with Terraform Cloud, Github Action, Gitlab-CI or just manually.

Reviewers and still the human failure

A good code review should smooth out these differences, but depending on how the review is done or how the reviewer is available to do it in depth, the result varies.

We talked about facts that are inherited from the organization (humans and priorities changes, not enough time is allocated to maintain code, etc.).
But we also talked about how our skills evolved since we started our first function.

The way we follow is to make onboarding simple and efficient.

As an IT consultant, when I start writing code, I don’t want to lose time in the linter choice, which testing library I have to use, what’s the good editorconfig configuration, etc. I want to focus on the business needs as soon as possible.

Have you ever encountered teams that use many functions (GCP Cloud Functions, AWS Lambda, Azure Functions)?

How many of them use templating to create the function skeleton?

Templates, an investment that generates a series of uniform deployments

Exemples in this story are based on a simple hello world function written in Python and deployed on Google Cloud Platform, but if you have read my previous stories, you will understand how it will be useful to generate Terraform modules skeleton, Puppet/Chef/Ansible declarations, CLI, or API servers as well!

I chose to use Cookiecutter because it’s very easy to start.
There exist a lot of stories to compare cookiecutter with Yeoman, GitHub templates, etc. If you continue to read this story, you will see that it’s not another fork ;)

Let’s build our hello function !

After this long introduction, let’s deep dive into the code.

Sources for this story can be found in my github repository :

GitHub - jnahelou/cookiecutter-gcf-python: Python Google Cloud Function generator. Base for medium…

1. Identify what you want to do.

• I would like to create a simple Python HTTP Cloud Function with PyLint, editorconfig, gcloudignore, and a code skeleton
• I also included a Pytest sample to start faster
• Sometimes, I want to configure continuous deployment and tagging using GitHub action and semantic-release
• And it would be great if I can configure the monitoring !

2. Create your template

• Create a new folder, this folder will contain the configuration of our template.
• The most important file is the cookiecutter.json.
  This json file is used by cookiecutter to configure all inputs/variables
• Write your template : keep in mind that cookiecutter use Jinja2 interpolation language.
  You can switch from text to interpolation using : {{ }} or {% %} or using a multi-line statement :

{% row %}
somthing_to_evaluate
{% endrow %}

In my code, you will see that I use the {%- expr -%} format to remove blank lines.

The code generated by cookiecutter is created in the {{cookiecutter.project_slug}} folder. Variables will be replaced by the value defined by the user or defined in the cookiecutter.json config file.

You can see the same files as the tree used in step 1 but the content has been refactored to use Jinja2 variables.

Let’s deep dive into the github action workflow ci.yml

Using variables previously declared, I can add or remove steps in my pipeline.

I can also choose what’s the tool I will use to monitor my function :

3. Add hooks to execute pre/post actions

In the documentation you can find 2 kinds of hooks :

• hooks/pre_gen_project.py: runs before the template rendering.
  Useful to validate the variable content, you can add a set of regular expressions that variables must match
• hooks/post_gen_project.py: runs after the template rendering.
  You can easily add notifications, webhooks, do file suppressions, execute commands, …

4. Template rendering

Like Terraform modules, you can use local or remote templates.
Once the access method is defined (here I used github link), you can run the command line to generate the project :

$ cookiecutter https://github.com/jnahelou/cookiecutter-gcf-python.git

How to fill variables ?

Variables will be prompted and you can use stdin to fill them.
Of course if you prefer to use a variable definitions file, you can set the file path in --config-file argument or COOKIECUTTER_CONFIG environment variable.

At the end, take a look at the two ~/.cookiecutter generated. The cookiecutter_replay folder saves input of previous deployments while cookiecutters contains the git clone of templates you already used.

Are you ready to start ?

Feel free to follow the official tutorial : https://cookiecutter.readthedocs.io/en/latest/tutorials.html#create-your-very-own-cookiecutter-project-template

There are so many GitHub repositories you can use to improve your templates !