Python project template with automations for better coding

I got me coping an old project that I've already worked before a lot of time to start a new project, for the sake of having some automations that I used need and make my job easier.

This automations that I mentioned are regarding styling, checking and testing my codes, things that in my opinion can't be ignored if you want to become a better developer and write better codes.

My python project template can be found on my gihub on here: https://github.com/jjunior84/python-project-template.

I've tested it with python 3.9 and 3.10 and it's also required virtualenv library (this is the way I used to work).

Project structure organization

What will you find in this template:

make setup-env
├── .github
│ ├── config
│ │ ├── bandit.yaml
│ │ ├── flake8.conf
│ │ ├── mypy.ini
│ │ ├── pre-commit-config.yaml
│ │ └── pyproject.toml
│ ├── workflows
│ │ └── cicd.yml
│ └── requirements.txt
├── python_project_template
│ ├── __init__.py
│ └── dummy.py
├── tests
│ ├── __init__.py
│ └── test_dummy.py
├── .gitignore
├── LICENSE
├── Makefile
└── README.md

The project is using pre-commit, a library that’s coming with a client which function is to call Git hooks before your commit is performed, and if one of these hooks fails, your commit is not executed. Great, right?

expectations to see

Hooks that are set on .github/config/pre-commit-config.yaml:

• isort: sorts the imports alphabetically, and automatically separated into sections and by type.
• black: automatically formats your code and makes it compliant with PEP8 style coding convention.
• flake8: checks if your code is compliant with PEP8, the difference to black is that black formats what is there but never will add anything and flake8 checks and warns you what is wrong and/or missing.
• mypy: while flake8 checks code styling, mypy checks for code typing, it is a great complement to enforce you with development good practices.
• interrogate: checks for coverage of your code about Python function and class documentation, shows a summary and can enforce a minimum coverage.
• bandit: security check for your code, if you are not using libraries/versions that have some indication of security problems.

In the same folder where we have pre-commit config, you can also find the config files used for the helpers mentioned before.

In the .github folder, we also have the requirements.txt file used to set up a local environment with a fixed version of the libraries already mentioned and also pytest.

We have yet two another folders, one represents the project module and the other represents the tests module, both with only __init__ file.

CI/CD

The project template is using github actions to handle CI (continuous integration) and CD (continuous delivery), on workflow folder we have cicd.yaml file that describe how we want to execute our process when push commits.

Only ci job is set because the cd jobs depends on of your project, but as you can see we install the python version we need to work, the dependencies through requirements.txt file, lint (flake8, mypy, interrogate), security check (bandit), test (pytest).

Makefile

It’s an important file in the project template and deserves its section!

This file holds some useful target helpers and could and should be used to accelerate your development process when needs to validate your code. I will detail each one of them.

make setup-env

Setup a local environment, create a virtual environment on .venvfolder in the project, install all requirements, and install pre-commit with the hooks.

make style

Execute isort and black in folders with Python code, it ignores hidden folders (so .venvand .github folders are rightfully ignored).

make lint

Execute flake8, mypy and interrogate to check your code, the same strategy followed by style with folders.

make security-check

Execute bandit to ensure your project is safe from use libs marked as unsafe.

make test

Executes pytest over your folders, during execution create .artefact folder to store results

All this extra folders created to setup you local environment are already in .gitignore file, so don't worry of push them to your repository.