Commands to install and run Luigi

Luigi is a Python (2.7, 3.6, 3.7 tested) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.

Installation

In this step, you will create a clean sandbox environment for your Luigi installation.

First, create a project directory. For this tutorial luigi-demo:

mkdir luigi-demo

Navigate into the newly created luigi-demo directory:

cd luigi-demo

Create a new virtual environment luigi-venv:

python3 -m venv luigi-venv

And activate the newly created virtual environment:

. luigi-venv/bin/activate

You will find (luigi-venv) appended to the front of your terminal prompt to indicate which virtual environment is active:

Output:

(luigi-venv) username@hostname:~/luigi-demo$

Using the command below you can install the stable version of Luigi.

pip install luigi

To Run

To execute the code, type the following on your command line:

Local Schedule

python -m luigi — module file_name TaskName — local-schedule

Central Schedule

luigid — port 8082 > /dev/null 2> /dev/null &
python -m luigi — module file_name TaskName

Thanks for reading!!