Triggering n8n workflow from Airflow DAG

Both Airflow (https://airflow.apache.org/) and n8n (https://n8n.io/) are open-source workflow management platforms. While airflow allows to programmatically author the workflows in python, n8n provides a no-code/low-code option for creating workflows in a visual way through drag and drop and configuration capabilities. As n8n is based on nodeJs, any custom code can be written in JavaScript as part of workflow.

While both tools have their pros and cons, this article describes how we can bring both tools together to get best of both the worlds. Here we’ll learn about triggering n8n workflow from Airflow. However, before getting into that we need to understand few important concepts related to these tools.

Airflow DAG

DAG (Directed Acyclic Graph) is a collection of tasks that are connected via directed lines such that same task is not reached again while traversing the graph.

Airflow operators

Operators are one type of DAG tasks that allow integration with other services. Some common operators are listed below.

• BashOperator — Execute bash commands on the machine it runs on
• PythonOperator — Take python function as an input and call the same
• EmailOperator — Send emails using SMTP server configured
• SimpleHttpOperator — Make an HTTP request that can be used to trigger actions on a remote system
• MySqlOperator, SqliteOperator, PostgresOperator, MsSqlOperator, OracleOperator, JdbcOperator — Run SQL commands

n8n webhooks

n8n webhook node allows to create webhooks which can be used to receive data from apps and services when an event occurs. It is a trigger node, which means that it serves as the starting point for an n8n workflow. This allows several different services to connect to n8n and run a workflow when data is received.

Trigger n8n workflow from Airflow DAG

For this, we’ll make use of Airflow SimpleHttpOperator and n8n webhook. Step by step process is described below.

• Create webhook node in n8n, choose appropriate authentication mechanism and configure POST method and endpoint

• Create http connection in Airflow. Select connection type as HTTP and provide n8n host name

• Create SimpleHttpOperator task in Airflow DAG, pass the connection id created above, configure endpoint based on n8n webhook created above and the usual information such as method type, headers and payload

api_trigger_dependent_dag = SimpleHttpOperator(
    task_id=”api_trigger_dependent_dag”,
    http_conn_id=’n8n-webhook’,
    endpoint=’webhook/test-endpoint’,
    method=’POST’,
    headers={‘Content-Type’: ‘application/json’},
    data=json_body,
    dag=dag
    )

Make sure to configure the endpoint based on production url in n8n webhook node. Test url may need to be executed once in n8n to register the webhook.

When the Airflow DAG is executed, it will trigger n8n workflow via the webhook trigger node.

If you found value in this article, I invite you to heighten its impact — share it with your network, recommend it to friends and colleagues, and join me on this journey. Your support is invaluable. Additionally, I welcome your valuable feedback to enhance the content, ensuring it remains not only helpful but also accurate and engaging.