How to create a custom node in the Node-RED
Let’s start this article with a brief introduction to Node-RED. It’s a GUI-based Low-code programming platform for wiring together edge devices, IoT services, and APIs in new and exciting ways. As it is built on Node.js, we can run it on the local computer or low-cost hardware such as the Raspberry Pi as well as in the cloud to create event-driven applications. It uses flow-based programming that allows users to describe applications behavior by simply connecting high-level components, called “Node,” as building blocks. We can define functions and use pre-built modules in the nodes to process the input data they collect. We can easily install Node-RED on many kinds of platforms. For a definite guide about the Node-RED, I recommend visiting the official documentation and joining the Node-RED Forum or Slack channel.
Node-RED has mainly two logical parts: a GUI-based development environment called Flow Editor (or only Editor), which runs on a web browser, and a backend execution environment that runs on Node.js called Runtime. The Editor helps create and define the application flow, while actual code execution is done on the Runtime side. Let’s take a look at the Node-RED flow editor environment:
The main features of the Flow Editor are as follows:
- Node: The main building block of Node-RED applications, they represent well-defined pieces of functionality.
- Flow: A series of nodes wired together that represent the series of steps messages pass through within an application.
- Palette: A collection of nodes that are available within the Editor that you can use to build your application.
- Workspace: The main area where nodes are dropped and wired together via flows to define the application.
- Sidebar: It contains panels that provide various tools, such as configuration parameter settings, debugger display, information about nodes, and their help.
- Main menu: Flow deletion, definition import/export, project management, etc.
- Deploy Button: Press this button to deploy your apps once you’ve edited them.
Creating a flow in Node-RED
- We start by drag and drop our required nodes in the workspace from the palette. Here, each node has either zero or one input port and zero or more output ports.
- Configure all the needed parameters in the node.
- Wiring these nodes and build flows as per the application requirement.
- When we execute a flow using the deploy button, each node accepts the input message from the input port and produces the output message by processing this input, and that will be outputted from the output port.
In this whole process, we have used pre-built nodes and created flow among them to define the application. We didn’t require any programming in it, and this is the major feature of low code programming platforms like Node-RED.
Requirement of custom nodes
Node-RED has a wide variety of core nodes, each with their specific purpose that we can use to build our application. It includes common nodes(inject, debug, comment, etc.), network nodes (MQTT, HTTP, WebSocket, etc.), or parser nodes (CSV, HTML, JSON, etc.). But sometimes, we lack the functionalities of these nodes as per the requirements. So for that, we can also develop a custom node and publish it independently as Open Source. We can find many custom nodes created by other community users and published as npm modules on the public npm repository and the Node-RED Flow Library. The published node can be installed and used from the Node-RED editor. Let’s discuss the step-by-step process of creating a custom node for the Node-RED, which is the main goal of this article.
Before starting it, I want to inform you about some general principles to follow when creating new nodes. These reflect the approach taken by the core nodes and help provide a consistent user experience.
Node-RED nodes consist of two files: a JavaScript file that defines processing and an HTML file that provides a UI such as a setting screen.
- In the JavaScript file, the processing of the node is defined as a function. An object that contains node-specific properties is passed to this function.
- The HTML file defines the node’s properties, edit dialog and help text. The input values entered in the node properties will be called in the JavaScript file and processed to generate the output.
A package.json file is used for packaging it all together as an npm module.
We will learn how to create a custom node by creating a weather node as an example. This node will take the name of any place as an argument and return the weather data such as temperature and weather conditions.
Step1(Basics): First of all, we will create a directory where we will develop our code. In this directory, we will create weather.js and weather.html files. To generate a standard package.json file, we will run npm init command, which will ask a series of questions to help create the initial content for the file, using sensible defaults where it can. When it is prompted, give the name of our node module as node-red-contrib-weather-data.
We use node-red-contrib- as a prefix for the name to make it clear the Node-RED project does not maintain them. Alternatively, any name that doesn’t use node-red as a prefix can be used.
Then we must add node-red section in the package.json file. This tells the Runtime what node files the module contains.
{
"name": "node-red-contrib-weather-data",
...
"node-red": {
"nodes": {
"weather-data": "weather.js"
}
}
}If the node has external module dependencies, they must be included in the dependencies section of its package.json file. For our weather node, we will use the request npm module to make HTTP calls. To install the request module, we should run the following command:
npm install requestafter running it, we can see the request module has been added as dependencies in the package.json file, or we can add it manually(with the current version number) as shown below:
{
...
"dependencies": {
"request": "^2.88.2"
},
...
}Same as this process, we can also install other required npm modules for our custom node.
Step2(JavaScript file): Now, we will write our JavaScript code in the weather.js file. In this file, we will define the functionalities of our node as mentioned below:
// to provide the module access to the Node-RED runtime API
module.exports = function(RED) {
const request = require('request'); // to get latitude and longitude coordinates of the given
location name.
function geocode (address, access_token, callback) {
const url = 'https://api.mapbox.com/geocoding/v5/mapbox.
places/' + encodeURIComponent(address) +
'.json?access_token=' + access_token +
'&limit=1';
request({ url, json: true }, (error, {body}) => {
if (error) {
callback('Unable to connect to location service!',
undefined);
} else if (body.features.length === 0) {
callback('Unable to find location. Try another
search.', undefined);
} else {
callback(undefined, {
latitude: body.features[0].center[1],
longitude: body.features[0].center[0],
location: body.features[0].place_name
})
}
})
} // to get the weather information from the coordinates of the
location provided
function forecast(latitude, longitude, access_key, callback) {
const url = 'http://api.weatherstack.com/current?
access_key=' + access_key + '&query=' +
latitude + ',' + longitude;
request({ url, json: true}, (error, {body}) => {
if (error) {
callback("Unable to connect to weather service.",
undefined);
}
else if (body.error) {
callback("Unable to find location.", undefined);
}
else {
callback(undefined, {
"weatherDescription":
body.current.weather_descriptions[0],
"actualTemp": body.current.temperature,
"feelsLikeTemp": body.current.feelslike
})
}
})
}
// weather-data node
function weatherNode(config) {
RED.nodes.createNode(this, config);
const node = this;
node.location = config.location;
const mapbox_key = this.credentials.mapbox_key;
const weatherstack_key = this.credentials.weatherstack_key;
node.on('input', function(msg) {
let locationName = node.location ? node.location :
msg.payload; if (!locationName || !mapbox_key || !weatherstack_key) {
node.error("Input data not provided.", msg)
} else {
geocode(locationName, mapbox_key, (error, {latitude,
longitude, location} = {}) => {
if (error) {
node.error(error, msg);
} else {
forecast(latitude, longitude,
weatherstack_key, (error, forecastData) => {
if (error) {
node.error(error, msg);
} else {
msg.payload = {
place: location,
weather:
forecastData.weatherDescription,
actualTemperature:
forecastData.actualTemp,
feelsLikeTemperature:
forecastData.feelsLikeTemp
}
node.send(msg);
}
});
}
})
}
})
}
// to register the weatherNode function as a node
RED.nodes.registerType("weather-data", weatherNode, {
credentials: {
mapbox_key: {type: "password"},
weatherstack_key: {type: "password"}
}
});
}
The node is wrapped as a Node.js module. The module exports a function that gets called when the Runtime loads the node on start-up. The function is called with a single argument, REDthat provides the module access to the Node-RED runtime API. In this code,
- The first function, ‘geocode,’ uses mapbox API to get the latitude and longitude coordinate information about the provided location name by the user. (Access token required from the mapbox website as a credential)
- The second function, ‘forecast,’ uses weatherstack API to get all the weather information (current temp, feels like temp, humidity, etc.) through the location’s latitude and longitude coordinates. (Access key required from the weatherstack website as a credential)
- The node itself is defined by a third function,
weatherNodethat gets called whenever a new instance of the node is created. The function calls theRED.nodes.createNodefunction to initialize the features shared by all nodes. After that, the node-specific code lives. - It is passed an object containing the node-specific properties (e.g., location name) set in the flow editor. In this instance, the node registers a listener to the
inputevent, which gets called whenever a message arrives from the upstream nodes in a flow. We are also getting credentials for mapbox and weatherstack API. We will talk about it later in Step5. - Then we check for the error if the given location name is empty or incorrect. If there is an error in the flow, it will trigger a
catch nodepresent on the same tab, allow us to build flows to handle the error. - If there is no error, we will get the weather information using geocode + forecast function and then calls the
sendfunction to send a message to the output port. At the last part of the code, theweatherNodefunction is registered with the Runtime using the name for the node,weather-data.
Step3(HTML file): Now, let’s write our HTML code in the weather.html file. This is responsible for the UI and design of the node we create. Following is the code for the HTML file:
<!-- register weather-data node -->
<script type="text/javascript">
RED.nodes.registerType('weather-data',{
category: 'weather',
color: '#a6bbcf',
defaults: {
name: {value:""},
location: {value: ""}
},
credentials: {
mapbox_key: {type: "password"},
weatherstack_key: {type: "password"}
},
inputs:1,
outputs:1,
icon: "font-awesome/fa-cloud",
label: function() {
return this.name||"weather-data";
}
});
</script><!-- Define setting UI for weather-data node -->
<script type="text/html" data-template-name="weather-data">
<div class="form-row">
<label><i class="fa fa-tag"></i><span data-i18n="weather-
data.label.name"></span></label>
<input type="text" id="node-input-name" data-i18n="
[placeholder]weather-data.label.name">
</div> <div class="form-row">
<label><i class="fa fa-map-marker"></i><span data-
i18n="weather-data.label.location"></span></label>
<input type="text" id="node-input-location" data-i18n="
[placeholder]weather-data.label.location">
</div> <div class="form-row">
<label for="node-input-mapbox_key"><i class="fa fa-globe">
</i><span data-i18n="weather-data.label.mapbox_key"></span>
</label>
<input type="password" id="node-input-mapbox_key" data-i18n=
"[placeholder]weather-data.label.mapbox_key_placeholder">
</div> <div class="form-row">
<label for="node-input-weatherstack_key"><i class=
"fa fa-thermometer-three-quarters"></i><span data-i18n=
"weather-data.label.weatherstack_key"></span></label>
<input type="password" id="node-input-weatherstack_key"
data-i18n="[placeholder]weather-data.label.
weatherstack_key_placeholder">
</div>
</script>
Mainly HTML file contains three distinct parts, each wrapped in its own <script> tag. But here in the code, we have defined only two parts:
- The first part defines the main node definition that is registered with the Editor. This defines things such as the palette category, the editable properties (
defaults) and what icon to use. In this partRED.nodes.registerTypefunction takes two arguments; the type of the node and its definition. The node type must match the value used by the call toRED.nodes.registerTypein the corresponding.jsfile. - The second part, the edit template, defines the content of the edit dialogue for the node. It is defined in a script of type
text/htmlwithdata-template-nameset to the type of the node. Heredata-i18nis for internationalisation of our node. This will be discussed in the next step. - The third part, which is not mentioned in the code, is the help text that gets displayed in the Info sidebar tab. It is defined in a script of type
text/htmlwithdata-help-nameset to the type of the node. We will talk about it in the next step.
Step4(Internationalisation): For better and convenient use of our node, we should provide translated content of different terms for our node to reach as many international users as possible. For this, we need to define a corresponding set of message catalogs, edit templates for the node setting UI, and help texts. We will add individuals .html and .json file for each of the language support in the current directory as shown below:
node-red-contrib-weather-data/locales/__language__/weather.json
node-red-contrib-weather-data/locales/__language__/weather.htmlThe __language__ part of the path identifies the language the corresponding files provide. By default, Node-RED uses en-US. We will add two folders in the locales folder, en-US (for the English language) and ja (for the Japanese language). In the .html file, we will define the help text for our node which was mentioned in the third part of the main weather.html file. This will provide translated versions of the node’s help text that gets displayed within the Info sidebar tab of the Editor. In the .json file, we will define the content of the edit dialogue, which will be used in our main weather.html file. Any HTML element provided in the node template of weather.html file can specify a data-i18n attribute to provide the message identified to use. By default, the text content of an element is replaced by the message identified. Following are the files for the English language support.
.json file:
{
"weather-data": {
"label": {
"name": " Name",
"location": " Location",
"mapbox_key": " mapbox",
"mapbox_key_placeholder": "Enter mapbox access token",
"weatherstack_key": " weatherstack",
"weatherstack_key_placeholder": "Enter weather access
key"
}
}
}.html file:
<!-- Define help text of the node in the English language -->
<script type="text/html" data-help-name="weather-data"> <p>Give the weather information about the provided location.</p> <h3>Inputs</h3>
<dl class="message-properties">
<dt class="optional">Name <span class="property-
type">string</span></dt>
<dd>If not set in the node configuration, this property will
be sets as the "weather-data" to read.</dd>
<dt>Location <span class="property-type">string</span></dt>
<dd>If not set in the node configuration, this property will
be sets as the msg.payload provided to it.</dd>
<dt>mapbox Access Token <span class="property-
type">string</span></dt>
<dd>It can be generated from this <a
href="https://www.mapbox.com/">website</a></dd>
<dt>weatherstack Access Key <span class="property-
type">string</span></dt>
<dd>It can be generated from this <a
href="https://weatherstack.com/">website</a></dd>
</dl> <h3>Outputs</h3>
<dl class="message-properties">
<dt>payload <span class="property-type">object</span></dt>
<dd>The contents of the output will be an array object of
different weather properties.</dd>
</dl> <h3>Details</h3>
<p>If the given location name is not an appropriate name or
location field in node configuration and msg.payload both are
empty, then output msg will be an error.
</p>
<p>The output msg will contain an object of <code>place</code>,
<code>weather</code>, <code>actualTemperature</code>, and
<code>feelLikeTemperature</code>.
</p></script>
Following are the files for the Japanese language support:
.json file
{
"weather-data": {
"label": {
"name": " 名前",
"location": " 場所",
"mapbox_key": " mapbox",
"mapbox_key_placeholder": "入力 mapbox access token",
"weatherstack_key": " weatherstack",
"weatherstack_key_placeholder": "入力 weather access key"
}
}
}.html file:
<!-- ノードのヘルプテキストを日本語で定義する -->
<script type="text/html" data-help-name="weather-data"> <p>入力された場所の気象情報を提供します。</p><h3>Inputs:</h3>
<dl class="message-properties">
<dt class="optional">名前 <span class="property-
type">string</span>
</dt>
<dd>ノード・コンフィグレーションで設定されていない場合、このプロパティが
"weather-data"の名称として設定されます。</dd>
<dt>場所 <span class="property-type">string</span></dt>
<dd>ノード・コンフィグレーションで設定されていない場合、このプロパティ
がノードのmsg.payloadとして設定されます。</dd>
<dt>mapbox Access Token <span class="property-
type">string</span></dt>
<dd>ここで生成できます<a href="https://www.mapbox.com/">ウェブ
サイト</a></dd>
<dt>weatherstack Access Key <span class="property-
type">string</span></dt>
<dd>ここで生成できます<a href="https://weatherstack.com/">ウェ
ブサイト</a></dd>
</dl> <h3>Outputs:</h3>
<dl class="message-properties">
<dt>Payload <span class="property-type">object</span></dt>
<dd>出力の内容は、異なる気象プロパティの配列オブジェクトになります。
</dd>
</dl><h3>Details:</h3>
<p>指定されたロケーション名が、ノード・コンフィグレーションの適切な名前または
ロケーショ ン・フィールドではなく、msg.payload が共に空の場合、出力
msg はエラーとなります。
</p>
<p>出力されるmsgは、<code>place</code>、<code>weather</code>、
<code>actualTemperature</code>
<code>feelLikeTemperature</code>のオブジェクトを含みます。
</p></script>
So now, the user will get the node details as per the system language configuration. This will reduce the complexity of how to use our node.
Step5(Credentials): A node may define a number of properties as credentials. These properties are stored separately from the main flow file and do not get included when flows are exported from the Editor. Here, our node's access token for the mapbox API and the access key for the weatherstack API are password-type credentials. For these two, we have to add a separate credentials property to the node’s definition as shown in the weather.html file’s first part. Next, we need to add the edit template for the setting UI. In the weather.js file’s RED.nodes.registerType part, we must include the credentials, and then we can access the credentials in the Runtime weatherNode function using the credentials property.
Step6(Appearance): We can customize our node appearance in terms of three aspects; the icon, background color, and its label.
- The node’s icon is specified by the
iconproperty in the first part of the mainweather.htmlfile. The icon can be either a stock icon (by Node-RED), custom icon, or Font Awesome(FA) icon. We have used the ‘cloud’ FA icon, which is described asicon: “font-awesome/fa-cloud". - The node background color is specified by the
colorproperty in the mainweather.htmlfile ascolor: '#a6bbcf' - There are four label properties of a node;
label,paletteLabel,outputLabelandinputLabel. Here we have definedlabelproperty only. The value of the property can be either a string or a function. We have defined it as a function; it will get evaluated when the node is first loaded or after it has been edited. The function is expected to return the value to use as the label.
Step7(Installation): We have created all the required files for our custom node as described above. Now we can install it into our Node-RED Runtime. First of all, we will run npm install command in the main directory of our custom node module. This will install a package and any packages that it depends on. Then by running pwd command, we can get the exact path (e.g., /Users/[username]/node-red-contrib-weather-data) of our node module. We will copy this path and then go to the Node-RED user directory, typically ~/.node-red. At this place, we will run the following command to install our custom node to the Node-RED library:
npm install <path of the node module>// Example:
npm install /Users/[username]/node-red-contrib-weather-data
We can see our installed custom node in the editor palette and its files in the ~/.node-red/node_modules folder.
We will create a flow using weather-data node as shown in the figure. We have added a catch node separately to catch the errors while running the flow. In the edit dialogue box of the weather-data node, we will configure all the details. When we inject the inject node, we will get the output object in the debug panel of the Sidebar as shown below:
{"place":"Tokyo Prefecture, Japan","weather":"Partly cloudy","actualTemperature":26,"feelsLikeTemperature":27}If we leave the Location name empty or enter the wrong name (e.g., “!”) It will give the following error:
// Error while no location name
{"message":"Input data not provided.","source":{"id":"fa31457292370b9c","type":"weather-data","count":1}}// Error while wrong location name
{"message":"Unable to find location. Try another search.","source":{"id":"fa31457292370b9c","type":"weather-data","count":1}}
Step8(Testing): For the unit testing of our custom node, we can use the npm module called node-red-node-test-helper. This test-helper, start the Node-RED Runtime, load a test flow, and receive messages to ensure our node code is correct. Node-RED is also required by the test-helper as a peer dependency, meaning it must be installed along with the test-helper itself. To install this, we will run the following command in the root directory of our node module:
npm install node-red-node-test-helper node-red --save-devWe will use mocha JavaScript test framework for our testing. We will also install it by using the npm install mocha --save-dev command. After running these two commands, we can see the mocha, test-helper module and Node-RED in our package.json file’s devDependencies part:
{
...
"devDependencies": {
"mocha": "^9.1.1",
"node-red": "^2.0.6",
"node-red-node-test-helper": "^0.2.7"
}
...
}To run our tests, we will add a test script to our package.json file in the scripts section. To run all of the files with the _spec.js prefix in the test directory:
{
...
"scripts": {
"test": "mocha \"test/**/*_spec.js\""
},
...
}This will allow us to use npm test on the command line to run all the test files and see the results for it.
For unit testing, first we need to set environment variables for mapbox API and weatherstack API access keys. We will use dotenv npm module that loads environment variables from a .env file into process.env. We will install this module by running npm install dotenv --save-dev command and then add .env file in the root directory of our node module.
# Set your API connection information here
MAPBOX=[mapbox access token(without brackets)]
WEATHERSTACK=[weatherstack access key(without brackets)]To add a unit test to the weather node, we will add a test folder to our node module package containing a file named weather_spec.js. In that file, we will add the following code:
const helper = require("node-red-node-test-helper");
const dotenv = require('dotenv');
const weatherNode = require("../weather.js");dotenv.config();helper.init(require.resolve('node-red'));describe('weather-data Node', function () {
beforeEach(function (done) {
helper.startServer(done);
}); afterEach(function (done) {
helper.unload();
helper.stopServer(done);
}) it('should be loaded', function (done) {
const flow = [{id: "n1", type: "weather-data", name:
"weather-data"}];
helper.load(weatherNode, flow, function() {
const n1 = helper.getNode("n1");
try {
n1.should.have.property('name', 'weather-data');
done();
} catch(err) {
done(err);
}
});
}) it('should check for empty location name', function (done) {
const flow = [
{ id: "n1", type: "weather-data", name: "weather-data",
wires: [["catchNode"]] },
{ id:"catchNode", type: "catch", scope:null, uncaught:
false, wires: [["n2"]]},
{ id: "n2", type: "helper" }
]
helper.load(weatherNode, flow, function () {
let n1 = helper.getNode("n1");
let n2 = helper.getNode("n2"); n2.on("input", function (msg) {
try {
msg.should.have.property('error');
msg.error.should.have.property('message','Input
data not provided.');
done();
} catch (err) {
done(err);
}
})
n1.receive({payload: ""})
})
}) it('should check for wrong location name', function (done) {
const flow = [
{ id: "n1", type: "weather-data", name: "weather-data",
wires: [["catchNode"]] },
{ id:"catchNode", type: "catch", scope:null, uncaught:
false, wires: [["n2"]]},
{ id: "n2", type: "helper" }
]
const credentials = { "n1": {
mapbox_key: process.env.MAPBOX,
weatherstack_key: process.env.WEATHERSTACK
}}
helper.load(weatherNode, flow, credentials, function () {
let n1 = helper.getNode("n1");
let n2 = helper.getNode("n2"); n2.on("input", function (msg) {
try {
msg.should.have.property('error');
msg.error.should.have.property('message','Unable
to find location. Try another search.');
done();
} catch (err) {
done(err);
}
})
n1.receive({payload: "!"})
})
})
})
These tests will check our node module for three things: If the node is loaded correctly or not in the Runtime, If we don’t input the location name, it should give an error, and if we enter the wrong location name, it should also give an error. These tests load the node into the Runtime using helper.load supplying the node under test and a test flow.
Step9(Publishing): We have created our custom node, and now we can publish it to the Node-RED library so that other users can also use it. To publish our node module, first we need to publish it to the public npm repository. Before publishing the node, there are few things that need to be taken care of. The name of the node module should follow the project’s naming guidelines provided by the Node-RED. It should contain a README.md file that describes what our node does and how to use it. We also need to add "node-red" in its list of keywords in a package.json file. After that, we need to create an account on the public npm repository. Then we will log in to our npm account from the terminal using npm login command. After that, from the root directory of our node module, we will run npm publish command. This will publish our node module on the public npm library. Once the node module is published to npm, the node module can be added to the Flow Library using this form. So by this process, any Nodes published correctly to npm will appear on the Node-RED Library.
That’s it. We have created our custom node from scratch and published it to the Node-RED Library with nine easy steps. Same as this, we can create any other node as per the requirements. You can get more detailed information about this process on the Node-RED Documentation. and also get in touch with other users to discuss different topics on the Node-RED Forum or Slack channel. Thank you for READING.
