Sharing Variables Between Python & Node.js in Jupyter Notebooks

They live apart and speak different languages, but these variables hold common values

I created the pixiedust_node project to allow Jupyter Notebooks to contain a mixture of Python and Node.js cells. It’s installed with a simple command:

!pip install pixiedust_node
import pixiedust_node

After that, use %%node at the top of cells that are to contain JavaScript:

%%node
console.log("Today's date is", new Date());

Notebooks can use any npm package by using the npm.install Python command:

npm.install( ('request', 'request-promise') )

And then using the library in a Node.js cell:

%%node
const request = require('request-promise');
request.get('http://www.google.com').then(print);

Enabling notebooks to mix Node.js and Python is convenient for collaboration, but moving data between the different cells assumed the presence of a separate, external data source. No longer!

Share Python and Node.js variables in multiple directions. Photo by Brendan Church on Unsplash

Sharing variables (shariables?)

With the release of pixiedust_node v0.2 you can now share data interchangeably between Python and Node.js cells. Declare some variables in Python:

a = 'hello'
b = 2
c = False
d = {'x':1, 'y':2}
e = 3.142
f = [{'a':1}, {'a':2}, {'a':3}]

And they will magically appear in your Node.js cells:

%%node
console.log(a, b, c, d, e, f);
// hello 2 false {y:2, x:1} 3.142 [{ a:1 }, { a:2 }, { a:3 }]

Similarly, global variables declared in Node.js cells:

%%node
var i = 'world';
var j = 3;
var k = true;
var l = { z:99 }
var m = 2.7182;
var n = ['apples', 'oranges'];

Will be automatically copied to Python equivalents:

print(i,j,k,l,m,n)
# (u'world', 3, True, {u'z': 99}, 2.7182, [u'apples', u'oranges'])

If you wish transfer data from Node.js to Python from an asynchronous callback, make sure you write the data to a global variable:

%%node
var googlehomepage = '';
request.get('http://www.google.com').then(function(data) {
  googlehomepage = data;
  print('Fetched Google homepage');
});

How to upgrade

Simply run:

!pip --upgrade pixiedust_node

Why do this?

PixieDust already allows the sharing of data from Python cells to Scala cells. This feature is now extended to Node.js cells too.

The Node.js library ecosystem is extensive. Perhaps you need to fetch data from a database and prefer the syntax of a particular Node.js npm module. You can use Node.js to fetch the data, move it to the Python environment, and convert it into a Pandas or Spark data frame for aggregation, analysis and visualisation.

PixieDust and pixiedust_node give you the flexibility to mix and match Python and Node.js code to suit the workflow you are building and the skill sets you have in your team.

Mixing Node.js & Python code in the same notebook is a great way to integrate the work of your software development and data science teams to produce a collaborative report or dashboard.

Sounds interesting? Try it out.