Visualizing Graphs in 3D with WebGL

While looking for efficient graph visualization libraries for large scale rendering, I came across 3d-force-graph, a really neat wrapper around three.js for graph visualization. Check out that repository after reading this, they have many more examples and settings to explore.

Michael Hunger
Jul 23, 2018 · 6 min read

I was already aware of and impressed by three.js, especially the 3d and WebGL capabilities. I had seen and done some graph visualization using it years ago, but 3d-force-graph packages three.js nicely with a Graph API and adds useful options to quickly get good looking results.

If you want to try the different approaches directly, I put them into a GitHub Repository with live examples using the RawGit Service to serve HTML pages and assets directly from the repository.

Datasets

I started by using the Game of Thrones interaction graph, that my colleague Will Lyon wrote about here and which you can create yourself by running :play got in your Neo4j Browser.

Besides the basic graph of characters, it also has interaction “weights” on the relationships and the guide adds some additonal metrics like a pageRank property that we want to use in the visualization.

To take it to a larger dataset, you can explore the ICIJ’s Paradise Papers database, which is available as a Neo4j Sandbox, using this visualization.

But let’s get started.

Accessing the Data

As I outlined in the first article of this series, Neo4j’s Cypher over the binary Bolt protocol, can pull up to 1M records per second over the wire, enough to feed our graph visualization interactively with enough data.

In our experiments, we use again the Neo4j Javasscript Driver to query the graph for data and render it in the 3d-graph.

Please note that the JS driver uses a custom Number object, which we have to turn into JS integers with value.toNumber().

Thee following, interactive pages load 5000 relationships from your graph at bolt://localhost, you might need to set auth (default is user: neo4j, password: test)

Basic Loading

Basic loading here we’re just using the node ids because that is the fastest way to extract the graph structure.

Load relationship list

MATCH (n)-->(m) RETURN id(n) as source, id(m) as target LIMIT $limit

We load that data using the Javasscript driver as demonstrated before and transform it into a data structure in gDatathat 3d-force-graph understands.

const driver = neo4j.v1.driver("bolt://localhost", neo4j.v1.auth.basic("neo4j", "test"));
const session = driver.session();
const start = new Date()
session
.run('MATCH (n)-->(m) RETURN id(n) as source, id(m) as target
LIMIT $limit', {limit: 5000})
.then(function (result) {

We need one array with links from a source id to a target id. And one array for nodes containing our data to render. In the initial example that will just be the ids. We compute those id’s from the unique Set of source and target ids of our relationship-list.

const links = result.records.map(r => 
{ return {source:r.get('source').toNumber(),
target:r.get('target').toNumber()}});
session.close();
console.log(links.length+" links loaded "+(new Date()-start)+" ms.")
const ids = new Set()
links.forEach(l => {ids.add(l.source);ids.add(l.target);});
const gData = { nodes: Array.from(ids).map(id => {return {id:id}}),
links: links}

Then we just need to use that to initialize our 3d-graph.

ForceGraph3D()(document.getElementById('3d-graph')).graphData(gData)

The full code example can be seen and read here:

Render Example | Code

Incremental Loading

With incremental loading, we can add each row from the query result to the graph as it arrives. Especially with large results, that improves the responsiveness of the visualization and the user immediately sees feedback that they can interact with. Also it’s fun to watch the new nodes and relationships popping into existence.

The main difference is that we now create the 3d-graph upfront and then update its graphData with each arriving row, adding our new entries using the Array spread operator.

result.records.forEach(r => {
const { nodes, links } = Graph.graphData();
const link={source:r.get('source').toNumber(),
target:r.get('target').toNumber()}
Graph.graphData({
nodes: [...nodes, { id:link.source }, { id: link.target}],
links: [...links, link]
});
});

This is best shown in action, so test it out below.

Render Example | Code

Color and Caption

Of course a bland graph excites no one, so let’s some color and information. That requires us to provide that info for the nodes, so besides the id, we also provide a label for coloring and a caption for each node. color by label and text caption on hover

MATCH (n)-->(m)
RETURN
{ id: id(n), label:head(labels(n)), caption:n.name } as source,
{ id: id(m), label:head(labels(m)), caption:m.name } as target
LIMIT $limit

Fortunately 3d-force-graph comes with some hand option for auto-coloring based on an attribute, so we don’t have to handle colors or palettes manually. For the caption we just provide text to render in the nodeLabel callback and for feedback we also change the cursor while hovering over a node.

const Graph = ForceGraph3D()(elem)
.graphData(gData)
.nodeAutoColorBy('label')
.nodeLabel(node => `${node.label}: ${node.caption}`)
.onNodeHover(node =>
elem.style.cursor = node ? 'pointer' : null);

Render Example | Code

This time, we use the Paradise Papers as an example as it has more different labels (five) than the Game of Thrones interaction graph. Something that becomes quickly apparent in the 3d visualization are the clusters that form around officers (people in management roles for shell companies) and intermediaries (law firms and banks creating shell companies).

Weights for Node and Relationship Sizes

After running a graph algorithm like PageRank on our data, the nodes get a rank score as a weight. The relationships of the GoT graph already had interaction weights, both of which we can now render in addition to colors and captions.

We also want to color relationships by type. We use log(weight) for relationships as they would grow too thick otherwise.

So we return then as additional information from our graph query.

MATCH (n)-[r]->(m)
RETURN { id: id(n), label:head(labels(n)), caption:n.name,
size:n.pagerank } as source,
{ id: id(m), label:head(labels(m)), caption:m.name,
size:m.pagerank } as target,
{ weight:log(r.weight), type:type(r)} as rel
LIMIT $limit

We use the size attribute for nodes with the nodeVal method and the weight on relationships with the linkWidthmethods. For coloring the relationship-type we call linkAutoColorBy.

const Graph = ForceGraph3D()(elem)
.graphData(gData)
.nodeAutoColorBy('label')
.nodeVal('size')
.linkAutoColorBy('type')
.linkWidth('weight')
.nodeLabel(node => `${node.label}: ${node.caption}`)
.onNodeHover(node =>
elem.style.cursor = node ? 'pointer' : null);

Render Example | Code

Particles & Cluster Coloring

If we also ran a community detection or clustering algorithm like Louvain, we can use the cluster information for coloring nodes and relationships instead, making immediately visible to which cluster an entity belongs. So we return the louvain property as community (or cluster) entry for nodes and for relationships return the "smaller" community number as the one the relationship belongs to.

MATCH (n)-[r]->(m)
RETURN { id: id(n), label:head(labels(n)), community:n.louvain,
caption:n.name, size:n.pagerank } as source,
{ id: id(m), label:head(labels(m)), community:n.louvain,
caption:m.name, size:m.pagerank } as target,
{ weight:r.weight, type:type(r),
community:case
when n.community < m.community then n.community
else m.community end} as rel
LIMIT $limit

As an additional, fancy feature we replace the solid relationships with a particle stream instead, making them less obstructive.

In this case we use the original weight property, (not log(weight)) as it represents the number of interactions, i.e. the number of particles traveling between the two characters.

const Graph = ForceGraph3D()(elem)
.graphData(gData)
.nodeAutoColorBy('community')
.nodeVal('size')
.linkAutoColorBy('community')
.linkWidth(0)
// number of particles
.linkDirectionalParticles('weight')
.linkDirectionalParticleSpeed(0.001) // slow down
.nodeLabel(node => `${node.label}: ${node.caption}`)
.onNodeHover(node =>
elem.style.cursor = node ? 'pointer' : null);

Render Example | Code

That’s it for today from me.

There are many more options in 3d-force-graph, please check it out, play around and let me know in the comments what you came up with.

Big thanks to its author, Vasco Asturiano for creating such a great and easy to use library.

Neo4j Developer Blog

Developer Content around Graph Databases, Neo4j, Cypher…

Michael Hunger

Written by

A software developer passionate about teaching and learning. Currently working with Neo4j, GraphQL, Kotlin, ML/AI, Micronaut, Spring, Kafka, and more.

Neo4j Developer Blog

Developer Content around Graph Databases, Neo4j, Cypher, Data Science, Graph Analytics, GraphQL and more.

More From Medium

More from Neo4j Developer Blog

More from Neo4j Developer Blog

More from Neo4j Developer Blog

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade