Shindig: Integrating Neo4j with Meteor
- ccorcos:neo4j
- Previous article: Shindig: An event discovery app built with Meteor.js, React.js, and Neo4j
Meteor is an isomorphic full-stack Javascript framework and its hands-down the easiest way to get started building apps. There’s very little boilerplate and for the most part, it just works.
Meteor ships with MongoDB and has all this awesome fancy stuff that reactively publishes and latency compensates the entire interaction with the database through Minimongo. But MongoDB is not a good choice for every app.
Mongo is a non-relational database. Its a great solution when all you want to do is store and retrieve documents with some basic queries. But Mongo is not meant for modeling relationships. Shindig involves a network of users who follow other users who star or host events. In fact, this Neo4j Cypher query will return the entire network which semantically follows the previous statement.
MATCH (u:USER)-[f:FOLLOWS]->(p:USER)-[s:HOSTS|STARS]->(e:EVENT)
RETURN *Neo4j also has a nice browser admin tool. Check out the result of this query.
To do relational queries in Mongo, you end up having to do a big $in query which will only get less and less performant over time.
On top of that, you’ll need to do reactive joins in your meteor publications, possibly using reywood:publish-composite or one of many other popular packages which gives you even worse performance with Mongo.
I’m also sorting events based on a popularity score which would require a Mongo aggregation pipeline query. These kinds of queries are non-reactive in regards to Meteor so you would have to use Meteor.methods — the equivalent of a REST API — to send raw data to the client.
So my point is, Mongo is just not a suitable database for this app.
Interfacing with Neo4j
My first challenge was simply connecting to Neo4j and running Cypher queries. Neo4j has a simple HTTP interface and I built a package ccorcos:neo4j to abstract away parsing the response and handling any errors. It also comes with a shell script to start and stop the database locally and save the database in your project’s .meteor/local/neo4j directory. This isolates the databases between projects just like the Meteor build tool does with Mongo. If only Meteor would give me some build cycle hooks so I wouldn’t have to manually start and stop Neo4j separately!
All you have to do is instantiate the database and then you can run queries on the server within a Fiber using Neo4j.query.
Neo4j = Neo4jDb() # defualts to localhost:7474
str = Neo4j.stringifygetFollowers = (userId) ->
Neo4j.query """
MATCH (u:USER)-[:FOLLOWS]->(:USER {id:#{str(userId)}})
RETURN u.id
"""
You can set neo4j_url in your settings.json which will automatically connect and instantiate a global Neo4j object, but you can also connect to multiple different Neo4j instances by calling with Neo4jDb with different urls.
When writing Cypher queries, you can use Neo4j.stringify to escape any strings and properly format objects (which don’t translate exactly to JSON).
Now we need to get this data to the client. We can use Meteor.methods but these queries won’t be reactive and then Meteor isn’t all that cool all of a sudden. So lets do that next.
