Additionally — for this demo I am using Eclipse Java Neon 4.6.1, and Neo4j community edition 3.2.0-alpha06. I am also using Maven for this project, which is included with Eclipse (you do not need any previous experience with Maven. I will walk you through adding dependencies.) This tutorial should work fine for other versions of Eclipse and Neo4j though.

Project Setup

To start — lets make a new java project in Eclipse add a package, class, and target folder.

Then right click on the project and select Configure > Convert to Maven Project.

Next we need to add Neo4j dependency. Open the ‘pom.xml’ file from the package explorer, select the ‘Dependencies’ tab at the bottom of the window, then select ‘Add’

Fill in the ‘Select Dependency’ menu as seen, if you are using a different version of Neo4j adjust the ‘Version’ value to match yours.

Now, to add Maven Dependencies to the build path. Right click on your project within the Package Explorer and select Build Path > Configure Build Path. Select Maven from the left panel, and enter ‘Maven Dependencies’ in the Active Maven Profiles textbox.

For the last step in project setup, lets add a folder within our ‘target’ folder we made earlier. I’ve named mine ‘movies-db’ and this is where we’ll keep the movies database. Final project setup should look something like this –

Database Creation and Import

For this demo I am using the ‘Movies’ database provided by Neo4j. It comes with 171 nodes and 253 relationships. It holds a variety of data on movies, actors, directors, writers, and reviewers; and was very useful as I attempted to learn to use Cypher. The inclusion of this database within our application can be completed with just a few simple steps — first start Neo4j and select ‘Choose…’ from the launcher window then navigate to the ‘movies-db’ folder we just created.

Start the database, and navigate to http://localhost:7474 when the application is ready.

Within the browser, type ‘:play movie graph’ into the editor, and then use the left within the output to bring up the screen shown below. Click anywhere within the green outline to copy the database creation script into the editor and press play to run the script. We now have our database within our application folder, and can begin creating our application!

Application Code

First be sure to shut down your database via the Neo4J launcher window by selecting ‘stop’. This method only offers one connection at a time, and will error if the database is in use elsewhere when we attempt to use it.

Before we start looking at the code for this small application, lets take a look at making a connection with the database

Now lets look at the simplest method I’ve made so far.

This is a simple method to find either the name or the title of a node, given its ID number. Some queries made at the database will only ever return nodes, so this method lets us determine what was actually returned. The syntax should look somewhat familiar for anyone who has done basic SQL queries using java, with the query formatted as a String, with a variable indicating the specific value to find. The Neo4j method db.execute() returns a Result object, which functions similarly to a ResultSet, except that Result contains a Map object connecting a Node with a value. In this instance what we want is either the name (if the node was a person) or a title (if the node was a movie) so we select both, and afterwards replace all null values with empty strings with the line

name = name.replace("null", "");

This is a method to find an actors Bacon Path — that is the shortest path of actors someone has worked with that leads to Kevin Bacon

This time, when the query executes it will return a path of nodes instead of individual nodes. We will need to parse the path into a format that can be output and read by a user, so we’ll need the parsePath() method called above

The path string returned when a query provides a path looks like

(19)--[ACTED_IN,29]-->(15)<--[ACTED_IN,26]--(16);

Which is less than useful. So, we use regular expressions to remove all data from the string that isn’t a node ID number, and then use our previously defined getNodeByID() method to determine what the data means.

Application in Action

I have added a text based interface to this project, and it contains some functionality not discussed here, but I encourage you to attempt to create the queries that would generate the results needed for the rest of the application (The Kevin Bacon friends option shows a list of everyone within 4 degrees of Kevin Bacon. His friends of friends if you will)

Hint: Neo4J offers specific functionality just for queries like that, and I’ve discussed them before. Good luck!

As previously mentioned, the strength of a graph database lies in its relationships — the ability to directly model the connections of a set of data. For data sets that feature a large amount of relationships the overhead of modelling them can be significant. Let’s look at an example of this

This is an ERD I’ve used during my studies at SAIT. It features quite a large amount of connections between the tables, with bridging tables used to resolve many to many relationships. Within a traditional data model, the foreign keys are how the model defines relationships — for example, within the COURSE_REGISTRATION table there are a number of foreign keys (student_id, course_code, section, semester, year). These are used to create a relationship between a row from the STUDENT table and a row from the CLASS_SECTION table, in this case a ‘registered in’ relationship. Looking a little closer though, you realize that this is a table composed of six columns with only one containing data that is actually unique to this table (the grade column). The rest of the data is already stored within the STUDENT and CLASS_SECTION tables, and is repeated purely to define the relationships between objects, and link a single student to a single class section. It might not seem like much, but with any reasonably sized school this table could quickly balloon out of control.

(As an illustration, SAIT had 4,378 students in its graduating class of 2015. If each was enrolled in 5 classes for 4 semesters then that would be 5 classes x 4 semesters x 5 columns x 4,378 students — a total of 437,800 pieces of data within the COURSE_REGISTRATION table per graduating class.)

Using this model for data-sets like this can also create a headache for developers trying to perform complicated queries requiring a large amount of joins, and all those joins can slow down a system that regularly needs to perform those complicated queries. This is where a graph database can assist with the simplification of a data set like this.

RDBMS to Graph

Let’s convert the above database into a graph. To do this, we’ll need to complete 3 steps

Step 1: Find the nodes

Step 2: Find the relationships

Step 3: Manage bridging tables

First, let’s find the nodes. The simplest way to accomplish this is to locate the objects within the database. In this case the STUDENT, FACULTY, CLASS_SECTION, and COURSE tables are all representing real life objects and will be our nodes, while EXPERTISE and COURSE_REGISTRATION are bridging tables providing a basis for defining a relationship between some of those objects.

Second, we need to find our relationships. This is done by examining the foreign keys. Whenever a table contains a foreign key from another table, we can assume that there is some type of relationship between them. For this step, we’ll ignore bridging table relationships.

For our example, the relationships would be (using the cypher notation utilized by Neo4j)

· (FACULTY) –[INSTRUCTS]-> (CLASS_SECTION)

· (CLASS_SECTION) –[ONE_OF]-> (COURSE)

· (COURSE) –[PREREQUISITE]->(COURSE)

Next we need to turn the bridging tables into relationships as well. Graph databases are able to contain many to many relationships, so the process of converting these tables is as simple as making them a relationship that contains properties for any unique data they contain (like that grade value discussed earlier)

· (STUDENT) –[ENROLLED_IN]-> (CLASS_SECTION) /grade

· (FACULTY) –[EXPERTISE]-> (COURSE) /expertise_level

Now we can make our ERD

This looks much simpler than our original ERD, but is able to contain all of the information the original could.

Relational databases have been in use for a long time. Oracle was originally released in 1979 — and the relational model continues to see widespread adoption across a large variety of applications, for better or worse. For many of these applications, using a RDBMS may be the easiest option for now, but in the long run may be adding unnecessary difficulties to their systems. There are a variety of NoSQL database types available, and I believe many companies would benefit from taking a close look at their current system to see if there is any room for improvement.

Now that we’ve gotten some introduction into graph databases in general, let’s start using a graph database. I’ve decided to go with Neo4j and the companion query language Cypher for this series. Neo4j is the most popular graph database in use at this point in time, and so offers the largest amount of community resources available to assist while figuring your way around the system. Neo4j also offers a variety of tutorials and sample databases to practice with. That being said, there are plenty of options out there and it would be worth it to take some time to examine what each offers should you be considering implementing a graph backend into a system.

CYPHER

Before we dive into making nodes and relationships, we have to start with some basic Cypher commands. Cypher is a query language developed by Neo technology for use with Neo4j. It was inspired by SQL, and so will be fairly intuitive for anyone already familiar with SQL.

Some basic Cypher syntax from the ‘introduction to Cypher’ tutorial offered by Neo4j –

MATCH (node1:Label1)-[rel:TYPE]->(node2:Label2)

WHERE rel.property > {value}

RETURN rel.property, type(rel)

This first uses the ‘MATCH’ clause, which is used to indicate any relationship we would like to find. This is matched to the nodes and relationship we would like to include in the search. Within the parenthesis we state the name of the node (node1, node2), along with node Label (which could also be called its type — whatever object the node is representing)

It then uses a ‘WHERE’ clause to refine the search. In this instance, on a particular property held by the relationship between node1 and node2

The ‘RETURN’ clause states the data you would like returned by the query. Think SQL’s ‘SELECT’ statement.

If we wanted to delete a relationship or a node, we can simply replace the ‘RETURN’ statement at the end of the query with a ‘DELETE’ statement, and then specify the nodes or relationships to remove.

MATCH (node1:Label1)-[rel:TYPE]->(node2:Label2)

WHERE rel.property > {value}

DELETE rel

We can also build off of this basic structure to perform updates on our data — here changing the value of a property

MATCH (node1:Label1)-[rel:TYPE]->(node2:Label2)

WHERE rel.property > {value}

SET rel.property = ‘new value’

To create a new node or relationship you use the ‘CREATE’ keyword.

CREATE(node1:Label1{property1:’<something>’,property2:’<somethingelse>’})

As expected, this uses some of the same structure as the “MATCH” statement above. We replace ‘node1’ with the name of the node (some distinguishing feature of the data contained within), and replace ‘Label1’ with the type of node we want to create. We can also add however many properties we need to keep track of for our node within the curly braces.

DATABASE CREATION

To start, I’ve installed the Neo4j community edition from here and launched the program. Let’s make a new database by selecting ‘choose’ and creating a new folder ‘test.graphdb’ within the Neo4j folder

Then press start and navigate to localhost:7474 within your browser once the database has started.

Within the browser console you are able to perform any create, read, update or delete function as needed. It can be a useful tool for visualizing data and relationships — which makes it perfect for learning and demonstrating Neo4j. It is however a bit resource heavy for larger data sets. The Neo4j shell is significantly more useful for proper use, and I will be demonstrating it in later posts.

First, let’s start by adding some nodes. I’ll type the following three commands into the prompt within the web console —

CREATE(Billy:Person{name:’Billy’,id:’0000'})

CREATE(Susan:Person{name:’Susan’,id:’0001'})

MATCH(s:Person{name:’Billy’}),(d:Person{name:’Susan’})

CREATE (s)-[r:Friends_with]->(d)

Querying

If we now query the database using MATCH (n) RETURN (n) (a query that asks for everything and displays all results) we can see our nodes and relationship we just created!

I’ll add some more nodes and relationships so we can see some of the Cypher queries mentioned earlier in action

We can search for a specific node

Remove relationship

That was Bobs only relationship, so we might as well remove him from the database as well

Let’s change Tammys name.

And now an to show all data, including the changes I just made

Conclusion

As you can see, Cypher is very straight forward for simple queries, and Neo4j is fairly intuitive for basic use. Next time we’ll look at converting an existing RDBMS to a graph database, and I will begin to examine some of the drawbacks I’ve found in my time working with Neo4j.

What is a graph database?

We’ll start with a very basic introduction to graphs — a graph is an abstract data type typically used to model data that contains objects (referred to as nodes or vertices), and some sort of relationship between those objects (lines or edges).

A graph database uses its nodes to contain data. Typically a node is used to represent an entity object, and is comparable to a row in a relational database. Each node also contains the properties of that entity, which can be anything the business needs to keep track of (ID numbers, location, etc.) These properties correspond to the columns in a relational database. Graph databases also contain edges between each node. These are use to describe the relationships between the objects contained in the database, and in most they can be compared to the foreign key column of a row in a RDBMS.

Why use a graph database?

The ability to directly state the relationship between two objects within a graph database provides a strong advantage over relational database systems for certain datasets. Namely, those that contain a large amount of relationships within the data.

Graph databases may also offer increased efficiency for searches in some cases. The best example of this is when a business needs to perform a search that is more than one level deep. As an example, let’s consider a database storing corporate structure information.

What sort of query might we need to perform on this data? What if we wanted to find all employees who work in a department with a specific employee, Sarah? (that is, all of her direct co-workers). We would first need to find which department(s) she currently works for by searching the ‘Department_Employee’ table, then search that table again for all other rows matching the retrieved Department_id’s in order to retrieve the Employee_id numbers for her co-workers, then finally the ‘Employee’ table to retrieve the names of her co-workers. That’s a lot of searching! Using a graph database, the process becomes somewhat simpler

In this system, we would first search for Sarah, and see which departments she belongs to. Then simply follow the links into each department backwards in order to discover the other members. For searches such as this, the efficiency is significantly increased.

Graph databases may not be the optimal choice for every business, but they have their advantages in certain situations. If you’d like to find out more make sure to check back for the rest of the series. I will be demonstrating use of an actual graph database, Neo4J. We’ll look at generating a fresh database, transforming existing RDBMS into graph, and using Cypher to make queries.