Create a Data Marvel — Part 4: How To Design the Application

*Update*: All parts of this series are published and related content available.

Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7, Part 8, Part 9, Part 10

Completed Github project (+related content)

Throughout the last few blog posts in this series, you have seen how Mark Heckler and I began this coding project, chose a dataset, drafted a graph data model (Part 1), and imported the data (Part 2 and Part 3) from an API to a Neo4j database. This post will now focus on the application development and the choices and options available to us, as well as what we decided and why.

We already knew we wanted to work with Spring and Neo4j before we began the project, but we needed to figure out how best to approach integration between the application and the data layers. First, we needed to consider our limitations and goals for this demo.

We planned to submit it to conferences or meetups for presentation, so the whole kit-and-kaboodle needed to be explained in 30–60 minutes (90 minutes, if we had luxury). We also wanted to be able to live-code as much as possible and really show people how simple it is to use both technologies and what each brought to the table. This means that we needed to code as much as possible within the 30–60min time frame. Plus, we knew we would need some introduction to our project and data source.

Could we show what we wanted and give the audience a guidebook for start-to-finish development and inspire them to want to walk out of the session and build it themselves? This is really the crux of designing good conference content. We needed to find a short, sweet, and simple way to build an application and easily connect to a graph data source. There were a few different paths we could take.

1. We could write a Spring application and use a driver (such as Java) to connect from the application to a Neo4j instance. This means some configuration and setup code to connect the driver to the database and instantiate it in the application.
2. Our other option (which we took) was to use the integration project of Spring Data Neo4j. This project was built by employees and community members to join the capabilities and ease of Spring Data with the power of graphs in a Neo4j backend. It includes pre-configured setup, as well as additional built-in functionality that seemed perfect for our needs. We will talk a bit more in detail about what this project gives us.

The Joys of Spring Data

Spring Data (per their documentation) aims to “provide a familiar and consistent, Spring-based programming model for data access while still retaining the special traits of the underlying data store.” Now, this simply means that it was designed to plug and play easily with any datastore and not change the datastore’s unique structure or interactions.

Spring Data needs to be extremely flexible to work with a variety of data sources — including relational, document, key-value, graph, and others. This is something the project accomplished, providing simple integrations with sources such as JDBC, JPA, MongoDB, Redis, Couchbase, Elasticsearch, and many others! For a full list of supported sub-projects within Spring Data’s high-level umbrella project, you can check out the documentation.

Spring Data also provides features for various object-mapping methods per datastore needs and includes its fantastic repository! It also makes creating domain classes simple with several annotations. True to its goals, the project also includes simple integrations or more complex options, as well, based on needs.

If you are familiar with the Spring ecosystem, you may know one of its goals is to reduce (and sometimes eliminate) boilerplate code. One helpful feature for this is that some methods can be derived for you without writing a query. Spring Data (no matter what data source it’s using) can derive queries for you based on repository method names. This sounds like Spring’s typical “magic”, but we will look at this more in detail later.

This impressive list of Spring Data integrations includes Neo4j using the project name Spring Data Neo4j and includes a variety of helpful functionality. We will look at what Spring Data Neo4j (SDN) can add to our demo next!

The Simplicity of Spring Data Neo4j

Spring Data Neo4j combines the Spring functionality of passing and grouping data with the power of property graph data storage and retrieval in Neo4j. The project supports the property graph data model, which organizes data into nodes and relationships. SDN also maps a graph to objects using the Object-Graph-Mapper (OGM) with annotations for mapping POJO entities.

Another important functionality is its support of the Cypher Query Language. This allows the developer to write Cypher queries in the application and pass the query and data between layers over all the common Neo4j transport methods — binary protocol (BOLT), HTTP, and embedded.

* Note: SDN also is able to be used with Neo4j’s causal clusters (multi-server cluster).

Finally, Spring Data Neo4j helps us avoid writing so much boilerplate code, therefore streamlining the development process and allowing us to focus on the data and the application functionality! Remember that Spring Data also can derive dynamic queries based on the repository method names. This also means we don’t need to write a block of extra code for queries. We will show an example in upcoming paragraphs.

Just to show what all is involved in the Spring Data Neo4j package and what it provides out-of-the-box, the diagram below outlines each layer and how communication passes between the application and the database.

The Obvious Choice

Ok, so using Spring Data Neo4j completely made sense for what we needed. It would allow us to easily build a simple application with Spring “magic” and reduce hassle in connecting to our Neo4j data source. It would handle mapping from our domain objects to our Neo4j property graph model elements (nodes and relationships) using a few simple annotations, and it could shortcut some of our basic methods by automatically deriving those queries.

All-in-all, this integration could help us focus on the get-up-and-running steps within our allotted 30–60 minutes of presentation time! Let’s start building!

Creating the Application

Another added benefit to the Spring development ecosystem is the Spring Initializr. This page is a simple form that helps you design a Spring application with the needed dependencies, naming, language (Java, Kotlin, or Groovy), version, and format (maven or gradle). Based on your needs and choices, it generates the skeleton project for you with some basic folders and an application class outline. This is super helpful for hitting the ground running and creating a project before audience members’ eyes in mere seconds (go ahead, try it out!).

Once you choose your project details, you can click Generate Project, and it will create the project and download it to your local device. You can then open the project in your favorite editor and start coding!

For today’s post, we will generate a dummy project (leave the default group and artifact fields) with our 4 dependencies for the Marvel comic demo and step through the development.

We want to include Lombok to trim repetitive code for getters, setters, and constructors. Lombok helps us move a little faster and automatically configures your classes with the appropriate getters, setters, and constructors based on simple annotations you can provide instead. You will see these in our code.

We also need Neo4j so that Spring knows to connect to that as our data source, so we add that in, as well. The Web dependency handles several web client and server components with all supporting tasks, and Thymeleaf helps make our front end webpage pretty using templates.

With all this info plugged in shown in the image below, we are off to a great start!

The Project

We like to use JetBrains’ IntelliJ IDEA editor and typically use Maven projects, so we open the pom.xml file in IntelliJ, and the project structure opens up for us.

If you open the pom.xml file at the bottom, you will see Spring Boot is incorporated, as well. Spring Boot makes it incredibly simple to create applications that you can run with very little Spring configuration. Certain niceties such as metrics and health checks are also available, should you want/need them. The painless bootstrapping ability and dependency versioning management are the core of Spring Boot’s popularity.

That’s what the spring-boot-starter-parent handles for us. If you look on down into the dependency tree, you will see that we have spring-boot-starter-data-neo4j. That’s our Spring Data Neo4j module that includes Spring Boot!

We also see Thymeleaf, the web package, and Lombok. Everything looks good, and we have all of our dependencies.

Outside of the pom.xml, we have a bare-bones folder structure. Under the src folder, we have our main folder, and the test folder. We will not work much with the test folder at this time, but it is important to create tests for your applications and add them in this folder.

The main folder contains all of our core domain model classes and repository interfaces. It is also where we have our resources folder to put html and other files later. This main folder is where the bulk of our code will go for this project.

What I Learned

Before I stepped through this part of the process, I knew how to build an application, but I didn’t know exactly why or what Spring was handling behind-the-scenes. This helped me understand how and why to make certain choices in design.

I also relied much more on my dad’s wealth of knowledge on the Spring ecosystem and shortcuts, but we were both new to the Spring Data Neo4j integration. Below is a list of my key takeaways from the first step of the application development process.

1. It took some research and pair programming sessions to figure out the best integration solution. I relied some on my dad’s depth of knowledge in the Spring topic.
2. We had to plan out what we wanted to focus most on and what we wanted audience members to see and learn from it. Deciding to focus on code and building the solution was an easy choice for us, but determining what needed for context and what/how to trim other content was not so easy.
3. While we both had certain knowledge in Spring or in Neo4j, neither of us had dealt with the Spring Data Neo4j integration project. We had to read docs, look at some examples, and experiment with it some. It was fun and rewarding, as well as not overwhelming.

Next Steps

This part of the project often seems trivial and less important than data in the database or coding the application itself, but it’s still a critical piece of the puzzle. Knowing how and why two technologies integrate is part of every developer’s daily life these days.

Having an integration project that is built to shortcut some of the hassles of bridging gaps and connecting two different technologies often makes life much easier. One other benefit is that a team may be familiar with one technology, so the bridge project take something new and combines it with existing knowledge on the comfortable topic. It is easier to work with each technology’s strengths, as well as on-board new developers.

The next posts will step through the code in the application and show how it retrieves data from Neo4j and displays it nicely on a page. Stay tuned!

Resources

• Follow the duo on Twitter to see what’s coming: @mkheck and @jmhreif
• Download Neo4j
• Spring Data Neo4j docs
• Spring Data Neo4j Guide
• Project Lombok docs
• Previous parts of this blog series: Part 1, Part 2, Part 3