Create a Data Marvel — Part 8: Controlling and Servicing our Comic Endpoints

*Update*: All parts of this series are published and related content available.

Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7, Part 8, Part 9, Part 10

Completed Github project (+related content)

We will continue developing our ComicIssue classes for passing the ComicIssue objects, along with any entities related to them. This post will cover the ComicIssue controller and service classes for handling requests and shaping any results. As before, previous blog posts (Parts 1–7) have built upon information and code leading up to this. For review of code for the ComicIssue domain, repository, and controller classes, you can check out Part 7 of this series.

Let’s go ahead and dive in!

Controller vs. Service classes

As we have previously discussed, the controller class is the intermediary between the data layer and the user interface to accept requests from the user and send back responses. This is where the code for logic and data manipulation is typically placed.

One minor distinction for our ComicIssue is that we will use the controller to outline and map our ComicIssue endpoints, then use a service class to handle logic and formatting of the data for us. This type of architecture (creating an external and internal API) is mostly personal preference for division of labor and easier-to-read code. Hopefully, you will see why shortly. If we included all the upcoming code in one class, the waters could get murky very quickly. Creating a separate ‘service layer’ class adds passthroughs to a couple of operations, but it also keeps our controller class more readable and maintainable by placing the rather substantial data assembly/formatting in its own class, along with those remaining data access (repository) calls.

The controller class

package com.thehecklers.sdnmarvelcomics.comicissue;
import …
@AllArgsConstructor
@Controller
@RequestMapping(“/comicissues”)
public class ComicIssueController {
  private final ComicIssueService issueService;
  @GetMapping(“/findbyname”)
  @ResponseBody
  public ComicIssue findByName(@RequestParam String name) {
    return issueService.findByName(name);
  }
  @GetMapping(“/findbynamelike”)
  @ResponseBody
  public Iterable<ComicIssue> findByNameLike(@RequestParam String name) {
    return issueService.findByNameLike(name);
  }
  @GetMapping(“/buildgraph”)
  @ResponseBody
  public Map<String, Object> buildgraph(@RequestParam(required = false) Integer limit) {
    return issueService.graph(limit == null ? 100 : limit);
  }
  @GetMapping(“/graph”)
  public String graph(@RequestParam(required = false) Integer limit, Model model) {
    model.addAttribute(buildgraph(limit));
    return “issuesgraph”;
  }
}

In the code above, we start our class with an @AllArgsConstructor Lombok annotation to create a constructor with one parameter for each field in the class, in this case, the ComicIssueService bean we want Spring Boot to infect/autowire for our use. The next annotation for @Controller is different from our other entity controllers because we do not want to return JSON for every method in the ComicIssueController. Remember that the @RestController annotation we have been using is a combination of @Controller and @ResponseBody, the latter of which defaults to a JSON-format return. Instead, we want to specify which methods should return JSON and which could return an html page (through a template engine). Finally, our last annotation sets up the general endpoint for all of our ComicIssue calls (“/comicissues”).

Our class body starts by injecting our service class, then defines our methods. The findByName() and findByNameLike() methods are first. These are pretty straightforward, as we need to set up the endpoints for each using the @GetMapping annotation and then annotate with @ResponseBody to notify Spring that these methods will simply return JSON as the response.

The findByName() method is expected to only return a single result, and we need to pass in the searched name as a parameter. The method will call the service class method with the name parameter as the argument.

The findByNameLike() is handling a fuzzy search, so it could return several possible matching comic issues. It will expect an Iterable of ComicIssues in return and also pass the searched name as a parameter. Just like our findByName() method, this method calls the service class method and passes the name to it.

Our third method is buildgraph(). This method also uses the same annotations as our previous methods for mapping the endpoint and defining our JSON return. However, this method definition and call is a bit different. The code is repeated below for reference alongside the explanation.

@GetMapping(“/buildgraph”)
@ResponseBody
public Map<String, Object> buildgraph(@RequestParam(required = false) Integer limit) {
  return issueService.graph(limit == null ? 100 : limit);
}

First, we are expecting a Map<> to return from this method because it formats our data to be displayed for our visualization. D3 expects a map of nodes and relationships, so that is what must return here. Getting the data into that map format is what our service class will do. We will look at that code in a bit.

We are also passing an integer parameter called limit to this method. This will allow us to specify how much of the graph we want to return in our visualization. If we have a very large graph, rendering a visualization in our webpage could overload the page or cause performance impact. Depending on user preferences, we can pass in an arbitrary number to limit the volume of the return.

Just like our previous methods, buildgraph() will call the service class method, but it passes something more interesting in the parameter (limit==null ? 100 : limit). This bit of code is a ternary operator that checks if limit is equal to null. If it is, it uses the default value of 100. Otherwise, it uses the passed-in value. If the user forgets to pass in a value (or doesn’t need a specific amount returned), this ensures that the number of nodes and relationships returned to the visualization will not exceed 100. It will avoid an accidental browser overload or poor performance.

The last method is graph(). This code is what will call the template engine that searches for an html page and loads it to the browser, if it finds one. We have restated the method code below to use as reference with the following explanation.

@GetMapping(“/graph”)
public String graph(@RequestParam(required = false) Integer limit, Model model) {
  model.addAttribute(buildgraph(limit));
  return “issuesgraph”;
}

Our usual @GetMapping annotation sets up the endpoint for calling this method, and we can then define it. We are expecting a return type of String because our template engine will use the returned string to search for an html file with that same name to serve up to our browser. Our parameters for the method include the integer limit (again, for the visualization) and a model object that can supply attributes for rendering pages. In our case, we call the addAttribute() method on the model and pass our buildgraph() method with the limit parameter to convert our data to the expected visualization format, then add that as an attribute to the model for rendering. Finally, the method returns the issuesgraph String, which will cause the template engine to look for an issuesgraph.html file to match it. We will show the code for the html file later.

Now that we have our Controller class completed and all of our endpoints and methods defined, we can move on to our service class and show how we handle additional logic there.

The service class

This class has quite a bit of code, so we will break up the blocks into a couple of chunks for easier consumption. Let us start with the annotations, class declaration, and a couple of our simpler methods.

package com.thehecklers.sdnmarvelcomics.comicissue;
import …
@AllArgsConstructor
@Service
public class ComicIssueService {
  private final ComicIssueRepo issueRepo;
  @Transactional(readOnly = true)
  public Iterable<ComicIssue> findAllComicIssues() {
    return issueRepo.findAll();
  }
  @Transactional(readOnly = true)
  public ComicIssue findByName(String name) {
    return issueRepo.findByName(name);
  }
  @Transactional(readOnly = true)
  public Iterable<ComicIssue> findByNameLike(String name) {
    return issueRepo.findByNameLike(name);
  }
  @Transactional(readOnly = true)
  public Map<String, Object> graph(Integer limit) {
    return toD3Format(issueRepo.graph(limit));
  }
  ...
}

In the above segment, we use the @AllArgsConstructor to have Lombok include a constructor with all member variables as parameters. The @Service is an alias for @Component, which tells Spring to create a bean and make it available to the application for use. We use @Service instead of @Component as an indication that this is a stateless interface (a la DDD) that serves as a foundation for our internal API.

Within our class body, we inject our ComicIssueRepo into our service, allowing the service to call the repository methods. Next, we have 4 methods using the @Transactional annotation. This annotation sets boundaries for how much will run in a single transaction. We could run multiple calls to Neo4j within a transaction, but we want to separate each method into its own unit of work. We also specify that data in this method should be read-only and should not have any write transactions.

Our first method is findAllComicIssues(). This method simply calls the repo and executes the built-in findAll() method to return all ComicIssues in the database, using the same pattern from our other entity controller classes. The next two methods are for findByName() and findByNameLike(). These methods are straightforward and call the respective method from the issueRepo and both pass in the name value as a parameter.

The last method in this block is for graph(). It expects a map returned with the nodes and relationships for the visualization and expects the limit parameter. The return call is the most interesting piece. First, we call the issueRepo and associated method and pass in our limit value. However, in order to get our data into the format the D3 visualization expects (Map<>), we also need to transform and manipulate those results. That is what the toD3Format() method will do, and we will look at that code shortly. We wrap the issueRepo.graph(limit) call within the toD3Format() method, and we should end up with our Map<> structure for displaying the graph visualization on our webpage.

Now we can look at what the toD3Format() method does to transform the data into a Map<>.

private Map<String, Object> toD3Format(Iterable<ComicIssue> issues) {
List<Map<String, Object>> nodes = new ArrayList<>();
List<Map<String, Object>> rels = new ArrayList<>();
int i = 0;
Iterator<ComicIssue> result = issues.iterator();
while (result.hasNext()) {
  ComicIssue issue = result.next();
  nodes.add(map(“name”, issue.getName(), “label”, “issue”));
  int target = i;
  i++;
  for (Character character : issue.getCharacters()) {
    Map<String, Object> comicChar = map(“name”, character.getName(), “label”, “character”);
    int source = nodes.indexOf(comicChar);
    if (source == -1) {
      nodes.add(comicChar);
      source = i++;
    }
    rels.add(map(“source”, source, “target”, target));
  }
  for (Creator creator : issue.getCreators()) {
    //same code block as for Characters
  }
  for (Series series : issue.getSeries()) {
    ...
  }
  for (Story story : issue.getStories()) {
    ...
  }
  for (Event event : issue.getEvents()) {
    ...
  }
}
return map(“nodes”, nodes, “links”, rels);
}
private Map<String, Object> map(String key1, Object value1, String key2, Object value2) {
    Map<String, Object> result = new HashMap<String, Object>(2);
    result.put(key1, value1);
    result.put(key2, value2);
    return result;
}

I have condensed the code a bit to make it easier to read, but the same code block within the for (Character) loop is the same for each entity. The toD3Format() method will return a Map<> type, which is what d3 needs to render our visualization. Our Iterable<ComicIssue> is passed into the method, so we can iterate over the list and separate our objects into nodes and relationships.

We begin the method body by creating two new ArrayLists — one for nodes and one for relationships (rels). Then, we set up an iterator to loop through each ComicIssue object and grab the next comic (ComicIssue issue = result.next();). The next line adds two values to our nodes map for the issue name and the label on the issue (which is :ComicIssue). It does this by passing the name and label to the map() method that creates a HashMap with two keys and two values, then adding the resultant Map<> to the list of mapped nodes.

Now, for each ComicIssue entity, we have sub-entities for each Character, Creator, Event, Series, or Story related to that comic. We will need to loop through each of those lists and map the start and end node of the relationship, as well. This bit of code is repeated for clarity.

int target = i;
i++;
for (Character character : issue.getCharacters()) { 
  Map<String, Object> comicChar = map(“name”, character.getName(), “label”, “character”);
  int source = nodes.indexOf(comicChar);
  if (source == -1) {
    nodes.add(comicChar);
    source = i++;
  }
  rels.add(map(“source”, source, “target”, target));
}

We start with our target node, which is the current ComicIssue node that is on one side of the relationship. For each Character related to the particular ComicIssue, we call the map() method to create another HashMap from the name property and label of each Character. Then, we pull the the index of that entity in the list and assign that to our source variable (source = nodes.indexOf(comicChar);). At the end of the block, we also call the map() method to format the source and target values into our expected HashMap and add them to the relationship map. Now, each of the ArrayLists we initially set up (nodes and rels) will contain a list of indexes for the related objects and their connections.

We loop through the same logic for the rest of our entities (Creator, Event, Series, Story) that we did for Character, adding each of the nodes and relationships for these objects to the arrays.

Now we have a functional application that should return ComicIssues from our endpoints, along with the related Character, Creator, Event, Series, and Story entities! We can tap some of the /comicissues endpoints to ensure everything is working. Our last step will be to create our html page for viewing the data in a webpage and rendering our visualization!

What I Learned

This step was quite involved, as we starting knitting together all of the entities to return with a particular ComicIssue. The toughest part was probably setting up the endpoints, template engine structure, and d3 formatting for the visualization. I’ve outlined my challenges in the points below.

1. D3 looks complicated at first, but it actually expects a straightforward format of maps. I was able to use much of the existing code from some of our sample projects, and then modify it for our use case. The complexity was in the number of entities that we have for our Marvel Comics data.
2. The separation of Controller and Service was very confusing at first. I wasn’t sure why it was needed or what purpose it served. However, it became clear that I didn’t want to include all of the code from both classes in a single file (would’ve ended up around 140 lines of code)! Separating endpoints from the formatting and mapping logic of the d3 code made it much easier for me to sift through code and change things. Separating concerns into two classes (external-facing controller API and internal service API) made for cleaner code divisions and an API that was easier to reason about.

Next Steps

We are so close to have a full-stack application completed. Our next post will place the final (pretty!) puzzle piece into place to add the html page and allow us to interact with the data in a visual and friendly manner. Hope to see you then!

Resources

• Follow the duo on Twitter to see what’s coming: @mkheck and @jmhreif
• Download Neo4j
• Spring Data Neo4j docs
• Spring Data Neo4j Guide
• Project Lombok docs
• Previous parts of this blog series: Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7