Reactive multi-tenancy with Neo4j 4.0 and SDN/RX

Three for the price of one: A movie rental story

Denise Jans on unsplash

We are going to address three topics in this post, and they all deal with new features in Neo4j 4.0 and surrounding ecosystem. We have:

• Multi-database support in Neo4j 4.0 (Enterprise Edition)
• Reactive Database, drivers and Spring Integration (All Editions)
• Role-based access controls (RBAC) (Enterprise Edition).

And, as a bonus: Fabric; a new way of querying multiple Neo4j databases together.

In this post, I use the super secret password that we’ll use in testing Neo4j SDN-RX as well. The secret password is for the user neo4j will be secret.

The business domain

Neo4j offers the movie graph starter code sample through its browser interface, so we are going to use that. It’s an easy and quick way to generate some data. It contains a set of movies and people, and how those people interacted (acted, directed, etc.) with those movies.

Now imagine two rental streaming services that use this data, “Videos ‘R Us” and “Video City”. Both are tenants of a bigger corporate company, “Videokingdom”. Videokingdom runs a new Neo4j 4.0 Enterprise Edition instance and provides each tenant their own database. Both tenants use the movie streaming application provided by Videokingdom. This application provides a reactive, scalable REST-ful backend. We must make sure that each user of the backend is identified as a user of the specific tenant, and database interactions go to the correct backend.

Preparing the databases

To begin, you will need a locally available Neo4j 4.0 installation. You can either use the server version or use Neo4j Desktop. Navigate to the bin folder in the installation directory. Start the instance service as follows:

./bin/neo4j start

Now, bring up the Cypher Shell with:

./bin/cypher-shell -u neo4j -p neo4j

On the initial start, it will ask for a new password. I chose secret . Next, create two databases:

:USE system;
CREATE DATABASE `videos-are-us`;
CREATE DATABASE `video-city`;

The first command tells the Cypher shell to switch to the system database. The system database understands the CREATE DATABASE command.

Next step, create a role movie_streaming_application

CREATE ROLE movie_streaming_application;

It is always a good idea to create a dedicated database user for a web application, so we will create that too:

CREATE USER reactive_rental SET PASSWORD 'secret' CHANGE NOT REQUIRED;
GRANT ROLE publisher TO reactive_rental;
GRANT ROLE movie_streaming_application TO reactive_rental;

publisher is a built-in role that allows reads and writes of data but no changes to indexes or constraints.

Imagine now that users of Video City are not allowed to see who reviewed a movie because… whatever reason a business had. In Neo4j 4.0, we can use fine grained access control for that. Let’s have a look at the data model to remind ourselves what changes to access we’re going to be making:

graph data model for the movies database

So we’re going to prevent Video City’s database being able to traverse REVIEWED:

DENY TRAVERSE
    ON GRAPH `video-city`
    RELATIONSHIPS REVIEWED
    TO movie_streaming_application;

Whilst we are at it, we can also deny writes to the video-city database, at least for a movie_streaming_application :

DENY WRITE ON GRAPH `video-city` TO movie_streaming_application;

The next step is to get the movie dataset into both databases. You can do that in Neo4j browser via the :play movies command or use the movies.cypher script from SDN/RX. For that post here we stay in the terminal respectively in the command line and use the provided cypher file above. Leave the Cypher shell with :exit , download the above Cypher-Script and run the following two commands, adapting the path to the Cypher file as needed:

./bin/cypher-shell -u neo4j -p secret -d videos-are-us \
   -f movies.cypher
./bin/cypher-shell -u neo4j -p secret -d video-city -f movies.cypher

Both databases for the rental companies are now set up and have data for us to work with. To verify that our access rules are working, try the following. Open the Cypher shell as user reactive_rental (use ./bin/cypher-shell -u reactive_rental -p secret )

:use videos-are-us
MATCH (m:Movie {title: 'Cloud Atlas'})
OPTIONAL MATCH (m) <- [:REVIEWED] - (p) 
RETURN m, p;
:use video-city
MATCH (m:Movie {title: 'Cloud Atlas'})
OPTIONAL MATCH (m) <- [:REVIEWED] - (p) 
RETURN m, p;

Whilst the first query gives us Cloud Atlas and Jessica Thompson, the same query in a different database that has the same initial dataset, gives us only the movie.

Setting up the Spring Boot SDN/RX application

First of all, go to the Spring Initializr. As we want to create a reactive, secured web application, you will need to add reactive and security as dependencies. Don’t add Neo4j here, we will add that dependency later on.

Here is a link that selects all relevant dependencies on the initializer: multidatabase_movies. Click this and generate the project. Save it, open it in your favorite IDE. First of all, add Spring Data Neo4j⚡️RX as dependency in the pom.xml and make it look like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>

   <parent>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-parent</artifactId>
      <version>2.2.4.RELEASE</version>
      <relativePath/>
   </parent>

   <groupId>ac.simons.neo4j.examples</groupId>
   <artifactId>multidatabase-movies</artifactId>
   <version>0.0.1-SNAPSHOT</version>

   <properties>
      <java.version>1.8</java.version>
   </properties>

   <dependencies>
      <dependency>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-security</artifactId>
      </dependency>
      <dependency>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-webflux</artifactId>
      </dependency>
      <dependency>
         <groupId>org.projectlombok</groupId>
         <artifactId>lombok</artifactId>
      </dependency>
      <dependency>
         <groupId>org.neo4j.springframework.data</groupId>
         <artifactId>spring-data-neo4j-rx-spring-boot-starter</artifactId>
         <version>1.0.0-beta03</version>
      </dependency>
   </dependencies>

   <build>
      <plugins>
         <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
         </plugin>
      </plugins>
   </build>

</project>

I have omitted the test dependencies for brevity. SDN/RX is a mapping framework, so we can define the following entities and corresponding repositories

Movies…

package ac.simons.neo4j.examples.multidatabase_movies.domain;

import static org.neo4j.springframework.data.core.schema.Relationship.Direction.*;

import lombok.Getter;

import java.util.Set;

import org.neo4j.springframework.data.core.schema.Id;
import org.neo4j.springframework.data.core.schema.Node;
import org.neo4j.springframework.data.core.schema.Property;
import org.neo4j.springframework.data.core.schema.Relationship;

@Node("Movie")
@Getter
public class MovieEntity {

   @Id
   private final String title;

   @Property("tagline")
   private final String description;

   @Relationship(type = "ACTED_IN", direction = INCOMING)
   private Set<PersonEntity> actors;

   @Relationship(type = "DIRECTED", direction = INCOMING)
   private Set<PersonEntity> directors;

   @Relationship(type = "REVIEWED", direction = INCOMING)
   private Set<PersonEntity> reviewers;

   public MovieEntity(String title, String description) {
      this.title = title;
      this.description = description;
   }
}

SDN/RX repositories are 100% declarative, so we can define them as interfaces:

package ac.simons.neo4j.examples.multidatabase_movies.domain;

import reactor.core.publisher.Mono;

import org.neo4j.springframework.data.repository.ReactiveNeo4jRepository;

public interface MovieRepository extends ReactiveNeo4jRepository<MovieEntity, String> {

   Mono<MovieEntity> findOneByTitle(String title);
}

…and People

package ac.simons.neo4j.examples.multidatabase_movies.domain;

import lombok.Getter;

import org.neo4j.springframework.data.core.schema.Id;
import org.neo4j.springframework.data.core.schema.Node;

@Node("Person")
@Getter
public class PersonEntity {

   @Id
   private final String name;

   private final Long born;

   public PersonEntity(Long born, String name) {
      this.born = born;
      this.name = name;
   }
}

As we don’t query the people directly, so we won’t define a repository for them. The reactive REST interface will be defined with a more or less standard Spring controller, using the MoviesRepository . That controller however returns reactive data types:

package ac.simons.neo4j.examples.multidatabase_movies.web;

import ac.simons.neo4j.examples.multidatabase_movies.domain.MovieEntity;
import ac.simons.neo4j.examples.multidatabase_movies.domain.MovieRepository;
import lombok.RequiredArgsConstructor;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/movies")
@RequiredArgsConstructor
public class MovieController {

   private final MovieRepository movieRepository;

   @PutMapping
   Mono<MovieEntity> createOrUpdateMovie(@RequestBody MovieEntity newMovie) {
      return movieRepository.save(newMovie);
   }

   @GetMapping(value = { "", "/" }, produces = MediaType.TEXT_EVENT_STREAM_VALUE)
   Flux<MovieEntity> getMovies() {
      return movieRepository
         .findAll();
   }

   @GetMapping("/by-title")
   Mono<MovieEntity> byTitle(@RequestParam String title) {
      return movieRepository.findOneByTitle(title);
   }

   @DeleteMapping("/{id}")
   Mono<Void> delete(@PathVariable String id) {
      return movieRepository.deleteById(id);
   }
}

To start the application itself, we also need a main class:

package ac.simons.neo4j.examples.multidatabase_movies;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class MultidatabaseMoviesApplication {

   public static void main(String[] args) {
      SpringApplication.run(
          MultidatabaseMoviesApplication.class, args
      );
   }

}

Which is basically a standard Spring Boot application.

Configuration time

The pure database connection is easy and can be done via properties or the environment. Here is the properties solution:

org.neo4j.driver.uri=bolt://localhost:7687
org.neo4j.driver.authentication.username=reactive_rental
org.neo4j.driver.authentication.password=secret
spring.jackson.default-property-inclusion=non_null

The last line has nothing to do with SDN/RX, but prevents null fields in our JSON responses.

For the security configuration, we have a bit more manual work to do. The comments at the Bean methods explains in detail what we are doing here.

package ac.simons.neo4j.examples.multidatabase_movies;

import reactor.core.publisher.Mono;

import java.util.Locale;

import org.neo4j.springframework.data.core.DatabaseSelection;
import org.neo4j.springframework.data.core.ReactiveDatabaseSelectionProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.web.server.ServerHttpSecurity;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.ReactiveSecurityContextHolder;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.userdetails.MapReactiveUserDetailsService;
import org.springframework.security.core.userdetails.User;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.security.web.server.SecurityWebFilterChain;

@Configuration
public class SecurityConfig {

   /*
    * This bean configures relevant features of Spring security:
    * We want all requests to be authenticated and we don't use CSRF
    * protection in this application. We support only basic auth.
    */
   @Bean
   public SecurityWebFilterChain springSecurityFilterChain(
      ServerHttpSecurity http
   ) {
      http
         .authorizeExchange()
         .anyExchange().authenticated()
         .and()
         .csrf().disable()
         .httpBasic();
      return http.build();
   }

   /*
    * This creates two users: Michael that has a role of VIDEO-CITY,
    * and Gerrit, who is a VIDEOS-ARE-US user.
    */
   @Bean
   public MapReactiveUserDetailsService userDetailsService() {
      UserDetails userA = User.withDefaultPasswordEncoder()
         .username("Michael")
         .password("secret")
         .roles("VIDEO-CITY")
         .build();

      UserDetails userB = User.withDefaultPasswordEncoder()
         .username("Gerrit")
         .password("secret")
         .roles("VIDEOS-ARE-US")
         .build();

      return new MapReactiveUserDetailsService(userA, userB);
   }

   /*
    * This bean is part of SDN/RX. A DatabaseSelectionProvider 
    * can be used to determine the database to use. 
    * Here, we choose the reactive security context.
    */
   @Bean
   ReactiveDatabaseSelectionProvider reactiveDatabaseSelectionProvider() {

      return () -> ReactiveSecurityContextHolder.getContext()
         .map(SecurityContext::getAuthentication)
         .filter(Authentication::isAuthenticated)
         .map(Authentication::getPrincipal)
         .map(User.class::cast)
         .flatMap(u -> Mono.justOrEmpty(u.getAuthorities().stream()
            .map(r -> r.getAuthority().replace("ROLE_", "").toLowerCase(Locale.ENGLISH))
            .map(DatabaseSelection::byName)
            .findFirst()))
         .switchIfEmpty(Mono.just(DatabaseSelection.undecided()));
   }
}

The idea of this database selection is the following: We use the authorities granted to the users to determine the tenant of Videokingdom. This is done in a bean of type ReactiveDatabaseSelectionProvider . We have an imperative variant for it as well, but hey… Reactive all the things!

The reactive variant must of course return a reactive type as well. A Mono of DatabaseSelection . This fits nicely with the ReactiveSecurityContextHolder . This class is a part of Spring’s reactive security and gives you the authentication relevant at that point in time on which a reactive flow is executed. We check whether someone is authenticated and extract the roles assigned to the principal. These are the roles we defined when creating the user. Spring did prefix them with ROLE_ for internal Spring reasons.

Putting all this together and starting the main application will give you the following output at some point:

2020-01-29 21:27:04.256  INFO 2373 --- [           main] o.s.b.web.embedded.netty.NettyWebServer  : Netty started on port(s): 8080

And then try the following CURL request:

curl -u Michael:secret "http://localhost:8080/movies/by-title?title=Cloud%20Atlas"
curl -u Gerrit:secret "http://localhost:8080/movies/by-title?title=Cloud%20Atlas"

The first one returns:

{
  "title": "Cloud Atlas",
  "description": "Everything is connected",
  "actors": [
    {
      "name": "Halle Berry",
      "born": 1966
    },
    {
      "name": "Tom Hanks",
      "born": 1956
    },
    {
      "name": "Jim Broadbent",
      "born": 1949
    },
    {
      "name": "Hugo Weaving",
      "born": 1960
    }
  ],
  "directors": [
    {
      "name": "Tom Tykwer",
      "born": 1965
    },
    {
      "name": "Lana Wachowski",
      "born": 1965
    },
    {
      "name": "Lilly Wachowski",
      "born": 1967
    }
  ],
  "reviewers": []
}

While the second one returns:

{
  "title": "Cloud Atlas",
  "description": "Everything is connected",
  // Same actors and directors, but in addition, the reviewers
  "reviewers": [
    {
      "name": "Jessica Thompson"
    }
  ]
}

So we have an application in place that dynamically selects the tenant database based on the user role.

We also can verify that only user Gerrit can create new movies. While the following command:

curl -X "PUT" -u Gerrit:secret "http://localhost:8080/movies" \               
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "title": "Aeon Flux",
  "description": "Reactive is the new cool"
}'

succeeds with:

{"title":"Aeon Flux","description":"Reactive is the new cool"}%

Using Michael fails:

curl -X "PUT" -u Michael:secret "http://localhost:8080/movies" \               
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "title": "Aeon Flux",
  "description": "Reactive is the new cool"
}'
{
  "timestamp": "2020-01-29T20:39:43.913+0000",
  "path": "/movies",
  "status": 500,
  "error": "Internal Server Error",
  "message": "Write operations are not allowed for user 'reactive_rental' with roles [movie_streaming_application,publisher].",
  "requestId": "ba7cc39b"
}

I think role-based access controls in a database are a great way to secure things. The rental application might go away, replaced by another, along with different rules in it as well. The database is there to stay.

You’ll find the whole project as a zip file here: multidatabase-movies.zip.

But what about a complete view of all the movies?

This is where Fabric comes into play.

Fabric, introduced in Neo4j 4.0, is a way to store and retrieve data in multiple databases, whether they are on the same Neo4j DBMS or in multiple DBMSs, using a single Cypher query. Fabric achieves a number of desirable objectives:

• a unified view of local and distributed data, accessible via a single client connection and user session
• increased scalability for read/write operations, data volume and concurrency
• predictable response time for queries executed during normal operations, a failover, or other infrastructure changes
• High Availability and No Single Point of Failure for large data volume.

This is only a small teaser, and we are only interested in the first point.

Fabric is easy to set up, but you have to stop your Neo4j instance and open conf/neo4j.conf in your editor of choice. We’ll define a new Fabric database, named videokingdom, that will contain the two distributed graphs:

fabric.database.name=videokingdom
fabric.graph.0.uri=neo4j://localhost:7687
fabric.graph.0.database=videos-are-us
fabric.graph.0.name=moviesA
fabric.graph.1.uri=neo4j://localhost:7687
fabric.graph.1.database=video-city
fabric.graph.1.name=moviesB

The server is then restarted as usual, and you can use the Cypher shell again. Let’s get a list of all of the movies starting with ‘A’ held in both these databases. To do this, run the following query:

./bin/cypher-shell -u neo4j -p secret -d videokingdom
CALL {
  USE videokingdom.moviesA
  MATCH (movie:Movie)
  WHERE movie.title =~ 'A.*'
  RETURN movie.title AS title 
  UNION
  USE videokingdom.moviesB
  MATCH (movie:Movie)
  WHERE movie.title =~ 'A.*'
  RETURN movie.title AS title
} RETURN title  ORDER BY title ASC

This gives us an overview over all movies starting with “A”, including the newly created Aeon Flux.

I hope that gives you some ideas to play around with. Thank you and happy coding.

Thanks to the whole Neo4j engineering team for a fabulous Neo4j 4.0 release and my colleagues

Ljubica Lazarevic

,

Jennifer Reif

and

Gerrit Meier

for reviewing this article.