Graph to graph with Elixir

Moving data between semantic and property graphs

23 min readApr 12, 2019

Photo by Maarten van den Heuvel on Unsplash

In earlier posts I’ve shown how to access and manipulate RDF graphs using Elixir and separately also how to access and manipulate labeled property graphs (or LPG graphs). What I’d like to focus on here is conversions from one graph form to another in Elixir.

But first a quick recap on why.

Why graphs? Because graphs are everywhere. Because graphs are about connected data. Because graphs are essentially about large-scale data integration.

We heard recently at the London leg of the Neo4j GraphTour about the relationship between graphs and ML. Expect to see graphs figure more prominently in ML solutions.

And then again just a little earlier we also had the W3C Workshop on Web Standardization for Graph Data, a workshop on graph model alignment which looked at creating bridges between the RDF, LPG and SQL worlds.

So, why Elixir? Well of course because it’s a dynamic, functional language designed for building scalable and maintainable applications, especially distributed applications. Elixir has built-in support for fault tolerance and concurrency. But more than that where it truly shines is as a ‘sparks joy’ type of language. And that’s just got to be a good thing, right?

Now I was thinking I might have to roll up my sleeves and do some kind of a naïve translation between these two graph forms when Jesús Barrasa kindly pointed me to his neosemantics library. (He’s also published a great introductory blog post on importing RDF into Neo4j.) This operates as a simple plugin to Neo4j and the functions can be accessed as stored procedure calls through Cypher and also via an HTTP API as server extensions.

So, instead of writing something anew we’re going to create some wrapper functions onto the neosemantics stored procedures and extensions.

What else?

For accessing an LPG graph database we’re going to make use of the bolt_sips package from Florin Pătraşcu which implements a Neo4j driver for Elixir wrapped around the Bolt protocol.

And for accessing an RDF graph database we’re going to use the sparql_client package from Marcel Otto for dispatching queries to remote RDF models, which in turn brings in the rdf package for RDF processing in Elixir, as well as the sparql package for querying in-memory RDF models.

Roughly, the situation is as diagrammed below. Cypher queries are sent to the LPG graph database using the Elixir bolt_sips package, and may also call upon neosemantics stored procedures and extensions using the Elixir NeoSemantics wrappers we’ll define in this project. SPARQL queries are sent to the RDF graph database using the Elixir sparql_client package.

Elixir packages used by the :test_graph application to connect up LPG and RDF graph databases.

1. Create a ‘TestGraph’ project

First off, let’s create a new project TestGraph.

And we shall be building on top of some earlier work, especially reusing code from the TestNeo4j and TestQuery projects. (See the earlier posts Property graphs and Elixir and Querying RDF with Elixir for specific details.)

We create a new (supervised) project TestGraph:

% mix new test_graph --sup

a. Dependencies
We then declare dependencies on both bolt_sips and sparql_client in the mix.exs file (and we also make use of the hackney HTTP client):

defp deps do
  [
    # property graphs
    {:bolt_sips, "~> 1.5"},

    # rdf graphs
    {:sparql_client, "~> 0.2"},    # http client
    {:hackney, "~> 1.15"}
  ]

And we use Mix to add in the dependencies:

% mix deps.get

b. Configuration
We add this line to register the hackney client:

config :tesla, :adapter, Tesla.Adapter.Hackney

And we add these lines either to the main config.exs file or to any environment specific imports (e.g. dev.exs) with the details updated as required:

config :bolt_sips, Bolt,
  url: "bolt://neo4j:neo4jtest@localhost:7687"

Note that the url: option uses an explicit "bolt:" URI scheme and includes basic authentication credentials (here for testing neo4j:neo4jtest) within the URI userinfo component.

We’re also going to need this to declare the Neo4j service endpoint:

config :test_graph,
  neo4j_service: "http://neo4j:neo4jtest@localhost:7474"

And for managing SPARQL endpoints we’ll add three registered endpoints. (See the application setup below where we’ll use TestGraph.RDF.SPARQL.Client.sparql_endpoint/1 to set a :sparql_endpoint in the :test_graph application environment at startup.)

config :test_graph,
  :sparql, [ :sparql_dbpedia, :sparql_local, :sparql_wikidata ]config :test_graph, :sparql_dbpedia,
  url: "http://dbpedia.org/sparql"config :test_graph, :sparql_local,
  url: "http://localhost:7200/repositories/test-graph"config :test_graph, :sparql_wikidata,
  url: "https://query.wikidata.org/bigdata/namespace/wdq/sparql"

Finally, to simplify naming in IEx we’ll add in a .iex.exs configuration file so that a couple Bolt.Sips function names and the TestGraph function names (and delegates) can be treated as local. We’ll also alias the Cypher.Client and SPARQL.Client modules under TestGraph.LPG and TestGraph.RDF.

% cat .iex.exs
import Bolt.Sips, only: [config: 0, conn: 0]
import TestGraph
import TestGraph.Utilsalias TestGraph.LPG.Cypher.Client, as: Cypher_Client
alias TestGraph.RDF.SPARQL.Client, as: SPARQL_Client

Note that in aliasing the TestGraph.RDF.SPARQL.Client module we need to take care that we do not mask the public SPARQL.Client module. We finesse this by using the SPARQL_Client alias form, so that now both SPARQL.Client.* and SPARQL_Client.* function namespaces can coexist.

c. Application
We’ve created this project as a supervised application. So we now add the keyword option mod: line to our mix.exs file to automatically invoke the TestGraph.Application:

def application do
  [
    extra_applications: [:logger],
    mod: { TestGraph.Application, [] }
  ]
end

And we update the TestGraph.Application.start/2 function in lib/test_graph/application.ex as:

def start(_type, _args) do
  import Supervisor.Spec  TestGraph.RDF.SPARQL.Client.sparql_endpoint(:sparql_local)  children = [
    worker(Bolt.Sips, [Application.get_env(:bolt_sips, Bolt)])
  ]
  opts = [strategy: :one_for_one, name: TestGraph.Supervisor]
  Supervisor.start_link(children, opts)
end

The first line here just sets us up with a default SPARQL endpoint – in this case a local GraphDB instance – and adds a variable :sparql_endpoint to the :test_graph application environment when the application starts.

The application will now be started automatically and the Bolt connection can be tested by calling the Bolt.Sips.config/0 function:

% iex -S mix
Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]Interactive Elixir (1.8.1) - press Ctrl+C to exit (type h() ENTER for help)iex> config()
[
  socket: Bolt.Sips.Socket,
  port: 7687,
  hostname: 'localhost',
  retry_linear_backoff: [delay: 150, factor: 2, tries: 3],
  with_etls: false,
  ssl: false,
  timeout: 15000,
  max_overflow: 2,
  pool_size: 5,
  url: "bolt://localhost:7687",
  basic_auth: [username: "neo4j", password: "neo4jtest"]
]

See here for the project TestGraph code. (And note that the project TestGraph code also includes documentation which can be browsed here.)

2. Set up the databases

We’re going to be using Neo4j as the property graph database and public SPARQL endpoints to access remote RDF graph databases, although we have also used a GraphDB instance for local exploration.

a. Neo4j
See the previous post Property graphs and Elixir for details on setting up a local instance.

The neosemantics library is implemented as an extension mechanism. See installation instructions on the project page. Basically this amounts to copying the jar file into the <NEO_HOME>/plugins directory, adding a configuration directive to the <NEO_HOME>/conf/neo4j.conf file, and restarting the server. Simples.

And we also need to set up some indexes in the database (and restart the server):

CREATE INDEX ON :`Class`(`uri`);
CREATE INDEX ON :`Bnode`(`uri`);
CREATE INDEX ON :`URI`(`uri`);
CREATE INDEX ON :`Resource`(`uri`);

Neo4j also has a generic extension mechanism in its APOC (or, Awesome Procedures on Cypher) library. This is needed if you’re going to use the JSON-LD serialization format for RDF or to export Cypher graphs. There’s a handy video series to help with installation and use.

b. GraphDB
As mentioned in the post Working with SHACL and Elixir we make use of a GraphDB Free instance for local evaluation.

3. Lay out the project

So let’s get a quick overview of the project.

a. Project layout
Here’s the project module layout for the TestGraph and NeoSemantics modules.

% tree lib
lib
├── application.ex├── neo_semantics
│   ├── extension.ex
│   ├── inference.ex
│   └── mapping.ex
├── neo_semantics.ex├── test_graph
│   ├── graph.ex
│   ├── lpg
│   │   └── cypher
│   │       └── client.ex
│   ├── lpg.ex
│   ├── query.ex
│   ├── rdf
│   │   └── sparql
│   │       └── client.ex
│   ├── rdf.ex
│   └── utils.ex
└── test_graph.ex6 directories, 13 files

b. Data layout
Here’s the data layout for graphs and queries.

Note that graph models and queries are both expressed in Cypher (.cypher) for LPG databases, while for RDF databases graph models are expressed in an RDF serialization (e.g. Turtle, .ttl) and queries are expressed in the SPARQL Query language (.rq) or the SPARQL Update language (.ru). (The GraphGist format (.adoc) is used to document its embedded Cypher queries.)

:test_graph data layout for LPG and RDF graphs and queries.

We’ll define %TestGraph.Graph{} and %TestGraph.Query{} structs to help with access to the data files.

c. Graph struct
Let’s define a %TestGraph.Graph{} struct to keep some related data fields together. One of the fields :type will track whether this is a property graph (:lpg) or a semantic graph (:rdf). The other fields are straightforward data access fields:

:data – graph data
:file – name of graph file (within the graphs dir)
:path – absolute path of graph file
:uri – absolute URI of graph file (a file: scheme URI)

We can define the struct using the sigil syntax ~w for bare words, and the trailing a character marks these as atoms:

defmodule TestGraph.Graph do
  @moduledoc """
  Module providing a struct for graphs.
  """  defstruct ~w[data file path type uri]aend

We can test that this does the right thing by instantiating a vanilla struct:

iex> %TestGraph.Graph{}
%TestGraph.Graph{data: nil, file: nil, path: nil, type: nil, uri: nil}

And we can now define a %TestGraph.Graph{} constructor function TestGraph.Graph.new/3 which takes graph_data, graph_file, and graph_type as arguments for the :data, :file and :type fields, respectively. The :path and :uri fields are computed from the :file field.

def new(graph_data, graph_file, graph_type) do  graphs_dir =
    case graph_type do
      :lpg -> @lpg_dir <> "/graphs/"
      :rdf -> @rdf_dir <> "/graphs/"
      _ -> raise "! Unknown graph_type: " <> graph_type
    end  %__MODULE__{
    data: graph_data,
    file: graph_file,
    path: graphs_dir <> graph_file,
    type: graph_type,
    uri:  "file://" <> graphs_dir <> graph_file,
  }end

d. Query struct
We likewise define a %TestGraph.Query{} struct to keep some related data fields together.

defmodule TestGraph.Query do
  @moduledoc """
  Module providing a struct for queries.
  """  defstruct ~w[data file path type uri]aend

And again we can define a %TestGraph.Query{} constructor function TestGraph.Query.new/3.

def new(query_data, query_file, query_type) do  queries_dir =
    case query_type do
      :lpg -> @lpg_dir <> "/queries/"
      :rdf -> @rdf_dir <> "/queries/"
      _ -> raise "! Unknown query_type: " <> query_type
    end  %__MODULE__{
    data: query_data,
    file: query_file,
    path: queries_dir <> query_file,
    type: query_type,
    uri:  "file://" <> queries_dir <> query_file,
  }end

e. Data access
Now that we have our graph structs we can define data access functions to read and write the underlying data files.

We define a read/write pair read_graph/2 and write_graph/2 in TestGraph.LPG to access the data files and return a %TestGraph{} struct.

def read_graph(graph_file \\ graph_file()) do
  graphs_dir = @graphs_dir
  graph_data = File.read!(graphs_dir <> graph_file)  TestGraph.Graph.new(graph_data, graph_file, :lpg)
enddef write_graph(graph_data, graph_file \\ temp_graph_file()) do
  graphs_dir = @graphs_dir
  File.write!(graphs_dir <> graph_file, graph_data)  TestGraph.Graph.new(graph_data, graph_file, :lpg)
end

To read the saved LPG graph "books.cypher" we can use this call:

iex> graph = TestGraph.LPG.read_graph("books.cypher")
%TestGraph.Graph{
  data: "CREATE\n(book:Book {\n    iri: \"urn:isbn:978-1-68050-252-7\",\n    date: \"2018-03-14\",\n    format: \"Paper\",\n    title: \"Adopting Elixir\"\n}),\n(author1:Author { iri: \"https://twitter.com/bgmarx\" }),\n(author2:Author { iri: \"https://twitter.com/josevalim\" }),\n(author3:Author { iri: \"https://twitter.com/redrapids\" }),\n(publisher:Publisher { iri: \"https://pragprog.com/\" })\n\nCREATE\n(book)-[:AUTHORED_BY { role: \"first author\" }]->(author1),\n(book)-[:AUTHORED_BY { role: \"second author\" }]->(author2),\n(book)-[:AUTHORED_BY { role: \"third author\" }]->(author3),\n(book)-[:PUBLISHED_BY]->(publisher)\n\n;\n",
  file: "default.cypher",
  path: ".../test_graph/priv/lpg/graphs/default.cypher",
  type: :lpg,
  uri: "file:///.../test_graph/priv/lpg/graphs/default.cypher"
}iex> graph.data
"CREATE\n(book:Book {\n    iri: \"urn:isbn:978-1-68050-252-7\",\n    date: \"2018-03-14\",\n    format: \"Paper\",\n    title: \"Adopting Elixir\"\n}),\n(author1:Author { iri: \"https://twitter.com/bgmarx\" }),\n(author2:Author { iri: \"https://twitter.com/josevalim\" }),\n(author3:Author { iri: \"https://twitter.com/redrapids\" }),\n(publisher:Publisher { iri: \"https://pragprog.com/\" })\n\nCREATE\n(book)-[:AUTHORED_BY { role: \"first author\" }]->(author1),\n(book)-[:AUTHORED_BY { role: \"second author\" }]->(author2),\n(book)-[:AUTHORED_BY { role: \"third author\" }]->(author3),\n(book)-[:PUBLISHED_BY]->(publisher)\n\n;\n"iex> graph.uri
"file:///.../test_graph/priv/lpg/graphs/default.cypher"

To write a new LPG graph "new.cypher" with the same data:

iex> graph.data |> TestGraph.LPG.write_graph("new.cypher")
%TestGraph.Graph{
  data: "CREATE\n(book:Book {\n    iri: \"urn:isbn:978-1-68050-252-7\",\n    date: \"2018-03-14\",\n    format: \"Paper\",\n    title: \"Adopting Elixir\"\n}),\n(author1:Author { iri: \"https://twitter.com/bgmarx\" }),\n(author2:Author { iri: \"https://twitter.com/josevalim\" }),\n(author3:Author { iri: \"https://twitter.com/redrapids\" }),\n(publisher:Publisher { iri: \"https://pragprog.com/\" })\n\nCREATE\n(book)-[:AUTHORED_BY { role: \"first author\" }]->(author1),\n(book)-[:AUTHORED_BY { role: \"second author\" }]->(author2),\n(book)-[:AUTHORED_BY { role: \"third author\" }]->(author3),\n(book)-[:PUBLISHED_BY]->(publisher)\n\n;\n",
  file: "new.cypher",
  path: ".../test_graph/priv/lpg/graphs/new.cypher",
  type: :lpg,
  uri: "file:///.../test_graph/priv/lpg/graphs/new.cypher"
}

And similarly for LPG queries with a read/write pair read_query/2 and write_query/2 in TestGraph.LPG.

And then we replicate this pattern for RDF graphs and queries in TestGraph.RDF.

And to make these easier to access we define some top-level delegates:

defdelegate read_lpg_graph(arg), to: TestGraph.LPG, as: :read_graph
defdelegate read_lpg_query(arg), to: TestGraph.LPG, as: :read_query

Now we can simply call read_lpg_graph/1 and read_lpg_query/1 instead of TestGraph.LPG.read_graph/1 and TestGraph.LPG.read_query/1. And likewise for LPG writes and for RDF reads/writes.

And to list the graphs and queries we have some helper functions together with corresponding top-level delegates:

iex> list_lpg_graphs()
["movies.cypher", "books.cypher", "default.cypher"]iex> list_rdf_graphs()
["books.ttl", "urn_isbn_978-1-68050-252-7.ttl", "http___dbpedia.org_resource_London.ttl", "london100.ttl", "london.ttl", "nobelprizes.ttl", "bibo.ttl", "tony.ttl", "temp.ttl", "http___example.org_Elixir.ttl", "elixir.ttl", "default.ttl", "cypher.ttl", "neo4j.ttl", "hello.ttl"]iex> list_lpg_queries()
...iex> list_rdf_queries()
...

f. Data query
Likewise for our query structs we have read/write access functions which this time return a %TestGraph.Query{} struct:

def read_query(query_file \\ query_file()) do
  queries_dir = @queries_dir
  query_data = File.read!(queries_dir <> query_file)  TestGraph.Query.new(query_data, query_file, :lpg)
enddef write_query(query_data, query_file \\ temp_query_file()) do
  queries_dir = @queries_dir
  File.write!(queries_dir <> query_file, query_data)  TestGraph.Query.new(query_data, query_file, :lpg)
end

To read the saved LPG query "default.cypher" we can use this call:

iex> query = TestGraph.LPG.read_query("default.cypher")
%TestGraph.Query{
  data: "match (n) return n limit 1\n",
  file: "default.cypher",
  path: ".../test_graph/priv/lpg/queries/default.cypher",
  type: :lpg,
  uri: "file:///.../test_graph/priv/lpg/queries/default.cypher"
}

Query writes are the same as we earleir discussed for graph writes.

Functions to dispatch the queries are defined in the TestGraph.LPG.Cypher.Client module which we’ve aliased to Cypher_Client. There are two forms for making the remote query: rquery/2 and rquery!/2.

iex> Cypher_Client.rquery! query.data
[
  %{
    "n" => %Bolt.Sips.Types.Node{
      id: 26703,
      labels: ["Book"],
      properties: %{
        "date" => "2018-03-14",
        "format" => "Paper",
        "iri" => "urn:isbn:978-1-68050-252-7",
        "title" => "Adopting Elixir"
      }
    }
  }
]

We can simplify this with the top-level delegates cypher and cypher!:

iex> cypher! query.data
[
  %{
    "n" => %Bolt.Sips.Types.Node{
      id: 26703,
      labels: ["Book"],
      properties: %{
        ...
      }
    }
  }
]

And, as before, we have the same parallel functions and delegate declarations for RDF:

iex> TestGraph.RDF.read_query("default.rq").data |> SPARQL_Client.rquery! 
#RDF.Graph{name: nil
     ~I<http://dbpedia.org/resource/London>
         ~I<http://www.w3.org/2003/01/geo/wgs84_pos#lat>
             %RDF.Literal{value: 51.50722122192383, lexical: "5.150722122192383E1", datatype: ~I<http://www.w3.org/2001/XMLSchema#double>}}

Or in delegate form:

iex> read_rdf_query("default.rq").data |> sparql! |> RDF.Turtle.write_string!
"<http://dbpedia.org/resource/London>\n    <http://www.w3.org/2003/01/geo/wgs84_pos#lat> 5.15072212219238E1 .\n"

These queries are targeted at native graphs in their respective databases. We now need to consider how to convert graphs from one organization to another so that they may be imported and exported to and from different graph databases. And for that we’re going to use the neosemantics library.

4. Wrap the ‘neosemantics’ library

The neosemantics library provides stored procedures across three main functionalities:

ingestion → NeoSemantics
mapping → NeoSemantics.Mapping
inference → NeoSemantics.Inference

And there is also a server extension mechanism for RDF export:

extension → NeoSemantics.Extension

We store these NeoSemantics modules in parallel with the TestGraph module layout.

So, let’s take the semantics.importRDF procedure first. This is called in a Cypher query with three arguments as:

call semantics.importRDF(url, format, props)

We can wrap this call and dispatch it with this wrapper function:

def import_rdf(conn, uri, format) do
  cypher = "call semantics.importRDF(\"" <> uri <> "\", \"" <> format <> "\", {})"
  Bolt.Sips.query(conn, cypher)
end

Note that the wrapper functions use snake case which is more usual in Elixir, e.g. NeoSemantics.import_rdf/3 for semantics.importRDF.

Some other notes are necessary. We’ve added in one argument and subtracted another. In order to dispatch the query we need to add in the bolt_sips connection argument conn. This is added as a first argument so that it can be piped in regular Elixir style. We’re also omitting the props argument and for now will just supply an empty map.

We build a Cypher querystring using quoted strings for the uri and format parameters. This querystring is then passed to the Bolt.Sips.query/2 function.

Similarly for the other neosemantics ingestion procedures

call semantics.previewRDF(url, format, props)call semantics.previewRDFSnippet(url, format, props)call semantics.streamRDF(url, format, props)call semantics.liteOntoImport(url, format)

we define the wrapper functions NeoSemantics.preview_rdf/3 NeoSemantics.preview_rdf_snippet/3, NeoSemantics.stream_rdf/3, and NeoSemantics.lite_onto_import/3.

We’re going to be focussing on the import and export capabilities that neosemantics provides so will not discuss the mapping and inference functionalities further here, other than to note that all library procedures have been wrapped and are available for testing.

5. Test the ‘neosemantics’ wrapper library

So, let’s try this out.

Now we’ve got a default RDF file in the project under the priv/ dir. We’ll use the function TestGraph.RDF.read_graph/1 (or rather the TestGraph.read_rdf_graph/1 delegate) to get the URI for this file and save that to a uri variable. And since this is a Turtle file let’s save that format string to the fmt variable.

iex> uri = read_rdf_graph().uri
"file:///.../test_graph/priv/rdf/graphs/default.ttl"iex> fmt = "Turtle"
"Turtle"

Note also that the wrapper functions come in two flavours: with and without a trailing bang (!) character. The simple form returns a success or failure tuple which can be used in pattern matching, whereas the marked form returns a plain value or raises an exception. When a positive outcome is expected the marked form may be easier to use. (See Elixir naming conventions for more info.)

So, using the simple form first we have this usage pattern where we match on a positive outcome with the :ok atom and save the response into a resp variable:

iex> {:ok, resp} = (conn() |> NeoSemantics.import_rdf(uri, fmt))
{:ok,
 [
   %{
     "extraInfo" => "",
     "namespaces" => %{
       "http://purl.org/dc/elements/1.1/" => "dc",
       "http://purl.org/dc/terms/" => "dct",
       "http://purl.org/ontology/bibo/" => "ns0",
       "http://schema.org/" => "sch",
       "http://www.w3.org/1999/02/22-rdf-syntax-ns#" => "rdf",
       "http://www.w3.org/2000/01/rdf-schema#" => "rdfs",
       "http://www.w3.org/2002/07/owl#" => "owl",
       "http://www.w3.org/2004/02/skos/core#" => "skos"
     },
     "terminationStatus" => "OK",
     "triplesLoaded" => 8
   }
 ]}iex> resp
[
   %{
     "extraInfo" => "",
     "namespaces" => %{
       "http://purl.org/dc/elements/1.1/" => "dc",
       ...
     },
     "terminationStatus" => "OK",
     "triplesLoaded" => 8
   }
]iex> conn() |> Cypher_Client.test()
[%{"nodes" => 6, "paths" => 8, "relationships" => 4}]

Now with the marked form we can get at the response directly:

iex> conn() |> NeoSemantics.import_rdf!(uri, fmt)
[
  %{
    "extraInfo" => "",
    "namespaces" => %{
      "http://purl.org/dc/elements/1.1/" => "dc",
      ...
    },
    "terminationStatus" => "OK",
    "triplesLoaded" => 8
  }
]iex> conn() |> Cypher_Client.test()
[%{"nodes" => 6, "paths" => 8, "relationships" => 4}]

And as we’re dealing with Turtle here we could just use the wrapper function NeoSemantics.import_turtle!/2. Wrapper functions are defined for each of the recognized RDF formats: import_jsonld/2, import_ntriples/2, import_rdfxml/2, import_trig/2, import_turtle/2 (along with their marked forms).

iex> conn() |> NeoSemantics.import_turtle!(uri)

We can compare fetching nodes with Cypher (via bolt_sips) and SPARQL (via sparql_client):

iex> cypher! "match (n) return n skip 3 limit 2"
[
  %{
    "n" => %Bolt.Sips.Types.Node{
      id: 23448,
      labels: ["Resource"],
      properties: %{
        "uri" => "http://nobelprize.org/nobel_prizes/chemistry/laureates/2005/speedread.html"
      }
    }
  },
  %{
    "n" => %Bolt.Sips.Types.Node{
      id: 23449,
      labels: ["Resource", "ns1__University"],
      properties: %{
        "rdfs__label" => "Case Western Reserve University",
        "uri" => "http://data.nobelprize.org/resource/university/Case_Western_Reserve_University"
      }
    }
  }
]iex> sparql! "select ?s where {?s ?p ?o} offset 5 limit 2"
%SPARQL.Query.Result{
  results: [
    %{"s" => ~I<http://data.nobelprize.org/terms/fileType>},
    %{"s" => ~I<http://data.nobelprize.org/resource/filetype/Biographical>}
  ],
  variables: ["s"]
}

6. Create RDF, transform and store as LPG

Let’s try first creating some RDF and transforming to LPG, and then go on to using a published RDF dataset and transforming that.

a. Building a graph
As a first example let’s try building up some RDF by hand using the rdf package. This example is taken from the Early steps in Elixir and RDF post.

Here we define a TestGraph.RDF.book/0 function which makes use of a couple defvocab macros to define the DC and BIBO namespaces.

def book() do
    import RDF.Sigils    ~I<urn:isbn:978-1-68050-252-7>
    |> RDF.type(BIBO.Book)
    |> DC.creator(~I<https://twitter.com/bgmarx>,
         ~I<https://twitter.com/josevalim>,
         ~I<https://twitter.com/redrapids>)
    |> DC.date(RDF.date("2018-03-14"))
    |> DC.format(~L"Paper")
    |> DC.publisher(~I<https://pragprog.com/>)
    |> DC.title(~L"Adopting Elixir"en)
  end

We can now call this as:

iex> book = TestGraph.RDF.book()
#RDF.Description{subject: ~I<urn:isbn:978-1-68050-252-7>
     ~I<http://purl.org/dc/elements/1.1/creator>
        ~I<https://twitter.com/bgmarx>
         ~I<https://twitter.com/josevalim>
         ~I<https://twitter.com/redrapids>
     ~I<http://purl.org/dc/elements/1.1/date>
         %RDF.Literal{value: ~D[2018-03-14], datatype: ~I<http://www.w3.org/2001/XMLSchema#date>}
     ~I<http://purl.org/dc/elements/1.1/format>
         ~L"Paper"
     ~I<http://purl.org/dc/elements/1.1/publisher>
         ~I<https://pragprog.com/>
     ~I<http://purl.org/dc/elements/1.1/title>
         ~L"Adopting Elixir"en
     ~I<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
         ~I<http://purl.org/ontology/bibo/Book>}

And we can serialize this to Turtle and save to file as:

iex> book |> RDF.Turtle.write_string! |> write_rdf_graph("books.ttl")
%TestGraph.Graph{
  data: "<urn:isbn:978-1-68050-252-7>\n    a <http://purl.org/ontology/bibo/Book> ;\n    <http://purl.org/dc/elements/1.1/creator> <https://twitter.com/bgmarx>, <https://twitter.com/josevalim>, <https://twitter.com/redrapids> ;\n    <http://purl.org/dc/elements/1.1/date> \"2018-03-14\"^^<http://www.w3.org/2001/XMLSchema#date> ;\n    <http://purl.org/dc/elements/1.1/format> \"Paper\" ;\n    <http://purl.org/dc/elements/1.1/publisher> <https://pragprog.com/> ;\n    <http://purl.org/dc/elements/1.1/title> \"Adopting Elixir\"@en .\n",
  file: "books.ttl",
  path: ".../test_graph/priv/rdf/graphs/books.ttl",
  type: :rdf,
  uri: "file:///.../test_graph/priv/rdf/graphs/books.ttl"
}

We can read this back into a %TestGraph.Graph{} struct using the TestGraph.read_rdf_graph/1 function:

iex> graph = read_rdf_graph("books.ttl")
%TestGraph.Graph{
  data: "<urn:isbn:978-1-68050-252-7>\n    a <http://purl.org/ontology/bibo/Book> ;\n    <http://purl.org/dc/elements/1.1/creator> <https://twitter.com/bgmarx>, <https://twitter.com/josevalim>, <https://twitter.com/redrapids> ;\n    <http://purl.org/dc/elements/1.1/date> \"2018-03-14\"^^<http://www.w3.org/2001/XMLSchema#date> ;\n    <http://purl.org/dc/elements/1.1/format> \"Paper\" ;\n    <http://purl.org/dc/elements/1.1/publisher> <https://pragprog.com/> ;\n    <http://purl.org/dc/elements/1.1/title> \"Adopting Elixir\"@en .\n",
  file: "books.ttl",
  path: ".../test_graph/priv/rdf/graphs/books.ttl",
  type: :rdf,
  uri: "file:///.../test_graph/priv/rdf/graphs/books.ttl"
}

And now we can use the NeoSemantics.import_turtle!/2 function as:

iex> conn() |> NeoSemantics.import_turtle!(graph.uri)
[
  %{
    "extraInfo" => "",
    "namespaces" => %{
      "http://purl.org/dc/elements/1.1/" => "dc",
      "http://purl.org/dc/terms/" => "dct",
      "http://purl.org/ontology/bibo/" => "ns0",
      "http://schema.org/" => "sch",
      "http://www.w3.org/1999/02/22-rdf-syntax-ns#" => "rdf",
      "http://www.w3.org/2000/01/rdf-schema#" => "rdfs",
      "http://www.w3.org/2002/07/owl#" => "owl",
      "http://www.w3.org/2004/02/skos/core#" => "skos"
    },
    "terminationStatus" => "OK",
    "triplesLoaded" => 8
  }
]

And if we check this with Cypher_Client.test/0 we get this:

iex> Cypher_Client.test
[%{"nodes" => 6, "paths" => 8, "relationships" => 4}]

And yields this graph view in a Neo4j browser:

Simple RDF ‘books’ graph imported into Neo4j.

(The unattached node is a special node created by neosemantics to hold the RDF namespace table.)

Let’s try something more interesting.

b. Reusing a graph
We’ll use the LOD dataset nobelprizes from the Nobel Prize organisation which has 87,369 unique statements. This data is released under a CC0 waiver and is thus in the public domain. There’s a copy of this in the priv/rdf/graphs/ directory transformed from N-Triples to Turtle for ease of use. (Note that the dataset can be further explored at data.nobelprize.org.)

Let’s first import LOD dataset this into a new GraphDB database for validating and exploring. We can list out the unique RDF types with a simple SPARQL query:

GraphDB workbench window for SPARQL query and result set on ‘nobelprizes’ RDF graph.

We can again use the NeoSemantics.import_turtle!/2 function as:

iex> graph = read_rdf_graph("nobelprizes.ttl")
iex> conn() |> NeoSemantics.import_turtle!(graph.uri)

Or more simply we could also use this TestGraph.import_rdf_from_graph/1 call:

iex> import_rdf_from_graph("nobelprizes.ttl")

And if we check the state of the database:

iex> Cypher_Client.test
[%{"nodes" => 21027, "paths" => 86534, "relationships" => 43267}]

We can now query for distinct labels (which are mapped from RDF types):

iex> Cypher_Client.rquery!("match (n) return distinct labels(n)")[
  %{"labels(n)" => ["NamespacePrefixDefinition"]},
  %{"labels(n)" => ["Resource", "ns0__AwardFile"]},
  %{"labels(n)" => ["Resource"]},
  %{"labels(n)" => ["Resource", "ns0__Category"]},
  %{"labels(n)" => ["Resource", "ns0__FileType"]},
  %{"labels(n)" => ["Resource", "ns1__City"]},
  %{"labels(n)" => ["Resource", "ns1__University"]},
  %{"labels(n)" => ["Resource", "ns0__Laureate", "ns3__Person"]},
  %{"labels(n)" => ["Resource", "ns0__Laureate", "ns3__Organization"]},
  %{"labels(n)" => ["Resource", "ns1__Country"]},
  %{"labels(n)" => ["Resource", "ns0__NobelPrize", "ns1__Award"]},
  %{"labels(n)" => ["Resource", "ns1__Award", "ns0__LaureateAward"]},
  %{"labels(n)" => ["Resource", "ns0__PrizeFile"]},
  %{"labels(n)" => ["Resource", "rdf__Property"]},
  %{"labels(n)" => ["Resource", "rdfs__Class"]}
]

And we can now continue to explore this graph within Neo4j:

Neo4j browser pane showing Cypher query and results on imported ‘nobelprizes’ RDF graph.

7. Fetch RDF, transform and store as LPG

Our next step will be to query a remote datastore for an RDF dataset and to transform and import that into Neo4j.

We’ll use the demo query for SPARQL CONSTRUCT from the sparql_client project:

iex> read_rdf_query("elixir.rq").data |> IO.puts
prefix : <http://example.org/>
prefix dbo: <http://dbpedia.org/ontology/>
prefix dbp: <http://dbpedia.org/property/>
prefix foaf: <http://xmlns.com/foaf/0.1/>construct {
  :Elixir
    :name     ?name ;
    :homepage ?homepage ;
    :license  ?license ;
    :creator  ?creator .
}
where  {
  <http://dbpedia.org/resource/Elixir_(programming_language)>
    foaf:name     ?name ;
    foaf:homepage ?homepage ;
    dbp:creator   ?creator ;
    dbo:license   ?license .
}:ok

Note that this query modifies the data by rewriting subject and properties into its own example namespace. So the graph is being transformed on the way out.

We can retrieve this stored RDF query, send the query to the RDF database (by default DBpedia), get the data field and write this into a new graph with the following call:

iex> elixir = (
...>   read_rdf_query("elixir.rq").data
...>   |> SPARQL_Client.rquery!
...>   |> RDF.Turtle.write_string!
...>   |> write_rdf_graph("elixir.ttl")
...> )
%TestGraph.Graph{
  data: "<http://example.org/Elixir>\n    <http://example.org/creator> <http://dbpedia.org/resource/José_Valim> ;\n    <http://example.org/homepage> <http://elixir-lang.org> ;\n    <http://example.org/license> <http://dbpedia.org/resource/Apache_License> ;\n    <http://example.org/name> \"Elixir\"@en .\n",
  file: "elixir.ttl",
  path: ".../test_graph/priv/rdf/graphs/elixir.ttl",
  type: :rdf,
  uri: "file:///.../test_graph/priv/rdf/graphs/elixir.ttl"
}

And now we can transform this RDF graph into a property graph and import into Neo4j using the NeoSemantics.import_turtle!/1 function as:

iex> conn() |> NeoSemantics.import_turtle!(elixir.uri)
[
  %{
    "extraInfo" => "",
    "namespaces" => %{
      "http://example.org/" => "ns0",
      "http://purl.org/dc/elements/1.1/" => "dc",
      "http://purl.org/dc/terms/" => "dct",
      "http://schema.org/" => "sch",
      "http://www.w3.org/1999/02/22-rdf-syntax-ns#" => "rdf",
      "http://www.w3.org/2000/01/rdf-schema#" => "rdfs",
      "http://www.w3.org/2002/07/owl#" => "owl",
      "http://www.w3.org/2004/02/skos/core#" => "skos"
    },
    "terminationStatus" => "OK",
    "triplesLoaded" => 4
  }
]

Or more simply we could also use this TestGraph.import_rdf_from_query/1 call:

iex> import_rdf_from_query("elixir.rq")

And if we check the state of the database:

iex> Cypher_Client.test
[%{"nodes" => 5, "paths" => 6, "relationships" => 3}]

Demo RDF ‘elixir’ graph queried from DBpedia and imported into Neo4j.

As noted earlier, the unattached node here is for the RDF namespace table.

So, to summarize, we have two routes for querying RDF and importing into LPG: A – explicit, B – implicit.

Two routes for querying an RDF store and importing into an LPG database: A — explicit, B — implicit.

8. Roundtrip from RDF to LPG, and back to RDF

So far we have just looked at getting RDF into a property graph. Now, we’d also like to be able to export a property graph as RDF. And in that way we would then have the posssibilty of roundtripping a graph from one organization to another.

The neosemantics library makes RDF export available as server extensions that enable new endpoints to be created in the HTTP API.

There are two patterns – single node or arbitary query:

GET → return RDF for a specific node, identified by ID for vanilla LPG or by URI for imported RDF graphs:
a. GET /rdf/describe/id?nodeid=0
b. GET /rdf/describe/uri?nodeuri=http://dataset.com#id_1234
POST → return RDF for a given Cypher query, both for vanilla LPG graphs as well as for imported RDF graphs:
a. POST /rdf/cypher { “cypher” : “MATCH (n:Person { name : ‘Keanu Reeves’})-[r]-(m:Movie) RETURN n,r,m” }b. POST /rdf/cypheronrdf { “cypher”:”MATCH (a:Resource {uri:’http://dataset/indiv#153'})-[r]-(b) RETURN a, r, b”}

Note that 1a,b and 2a take optional parameters excludeContext and showOnlyMapped, respectively. See the neosemantics project for futher details.

The NeoSemantics.Extension module supports these functionalities with the following calls:

by node:
a. node_by_id/2, where second parameter exclude_context is optional
b. node_by_uri/2, where second parameter exclude_context is optional
by query:
a. cypher/2, where second parameter show_only_mapped is optional
b. cypher_on_rdf/1

By default the API calls return RDF in a JSON-LD serialization although this can be changed by modifiying the HTTP Accept header. The NeoSemantics.Extension functions currently just returns Turtle, although can be simply modified to allow for other serializations.

For better handling we’ll define the following top-level TestGraph composite functions:

by node:
a. export_rdf_by_id/2
b. export_rdf_by_uri/2
by query:
a. export_rdf_by_query/2
b. export_rdf_by_query_on_rdf/2

These calls will also save the RDF graphs, as well as providing some minor conversions as required.

a. Export RDF
We can try this out now. Let’s list out some LPG node IDs.

iex> Cypher_Client.node_ids
[
  %{"id(n)" => 14551},
  %{"id(n)" => 35096},
  %{"id(n)" => 35097},
  %{"id(n)" => 35098},
  %{"id(n)" => 35099},
  %{"id(n)" => 35100}
]

So, picking out a node by ID (35099, say) we can export RDF for this LPG node as:

iex> export_rdf_by_id(35099)
%TestGraph.Graph{
  data: "@prefix neovoc: <neo4j://defaultvocabulary#> .\n@prefix neoind: <neo4j://indiv#> .\n\n\nneoind:35099 a neovoc:Resource, neovoc:ns0__Book;\n  neovoc:dc__creator neoind:35096, neoind:35097, neoind:35100;\n  neovoc:dc__date \"2018-03-14\";\n  neovoc:dc__format \"Paper\";\n  neovoc:dc__publisher neoind:35098;\n  neovoc:dc__title \"Adopting Elixir\";\n  neovoc:uri \"urn:isbn:978-1-68050-252-7\" .\n",
  file: "35099.ttl",
  path: ".../test_graph/priv/rdf/graphs/35099.ttl",
  type: :rdf,
  uri: "file:///.../test_graph/priv/rdf/graphs/35099.ttl"
}

The RDF graph is saved into the RDF graphs directory. And we can retrieve this RDF as:

iex> read_rdf_graph("35099.ttl").data |> IO.puts
@prefix neovoc: <neo4j://defaultvocabulary#> .
@prefix neoind: <neo4j://indiv#> .neoind:35099 a neovoc:Resource, neovoc:ns0__Book;
neovoc:dc__creator neoind:35096, neoind:35097, neoind:35100;
neovoc:dc__date "2018-03-14";
neovoc:dc__format "Paper";
neovoc:dc__publisher neoind:35098;
neovoc:dc__title "Adopting Elixir";
neovoc:uri "urn:isbn:978-1-68050-252-7" .:ok

As a convenience the TestGraph.export_rdf_by_id/2 function writes the RDF (as Turtle) into a file named with the same ID:

% ls priv/rdf/graphs/35099.ttl
priv/rdf/graphs/35099.ttliex> cat priv/rdf/graphs/35099.ttl
@prefix neovoc: <neo4j://defaultvocabulary#> .
@prefix neoind: <neo4j://indiv#> ....

Likewise we can export by URI and the RDF will be written (as Turtle) into a file named with a safed form of the URI.

iex> export_rdf_by_uri("urn:isbn:978-1-68050-252-7")
%TestGraph.Graph{
  data: "@prefix neovoc: <neo4j://vocabulary#> .\n\n\n<urn:isbn:978-1-68050-252-7> a <http://purl.org/ontology/bibo/Book>;\n  <http://purl.org/dc/elements/1.1/creator> <https://twitter.com/bgmarx>, <https://twitter.com/josevalim>,\n    <https://twitter.com/redrapids>;\n  <http://purl.org/dc/elements/1.1/date> \"2018-03-14\";\n  <http://purl.org/dc/elements/1.1/format> \"Paper\";\n  <http://purl.org/dc/elements/1.1/publisher> <https://pragprog.com/>;\n  <http://purl.org/dc/elements/1.1/title> \"Adopting Elixir\" .\n",
  file: "urn_isbn_978-1-68050-252-7.ttl",
  path: ".../test_graph/priv/rdf/graphs/urn_isbn_978-1-68050-252-7.ttl",
  type: :rdf,
  uri: "file:///.../test_graph/priv/rdf/graphs/urn_isbn_978-1-68050-252-7.ttl"
}

And again we can retrieve the RDF as:

iex> read_rdf_graph("urn_isbn_978-1-68050-252-7.ttl").data |> IO.puts
@prefix neovoc: <neo4j://defaultvocabulary#> .
@prefix neoind: <neo4j://indiv#> .neoind:35099 a neovoc:Resource, neovoc:ns0__Book;
neovoc:dc__creator neoind:35096, neoind:35097, neoind:35100;
neovoc:dc__date "2018-03-14";
neovoc:dc__format "Paper";
neovoc:dc__publisher neoind:35098;
neovoc:dc__title "Adopting Elixir";
neovoc:uri "urn:isbn:978-1-68050-252-7" .:ok

And this also works for native LPG graphs. Let’s import the standard Movies graph.

iex> cypher_clear()
%{stats: %{"nodes-deleted" => 6, "relationships-deleted" => 4}, type: "w"}iex> lpg_movies().data |> cypher!
%{
  stats: %{
    "labels-added" => 171,
    "nodes-created" => 171,
    "properties-set" => 564,
    "relationships-created" => 253
  },
  type: "w"
}

Now let’s inspect a single node:

iex> Cypher_Client.node_by_id(6355)
[
  %{
    "n" => %Bolt.Sips.Types.Node{
      id: 6355,
      labels: ["Person"],
      properties: %{"born" => 1964, "name" => "Keanu Reeves"}
    }
 }
]

And we can export the RDF for that node as:

iex> export_rdf_by_id(6355).data |> IO.puts
@prefix neovoc: <neo4j://defaultvocabulary#> .
@prefix neoind: <neo4j://indiv#> .neoind:6355 a neovoc:Person;
neovoc:ACTED_IN neoind:6354, neoind:6363, neoind:6364, neoind:6365, neoind:6441, neoind:6454,
neoind:6508;
neovoc:born "1964"^^<http://www.w3.org/2001/XMLSchema#long>;
neovoc:name "Keanu Reeves" .:ok

b. Roundtrip RDF
We’re now in a position to string this all together, to roundtrip from RDF to LPG back to RDF.

We’ve defined a couple of example scripts with this project which can be run from the command line.The script london_roundtrip.exs has just two calls: TestGraph.import_rdf_from_graph/1 and TestGraph.export_rdf_by_uri/1, i.e. put RDF into an LPG store, and get RDF out:

import TestGraph# subject resource to query (dbr:London)
resource = "http://dbpedia.org/resource/London"# # read (and save) graph from DBpedia by querying
# import_rdf_from_query("london.rq")# read from saved graph
import_rdf_from_graph("london.ttl")# print out exported graph
IO.puts export_rdf_by_uri(resource).data

(For extra points we could go all the way and query the RDF from a remote database by using the TestGraph.import_rdf_from_query/1 call instead.)

Let’s first check the saved graph "london.ttl":

iex> read_rdf_graph("london.ttl").data
"<http://dbpedia.org/resource/London>\n    <http://www.w3.org/2000/01/rdf-schema#label> \"\\u0644\\u0646\\u062F\\u0646\"@ar, \"London\"@de, \"London\"@en, \"Londres\"@es, \"Londres\"@fr, \"Londra\"@it, \"\\u30ED\\u30F3\\u30C9\\u30F3\"@ja, \"Londen\"@nl, \"Londyn\"@pl, \"Londres\"@pt, \"\\u041B\\u043E\\u043D\\u0434\\u043E\\u043D\"@ru, \"\\u4F26\\u6566\"@zh ;\n    <http://www.w3.org/2003/01/geo/wgs84_pos#geometry> \"POINT(-0.12749999761581 51.507221221924)\"^^<http://www.openlinksw.com/schemas/virtrdf#Geometry> ;\n    <http://dbpedia.org/property/octLowC> 10.900000000000000355, 8.4000000000000003553 ;\n    <http://dbpedia.org/ontology/leaderTitle> \"Elected body\"@en, \"European Parliament\"@en, \"London Assembly\"@en, \"Mayor\"@en, \"UK Parliament\"@en ;\n    <http://dbpedia.org/property/junPrecipitationMm> 45.100000000000001421 ;\n    <http://www.w3.org/2003/01/geo/wgs84_pos#lat> \"51.507221221923828125\"^^<http://www.w3.org/2001/XMLSchema#float> ;\n    <http://dbpedia.org/property/yearPrecipitationMm> 601.70000000000004547 ;\n    <http://dbpedia.org/property/yearPrecipitationDays> 109.5 ;\n..."

This london_roundtrip.exs script is invoked at the command line by using mix run as:

% mix run examples/london_roundtrip.exs
@prefix neovoc: <neo4j://vocabulary#> .<http://dbpedia.org/resource/London>
  <http://dbpedia.org/ontology/PopulatedPlace/areaTotal>
"1572.0";
  <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> "5518.0";
  <http://dbpedia.org/ontology/abstract> "Ло́ндон (англ. London [ˈlʌndən] (инф.)) — столица и крупнейший город Соединённого Королевства Великобритании и Северной Ирландии. ...";
  <http://dbpedia.org/ontology/leaderTitle> "UK Parliament";
  <http://dbpedia.org/ontology/populationDensity> 5.518E3;
  <http://dbpedia.org/ontology/utcOffset> "±00:00UTC";
...
  <http://xmlns.com/foaf/0.1/name> "London" .

So that seems to have worked. We’ve got RDF being printed out which we can validate, etc.

But let’s check that we actually have imported the RDF graph into the LPG database. Back in IEx we can dump the LPG graph using the Cypher_Client.dump/1 function, or cypher_dump delegate. This uses the APOC call apoc.export.cypher.all() to export the complete database. (There is a corresponding Cypher_Client.dump/2 function which wraps the APOC call apoc.export.cypher.query() to export a particular graph defined by Cypher query.)

iex> cypher_dump("london.cypher")
[
  %{
    "batchSize" => 20000,
    "batches" => 1,
    "cleanupStatements" => nil,
    "cypherStatements" => nil,
    "file" => ".../test_graph/priv/lpg/graphs/london.cypher",
    "format" => "cypher",
    "nodeStatements" => nil,
    "nodes" => 2,
    "properties" => 151,
    "relationshipStatements" => nil,
    "relationships" => 0,
    "rows" => 2,
    "schemaStatements" => nil,
    "source" => "database: nodes(2), rels(0)",
    "time" => 11
  }
]

And we can read this graph to see that it is indeed the same kind of thing:

iex> read_lpg_graph("london.cypher").data
"CREATE (:`NamespacePrefixDefinition`:`UNIQUE IMPORT LABEL` {`http://dbpedia.org/ontology/`:\"ns2\", `http://dbpedia.org/ontology/PopulatedPlace/`:\"ns3\", `http://dbpedia.org/property/`:\"ns1\", `http://purl.org/dc/elements/1.1/`:\"dc\", `http://purl.org/dc/terms/`:\"dct\", `http://schema.org/`:\"sch\", `http://www.georss.org/georss/`:\"ns4\", `http://www.w3.org/1999/02/22-rdf-syntax-ns#`:\"rdf\", `http://www.w3.org/2000/01/rdf-schema#`:\"rdfs\", `http://www.w3.org/2002/07/owl#`:\"owl\", `http://www.w3.org/2003/01/geo/wgs84_pos#`:\"ns0\", `http://www.w3.org/2004/02/skos/core#`:\"skos\", `http://xmlns.com/foaf/0.1/`:\"ns5\", `UNIQUE IMPORT ID`:67776});\nCREATE (:`Resource`:`UNIQUE IMPORT LABEL` {`ns0__geometry`:\"POINT(-0.12749999761581 51.507221221924)\", `ns0__lat`:51.50722122192383, `ns0__long`:-0.1274999976158142, `ns1__aprHighC`:15.7, `ns1__aprLowC`:8.2, `ns1__aprMeanC`:12, `ns1__aprPrecipitationDays`:9.1, `ns1__aprPrecipitationMm`:43.7, `ns1__aprRecordHighC`:29.4, `ns1__aprRecordLowC`:-2.6, `ns1__aprSun`:168.7, `ns1__areaBlank1Km`:1737.9, `ns1__areaBlank2Km`:8382, `ns1__augHighC`:23.2, `ns1__augLowC`:13.7, `ns1__augMeanC`:19.4, `ns1__augPrecipitationDays`:7.5, `ns1__augPrecipitationMm`:49.5, `ns1__augRecordHighC`:38.1, `ns1__augRecordLowC`:5.9, `ns1__augSun`:204.7, `ns1__decHighC`:8.6, `ns1__decLowC`:5.4, `ns1__decMeanC`:7, `ns1__decPrecipitationDays`:10.2, `ns1__decPrecipitationMm`:55.2, `ns1__decRecordHighC`:17.4, `ns1__decRecordLowC`:-11.8,..."

9. Discussion

I’ve just skimmed the surface here showing how to read/write LPG and RDF graphs and queries, how to dispatch queries to LPG and RDF databases, and how to convert from RDF to LPG and back again using the neosemantics library. While roundtripping does indeed work, I have noted some apparent small differences in input and output graphs but these may be due to dataype conversions, language tags, etc. One would need to spend a lot more time to examine these issues in depth.

Summary

I’ve shown here in this post how we can use Elixir to move data between semantic and property graphs, and back again. As well as the worked examples in the text, a small number of sample scripts are also provided.

The conversions are handled by applying simple wrapper functions over the neosemantics library from Jesús Barrasa. We also use the bolt_sips package from Florin Pătraşcu and the sparql_client package from Marcel Otto to interface respectively with Neo4j and RDF graph databases.

This was just a preliminary look and much has not been tried (such as vocabulary mappings and inference) but the early findings on these initial small-scale experiments have been very positive. Especially the simplicity and ease of using Elixir to roundtrip graph data between two graph worlds has been a thing of joy.

Next post in this series

Native graph data in Elixir

Using libgraph for graph data structures

medium.com