Magic Sets and Custom Inference Rules in Virtuoso 8.x
The ability to materialize relations (i.e., sets of entity relationships grouped by a predicate) on a deductive basis is a critical feature of the genre of RDBMS application known as Deductive Database Management Systems — i.e., RDBMS equipped with Reasoning & Inference functionality. In recent times, this entire DBMS genre and its functionality have been obscured by the fanfare associated with NoSQL and Graph Databases.
In SQL RDBMS applications, materialized relations take the form of VIEWs which are described by SQL Queries and/or (in modern variants) by SQL Stored Procedures (a/k/a Persistent Stored Modules).
In RDF data management, materialized relations take the form of Inference Rules that inform the process of producing dynamically-materialized relations, colloquially referred to as Magic Sets. Magic Sets are basically the same concept as SQL VIEWs, but significantly enhanced by the semantic fidelity of relations represented as RDF sentence/statement graphs.
What?
Virtuoso supports the use of custom (magic) predicates to produce these Magic Sets or VIEWs or dynamically-materialized relations.
In Virtuoso 7.x and previous releases, the base product included only a few built-in magic predicates such as bif:contains. In those older versions, extending the collection of magic predicates required dropping down to SQL Stored Procedures — unnatural territory for those used to working with data represented as RDF sentence/statement graphs — under the misconception that RDF and SQL representations of structured data are mutually exclusive. (That is, it was thought that some structured data could be represented only as RDF sentence/statement graphs and not as SQL tabular relations; and that other structured data could be represented only as SQL tabular relations and not as RDF sentence/statement graphs. This has now been proven false; any structured data representable as SQL tabular relations may also be represented as RDF sentence/statement graphs, and vice versa.)
Virtuoso 8.0 introduced a simpler approach — an extension of its SPARQL-BI query functionality, implemented via a Macro Library abstraction. A Macro Library comprises custom inference rules that describe how to construct new Magic Sets, subject to conditions defined in those rules. A pragma declaration may now be used in the preamble of a SPARQL query to identify the desired Macro Library.
How?
Commonly used rule definitions can be grouped into Macro Libraries which enable a System’s Administrator to —
- Load a macro library into system metadata using a
SPARQL CREATE MACRO LIBRARYorSPARQL CREATE SPIN LIBRARYstatement, for invocation in SPARQL queries through thedefine input:macro-lib <macro-lib-name>pragma. - Attach a macro library to specific RDF storage using the
SPARQL ALTER QUAD STORAGE <storage-name> { ATTACH MACRO LIBRARY <macro-lib-name> }statement, thereby making the custom reasoning and inference context provided by that macro library available to every query without explicit pragma declaration per query.
Simple Example
Given an RDF Named Graph (Document or RDF Statements Container) identified by the IRI urn:spin:rule:geometry:lib, comprising RDF statements that describe a Custom Inference Rule for calculating the area of a rectangle as outline below:
@prefix shapes: <http://example.org/shapes#> .
@prefix spin: <http://spinrdf.org/spin#> .
@prefix sp: <http://spinrdf.org/sp#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .shapes:Rectangle
rdf:type owl:Class;
rdfs:label "Rectangle class";
spin:rule [
a sp:Construct;
sp:text """
CONSTRUCT { ?this <http://example.org/shapes#area> ?area }
WHERE {
{
SELECT ?this ?area
WHERE {
?this
<http://example.org/shapes#width> ?width ;
<http://example.org/shapes#height> ?height .
BIND ((xsd:float(?height) *
xsd:float(?width))
AS ?area) .
}
}
}
"""
] .
Here’s a one-liner (shown here on multiple lines, just for easier reading; each sequence of line feeds, spaces, tabs, and other whitespace may be collapsed to a single space character) for adding the rule described above to the Virtuoso Macro Library (where Inference Rules are stored):
EXEC
( 'SPARQL '||
SPARQL_SPIN_GRAPH_TO_DEFSPIN
('urn:spin:rule:geometry:lib')
);Basically, the example above leverages a built-in Stored Procedure for reading Rule Definition from an RDF document loaded into a Virtuoso Named Graph identified by the IRI <urn:spin:rule:geometry:lib>, courtesy of the following SPARQL LOAD command.
DEFINE get:soft "no-sponge"
LOAD <http://kingsley.idehen.net/DAV/home/kidehen/Public/Linked%20Data%20Documents/Tutorials/inference-rules/area-of-a-rectangle.ttl> INTO <urn:spin:rule:geometry:lib> To reveal the lower-level code generated by Virtuoso using the following SQL command:
SELECT
SPARQL_SPIN_GRAPH_TO_DEFSPIN
('urn:spin:rule:geometry:lib')That outputs the following lower level Rules Definition using Virtuoso’s Macro Library extension to SPARQL.
CREATE SPIN LIBRARY <urn:spin:rule:bind:geometry:lib>
{
CLASS <http://example.org/shapes#Rectangle>
{
RULE <urn:spin:rule:bind:geometry:lib#bnode-b2256179>
{
CONSTRUCT { ?this <http://example.org/shapes#area> ?area }
WHERE {
{
SELECT ?this ?area
WHERE { ?this <http://example.org/shapes#width> ?width ;
<http://example.org/shapes#height> ?height .
BIND ((xsd:float(?height) * xsd:float(?width)) AS ?area) .
}
}
}
UNION STORAGE
}
}
}
ASK WHERE { <nosuch> <nosuch> <nosuch> }Note: the UNION STORAGE clause ensures fusion of calcuated and existing values when inferring objects of the <http://example.org/shapes#area> relation identified by the variable ?area. Thus, if excluded inferred objects of the <http://example.org/shapes#area> relation would only comprise calculated values.
Here are links to Virtuoso SQL scripts for each of the Custom Inference Rules presented below:
- Rules for Inference of British Royal Family Relationship Types
- Rule for Geospatial Data Cleansing
- Rule for Calculating Area of a Triangle (short form)
- Rule for Calculating Area of a Triangle (long form)
- Rule for Identity Reconciliation — making use of Inverse-Functional Relationship Type Semantics
SPARQL Query to List Existing Rules across Macro Libraries
DEFINE output:valmode "LONG"
DEFINE input:storage "" PREFIX virtrdf:
<http://www.openlinksw.com/schemas/virtrdf#>SELECT DISTINCT ?sml AS ?macroLib
?o AS ?ruleDefinition
FROM virtrdf:
WHERE
{
?sml
a virtrdf:SparqlMacroLibrary ;
virtrdf:smlSourceText ?o .
}
ORDER BY ASC(STR(?sml))
Query Results Page
SPARQL Query to List Rules
Query Results Page
Rule Usage Query Examples
British Royal Family Relationships Rule
urn:spin:nanotation:demo:royal:family:lib2Query Demonstrating Usage of British Royal Family Relationships Rule
DEFINE input:macro-lib
<urn:spin:nanotation:demo:royal:family:lib2>PREFIX rel: <http://purl.org/vocab/relationship/>WITH <urn:spin:nanotation:demo:royal:family>SELECT DISTINCT ?person
?hasUncle
WHERE
{ ?person
a <#RoyalPerson> ;
<#hasUncle> ?hasUncle
}
ORDER BY ASC(?person)
Query Results Page
Rule for Calculating Area of a Rectangle
urn:spin:rule:geometry:libQuery Demonstrating Usage of Rule for Calculating Area of a Rectangle
DEFINE input:macro-lib <urn:spin:rule:geometry:lib>PREFIX shapes: <http://example.org/shapes#>
SELECT ?s
xsd:float(?w) AS ?width
xsd:float(?h) AS ?height
xsd:float(?area) AS ?area
FROM <http://geometry>
WHERE
{
?s a shapes:Rectangle ;
shapes:width ?w ;
shapes:height ?h .
OPTIONAL { ?s shapes:area ?area }
}
Query Results Page
Rule for Geospatial Data Cleansing
urn:spin:rule:geospatial:lib4 Query Demonstrating Usage of Rule for Geospatial Data Cleansing
DEFINE input:macro-lib <urn:spin:rule:geospatial:lib4>SELECT DISTINCT ?x
?y
?z
FROM <http://bostonopendata-boston.opendata.arcgis.com/datasets/465e00f9632145a1ad645a27d27069b4_2.csv>
WHERE
{ ?x a sioc:Item ;
<#hasLatitude> ?y ;
<#hasLongitude> ?z
}
Query Results Page
Identity Reconciliation Rule
urn:spin:rule:foaf:ifp:lib1Query Demonstrating Usage of Identity Reconciliation Rule
DEFINE input:macro-lib <urn:spin:rule:foaf:ifp:lib1>SELECT DISTINCT ?s
?sameAs
?mainEntityOfPage
FROM <urn:kidehen:ifp:test:2>
WHERE
{ ?s a foaf:Person ;
owl:sameAs ?sameAs ;
schema:mainEntityOfPage ?mainEntityOfPage .
FILTER ( ?s = <https://twitter.com/kidehen#this> )
}
Query Results Page
Conclusion
As a multi-model RDBMS, Virtuoso offers the best of multiple worlds.
Effectively covered by the examples above, the familiar concept of a tabular SQL VIEW has been extended to exploit the fine-grained semantics offered by entity relationship types represented as RDF sentence/statement collections that coalesce around a specific magic-predicate.
Most importantly, Virtuoso delivers all of this functionality — without undue complexity or compromised performance — to any application that supports HTTP, ODBC, JDBC, ADO.NET, or OLE DB.
Related
- Creating Custom Reasoning & Inference Rules using SPARQL CONSTRUCT and terms from SPIN Vocabulary — step-by-step guide to custom inference rules in Virtuoso 8.x
- Custom Reasoning & Inference Demonstrated using British Royal Family Relationship Types — incorporates data from DBpedia & Relationship Vocabulary in a practical functionality usage exercise
- Built-in Reasoning & Inference using British Royal Family Relationship Types — also incorporates DBpedia and Relationship Vocabulary data
- Exploiting Virtuoso’s Custom Reasoning & Inference using Jena or RDF4J Frameworks
- Class Equivalence Reasoning & Inference using Virtuoso’s Built-In Rules
- About SQL Persistent Stored Modules
- About SQL Stored Procedures
- How do you know that you need a Multi-Model RDBMS?
