Dynamically Generating Documents from SPARQL Queries
Using Virtuoso to Increase the Power of Desktop Analysis Tools
End-user client applications for data analysis often have some hard-wired limitations, such as requiring a particular combination of data document type and document location.
Typically, these client applications do support structured data in some tabular form (HTML, CSV, JSON, CXML, etc.), but even these are often only consumable through the local filesystem (i.e., no direct access via HTTP or other network protocol).
Until recently, there was no easy way to bring such limited-focus applications to bear on RDF. We at OpenLink Software decided to change that.
Enter, Virtuoso
Virtuoso, our multi-model hybrid RDBMS application, lets you create change-sensitive structured-data documents from data queries — that is, SPARQL and other query result documents, whose contents change when the data in the back-end DBMS changes.
These documents can be generated in various formats, and automatically exposed to clients that support HTTP and/or WebDAV protocols. Since the filesystems of modern macOS, Windows, and Linux are all capable of mounting WebDAV directories, these Virtuoso-generated documents are also available to client tools that only know how to work with the local filesystem.
Putting it to the test
[0] We recommend that you first update to the latest Virtuoso binary for your major version — whether VOS (Open Source Edition) or Enterprise Edition.
[1] Log in to the Conductor on your instance, http://{fqdn}:{port}/conductor/, as the dba user (or any user with DBA permissions). Drill down to the System Admin → Packages tab, and confirm that you have a recent Conductor (at least 1.00.8774 as of 2018‑03‑02) installed on your instance. If needed, you can download and install the latest Conductor VAD for VOS v7.x, for Enterprise Edition v7.x, or for Enterprise Edition v8.x).
[2] Log in to the Conductor on your instance, http://{fqdn}:{port}/conductor/, as the dba user (or any user with DBA permissions).
[3] Drill down to Web Application Server → Content Management.
[4] Navigate to the intended DAV location for your new folder of SPARQL result documents. For this exercise, we’ll put it in DAV/home/demo/.
[5] Create a new folder (called dynamic-sparql-resultsfor this exercise), and set the DET Folder type to Dynamic Resources. Set ownership on the new folder to the appropriate Virtuoso SQL user and group (similar to in the filesystem); here we’ll use demo for both user and group. If you want the documents in this folder to be readable by any visitor, with or without authentication (as in this exercise), grant +Read privileges to Public by ticking the r box under Other .
[6] Go to the SPARQL endpoint for the same instance, http://{fqdn}:{port}/sparql/, and tick the check-box labeled “Save resultset to WebDAV folder on the server.” Note: You may need to be logged in to reveal this checkbox.
[7] This will reveal some additional options.
[8] Assign a file name via the File name input field — for instance, first-25-concepts-csv.
[9] The Refresh periodically check-box defaults to ticked, which means the document will automatically reflect changes in the underlying data, according to the timeout.
[10] Select your desired document format/type (CSV, HTML, JSON, Turtle, etc.) from the Results Format menu.
[11] Input a SPARQL SELECT Query, and click the Run Query button.
[12] The results of SPARQL query will not be displayed directly. Instead, you’ll get a notification with the URL of the file that’s been generated; e.g., http://demo.openlinksw.com/DAV/home/demo/dynamic-sparql-results/first-25-concepts-csv
[13] You may now mount the dynamic-sparql-results folder using your host operating system's built-in WebDAV functionality to work directly with the file in the DAV repository, which contents will be updated periodically as noted in the Saved to DAV announcement shown above.
Alternatively, you may download a copy of the file to work with locally. The content of the downloaded copy will not update, but a fresh download will bring the latest query results.
[14] Any tool that works on your chosen results format (e.g., Data Visualization and Analytics apps — such as Tableau, Microsoft Excel, or Qlik Sense — commonly handle CSV) can now be used against either the DAV-resident or downloaded file.
Live Examples
The dynamic-sparql-results DAV folder on the demo.openlinksw.com instance holds various dynamic result documents created via SPARQL endpoint (while logged in as user demo).
Related
- Conceptual Relational Data Virtualization — leveraging existing open standards (SQL, SPARQL, RDF, HTTP, ODBC, JDBC)
- Integrating Dropbox, OneDrive, Google Drive, Box, Amazon S3, and other storage services with Virtuoso’s WebDAV functionality
