Swagger, the API Economy, REST, Linked Data, and a Semantic Web

Source: http://blogs.forrester.com/f/b/users/MGILPIN/api_and_data_economy.jpg

Situation Analysis

The network commonly referred to as the World Wide Web (“Web”) is built upon the powerful duality delivered by a HTTP URI (“Hyperlink”).

One Hyperlink packs two powerful features:

  1. Denotation —referent signification using a symbol
  2. Connotation — referent description using a form of media (for instance, a document)

The Hyperlink that brings you to this post demonstrates Medium’s use of said Hyperlink as the document’s name; i.e., how it identifies this document.

A Name is always about the dual act of denotation and connotation via some form of indirection. In the real-world, this denotation → connotation indirection happens in our minds, while on the Web it occurs by way of hyperlinks that return documents (with negotiable content-type).

Hypermedia (Webby Content or Data)

Hypermedia (Webby Content) is Document Content that includes hyperlinks, which may or may not include explicit controls for Hyperlink lookup (or dereference).

For instance, an HTML document is an example of a Hypermedia resource equipped with explicit controls that inform a User Agent about Hyperlink-lookup — in this case, the <a/> tag. The fact that instructions and content are intermingled is what makes HTML a Markup Language, rather than an Abstract Language (e.g. RDF where information is relayed by way of relation relationship type semantics defined by vocabularies and ontologies).

Notations for creating RDF document content e.g., (RDF-Turtle or JSON-LD) don’t include explicit Hyperlink-lookup controls. Why? For the following reasons:

  1. Both are notations for constructing RDF Language sentence using subject → predicate → object structure. Thus, the role-semantics of sentence subject, predicate, or objects are the keys to understanding what’s encoded in a sentence (just like the real-world);
  2. When Linked Data principles are applied, both notations enable the use of hyperlinks to identify the subject, predicate, and object (optionally) of an RDF sentence/statement.
  3. RDF in its most basic form is driven by a foundation vocabulary (That all RDF-aware applications MUST understand) with regards to the nature of a sentence i.e., its subject, predicate, object components
  4. The sophistication of the information encoded in an RDF Language sentence is a function of the vocabulary of terms applied (again just like the real-world were he vocabulary of a three year old would difference significantly from that of an adult).

Here’s a visualization of a snippet of statements from RDF’s foundation vocabulary, with emphasis on the description of terms such as rdf:Statement rdf:subject, rdf:predicate, and rdf:object.

RDF Vocabulary Snippet courtesy of the OpenLink Structured Data Sniffer (which adds RDF content rendering to browsers)

REST Interaction Pattern

As stated earlier, the Web is fundamentally a publicly accessible network where every de-referencable node on the network is identified by a Hyperlink. Thus, Agents and Servers can exchange information intelligently by guided de-reference (lookup) of relevant segments of hypermedia.

For Agents in particular, this provides a powerful mechanism for application state transition; i.e., the results of Hyperlink lookups actually drive the state transitions a user perceives as application behavior.

REST vs a Semantic Web of Linked Data — Artificial Dichotomy

Unfortunately, a false dichotomy has arisen across the communities of so-called “Web Developers” (who prefer explicit Hyperlink-lookup controls) and don’t necessarily see the need for HTTP URI opacity.

Roy Fielding — espousing a view on HTTP URLs that Identify Document Locations/Addresses regarding REST interaction pattern

Here is the world-view associated with so-called “Semantic Web Developers” who prefer to leave understanding the nature of controls to context provided by RDF sentences.

Tim Berners-Lee — espousing a generic view of URIs as Names used to identify anything re., Linked Data Principles

Like all paradoxes, Tim Berners-Lee and Roy Fielding aren’t contradicting one another. In fact, they are speaking about different things that are mutually compatible i.e., denotation and connotation with regards to HTTP!

Anyway, sticking with my solution focus in this post, the issue outlined above is beginning to reach a point of resolution, thanks to the obvious need to document Web APIs (what “Web Developers” are interested in) without covertly imposing the English Language on every developer.

API Documentation

In recent times, Swagger has emerged as an open standard for Web API documentation. Its adoption by API publishers has already reached the point of critical mass.

Using a Smart Agent to bridge between REST APIs and the Semantic Web of Linked Data

The remainder of this post demonstrates how API documentation can effectively bridge the artificially separated “Web Developer” and “Semantic Web Developer” communities.

The tools used in this exercise are:

  1. OpenLink Smart Data Bot — a Smart Agent that dynamically generates interaction interfaces for both humans and machines via webby data (a/k/a Linked Data) derived from APIs documented using Swagger (specifically, JSON Content-Type)
  2. Terms from the RDF and Schema.org vocabularies
  3. URIBurner Service — a live instance of Virtuoso (our platform for modern data access, integration, virtualization, and management) that has the Linked Data module (a/k/a the “Sponger”) enabled
  4. OpenLink Data Explorer Extension (ODE) — adds Linked Data transformation services to existing Web Browsers
  5. OpenLink Structured Data Sniffer (OSDS) — discovers Structured Metadata embedded in HTML documents
  6. DBpedia — the kernel that spawned the massive Linked Open Data Cloud enclave within the World Wide Web
  7. A DBpedia Query API — documented using Swagger and published via Swagger Hub

Generating API Description from Swagger API Documentation

List of DBpedia APIs currently deployed via SWAGGERhub.

Looking at the DBpedia Query API labeled “DBpedia Virtuoso” which is basically a DBpedia API that exposes the SPARQL Query services provided by Virtuoso instance from which it is deployed.

Using the ODE browser extension to select JSON edition of Swagger API documentation for “on the fly” transformation into RDF-based Linked Data.

Result of Transformation via URIBurner service to which ODE is bound (by way of configuration i.e., you can bind ODE to any Linked Data Service be it Virtuoso-based or from other solution providers).

Having an RDF-based Linked Data transformation of an API documented using Swagger implies the following:

  1. A document exists at a Location identified by URL on the Web (in this case URIBurner is the document server — so the URL is grounded in that domain i.e., http://linkeddata.uriburner.com/about/html/https/app.swaggerhub.com/apiproxy/schema/file/DBpedia/Virtuoso/1.0.0/swagger.json
  2. A URI also exists that explicitly identifies the API described by the description derived from Swagger — this identifier is also URIBurner domain grounded i.e., http://linkeddata.uriburner.com/about/id/entity/https/app.swaggerhub.com/apiproxy/schema/file/DBpedia/Virtuoso/1.0.0/swagger.json

Recalling my earlier points about denotation and connotation, this is why looking-up (clicking or de-referencing) the two aesthetically distinct HTTP URIs above lead to the same content being presented in your browser. Here’s additional information to emphasize this very important point:

  1. http://linkeddata.uriburner.com/about/id/entity/https/app.swaggerhub.com/apiproxy/schema/file/DBpedia/Virtuoso/1.0.0/swagger.json — a URI that identifies API while also exhibiting denotation & connotation duality by way of indirection (again: this is HTTP URI’s inbuilt killer-feature and the reason by we have a World Wide Web)
  2. http://linkeddata.uriburner.com/about/html/https/app.swaggerhub.com/apiproxy/schema/file/DBpedia/Virtuoso/1.0.0/swagger.json —a URL (actually a kind [subcategory] of URI) that identifies a Document (Content Container Location) .

OpenLink Smart Data Bot (OSDB) Binding

Now that we have an HTTP URI that unambiguously identifies an API, we can now use that URI for all subsequent references e.g., when loading it into OSDB, as per depiction below.

List of Web Services (Services) Registered with OSDB — each Service is associated with one or more Actions (Operations)

Once the description of the API has been processed by OSDB it is then added to the list of registered Web Services (Services), and from this point onwards you simply click for further guidance.

Here’s a list of options (arguments) available to a human or software agent seeking to interact with a specific Action (labeled “find_related_dbpedia_resources_for_a_given_string”) associated with a registered Service (labeled “dbpedia_query_api”) using OSDB.

Here’s an interaction console (oriented to the human user) that provides one interaction option.

Note:

All default values and help-text are generated from the API description (this is a pattern also used by many Swagger tools, the difference here is that OSDB produces the console from RDF-based Linked Data i.e., leverage relationship type semantics that it understands by way of term definitions from the RDF and Schema.org vocabularies).

Here’s the console with input which in this case takes the form of SPARQL Query text.

Result of hitting the “Execute Action” button which leads to the content of a Query Results document being presented in your browser.

Using our Structured Data Sniffer’s SPARQL Editor feature, you can jump straight in to Query view and/or modification mode, if you choose.

By changing the &query parameter name to &qtxt — in the HTTP URI/URL presented in your browser — you can also retrieve the content of DBpedia’s SPARQL Query editor which also enables you view and/or modify query definitions text.

REST-ful Interaction

You can repeat the human-centric interactions covered so far, by emulating the behavior of an HTTP User agent using CURL. Basically, the state of your application experience is guided by the document content retrieved i.e., you can work you way through various OSDB actions, courtesy of it registered Web Services API and their associated actions (operations).

Examples:

Given an OSDB registered API (Web Service) labeled dbpedia_query_api, here’s how you obtain is associated Actions, using CURL.

curl -ik https://osdb.openlinksw.com/osdb/api/v1/services/dbpedia_query_api
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 510
ETag: W/”1fe-qyBtEiQ9D6Qf9Aa8S1g5FA”
Date: Fri, 23 Jun 2017 13:27:05 GMT
Connection: keep-alive
{“status”:”success”,”method”:”describe_service”,”api”:”/osdb/api/v1/services/:service_id”,”response”:{“service_id”:”dbpedia_query_api”,”service_name”:”dbpedia_query_api”,”description”:”Virtuoso is a modern enterprise grade solution for data access, integration, and relational database management (SQL Tables and/or RDF based Property/Predicate Graphs).\n”,”import_source_uri”:”http://linkeddata.uriburner.com/about/id/entity/https/app.swaggerhub.com/apiproxy/schema/file/DBpedia/Virtuoso/1.0.0/swagger.json"}}

Given OSDB Action labeled find_related_dbpedia_resources_for_a_given_string, here’s its call invocation structure (or signature) is obtained.

curl -ik https://osdb.openlinksw.com/osdb/api/v1/actions/dbpedia_query_api/find_related_dbpedia_resources_for_a_given_string
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 770
ETag: W/"302-z3DKgwJs73ywTabTRulMiA"
Date: Fri, 23 Jun 2017 01:26:59 GMT
Connection: keep-alive
{"status":"success","method":"describe_action","api":"/osdb/api/v1/actions/:service_id/:action_id","response":{"action_id":"find_related_dbpedia_resources_for_a_given_string","description":"Find related DBpedia resources for a given string","entry_point":{"http_method":"GET","url_template":null,"url":"http://dbpedia.org//sparql","name":null,"description":null,"encoding_types":["application/json","application/xml"],"content_types":["text/html"],"parameters":[{"parameter_name":"default-graph-uri","display_name":"default-graph-uri","description":"Default Data Set Name (Graph IRI)","type":"query","required":0,"permitted_values":null},{"parameter_name":"query","display_name":"query","description":"Query Text","type":"query","required":1,"permitted_values":null}]}}}

Invoking Action labeled find_related_dbpedia_resources_for_a_given_string with a preference to return a document URI/URL rather than content.

curl -ki -X POST -d ‘{ “query”:”SELECT DISTINCT ?o WHERE {?s a ?o} limit 50", “default-graph-uri”: “http://dbpedia.org", “osdb:output_type”: “url_only” }’ -H ‘Content-Type: application/json’ https://osdb.openlinksw.com/osdb/api/v1/actions/dbpedia_query_api/find_related_dbpedia_resources_for_a_given_string/exec
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/plain; charset=utf-8
Content-Length: 148
ETag: W/”94-xo5Bj+qoWgP6ChHqua9RaQ”
Date: Fri, 23 Jun 2017 13:22:42 GMT
Connection: keep-alive
http://dbpedia.org//sparql?query=SELECT%20DISTINCT%20%20%3Fo%20WHERE%20%7B%3Fs%20a%20%3Fo%7D%20limit%2050&default-graph-uri=http%3A%2F%2Fdbpedia.org

Invoking Action labeled find_related_dbpedia_resources_for_a_given_string with a preference to return HTML content.

curl -ki -X POST -d ‘{ “query”:”SELECT DISTINCT ?o WHERE {?s a ?o} limit 50", “default-graph-uri”: “http://dbpedia.org", “osdb:response_format”: “text/html” }’ -H ‘Content-Type: application/json’ https://osdb.openlinksw.com/osdb/api/v1/actions/dbpedia_query_api/find_related_dbpedia_resources_for_a_given_string/exec

Invoking Action labeled find_related_dbpedia_resources_for_a_given_string with a preference to return RDF-Turtle content..

curl -ki -X POST -d ‘{ “query”:”SELECT DISTINCT ?o WHERE {?s a ?o} limit 50", “default-graph-uri”: “http://dbpedia.org", “osdb:response_format”: “text/turtle” }’ -H ‘Content-Type: application/json’ https://osdb.openlinksw.com/osdb/api/v1/actions/dbpedia_query_api/find_related_dbpedia_resources_for_a_given_string/exec

Bearing in mind that OSDB is an aggregator of APIs (documented using Swagger or described from onset using RDF Language), for each API registered it gains N number of Action Invocation capabilities (or skills).

Here’s an example using CURL to explore Uber Product Offers, via OSDB:

curl -ikL -X POST -d ‘{ “latitude”:”37.7759792", “longitude”:”-122.41823", “osdb:output_type”:”generate_rdf”, “osdb:response_format”:”application/ld+json” }’ -H ‘Content-Type: application/json’ https://osdb.openlinksw.com/osdb/api/v1/actions/uber/products/exec
Content-Type: application/ld+json; charset=utf-8
Content-Length: 33701
ETag: W/”83a5–0aNzDsCESshzkpLcKYtpQQ”
Date: Fri, 23 Jun 2017 17:09:54 GMT
Connection: keep-alive
{ “@graph”: [
{ “@id”: “http://schema.org/",
“@type”: “http://www.openlinksw.com/schema/attribution#DataSource",
“http://www.openlinksw.com/schema/attribution#hasNamespacePrefix": “schema” },
{ “@id”: “https://sandbox-api.uber.com/v1/products?latitude=37.7759792&longitude=-122.41823",
“@type”: “http://schema.org/CreativeWork",
“http://schema.org/about": { “@id”: “http://ods-qa.openlinksw.com:8896/about/id/entity/https/sandbox-api.uber.com/v1/products?latitude=37.7759792&longitude=-122.41823"},
“http://schema.org/mainEntity": { “@id”: “http://ods-qa.openlinksw.com:8896/about/id/entity/https/sandbox-api.uber.com/v1/products?latitude=37.7759792&longitude=-122.41823"},

Conclusion

I’ve used OSDB’s functionality to demonstrate the fact that the notion of a Semantic Web of Linked Data and REST-ful interactions using APIs aren’t in anyway mutually exclusive. In fact, they are symbiotic aspects of what makes the Web’s underlying architecture so powerful.

A single instance of OSDB provides a REST-ful interaction point for engaging a variety of Actions extracted from a variety of APIs documented using Swagger and/or RDF documents.

Fundamentally, the architecture that delivered the Web is also a vehicle for loosely-coupled exploitation of the time-test M-V-C pattern along the following lines:

  • Model — Semantic Web of Linked Data
  • Viewer — HTML-based Web of Documents
  • Controller — REST-ful interaction with APIs where operations are part of a Web of Actions.

Links

  • Open Smart Data Bot — a Smart Agent that dynamically generates action (operation) interaction interfaces (for both humans and machines) via webby data (a/k/a Linked Data) derived from APIs documented using Swagger (specifically, JSON Content-Type)
  • Terms from the RDF and Schema.org vocabularies
  • URIBurner Service — a live instance of Virtuoso that has the Linked Data module (a/k/a “Sponger”) module enabled
  • OpenLink Data Explorer Extension (ODE) — adds Linked Data transformation services to existing Web Browsers
  • OpenLink Structured Data Sniffer (OSDS) — discovers Structured Metadata embedded in HTML documents
  • DBpedia — kernel that spawned the massive Linked Open Data Cloud enclave within the World Wide Web
  • A DBpedia Query API — documented using Swagger and published via Swagger Hub
  • APIs & REST — by Roy Fielding
  • Linked Data Principles — by Tim Berners-Le

Related