GSoC — Phase 3

3 min readAug 24, 2019

Synchronization Mechanism

At the end of phase 2, I started working on synchronization mechanism. Still then, we were storing graph representation of the server data at client-side with the help of redis-graph. But there wasn’t any way to make the client aware of data changes being made at the server (by other clients), the only way client had around it was to make an explicit request to the server.
What we needed was a lightweight pushing mechanism to push changes to all the clients. So the client does not end up with stale data. We decided to use WebSockets to make it ‘lightweight’ in the true sense of the word.

At hydrus I needed to implement a push event which broadcasts every state changing update to all the clients connected through socket. Besides this we needed some sound mechanism to allow clients to reconnect. When a client losses the connection and tries to reconnect, it sends a reconnect event to the server with the last job_id (id of the last updated it had received), on receiving reconnect event hydrus uses the last_job_id provided by the client to search through all the modification records and send the client a list of modifications done after client’s disconnection, then client can use that list to update local data with related data at the server. For this we needed to keep track of all the modification records, storing and searching all such records might become too expensive, so we added a thread to run at every 15 minutes and remove all old modification records except the latest 1000 records. By default the clean up routine runs on every 900 seconds (15 minutes) but user can change it by using the stale_records_removal_interval argument while launching hydrus through the CLI.

Adding support for Hydra:Link

It involved providing support for creation of properties with type hydra:Link, and use of such properties at hydrus to properly handle nested resources.

Adding HydraLink:
The first step was to create a class at hydra-python-core which can be used as template to create new hydra:Link predicates. Then user can import HydraLink from the doc_writer module to create new predicates and use them in defining SupportedProperty of any Class.

Example:

dummy_prop_link = HydraLink(id_="singleClass/dummyProp",title= "dummyProp", domain="vocab:singleClass", range_="vocab:dummyClass")

Then we can use this link as property value of any SupportedProperty

class_2.add_supported_prop(HydraClassProp(
    prop=dummy_prop_link, title="dummyProp", required=False, read=False, write=True))

Result of API generation

"@type": "SupportedProperty",
"property": {
              "@id": "vocab:singleClass/dummyProp",
              "@type": "hydra:Link",
              "description": "",
              "domain": "vocab:singleClass",
              "range": "vocab:dummyClass",
              "supportedOperation": [],
              "title": "dummyProp"
            },
 "readable": "false",
 "required": "false",
 "title": "dummyProp",
 "writeable": "true"
                },

Using hydra:Link to handle nested resources:
While inserting a new element the string value of hydra:Link property is handled differently than the string value of other properties. Instead of storing the link directly as GraphIIT triple, hydrus extracts the instance_id from the link and creates a new GraphIII triple to store this new relation. So links are only used for representation, hydrus uses existing graph schema to store relations implied by links.

Working on middle-layer of the agent GUI

Agent GUI at hydra-python-agent-gui has a Flask middle-layer which receives requests from the the GUI and uses them to call hydra-agent methods. For this I had to create two endpoints one was send_command which handled commands and the other was apidoc_graph which returned a graph representation of the APIDoc.

Extending and improving tests:

I had already made few PRs to extend tests in the first phase(#399 and #398) but still we had some pending issues related to tests (like #317). Besides that we needed to extend the nested class test in crud to test hydra:Link properties. Solving #317 revealed some places where we were using type of the class instead of the path of the class to perform some operations, which can lead to errors when path(endpoint `@id`) of the class is different than the type of the class. After correcting this we finally have a version of hydrus which will work for any Hydra API having classes with independent paths and it is also tested for the same.

GSoC — Phase 3

Synchronization Mechanism

Adding support for Hydra:Link

Working on middle-layer of the agent GUI

Extending and improving tests:

Written by Vishal Desai