Time-Travel with Fluree

Fluree’s time travel capabilities allow you to join the ranks of time-traveling phenoms like Marty McFly and Dr. Who. Because Fluree is a blockchain database, time-travel is not a feature that we have built on top of FlureeDB, but rather it is core to the very way Fluree is constructed.

Why Time Travel is An Integral Part of Fluree

Because each Fluree database is a blockchain, it stores the entire history of every transaction performed. This is part of how a blockchain ensures that information is immutable and secure.

A key 🔑 aspect of what gives blockchain its “immutability” characteristic involves its chronological method of storing and transacting data. Once the network verifies a transaction with consensus, the update is time-stamped, cryptographically hashed, and added to the blockchain as a chronological update. This allows participants in the network transparent access to the full history of the distributed ledger, from the genesis block to the most recent transactional data. What makes this chain “unbreakable,” or immutable, is the method of cryptology: the hashing process of a new block includes meta-data from the previous block’s hash, and so on and so forth. This allows for a source for data truth — manipulation is near impossible.

And such is the case with Fluree — every transaction represents a state change to your database, and thus a new block is formed, secured with hashing, and added to the existing chain as its most recent update. This new block acts as a time-stamped snapshot of the database at the exact point in time of the transaction.

Rather than view this history as a problem to be solved, Fluree views this history as an asset. It is what enables efficient, lightweight time-travel.

Time Travel is simply the ability for your applications to query FlureeDB against any point in time and retrieve an instantaneous response. It effectively gives your apps an instant “rewind” dial with no added development overhead.

In this article: first, we’ll look at a few examples of querying FlureeDB with down-to-the-millisecond accuracy, and then we’ll go into detail about how blockchain enables time-travel queries. All in due, ahem, time.

How It Works

Whenever you issue a query on FlureeDB, you can optionally specify either:

• a block number
• a duration in ISO-8601 format (i.e. PT1H would be as of one hour ago)
• a wall clock time in ISO-8601 format (2017–11–14T20:59:36.097Z)

The query result will show you the data that you requested as of that date and time.

You are also able to query a subject’s entire history (or a subset of the history). For example, with a single query, you can see every single update you’ve made to, say, the price of a piece of clothing on your website.

Before we can see these features in action, we will need to create and populate a new database.

Populating Our Database

Let’s say we have an application where people can add their favorite movies to their profile. We’ll call it ReelFriends. The database schema underlying our application is:

[{
"_id": "_collection",
"name": "user"
},
{
"_id": "_collection",
"name": "movie"
},
{
"_id": "_predicate",
"name": "user/handle",
"type": "string",
"unique": true
},
{
"_id": "_predicate",
"name": "user/favMovies",
"type": "ref",
"multi": true,
"restrictCollection": "movie"
},
{
"_id":"_predicate",
"name": "movie/title",
"type": "string",
"unique": true
},
{
"_id":"_predicate",
"name": "movie/genre",
"type": "tag",
"multi": true
},
{
"_id": "_predicate",
"name": "movie/budget",
"type": "int"
}]

Now, we can populate the database with some initial users and favorite movies.

[{
"_id": "user$sammi",
"handle": "sammi",
"favMovies": ["movie$1", "movie$2", "movie$3"]
},
{
"_id": "user$lee",
"handle": "lee",
"favMovies": ["movie$4", "movie$5", "movie$6", "movie$1"]
},
{
"_id": "user$julissa",
"handle": "julissa",
"favMovies": ["movie$1", "movie$7", "movie$8", "movie$9"]
},
{
"_id": "movie$1",
"title": "The Shawshank Redemption",
"genre": ["drama", "crime"],
"budget": 25000000
},
{
"_id": "movie$2",
"title": "Hot Fuzz",
"genre": ["crime", "action", "comedy"],
"budget": 12000000
},
{
"_id": "movie$3",
"title": "Gran Torino",
"genre": ["drama"],
"budget": 33000000
},
{
"_id": "movie$4",
"title": "The Last Airbender",
"genre": ["action", "adventure", "family"],
"budget": 45000000
},
{
"_id": "movie$5",
"title": "Napoleon Dynamite",
"genre": ["comedy"],
"budget": 400000
},
{
"_id": "movie$6",
"title": "Stranger Than Fiction",
"genre": ["comedy", "fantasy", "comedy", "romance"],
"budget": 30000000
},
{
"_id": "movie$7",
"title": "Titanic",
"genre": ["drama", "thriller", "romance"],
"budget": 200000000
},
{
"_id": "movie$8",
"title": "Avatar",
"genre": ["action", "adventure", "fantasy", "science fiction"],
"budget": 400000
},
{
"_id": "movie$9",
"title": "The Wrestler",
"genre": ["drama", "romance"],
"budget": 6000000
}]

Over time, the users discover new content that they like or their tastes have changed. Note, in order for the subsequent examples to work, each of these transactions has to be issued separately.

[{
"_id": ["user/handle", "julissa"],
"favMovies": [["movie/title", "The Shawshank Redemption"]],
"_action": "delete"
},
{
"_id": ["user/handle", "lee"],
"favMovies": [["movie/title", "The Shawshank Redemption"]],
"_action": "delete"
}]
[{
"_id": ["user/handle", "julissa"],
"favMovies": ["movie$1", "movie$2"]
},
{
"_id": ["user/handle", "lee"],
"favMovies": ["movie$2", ["movie/title", "Stranger Than Fiction"]]
},
{
"_id": ["user/handle", "sammi"],
"favMovies": ["movie$3"]
},
{
"_id": "movie$1",
"title": "Cleopatra",
"genre": ["drama", "history", "romance"],
"budget": 31115000
},
{
"_id": "movie$2",
"title": "Troy",
"genre": ["adventure", "drama", "war"],
"budget": 175000000
},
{
"_id": "movie$3",
"title": "Quantum of Solace",
"genre": ["action", "adventure", "crime", "thriller"],
"budget": 200000000
}]
[{
"_id": ["user/handle", "sammi"],
"favMovies": [["movie/title", "Gran Torino"]],
"_action": "delete"
}]
[{
"_id": ["user/handle", "lee"],
"favMovies": [["movie/title", "The Last Airbender"]],
"_action": "delete"
}]

When we query for a given user, expanding out their movie preferences, by default we only see the user’s current favorite movies.

{
"select": ["*", {"user/favMovies": ["*"]}],
"from": ["user/handle", "sammi"]
}

Result:

{
"status": 200,
"result": {
"user/handle": "sammi",
"user/favMovies": [
{
"movie/title": "The Shawshank Redemption",
"movie/genre": [
"drama",
"crime"
],
"movie/budget": 25000000,
"_id": 4303557230593
},
{
"movie/title": "Hot Fuzz",
"movie/genre": [
"crime",
"action",
"comedy"
],
"movie/budget": 12000000,
"_id": 4303557230594
},
{
"movie/title": "Quantum of Solace",
"movie/genre": [
"crime",
"action",
"thriller",
"adventure"
],
"movie/budget": 200000000,
"_id": 4303557230604
}
],
"_id": 4294967296001
},
"fuel": 20,
"block": 8,
"time": "1.55ms",
"fuel-remaining": 99999976685
}

We can also issue that same query as of the last block (block 7).

{
"select": ["*", {"user/favMovies": ["*"]}],
"from": ["user/handle", "julissa"],
"block": 7
}

Or as of 5 minutes ago.

{
"select": ["*", {"user/favMovies": ["*"]}],
"from": ["user/handle", "julissa"],
"block": "PT5M"
}

Using Our Time-Travel Capabilities

Now that we have our database in place, we can look at a few examples for how we can harness FlureeDB to gain (super important) insights into our friends’ movie tastes. Let’s consider a few scenarios.

Scenario 1: Your Friend, Lee, Swears He Never Liked The Movie “The Last Airbender”

With a single query, you’ll be able to prove that your friend is a dirty liar. However, before we can build that query, we’ll need to discuss a bit of background on flakes.

Flakes

In Fluree, every fact is stored in the database as a six-tuple, where each element in the tuple corresponds to: a subject-id, a predicate-id, an object, a time, a boolean (true or false), and a metadata object.

You can think of a subject-predicate-object combination as a fact. For example:

[["user/handle", "sammi"], "user/favMovies", "Quantum of Solace"]

The above triple asserts the fact that the subject, [“user/handle”, “sammi”] has a predicate (user/favMovies) with the object “Quantum of Solace.” In a normal, triple-store database, the subject, predicate, and object contain sufficient information. However, because FlureeDB has a time travel element, every piece of information includes additional information, namely:

• Time: A negative integer (t), which corresponds to the block during which a fact is asserted.
• true/false: Whether a given fact is true or false.

History Queries

In order to view history, we need to provide a three-tuple with the subject, predicate, or object information we want to query.

For example, to see every single update that Lee has made to his favorite movies, we can specify the subject as [“user/handle”, “lee”] and the predicate as “user/favMovies”. If we don’t want to specify an object, we can simply exclude the third item in the tuple.

{
"history": [["user/handle", "lee"], "user/favMovies"]
}

(Note that we can also specify an exact subject-id, i.e. 4294967296002, and a predicate-id, 1001. However, if we do not know a subject’s id, we can specify a two-tuple of a unique predicate object, i.e. [“user/handle”, “lee”], and for a predicate, we can simply specify the predicate name).

This query returns a long list of flakes that correspond to every update Lee has made to his favorite movies.

While we can find our answer in this list of flakes, we can issue a more specific query to determine whether Lee is lying about his history with the “The Last Airbender” movie.

{
"history": [["user/handle", "lee"], "user/favMovies", ["movie/title", "The Last Airbender"]]
}

Looks like he was caught red-handed! Below, we see that there are only two flakes that match our query. The first flake shows that Lee did in fact list “The Last Airbender” as a favorite movie at block 3. The second flake shows that he removed the movie in block 7.

{
"result": [
{
"flake": [4294967296002, 1001, 4299262263300, -15, true, null],
"block": 3
},
{
"flake": [4294967296002, 1001, 4299262263300, -15, false, null],
"block": 7
}
],
"fuel": 2,
"status": 200,
"block": 7,
"time": "4.39ms",
"fuel-remaining": 99999976702
}

Scenario 2: Did Your Users’ Tastes Get More Expensive?

You’re curious about whether users’ tastes in movies have gotten more expensive between the time they joined ReelFriends and now. In order to determine this, we can build an analytical query that returns the average budget for a user’s favorite movies at a given point in time. We’ll compare users’ average movie budgets in block 3 (when they first created their ReelFriends accounts) to now.

Analytical queries require a “select” key-value pair and a “where” key-value pair. The value for the “select” key will be [“?handle”, “(avg ?budget)”]. The ? at the beginning of ?handle and ?budget indicate that those are variables. We will declare these variable in our “where” value. avg is a built-in function that returns the average.

"select": ["?handle", "(avg ?budget)"]

The value for the “where” key is going to be an array of subject-predicate-object triples, similar to the ones we used in the historical queries. The components of these triples can also be variables, which gives us a lot of power in filtering and querying our data. Although, I present these clauses in a particular order, the order of the clauses doesn’t matter (unless you are querying multiple sources, which we are not doing in this example).

The first clause gets every user’s favorite movies, and it binds the user entities to ?user and the movies entities to ?movie.

[ "?user", "user/favMovies", "?movies"]

Specifically, we want to retrieve each movie’s budget, with our next clause binds the movies’ budgets to ?budget.

[ "?movies", "movie/budget", "?budget"]

Finally, we want our query to display user’s handles, rather than their subject-id, so we bind handles to ?handle.

["?user", "user/handle", "?handle"]

Our full query is:

{
"select": ["?handle", "(avg ?budget)"],
"where": [
[ "?user", "user/favMovies", "?movies"],
[ "?movies", "movie/budget", "?budget"],
["?user", "user/handle", "?handle"]]
}

When we issue this query, we find out that Julissa has the most expensive taste in movies, with an average movie budget of over $82 million. Lee has the least expensive taste, with an average budget of $68 million.

{
"status": 200,
"result": [
[ "julissa",82,503,000],
["lee", 68,466,666.66666667],
["sammi", 79,000,000]
],
"fuel": 26,
"block": 7,
"time": "3.17ms",
"fuel-remaining": 99999963388
}

If we issue the same query as of block 3:

{
"select": ["?handle", "(avg ?budget)"],
"where": [
[ "?user", "user/favMovies", "?movies"],
[ "?movies", "movie/budget", "?budget"],
["?user", "user/handle", "?handle"]]
}

We find out that users’ tastes did, in fact, get more expensive!

{
"status": 200,
"result": [
["lee", 25,133,333.33333333],
["julissa", 57,850,000],
["sammi",23,333,333.33333333]
],
"fuel": 22,
"block": 3,
"time": "1.55ms",
"fuel-remaining": 99999963366
}

Scenario 3: Sammi Doesn’t Have Anyone To Watch Crime Movies With!

Sammi is upset, because she is a big fan of crime movies. However, she feels like she doesn’t have anyone to watch them with anymore. Sammi is curious to see whether her friends have moved away from crime movies.

We already know how to run analytical queries, so we can easily find out if Sammi’s suspicions are true. In the below query, we select ?handle, and (count ?movie) specifically where the movies’ genre is movie/genre:crime. Note that because movie/genre is of type tag, we have to follow the ?genreEnt to get its _tag/id.

{
"select": ["?handle", "(count ?movie)"],
"where": [ ["?user", "user/favMovies", "?movie"],
["?movie", "movie/genre", "?genreEnt"],
["?genreEnt", "_tag/id", "movie/genre:crime"],
["?user", "user/handle", "?handle"]],
"block": 3
}

Results:

{
"status": 200,
"result": [
[ "julissa",1 ],
["lee",1],
["sammi", 2]
],
"fuel": 36,
"block": 3,
"time": "6.56ms",
"fuel-remaining": 99999999411
}

Back when her friends joined ReelFriends, both Julissa and Lee had one favorite movie in the crime genre. What about now?

{
"select": ["?handle", "(count ?movie)"],
"where": [ ["?user", "user/favMovies", "?movie"],
["?movie", "movie/genre", "?genreEnt"],
["?genreEnt", "_tag/id", "movie/genre:crime"],
["?user", "user/handle", "?handle"]]
}

Results:

{
"status": 200,
"result": [["sammi", 3 ]
],
"fuel": 47,
"block": 7,
"time": "1.77ms",
"fuel-remaining": 99999999364
}

Looks like Sammi needs some new friends. As of the most recent block, she has three favorite movies in the crime genre, but no one else has any!

A Little More About How Time Travel Works

As previously discussed, every transaction in FlureeDB is stored as a six-tuple flake. Flakes are the changes that happen to a database at a given point in time. If our database is currently at block 130, we can retrieve the database as of block 120 by simply “playing” the flakes of the last 10 blocks backwards. Moving our database to block 132 is only a matter of “playing” two-blocks-worth of flakes forward. By structuring our database as flakes, we can easily move back and forth in time, and even query across multiple points in time at once.

Not only can you use time travel to perform complex queries about various points in time, but you can even rewind entire applications with the click of a button. For example, we built a stock-option dashboard where you can view the stock options that employees are exercising. Rather than having to query for a single piece of information at say, March 8, 2013, you can view the entire database (and, in this case, dashboard, at that point in time).

Maybe you can’t use FlureeDB to go back in time and date your mother, but I think FlureeDB’s time-travel capabilities provide more practical business use cases!

Bonus Content: Find Out the Location For Your Favorite Movies

The below query connects FlureeDB with Wikidata and returns a list of the narrative locations for each user’s favorite movies. See if you can understand the query, and run it for yourself. If you’re curious about this feature, use a time machine to travel to my next blog post. ;)

{
"select": ["?handle", "?title", "?narrative_location"],
"where": [ ["?user", "user/favMovies", "?movie"],
["?movie", "movie/title", "?title"],
["$wd", "?wdMovie", "?label", "?title"],
["$wd", "?wdMovie", "wdt:P840", "?narrative_location"],
["$wd", "?wdMovie", "wdt:P31", "wd:Q11424"],
["?user", "user/handle", "?handle"]]
}

Abbreviated Results:

{
"status": 200,
"result": [ [ "julissa", "Troy","Ancient Greece"],
["julissa","Titanic", "New York City"]
…
}

Hey, time traveler… Want to learn more about Fluree? Head on over to our brand new Documentation and dig in!

While you are at it, check out:

Quick Links:

• Website
• Linkedin
• Twitter
• Slack