Notes on OrbitDB and Textile

OrbitDB and Textile are peer-to-peer databases built on IPFS and CRDTs. Here are some neat bits and differences we found while researching peer-to-peer storage with Autark and Aragon, and for our projects at Open Work Labs.

Environment and philosophy

Orbit is written in JavaScript, so it works in node and the browser. Their design philosophy emphasizes maximal developer flexibility. This means staying agnostic to underlying schema design, encryption, and identity.

Textile’s initial Go implementation was designed for mobile and has encryption and schema tools built in. They’re quickly building a JavaScript api for connecting to local or remote Textile nodes. More on their philosophy.

Schemas

Schemas help peer-to-peer databases keep data consistent. This makes way for features like reliable querying and adds control over IPLD construction.

Textile has protocol level support for schemas, which define how data is processed and stored. Each thread (a decentralized database, shared between specific peers) can only have one schema type that defines a DAG structure and transformations needed to construct it. There are a few things written on this, including their use of “mills” to transform data into DAGs, and the use of a ‘/blob’ schema to handle arbitrary data types.

In Orbit, a database can be instantiated with one of five data models, which generally defines their schema. Enforcing other or more specific schema definitions — such as data formats or DAG structures within a specific database — is left up to the developer via a custom store. More on this will be coming out soon in their field-manual. According to Mark Henderson from Orbit, DAG construction and general schema enforcement are features that can be easily added to the protocol.

Access control and encryption

Textile implemented a “wallet” based off of Stellar’s wallet — the wallet generates different accounts (each with their own private key) for different applications/devices. Unique keys are generated for each thread and database entry. Information in a thread is encrypted by default, with encryption being built into the protocol. Here’s a description of Textile’s encryption. Wallets allow for account backup and recovery in case accounts are lost (e.g. a user loses their phone in the mud). In terms of dynamic access control, threads provide a robust framework to specify read/write and sharing permissions.

In Orbit, dynamic access control is handled for writing to a database, but by default, if a user gets access to the Orbit address, they can read the data. Entries can be encrypted before going into Orbit, but databases are unencrypted by default. Orbit uses key-pairs within their identities — an identity public key is needed to give that identity write access to a database. You could use this identity to generate other keys (for other browsers or devices) and/or encrypt data, but it’s unclear if this would cause security threats. Additionally, identity key pairs are stored in the browser (and as of recently in leveldb to run Orbit in web-workers), which raises issues — for example, what if a user cleans their browser or uses a different one? This is a known issue, with good ideas for solutions, but much is left up to the developer and there may be security implications.

The browser introduces different security challenges than those in native environments. Key pairs can be stored more securely and permanently on a user’s machine than in the browser.

Always on nodes

One similarity between Orbit and Textile is that they both rely on “always on” nodes. Since clients can sign on and off often and spontaneously, we keep a node always running that can update a peer if no other peers are online with their data.

3Box has open sourced an example Orbit pinning node here.

Textile provides an open source service called “cafes” (with instructions on deploying them), which are nodes that offer offline inboxing and backup for this purpose. A cafe peer can be also used by JWT authenticated clients as a standalone server to pin files. JWT authenticated clients could include browser clients as well.

Conflict resolution

Peer-to-peer databases need a strategy to avoid conflicts from updates from members of the group. Orbit built a solution with CRDTs called ipfs-log and threads’ approach is similar (more on that here). Both are append-only logs that help peers share a mutable state. There’s been a lot of research on CRDTs by Protocol Labs and others.

Thank you to the OrbitDB and Textile teams for reviewing this blog post and being super helpful throughout our research.