A journey towards privacy. The usability.
Since the beginning of time in 2008, privacy and security have been in constant battle with usability. You know the drill: “Not your keys, not your coin”. This problem has grown more significant with the advent of blockchains that promise computational privacy.
In this post, I will share how we at Obscuro Labs are building the first decentralised platform that keeps data private but does not change the user and developer experience. After reading this article, as a developer, you should understand how to build a dApp where protecting data access is a first-level concern. As a future user, you will learn the very simple concepts behind the tech that keeps your data confidential.
I won't explain in detail why the transition from fully transparent chains is beneficial, but, to make a long story short, we think blockchains are revolutionary because they are permissionless and offer transparency in the validation process, not in the data. In other words, the desirable property is that they remove the trust and thus the power that a central operator has, not that all balances are visible.
A blockchain with confidential data
To build a successful permissionless platform where data is private, these conditions must be met:
- Data and processing must remain confidential.
- Validation must be transparent.
- The user experience must be good.
This article will focus on the last point because it is a crucial ingredient to success but receives much less attention.
If you're interested in how we achieve the first two conditions, have a look at our whitepaper. In a nutshell, we use Trusted Execution Environments (TEEs) which are encrypted computing enclaves within a CPU and memory into which no one, including node operators, can see.
Confidential data
What do we mean by “data” in the context of a blockchain?
It all starts with user transactions, which are signed commands. These transactions will be executed by nodes where they fail or succeed. The result of executing transactions is a change in the internal state of the node, kept in a local database, reflected in each block as a state root hash. Nodes will expose their internal database via RPC. Executing a transaction also results in a transaction receipt and event logs.
All these elements are referred to as “data” and must be kept confidential on behalf of the user who owns them.
The fully transparent chain
Before discussing the user experience (UX) and developer experience(DX) of chains with confidential data, let’s first look at how things work in transparent chains.
Note: you can skip this section if you understand the UX of existing chains.
There are no secrets in a typical EVM chain like Ethereum, Optimistic Rollups, Avalanche or ZK EVM rollups (yes, even that).
Any user, web application or smart contract can call any function of any smart contract as long as it knows its address and ABI. If no function returns the desired data, a user running a node can query the internal database directly for any information.
Another way of accessing data is querying or subscribing to event logs. These are pieces of information emitted during the execution of transactions. In a typical chain, anyone is free to query or subscribe to anything.
For example, if we look at the ERC20, anyone can call "balanceOf" for any address or subscribe to the "Transfer" event to get a live feed of all activity.
Note: This article will use the ERC20 as our primary example because it is the most used and well-understood smart contract.
The user experience
Typically, users have a wallet like MetaMask, which runs as a browser extension and has a javascript API.
There are dApps written in javascript that can connect and interact with the installed MetaMask. These dApps query data from the blockchain and aggregate it in dashboards. They can also present real-time data based on the live events emitted by contracts. The ultimate goal of any dApp is to create the right environment where the user can build, sign and submit transactions, which will modify the chain's data. Transactions are always signed in MetaMask, which pops up at the right time.
The developer experience
DApp developers worry a lot about protecting the integrity of their applications. That transactions submitted by users can only follow specific rules and perform certain actions.
They don't have to worry about preventing users from accessing protected data because everything is public.
The UX and DX of the confidential chain
Now that we understand the context, the problem we have to solve is how to maintain the experience described above in a world where data is opaque.
Users should still be able to use their favourite wallet, have a web dashboard with all the information they need, and have an overview of all their assets. Developers should not have to learn new syntax and use the same tooling.
Our ambition is to achieve that while data is kept private from everyone except the Trusted Execution Environments.
I propose we start by defining a fundamental principle.
The Rule
Data should be visible only to the account owners to which it is relevant.
This is a straightforward but potent rule. Our thesis is that as long as this rule is implemented correctly, there should be no UX degradation for most existing applications.
Note: Adding confidentiality means that Maximum Extractable Value (MEV) is no longer possible, and other classes of applications, like on-chain games with hidden information, become possible.
For the rule to be complete, we need to define data relevancy:
A piece of data (like a balance or an event) is considered relevant to a user if that data belongs to that user, was created by that user or has somehow affected that user. Also, if the creator of the data explicitly or implicitly decides that a user is entitled to see it.
You may notice that this is more like a set of guidelines than a strict definition. In practice, though, it is pretty straightforward to define it.
For an ERC20, for example, it is evident that the balance of an externally-owned account “belongs” to that user. In an on-chain game, the position of a player belongs to that player. An NFT minted by an artist belongs to the artist. And so on.
A developer working on a dApp has to go through the exercise of defining relevancy for every piece of information exposed.
The types of blockchain data
A confidential platform, like Obscuro (now known as Ten), must implement the relevancy rule on the data it controls and enforce the relevancy rules decided by the dApp developers.
There are multiple categories of blockchain data. Let’s look at them individually.
1. Data controlled by the platform.
As the name suggests, this data exists behind the scenes from the point of view of the dApps running on the platform.
Some examples are:
- Submitted transactions.
- Transaction receipts.
- Account information like balance, code, nonce, etc.
The confidential platform has to manage access to this data; otherwise, if the transactions were public, anyone could recreate the entire ledger by just replaying them.
Let’s apply the relevancy rule to these practical cases:
- The platform considers transactions and receipts relevant (and thus visible) only to the transaction signer.
- Account data is only relevant to the account owner.
Note that this is a direct application of the principle without any nuances or surprises.
2. Data exposed by smart contracts via "view" function calls.
Smart contracts can maintain an internal state and expose it via read-only functions. One example is: `ERC20.balanceOf`
The platform has no way of knowing the meaning of this data, so it cannot make any “relevancy” judgements. But the developers writing those functions do, so it makes sense for the platform to delegate this responsibility to them.
To allow the developers to do that, the platform has to provide a secure way for the smart contract to access who made the original call and to know whether there are intermediate contracts between the initial and current calls.
We'll have a closer look at this topic in the next section. To summarise the solution, the developer can use typical programming constructs, like if statements, to check whether the caller has the correct rights. This is a concept that developers of backend applications are very familiar with.
3. Data created in smart contracts but exposed by the platform.
This category is composed only of event logs.
Note: If you want an in-depth understanding of the inner workings of events, I recommend this article.
An example is an ERC20 `Transfer(from, to, value)` event, emitted when the token is transferred between two accounts.
Events are similar to view functions because they are application specific; on the other hand, they are different because they are asynchronous. They are emitted when transactions are executed but can be read at a later time by someone else.
This difference means that the data relevancy rules can’t be implemented programmatically when the event is emitted because it’s impossible to know who the requester is.
Let’s look at what design options we have to move forward:
Option 1 — Consider events relevant to everyone, but obfuscate the data
In this approach, the developer is put in control at the time of event emission and is responsible for ensuring no confidential data is leaked.
This approach appears simple and somewhat consistent with function calls but has a significant disadvantage: it severely degrades UX and breaks existing standards such as the ERC20.
If the developer obfuscates the `from`, `to`, and `value` fields of the Transfer event, then the smart contract is no longer a compliant ERC20. If they don’t, all data is effectively public and can be reconstructed.
According to the spec, if they choose not to emit the `Transfer` event, then the token is no longer an ERC20.
Option 2 — Implement a mechanism to reveal events selectively
One approach is to create a declarative mechanism using metadata, which the developer must populate and is available at runtime. This requires changes to the virtual machine, the programming language, and the tooling, which degrades the developer experience.
Our approach with Obscuro is based on the insight that events naturally contain externally-owned addresses as topics because users must be able to discover their events. This information is almost logically equivalent to the definition of data relevancy.
Rule 1: An event is relevant to all the externally-owned addresses it contains as topics.
According to this rule, the ERC20 Transfer(from, to, value) event will be visible only to the “from” and the “to” addresses.
There is one more rule required for confidential events.
If we look at an event like the “Uniswap.PoolCreated”, we notice that it is of general interest to all users looking to use Uniswap. It doesn’t contain any externally-owned address because it is a “Lifecycle event” of the contract. This brings us to:
Rule 2: An event is relevant to everyone if it contains no externally-owned addresses topics.
Note: We won't describe how the Obscuro platform implements these rules, as it would make for a much longer article. Shortly, it achieves this using viewing keys and encryption transparently to the end user and the dApp developer.
Writing a confidential dApp
So far, we have looked at the theory. It’s time to apply it and write a smart contract for a confidential platform.
The most important thing is for an app developer to always keep in mind that it is considered a bug if users can't access relevant data or if they expose irrelevant data.
The Obscuro ERC20
Let's start by writing a confidential ERC20, our go-to example.
The primary role of an ERC20 token is to maintain users' balances and to ensure that the common-sense rules of fungible tokens are implemented correctly. On top of that, a confidential ERC20 has to define and enforce data relevancy.
Any ERC20 has to maintain two maps:
- balances (account_owner => balance)
- allowances ([account_owner, account_spender] => amount)
Common sense tells us that the balance is only relevant to the account owners. And that the allowance is relevant to both the owner and the spender.
One interesting aspect is that both Externally Owned (EOA) and Contract accounts (CA) can hold a balance, so they should be able to query their balance.
Below you can see the "balanceOf" implementation from the reference Obscuro ERC20:
Before dissecting the function, let's examine the two implicit variables available inside the EVM.
`tx.origin` — is the address of the account which made the "eth_call" RPC call.
`msg.sender` — is the contract's address calling the current contract.
Note: This short article describes the difference between them. These are available in all EVM implementations.
Note: The typical EVM does not authenticate `tx.origin` for "eth_call" because these RPC calls are not signed. A confidential platform like Obscuro must create an authentication mechanism.
The “balanceOf” implementation is straightforward. It returns the balance only if the transaction originator or the calling contract asks for relevant data (as defined above).
Note: the security-minded reader will rightfully be concerned that any intermediary contract can leak all the data that passes through it. That is undoubtedly true, but remember that the user is ultimately responsible for the app calls they make, as they are responsible for the transactions they sign. If the user makes a call to smart contract A, it means that it trusts that A will not leak data either directly or indirectly.
The Transfer event
There is nothing for the developer to change on the transfer event because the information it contains ensures that the platform will only make it visible to the “from” and “to” parties.
It is good practice for the developer to double-check that the topics correctly capture the intent and should also add comments to describe the desired behaviour so that it becomes explicit.
Conclusion
At Obscuro, we focus on bringing usable confidential smart contracts to Ethereum.
Our goal is very ambitious. We want to hide all confidentiality details from end-users and allow developers to use their favourite Ethereum tooling without learning any new syntax.
By introducing the simple and intuitive concept of “Data Relevancy”, we are replicating the UX of web2 on top of a decentralised blockchain. Users in the web2 world take for granted that their data will only be displayed to them and that they can access all relevant data. DApp developers deploying on Obscuro who follow our guidelines will be able to give their users the same experience.
We are keen to hear your feedback about our thinking.
Find out more
To learn more about Ten, dive into the whitepaper, chat with the community on Discord, and follow us on Twitter.
