Decentralized Communications
Problem and Motivation
As most internet users are intermittently connected and behind NATs, for reliable messaging they must relay messages to other users using centralized services that are always available (e.g. messaging, forums, social networks, file sharing). The network effects and perverse incentives of vertically integrating message relay via centralized services has resulted in:
No privacy: private messages are readable by services.
No trust: recipients cannot verify senders.
No identity ownership or mobility: identities are service specific or outside of user control.
No message mobility: messaging only within a service.
No ownership: services own and control data and communications.
No backups: data is lost if a service goes down.
Single point of failure: communication stops if service goes down.
Goal
A system which solves these problems given the following constraints:
- no trusted third parties
- reliable, real time, persistent and redundant messages
- works without sender & receiver being online at same time
- scalable to current messaging traffic (>30B messages/day)
- strong spam and malicious agent resistance sans human intervention
- direct uncheatable economic incentives — no assumption server operators are charities
- user should have option to run their own server (on static IP) to ensure reliability for sending their own messages even if all other servers disappear
Existing services attempting to address either file sharing or messaging networks fail to meet many or all of these constraints.
Proposed Solution: Pubkey Ids & Message Relay Market
By making identities public keys and encrypting and signing messages in a common format, we can solve the privacy and trust problems as well as gaining identity mobility and message mobility.
By separating message relay and application functions of traditional services and creating a market of message relay servers, we can solve the no ownership, no backup and single point of failure problems.
The problem of uncheatable economic incentives (that user’s will pay for purchased relay services and services will provide them) is solved using Bitcoin joint escrow and user auditing via receipt acknowledgement messages.
Servers broadcast their availability, capabilities and prices on a public channels (e.g. Bitmessage, IRC, etc) so clients can discover them to contract message relay services. Clients can then share their contracted server list with their contacts via other channels in order to send messages.
Servers do not communicate with one another as email servers do and need have no knowledge of the existence of other servers. Instead, clients pull messages from the contracted servers of their friends/connections, which effectively eliminates spam. A server can also limit how many times a message may be pulled, which allows a user to limit their bandwidth costs for with a given message send.
Clients provide easy means to publish their user’s pubkeys and contracted server addresses via a standard format on existing networks (via a json file with a custom extension) and automatically post structured notifications within the network to friends when their contracted server list changes.
All messages are structured and email or IM like messages are just a few of the potential cases.
Messages and Servers
The server acts like a simple file system where all records are on paths of the form:
/userPubKey/outbox/receiverPubKey/recordId
Every record is of the form:
{ msg: { from: senderPubkey, to: receiverPubkey, timestamp: ts, request: { type: “write”, recordId: id, ttl: dt, tag: X, content: encryptedNonceAndContentUsingReceiverPubkey }, sig: msgSignatureUsingSenderPrivateKey, }
Where content is base64 encoded before encryption.
A read is only allowed if the read request is signed with the appropriate receiverPrivateKey. A write is only allowed if a record is signed by the sender and the senderPubKey has an active account on the server. This means the system is pull-based and clients are responsible for pulling from servers which may have messages for them.
As each record is encrypted using the receiver’s pubkey, it’s content can only be read by it’s intended recipient and not by the server or anyone passively or actively intercepting network traffic.
When a friend retrieves a message, they place an acknowledge response on their own contracted servers. In this way, user’s can periodically verify bandwidth usage and server charges.
Scaling and costs
A market in services ensures this architecture scales in proportion to usage and dynamically adjust prices ensuring value provided, resource costs and vender margins meet.
Uses
The general nature of these records would allows a wide range of uses. Facebook, Twitter (using broadcast channels), Dropbox, email, IM, etc.
API Details
All messages and responses are in the form:
{ msg: { type: “write”, from: senderPubkey, to: receiverPubkey, timestamp: ts, request: REQUEST, sig: msgSignatureUsingSenderPrivateKey, }
For brevity, we’ll only show the REQUEST component in following API specs. The sender or receiver can also be the server.
Server Record API
receive: { type: “write”, recordId: id, ttl: dt, tag: X, content: encryptedUsingReceiverPubkey }
respond: true or error string
receive: { type: “read”, path: path }
respond: Record or null
receive: { type: “inventory”, afterTs: ts, beforeTs: ts }
respond: [recordId1, recordId2, …]
receive: { type: “listen”, afterTs: ts }
receive: { type: “delete”, recordId: id }
respond: an array of recordId’s whenever new one’s are available and have not already been sent on this connection
Server Broadcast Message
send: { type: “serverBroadcast”, serverPubKey: K, address: ipOrOnionAddress, storageCostPerGigPerDay: btcX, bandwidthCostPerGig: btcY, proofOfWork: pow, timestamp: ts, signature: S }
Server Contract API
receive: { type: “contractOpen”, userPubkey: pubkey, tx: txWithUserinputs }
send: { type: “ContractServerAddInputs”, tx: twWithServerInputs }
receive: { type: “ContractClientSigned”, tx: txSignedByClient }
The server then signs and submits to Bitcoin network. When usage absorbs payment amount.
send: { type: “ContractRequestPayment”, tx: txSignedByServer }
If client accepts, it signs and broadcasts.
send: { type: “ContractRefundPayment”, tx: txSignedByClient }
If server accepts, it signs and broadcasts it to the Bitcoin network.
Server Usage API
receive: { type: “RequestUsage”, afterTs: ts1, beforeTs: ts2 }
respond: { type: “UsageReport”, [{ receiver: , path: }, …]}
Client Identity sharing format
{ type: “Identity”, label:””, pubkey: “”, relays: [address1, …] }
