In Part one, before talking specifically about Overledger, I need to explain some important concepts to help you appreciate what makes Overledger different from other Interoperability platforms. You may frequently see / use words such as DAPP, Blockchain, Private Keys etc but not necessarily understand what they are and how they function. My goal is not to provide a very technical, low level explanation, but instead hopefully fill in the missing pieces of how they all come together. This is part one of Eight on a series taking a closer look at Overledger.
What is an API?
API stands for Application Programming Interface and they are used everywhere in the world today. They define how a developer can program an application to interact with another and process their requests and responses. APIs have been around for a long time, whenever you use a desktop or laptop, APIs are what make it possible to move information between programs — for instance, by cutting and pasting a snippet of a LibreOffice document into an Excel spreadsheet. System-level APIs makes it possible for applications like LibreOffice to run on top of an OS like Windows in the first place.
On the Web, APIs make it possible for big services like Google Maps or Facebook to let other apps “piggyback” on their offerings. Some of the most popular API’s are the ones offered by Facebook, Google maps, YouTube etc where they allow other apps to use their services. Think about the way Yelp, for instance, displays nearby restaurants on a Google Map in its app or my post here on Medium embedding a video on YouTube below.
API’s when connecting to third parties over the Internet can significantly change the landscape for the services they provide. Every comparison site you go to connects to an API of the third party to provide a price. A more recent example, which will change banking dramatically, is the recent introduction of the EU PSD2 regulation.
In short, PSD2 enables bank customers, both consumers and businesses, to use third-party providers to manage their finances. In the near future, you may be using Facebook or Google to pay your bills, making P2P transfers and analyse your spending, while still having your money safely placed in your current bank account. Banks, however, are obligated to provide these third-party providers access to their customers’ accounts through open APIs (application program interface). This will enable third-parties to build financial services on top of banks’ data and infrastructure.
Banks will no longer only be competing against banks, but everyone offering financial services. PSD2 will fundamentally change the payments value chain, what business models are profitable, and customer expectations. Through the directive, the European Commission aims to improve innovation, reinforce consumer protection and improve the security of internet payments and account access within the EU and EEA.
PSD2 — the directive that will change banking as we know it
2018 is set to be a game-changing year for retail banking. As the PSD2 (Revised Payment Service Directive) becomes…
What is a DAPP?
I’m sure you all know what a DAPP refers to — A Decentralised Application, but what does that involve?
An Application consists of multiple layers — Front End and Back End.
Front end is where the user interface is and is what the end user sees to interact with the application. When you browse to twitter.com in a web browser or open the app on your phone that is the front end.
The back end handles business logic and data storage for the App. It is not visible to the user and generally associated with a database etc. When you sign into twitter and you can see your tweets, and people you are following etc, this information is stored in the back end.
For all but very basic testing environments, the front end and back end will reside on separate servers. The front end will generally be installed on a group of servers at a cloud provider for high availability and the back end will also consist of separate servers at a cloud provider for high availability. The front end requests data from the back end via an API and the API provides the back end responses to the front end. Even if they were located on the same server they would consist of separate applications and would still use an API to interact with each other.
Both the front end and back end are Centralised.
A DAPP is where the back end consists of a Smart Contract stored on the blockchain and is run on a Virtual machine such as Ethereum’s EVM. The Smart Contract and the blockchain are now used for the Business Logic and Data rather than stored on a database.
A smart contract is a computer protocol intended to digitally facilitate, verify, or enforce the negotiation or performance of a contract. Smart contracts allow the performance of credible transactions without third parties. These transactions are trackable and irreversible
An API is used for the Front end application to communicate to the back end application (Smart Contract and Blockchain) as well as being used directly to move data on and off an underlying chain.
With a DAPP, generally the front end remains centralised and the back end is decentralised however the front end can also be decentralised by using services such as IPFS (Inter Planetary File System)
IPFS is a distributed file system that functions similar to a Bittorrent swarm. Each file and all blocks within it are given a unique fingerprint called a cryptographic hash. Each network node stores only content it is interested in and when looking up files you’re asking the network to find nodes storing the content behind the unique hash.
What is Hashing?
A Hashing function converts any form of data into a unique string of text of a fixed size. The Output produced is known as a Digest (Also referred to as a Hash or a fingerprint) and is always the same fixed size regardless of the size of input data. So, for example SHA256 will always be 256 bits long. If you are using Hexadecimal representation, then each digit represents 4 bits. So, the Digest will be 64 digits long.
As an example, the message “Hello World” when converted using SHA256 Hashing algorithm outputs the following digest:
If I change any part of the message (for example use a lower case w instead of capital) it completely changes the output of the digest.
The message “Hello world” when converted using SHA256 Hashing algorithm outputs the following digest:
The contents of the bitcoin whitepaper converted using SHA256 produces the following digest:
The maximum size of Data that can be converted in a single digest with SHA256 is (264–1)/8 bytes = around 2,147,473,648 terabytes of data, so no need to worry about hitting any limits there any time soon.
It’s important to note that if you convert the exact same piece of data it will output the same digest each time. So you can go to a site such as https://passwordsgenerator.net/sha256-hash-generator/ and copy the “Hello World” message you should receive the same hash output as above. A Hash is a one way function in that you can’t determine what the original data is by just having the Digest.
Hashing provides integrity of the message to ensure it hasn’t been modified (as any tiny modification would completely change the fingerprint output)
What is Symmetric Cryptography?
Symmetric cryptography is where a shared secret is shared between the two parties. The same Key is used to encrypt the data on the senders side and decrypt the data on the receiving side. As only the two parties know the shared secret then nobody else can decrypt the message and so provides confidentiality. Symmetric encryption offers fast encryption with little overhead but securely sharing the key over the internet isn’t very easy and as a result is not as widely used anymore.
What is Asymmetric Cryptography?
Asymmetric cryptography is where each user has a pair of keys, a private key and a public key. The Private key is never meant to be shared and strictly only the user knows their own Private key, whereas the public key can be freely shared. If Bob wants to encrypt a message and send it to Fred then Bob encrypts the message using Fred’s Public Key, as the only way to decrypt a message encrypted with a public key is using the matching Private Key. As Fred is the only person to know their private key they are the only ones able to decrypt the message, providing confidentiality. This allows for secure sharing of keys over untrusted mediums such as the Internet and is widely used throughout the world.
What is a Digital Signature?
A Digital Signature uses a combination of Hashing and Asymmetric Cryptography to provide Non-Repudiation(Sender can’t deny having sent the message) and Integrity (ensures the message hasn’t been modified).
First the Data is converted to a Digest using a Hashing algorithm. Then the Digest is encrypted with the Senders Private Key. It’s important to note that as it is encrypted with the Senders private key, this does not provide confidentiality as anyone with the Senders Public Key (widely accessible) can decrypt the message. Instead it provides non-repudiation, meaning that the signer can’t claim they didn’t sign the message as they are the only ones with the private key.
How does this relate to Blockchain?
Your cryptocurrency wallet uses Asymmetric encryption. Your wallet address that you receive cryptocurrency from is a hash of your Public Key. Your wallet’s private key is something only you should know and is what enables you to spend the currency held in the wallet. It’s recommended to use a hardware wallet such as a ledger to store your wallet’s private keys to protect them.
Following on from the example above if Bob wants to send 1 Bitcoin to Fred he would need to create a Bitcoin Transaction. A Bitcoin Transaction consists of Inputs and Outputs, inputs contain details of previous transactions to validate Bob does indeed have the funds to send 1 BTC. Bob also adds his Public Key to the inputs. In the outputs he adds the receiving address and funds to transfer. Bob then performs a Hashing function on this transaction data to receive a Digest (Hash) of the transaction details. As mentioned in the Hashing section above, this provides integrity that the transaction details haven’t been modified.
Bob then encrypts the resulting Hash of the transaction details with his private key which results in creating a Digital Signature. Note only the Hash is encrypted and not the actual transaction. As discussed in the Digital signature section above, this provides Non-Repudiation as only Bob has access to his private key and as Bob sends his matching public key within the transaction a miner is able to validate that they are able to decrypt the digital signature using Bob’s public Key to view the Hash of the transaction details. If they are unable to decrypt the digital signature then the transaction is marked as invalid and dropped.
The Digital Signature is added to the Input of the transaction and then all the data is passed through another Hashing function which results in a Hash that is referred to as the Transaction ID. This is what is commonly used when using various explorers to locate a transaction on the blockchain.
Before explaining how the transaction then enters the Bitcoin Network i will just briefly explain a what a node is and overview of blockchain consensus.
A Full node is a computer that runs software for a particular blockchain. Each full node in the blockchain needs to download and verify every single block, and therefore every single transaction in each block. As a result, this uses significant amounts of Memory, Network bandwidth and Storage. The Ethereum blockchain requires more than 1 TB of storage and needs a Solid State Drive rather than standard hard drive to keep up with the changes. Each day this requirement increases as more is added to the blockchain. Also worth bearing in mind is that these requirements are when blockchain isn’t mainstream yet. The requirements are going to increase significantly with more and more usage.
A Full node is a participant in the peer to peer network for that particular blockchain, the more full nodes there are, the more decentralised it is considered to be.
For Proof of Work Blockchains like Bitcoin, a full node by itself is known as a validator. A validator does not add new Blocks to the blockchain instead it validates new blocks and informs other nodes once the new block has been verified. This replicates throughout the network until consensus is reached. There are no rewards for being a validator node and are only an expense. Full nodes can also use significant computer resources and run software to be able to create new blocks in the process known as mining.
Full nodes are complicated to interact with and due to computer resources required, make them unavailable for mobile devices etc. This is where light clients come in (and is what the vast majority of people use to send a transaction through their wallet app such as Myetherwallet, Metamask, various mobile wallets etc. Light clients do not store every block on the blockchain only the headers which means the computer resources required are sufficient with a mobile device. The trade off though is that it must trust a full node and a light client does not participate directly in the decentralised network validating blocks.
In short, consensus mechanisms are protocols that make sure all nodes (device on the blockchain that maintains the blockchain and (sometimes) processes transactions) are synchronised with each other and agree on which transactions are legitimate and are added to the blockchain.
These consensus mechanisms are crucial for a blockchain in order to function correctly. They make sure everyone uses the same blockchain. Everyone can submit things to be added to the blockchain, so it’s necessary that all transactions are constantly checked and that the blockchain is constantly audited by all nodes. Without a good consensus mechanism, blockchains are at risk of various attacks.
There are various consensus mechanisms all with their own advantages and disadvantages, Bitcoin for example uses Proof of work which involves solving complex mathematical puzzles to be able to determine which node is able to add the block to the network and receive Bitcoin as a reward. There are many others such as Proof of stake, Delegated Proof of stake etc but not in scope for this article. All you need to know is that each has their advantages and disadvantages and there is not one which is seen clearly better than others, which is the reason why there are so many different models.
The genesis block is the first block on a blockchain, also known as block 0 and is the only block that does not point to a predecessor block. Every other block on the network can trace back its lineage to the genesis block. It defines the settings for the blockchian.
So to wrap it up with an overview of how Bob’s transaction is added to the Bitcoin blockchain
Bob has created and signed his transaction (likely using a light client) which then contacts a full node to obtain the remaining required information and the transaction is placed in a pool of pending transactions.
A Full node that is running mining software and expensive computer hardware takes a selection of transactions (as many that will fit) and places them into a block. The miner then runs a resource intensive computational puzzle and if it is the first to do so out of the other nodes then it is selected as the one who can append the block onto the Bitcoin blockchain.
Before the block is appended though the miner looks at the last block committed to the blockchain. It does this because each “block” contains a Digest (Hash) of the previous “block” and that is what makes it part of a chain, hence the term “Blockchain”.
It then adds the Digest of the previous block to the new block it wants to add, which is then validated by all of the other nodes to confirm it is correct based on the consensus rules. Once 2/3rds of the nodes have confirmed the new block to chain, it is committed, and consensus is reached.
If any of the transactions or any data within the Block are changed the Digest (Hash) will be completely different. Then any additional Blocks after will also have a completely different Digest as they contain the Digest of the previous Block. This is what makes Blockchain so secure, it is extremely difficult to tamper with any of the data. Blockchain creates an immutable, persistent, auditable platform that also includes timestamps and removes the need to trust a third party.
OK, that was quite a lot of info to go through but important concepts to understand. In Part Two we will look into the Layers of Overledger
Part One — Blockchain Fundamentals
Part Two — The Layers Of Overledger
Part Three —Verification and the Tokenisation of data
Part Four — Features Overledger provides to MAPPs
Part Five — Creating the Standards for Interoperability
Part Six — The Team behind Overledger and Partners
Part Seven — The QNT Token
Part Eight — Enabling Enterprise Mass Adoption