Middleware Solutions for the Ethereum Blockchain

Published in

Deeplink Labs

9 min readAug 6, 2021

image: https://thecurrencyanalytics.com/blockchain/ethereum-eth-vs-chainlink-on-right-tool-for-right-job-14726.php

Abstract

The following article is a collection of information from a research sprint into middleware. More specifically, the team focused on enterprise-grade approaches to middleware services connecting developers to the Ethereum blockchain, and investigated the challenges they face and how they overcome them. We first discuss what middleware is, how it works, and why it is important. Then, we cover a few Ethereum middleware solutions. Lastly, we explore the fundamentals of implementing your own middleware solution linked to the Ethereum network.

Authors

Gavin Stein: Researcher/Editor

William Godfrey: Researcher

Jack Lodge: Researcher

Stephanie Shen: Researcher

Cameron Dimovski: Researcher

What is Middleware?

Middleware is a type of software that provides a common service or method of communication between two or more applications within a distributed network. It simplifies the process of trying to connect applications which weren’t designed to communicate with one another in turn helping streamline app development and reducing the time a product is brought to market¹.

How it Works

Middleware solutions allow applications to be developed without the need of having to create custom integrations whenever you want to communicate between different or foreign application components². To do this, they allow applications to connect by utilising messaging frameworks such as REST, JSON or XML to name a few.

In addition to this, middleware also provides services that let components which can be written in multiple programming languages to communicate with one another¹.

Types of Middleware

Middleware comes in many different implementations¹. To name a few:

APIs

Developers are provided tools to create and manage APIs for application use, allowing other devs to be able to connect to them

Databases

Makes the process of accessing and interacting with a back end database more simplified

Message Oriented

Application components whether that be data sources or devices can exchange messages by using different messaging protocols.

Remote Procedure Call

An application is able to trigger a procedure within another application on a different network as if they both running on the same computer

Within Enterprise

One use case for middleware is within industrial enterprises. It allows an organisation to connect all the business processes, back end data sources and application components within an integration hub. It can also be used for intelligent automation, helping a business to improve resource management and overall efficiency as it eliminates the manual processes.

An Enterprise Service Bus used to be the most commonly used middleware serving as an integration hub. In the present day however, cloud based models are used more and allows a business to connect all their services in a cloud environment. This is more efficient, eliminating the costs used for installation and maintenance of the middleware hardware within their own data centre¹.

Why Is it Important?

To summarise what was said so far, middleware helps businesses manage the challenges of using different tools, frameworks, programming languages or wanting to deploy an application across multiple platforms in which an architecture can vary. The key thing to remember is that it helps make application development cost-effective, efficient and helps to support environments, ensuring services will run consistently across a distributed network².

Existing Middleware Services for Ethereum

The following are a few different types of middleware solutions that currently exist to help you connect with the Ethereum blockchain depending on your needs. This is by no means an exhaustive list however.

Nansen.ai

Nansen.ai is an analytics platform for Ethereum. They combine blockchain data with wallet labels. Their blockchain data comes from an open source platform Ethereum ETL. This platform allows you to download up-to-date blockchain data in CSV formats via command line or you can access the latest database in Google BigQuery with SQL queries³.

Nansen.ai does more than just provide a nice interface for blockchain data analytics. It also provides labels for crypto wallets. An example of the label could be “Millionaire”, indicating that the wallet handles more than 1 million USD worth of cryptocurrencies. These labels are not originally a feature in the blockchain data. Over 99 percent of the labels are inferred with high precision from methodologies like machine learning. Nansen.ai currently tracks over 90 million labeled Ethereum wallets and their activities⁴.

image: https://docs.nansen.ai/Wallet-Labels-Emojis-What-do-they-mean-f55ee986ee724f4ab864a6a9c90207cf

Settlemint

Developing your own blockchain solution from scratch is hard. SettleMint aims to bridge the gap between companies and blockchain by offering minimised configuration APIs. It can help you by offering a layer three solution, from providing smart contract templates and dAPIs to a clean and informative user interface (dAPP)⁵.

Chainlink

Oracles

Oracles are becoming an increasingly foundational 3rd party of the decentralised financial ecosystem (DeFi), providing off-chain data for blockchain smart contracts. Oracles address crucial problems:

1. Blockchains struggle to allow native communication to external network systems off-chain.

2. Oracle networks must be decentralised in order not to nullify the benefits of the blockchain⁶.

image: https://www.gemini.com/cryptopedia/what-is-chainlink-and-how-does-it-work

Enter Chainlink

Chainlink is a decentralised oracle network established by Steve Ellis and Sergey Nazarov in June 2017. Chainlink functions off-chain as a network of oracle nodes connecting to on-chain protocols. On-chain, Chainlink functions a series of smart contracts. The Chainlink process begins with a requesting contract that triggers an event and concurrently three other contracts:

1. Reputation contract (verifies the reputability of nodes)

2. Order-matching contract (matches smart contract orders)

3. Aggregating contract (collects data from oracle node)⁶

Together these three pieces make up the Service Level Agreement (SLA) initiated when a requesting contract is made on Chainlink. Contract holders that would like to request a smart contract for a task pay LINK tokens. The rates established by the Chainlink node operator are dependent on the market relevance of the information given. Furthermore, node operators will stake on the network and deposit LINK as proof of their commitment. Node operators are penalised for malicious behaviour through a tax on the stake of the node.

Chainlink Adoption and Use case

Chainlink’s products and uses are currently being utilised by a diverse range of protocols to facilitate certain tasks. These tasks include, empowering decentralised lending and borrowing applications, derivative markets, price feeds to settle trades, insurance, pegging assets, connecting real world assets, payments, asset management and even NFT’s and Verifiable Random Function (VRF). More recently Chainlink has been deployed within Binance Smart Chain (BSC) to provide VRF. Another recent project is Chainlink being used by yearn.finance for the v2 LINK yvault (yvLINK)⁷. Chainlink provides the yearn protocol with a decentralised oracle network to deploy three strategies:

1. Supplying LINK to Vesper Finance LINK Pool to earn rewards

2. Lending LINK to AAVE and gaining interest while staking the accumulated AAVE

3. Locking LINK at MakerDAO then using minted DAI as a deposit for earning yield.

Chainlink 2.0

Chainlink Labs issued the Chainlink 2.0 white paper in April 2021, outlining increased network capabilities and possibilities. Scalable off-chain computing, explicit staking, and privacy-preserving features all contributed to Chainlink Lab’s long-term goal of creating a Decentralized Meta layer. The key functions were as follows: hybrid smart contracts, abstraction of complexity, scalability, secrecy, order-fairness, minimisation, and incentive-based crypto-economic security, as stated in the white paper⁸.

If you want to learn more about Chainlink, Cameron Dimovski has written an in depth piece you can read here.

Implementing your own Middleware Solution

Implementations exist across multiple languages such as Python, Javascript, Java, Swift (for iOS), and more. This article will focus on a Python implementation. You should become familiar with the following tools/concepts which are used in a variety of implementations.

Web3.py

Web3.py is a python module that simplifies interactions with the Ethereum blockchain for developers. With Web3.py you can create and send transactions, query smart contracts, and gather all sorts of on-chain data. If you are using google colab for any dev work, you will need to install jsonschema version 3.2.0.

Infura.io

Infura.io is a service that runs a node on the Ethereum blockchain and interacts with the network. This allows users of this service to connect to an Ethereum node over http. While this service has been criticised for introducing centralisation to a decentralised environment, for our purposes it is ideal.

GraphQL

Graph Query Language is an open-source language for data querying and manipulation within APIs.

Application Binary Interface (ABI)

ABI is a format defined for interacting with contracts and describes how data should be encoded, decoded for executable bytecode on the Ethereum Virtual Machine. It is represented in JSON format. The ABI will tell the EVM the function to call, what parameters to pass in, and outputs to receive, amongst other things.

ERC20 Standard

Refer to the official documentation for more information⁹, however this standard sets a minimum level of functionality for complying tokens.

First and foremost, any implementation needs to be able to connect to the blockchain, in this case Ethereum. While you could run your own node and connect that way, for R&D and small scale operations, this is too laborious and difficult. Instead, you could use a service such as Infura.io.

Now that we have the ability to connect with the Ethereum blockchain, we need a way to interact with it. That is where Web3.jy comes in. This python library gives you much more simple functionality with the Ethereum blockchain.

With both Infura.io and Web3.py, we have all the functionality to implement a middleware solution. However, there is one more piece of information that anyone implementing a middleware solution should be comfortable with; Application Binary Interface (ABI).

Below are a few examples of using these concepts to connect to the Ethereum blockchain and pull some information. These have been adapted from another article which you can find here.

First, make sure you have web3.py installed.

pip install web3

If you are using Google Colab, you will need to set the version of jsonschema to 3.2.0

!pip install --force-reinstall jsonschema==3.2.0

Depending on what method you use to connect to the blockchain, the next step is to create a web3 object. For this example, we used an Infura.io project.

w3 = Web3(Web3.HTTPProvider('https://mainnet.infura.io/v3/PROJECT_ID'))

You can information about the latest block like this.

w3.eth.getBlock('latest')

To get the balance of ether at an address, you can use the getBlanace() function.

address = Web3.toChecksumAddress('ADDRESS')# Get balance in wei (1/10^18 ETH)wei = w3.eth.getBalance(address)# Convert to ETHether = wei / 1000000000000000000

In the above example, the toChecksumAddress() function makes sure that it is a valid address.

The next example will query the balance of an ERC20 token at a given address.

# Import json moduleimport json# Establish ABI for handling smart contract inputs and outputs
ABI = json.loads('[{"constant":true,"inputs":[{"name":"_owner","type":"address"}],"name":"balanceOf","outputs":[{"name":"","type":"uint256"}],"payable":false,"stateMutability":"view","type":"function"}]')# Checksum wallet address of your choice
wallet_address = Web3.toChecksumAddress(ADDRESS)# Create an object of the ERC20 token contract
token_address = Web3.toChecksumAddress('0xc011a73ee8576fb46f5e1c5751ca3b9fe0af2a6f')contract = w3.eth.contract(token_address, abi=ABI)# Get balance of ERC20 token at given wallet address
raw_balance = contract.functions.balanceOf(wallet_address).call()# convert the value from wei to ether
value = Web3.fromWei(raw_balance, 'ether')

In this next example, we will get price data for the ERC20 token and convert this value to USD (well actually DAI). The ERC20 token we have as an example above is the SNX token or Synthetix network token. So we will get price data for SNX from on-chain (we will use UniSwap) and convert it to a USD value. Install gql if you haven’t already.

pip install gql
from gql import gql, Client
from gql.transport.requests import RequestsHTTPTransport

Now we will use gql to send a request to get price data from UniSwap on chain. Because UniSwap does not have a SNX/DAI liquidity pool, we will find the SNX/ETH value and then the ETH/DAI value.

sample_transport=RequestsHTTPTransport(
url='https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v2',
verify=True, retries=5,)client = Client(transport=sample_transport)query = gql('''
query {
  pair(id: "0x43ae24960e5534731fc831386c07755a2dc33d47"){
    reserve0
    reserve1}
}
''')# Get reserves of SNX/ETH liquidity pool and find the priceresponse = client.execute(query)
snx_eth_pair = response['pair']
eth_value = float(snx_eth_pair['reserve1']) / float(snx_eth_pair['reserve0'])# Now do the same for the ETH/DAI poolquery = gql('''
query {
  pair(id: "0xa478c2975ab1ea89e8196811f51a7b7ade33eb11"){
    reserve0
    reserve1
}
}
''')response = client.execute(query)eth_dai_pair = response['pair']
dai_value = float(eth_dai_pair['reserve0']) / float(eth_dai_pair['reserve1'])snx_dai_value = eth_value * dai_value