The Python Ethereum ecosystem

5 min readApr 6, 2017

Given the recent influx of new users to the Ethereum ecosystem it feels appropriate to give an update on the state of Ethereum tools for Python developers.

High Level Tools

The following tools sit at the top of the python stack, exposing high level interfaces and abstractions for application development and interaction with the Ethereum blockchain.

Populus: http://populus.readthedocs.io/

Populus is a smart contract development framework. It can be used as a command line tool to compile and deploy your contracts as well as a python library for high level scripting and automation. The framework also includes tools that make testing your smart contracts simple and painless.

Web3.py: http://web3py.readthedocs.io/

Web3.py is a python library inspired by the original Javascript based web3 library. This library exposes a standard and familiar way to interact with the JSON-RPC interface exposed by Ethereum nodes. In addition, it exposes a number of common utilities such as Ethereum currency denomination conversions, a contract class for interacting with smart contracts, and encoding/decoding utilities.

Eth-TestRPC: https://github.com/pipermerriam/eth-testrpc

This is the Python implementation of the TestRPC server (not to be confused with the Javascript implementation). This can be run as either a command line tool or programmatically from your code to create a transient Ethereum blockchain that can be used for testing.

PyEthApp: https://github.com/ethereum/pyethapp

PyEthApp sits on top of the PyEthereum library as a full Ethereum node written in python. It has atrophied a bit in the last year due to a lack of active maintenance and development. While I do not believe that this application can be run as a viable full Ethereum node, there is work being done now to fix this.

Low Level Tools

These lower level tools attempt to “do one thing well”. They are likely to prevent you from re-inventing the wheel as you develop your Ethereum applications.

Ethereum Utils: https://github.com/pipermerriam/ethereum-utils

This library is a “swiss-army-knife” for Ethereum projects. It exposes a number of useful utility functions that solve problems you are likely to run into. The functions are all designed to gracefully handle both unicode and bytestrings as well as having consistency in the types each function returns.

Ethereum ABI Utils: https://github.com/pipermerriam/ethereum-abi-utils

Sometimes the high level interfaces exposed by libraries like web3 are not quite what you need. When you need to dig deeper into transactions and contracts you will likely need to encode and decode the ABI encoded data. This ABI utils library exposes easy to use functions for performing these encoding and decoding operations.

Py-Solc: https://github.com/pipermerriam/py-solc

This is a simple wrapper around the solc command line compiler providing a nice to use API for compiling contracts.

Py-Geth: https://github.com/pipermerriam/py-geth

This is a simple wrapper around the geth client. It provides some nice abstractions for running nodes as subprocesses within a python script.

Base Layer Libraries

PyEthereum: https://github.com/ethereum/pyethereum

PyEthereum is the defacto Python implementation of the EVM. It contains useful utilities for testing as well as embedded versions of many of the utilities listed above such as ABI encoding/decoding, contract abstractions, utility functions, and solidity compilation. This library has primarily been authored by Vitalik Buterin who continues to use it as a tool for advancing his research and development of the base protocol.

PyRLP: http://pyrlp.readthedocs.io/en/latest/

This library implements encoding and decoding for the encoding scheme used by Ethereum for all of it’s different data types. This tool is necessary for both communicating with other Ethereum nodes as well as interacting with most any low-level data store used to store blockchain data. You’ll likely need this if you want to do things like manual signing of transactions.

PyDevP2P: http://pyrlp.readthedocs.io/en/latest/

This library implements the DevP2P wire protocol that is used by Ethereum nodes.

What’s Coming?

Now that you have a relatively full picture of the current state of the ecosystem lets talk about what’s in the works.

I’ll note that the following efforts are purely my own and are not being pushed forward by any other entity (including the Ethereum Foundation and the Enterprise Ethereum Alliance).

1) Extracting PyEthereum’s Goodies

As mentioned above, the PyEthereum codebase includes a myriad of useful tools. I’m currently in the process of extracting these into their own libraries with clearly defined public APIs and documentation. So far this has produced:

py-trie: Implementation of the tree structure used by Ethereum to store the EVM state. https://github.com/pipermerriam/py-trie
py-bloom: Implementation of the bloom filter used for log entries produced by the EVM. https://github.com/pipermerriam/ethereum-bloom

2) An alternate Python implementation of the EVM

The current Python implementation of the EVM has not proved to be a very stable foundation layer for Ethereum applications. As the EVM gains broader adoption we will need an implementation which can serve the ever growing diversity of needs. Things like pluggable consensus and the ability to easily configure the EVM with different opcodes and rules without requiring a change to the core library.

Introducing py-evm: https://github.com/pipermerriam/py-evm

This is a greenfield project to write a completely new EVM implementation in Python. It is currently focused heavily on providing the following things.

Composability: Rather than having the Homestead, DAO, and Anti-DOS fork rules hard coded, the EVM is instead a composition of the desired functionality. If you wanted to create your own EVM with only the Homestead and Anti-DOS changes but without the DAO rules you can do that without making any changes to the core library. The same goes for implementing new opcodes or alternate opcode logic. This can be done through the publicly exposed API rather than requiring changes to the core library.

Testing: Testing is going to be a 1st class feature in this EVM. This functionality will actually be a result of the composability since the test EVM will simply be a slighly modified composition of the primary EVM with a swapped out consensus mechanism that allows for commonly needed testing functionality like fast-forwarding time, mining ahead some number of blocks, or sending transactions from addresses for which you do not hold the private keys.

Stability: Once Py-EVM is released it will have a clearly documented public API. Any breaking API changes will undergo a deprecation cycle across multiple releases to ensure users have the appropriate warning to adjust their code.

The work on Py-EVM is ongoing. I expect to have early releases available within a small number of months but as with any estimates in the software world, you should take that with a grain of salt.