Why package management is good, why Ethereum presents a new challenge, and how EthPM hopes to solve this challenge.
Also included: devcon iv. workshop preview
The world’s computer affords us a tightly interconnected platform for trustless, verifiable software. With their unique addresses and ability to delegate to one another, smart contracts (just running blocks of code) should have the means for vast reuse. Concepts representing reusable components are even built right into smart contract languages (e.g., Solidity’s library). Despite all this, there is a strong need for better systems and processes to facilitate sharing code.
It’s been my and all of Truffle’s honor to participate in a community effort to build an Ethereum package management standard, an impressive work laid out originally by Piper Merriam, Tim Coulter, and others¹, and now funded thanks to a generous bounty through ETHPrize².
In this post, I offer some rationale on the importance of software package management (for any computing platform), outline a bit about what’s special about Ethereum as a development platform, and describe the ongoing work to effect EthPM.
Why package management is good
A great fundamental feature of software is that it can be copied essentially for free — when some software is created to solve a problem, that same software can be reused to solve that problem again and again, for many different people. That same software can often be extended to solve all problems of a similar nature. And even further, lots of software can be collected and bundled repeatedly, embedded inside other software, until that software and that solution are so ubiquitous that we can take them for granted and proceed to address more interesting problems.
The process of this reuse itself is a software problem: how do developers share code? How do developers borrow others’ code?
For instance, if one developer implements a simple authorization mechanism for their smart contract (perhaps where the “contract owner” has privileged access), then how can that same solution be widely available for other developers to use, without having to write it again or copy and paste the code?
To show some of the complexity of the software reuse problem, consider what happens next, after a piece of software becomes widely shared and reused. Suppose the “contract ownership” solution becomes ubiquitous and used by lots of developers. What if a flaw is found in that solution? Or if that solution can simply be improved in some way? Shouldn’t it be easy to share the outcome of this knowledge, to redistribute these updates with minimal effort, to everyone who is using the solution already?
This is a familiar idea in contemporary software development. Most programming languages and most computing environments have mature tools that make it very easy to do both packaging (the “sharing and reuse” problem) and versioning (the “software tends to change” problem). Most programmers are even used to solving problems by looking for existing solutions first.
Unfortunately, Ethereum and smart contract development (more generally) are new in computing, and so the development ecosystem has suffered from a lack of a well-tailored solution to this problem. Code is often copied and pasted (for example, I have personally seen more copies than I can count of the same Owned.sol file, in various projects, with minor modifications), or other languages’ package management systems are used (NPM, for instance³), and they do not completely handle Ethereum’s unique requirements for code sharing.
Why Ethereum presents a new challenge
Smart contracts are unlike other kinds of shareable code because they allow us to share them in a new way…
In contemporary software development process, the notion of a “package manager” is well-understood to represent a tool or service that lets you package up some code: give it a name, a description, a license, and importantly, an interface by which others can use your code, without their needing to understand your code’s specifics.
Code of this form is a static, lifeless blob of computer-encoded text, a written set of instructions to tell the computer to do a certain thing at a certain time. It might be a stand-alone program, like a web-browser or a tool for running automated tests, or it could be a software library, a building block for larger programs, something that provides the interface for a human to express an abstract idea in a way the computer understands.
This kind of code, the “set of instructions” metaphor, is still extremely useful in Ethereum. Owned.sol is a software library that provides the building block abstraction of “contract ownership.” If you are a smart contract developer who wants to leverage the idea of contract ownership, you need look no further than this.
But code on Ethereum, a smart contract, is more than that…
A smart contract⁴ persists as a unique and active component of the blockchain.
It persists as a continuous entity with its own history and memory — an internal state.
It gets its own wallet and can be programmed to spend its own money.
And anyone, human or machine, can interact with it, through its very own programmed external interface.
A smart contract provides all of the features of shared static code, plus this new feature: this persistent state. Using smart contracts, we can build, for example, a scarce digital asset, a fungible token⁵. This is not just some code for executing transfers and approvals, but is also a historically consistent record of everyone’s balance.
On Ethereum, to package code for reuse, we must be able to package it in its static form, and we also must be able to describe these running smart contracts. We might want to share the abstract notion of a token contract, or we might want to share a specific token contract, with its own address, maintaining a fixed supply of tokens and reliably tracking balance.
Aside: There is a strong analog to smart contracts in traditional computing. Modern computers represent the idea of a process, a running program that is currently active, possibly waiting for user input or crunching numbers. Smart contracts are like running processes, except that the process is designed to be public, interacted with by the world, with the sense that the same process will run forever in continuity. Hence, “the world’s computer.”
EthPM
The Ethereum Package Management System is an attempt to solve this problem. EthPM seeks to provide the groundwork for a system of describing and distributing reusable smart contracts and smart contract code. It is designed to meet the specific requirements listed above, as well as to remain general-purpose: to support multiple blockchain networks, to support and encourage a loose federation of package registries, and to be compatible with multiple storage networks, including IPFS⁶ and Swarm⁷.
At an architectural level, this comprises:
- A schema for describing a package
- A standard interface for a package registry
- A collection of core libraries in multiple languages
EthPM Package Manifests
Packages in EthPM are described via a manifest, a representation of the bundled collection of code and smart contracts. It represents the package’s name, the version, a description, a license, etc., as well as any code inside the package, and/or any smart contract deployments to blockchain networks.
Packages may depend on other packages, on specific building block libraries, or on various deployed smart contracts. The package manifest format also provides a means of representing these relationships between packages in a verifiable way.
This format is described in the draft EIP-1123 Revised Ethereum Smart Contract Packaging Standard, authored by Nick Gheorghita, myself, and others. This description is also maintained and available on the Manifest Specification website and in a formal, machine readable version.⁸
Registry Interface Standard
Package registries contain packages listed by name and version. By describing a standard interface for a package registry, EthPM seeks to encourage developers, auditors, and even entire organizations to build and maintain a decentralized collection of package registries. The goal in providing a standard interface is to be as general-purpose as possible, in hopes that the development community may reuse and build atop this interface.
The registry interface standard is described in the draft EIP-1319: Smart Contract Package Registry Interface, authored by Piper Merriam, Christopher Gewecke, and myself.
Core Language Libraries
In order to enable ease-of-use, EthPM hopes to foster implementation across a wide variety of programming languages and computing environments. In addition to describing package formats and interface standards, EthPM must essentially also provide software libraries of its own, to make reading and writing packages straightforward, as well as publishing packages to registries and installing packages from a registry.
The goal for these libraries is to enable the ecosystem of blockchain development tools to adopt these libraries to enable all developers to package and reuse their smart contracts.
Work in progress implementations are being actively developed for Python, Go, and Javascript.
Background and Current Status
The rationale for EthPM’s design is maintained via case study analysis of a number of core use cases. Through these examples, EthPM aims to understand its own requirements and to convey these requirements to others. Please see these Use Cases for more information.
The current work on EthPM represents the second iteration of this manifest format and of earlier implementations for package registries and tooling integrations. The project seeks to apply the lessons learned in its first interaction, to provide a more intuitive and more available system for software reuse.
Current work proceeds thanks to its community of developers, including the outstanding efforts of Nick Gheorghita, Mitch Kosowski, Christopher Brown, Christopher Gewecke, Piper Merriam, and many others. For more information, to participate in the project, or even just to say hello, please join the project’s Gitter channel.
Workshop Preview — devcon iv.
Nick Gheorghita, Christopher Brown, and I are pleased to present EthPM at our upcoming workshop at the Ethereum Developers’ Conference.
Our workshop will provide developers with a hands-on look at EthPM, including:
- An examination of the package manifest format
- A walk-through to deploy your own package registry
- Building your own EthPM package
- Interacting with your package via Web3
We hope to see you there!
References
^ ¹ EIP 190: Ethereum Smart Contract Packaging Standard https://eips.ethereum.org/EIPS/eip-190
^ ² ETHPrize http://ethprize.io/
^ ³ NPM https://www.npmjs.com
^ ⁴ More precisely, EthPM refers to this concept as a smart contract instance, to distinguish it from a smart contract type. This article uses the term smart contract to mean instance, implicitly.
^ ⁵ ERC-20 Token Standard https://eips.ethereum.org/EIPS/eip-20
^ ⁶ Inter-planetary File System https://ipfs.io/
^ ⁷ Swarm https://swarm-guide.readthedocs.io/en/latest/introduction.html
^ ⁸ Formal manifest specification is maintained via the JSON-Schema standard format for formal specifications http://json-schema.org/
