Walking Through the ERC721 Full Implementation
A Deep Dive into Managing ERC721 Assets
TL;DR
Basically, since ownership of ERC721’s is based on the ownership of a unique index or id, the fundamentals of token creation and transfer need to be extrapolated to accommodate that case. Also, the latest full implementation includes a
safeTransferFrom()function that checks for implementation of a standard interface prior to transferring tokens.
ERC721 Tokens
I have seen a lot of material around the interest in ERC721’s as non-fungible tokens that can hold metadata, but the depth of the material that I have found has left me looking for more details. My interest in ERC721’s started back at EthDenver back in February — you can read about about our project using ERC721’s here. I have since followed updates on the implementation of the ERC721 standard because I see a lot more value arising out of this design. As a full implementation of ERC721 rolled out from Open Zeppelin this week, I wanted to write up a resource for developers interested in creating their own ERC721 tokens. It took me a while to wrap my head around it, so hopefully, this can help get you there faster if you haven’t look at the ERC721 EIP yet. I tried to balance elements of Solidity and ERC20’s while introducing ERC721’s here. Side note, there is an awesome new site that consolidates all merged EIPs that I highly recommend checking out.
To me, token standards can be summarized and compared in the following ways:
- Ownership — How is token ownership handled?
- Creation — How are tokens created?
- Transfer & Allowance — How are tokens transferred, and how do we allow other addresses (contracts or externally owned accounts) transfer capability?
- Burn — How do we burn or destroy a token?
Understanding how these operations work helps to put a complete picture of how a token standard works. The below follows the OpenZeppelin ERC721Token.sol full implementation and amalgamates some additional knowledge of Solidity and other EIPs. As a first stab at documentation, I’m sure this will continue to evolve, and I will try to keep it updated. Any recommendations appreciated.
Token Ownership
As the the most popular token standard to date, ERC20’s have become a standard of comparison for new token proposals. They are fairly easy to understand, at least now that I look back anyways. In terms of ownership, what ERC20’s come down to is a mapping the balances of tokens to their respective owners’ addresses:
mapping(address => uint256) balancesIf you have purchased an ERC20 token, your ultimate ownership of that token is verified through the contract in which you bought it as the contract maintains a record of how many tokens (uint256) each address (address) holds. If we want to transfer our ERC20 tokens, then our balance is verified through the balances mapping so that we do not try to send more than we own. A question that may come to mind is, if I have never interacted with a particular token contract, how does it know my balance is zero? The above balances mapping initially defaults to zero, so even if you have never touched a particular token contract before, if you want to check your balance of that token, your balance will be appropriately verified as zero.
We have heard over and over again how ERC721’s are non-fungible, which again, for the millionth time means that tokens of the same class or contract can hold a different value. A value of one ERC721 Cryptokittie does not equal the value of another ERC721 Cryptokittie because they are each unique. In order for this to be true, we can no longer simply map an address to a balance. We have to know each unique token that we own.
For this reason, in the ERC721 standard, ownership is determined by an array of token indexes or ids that is mapped to your address. Since each token value is unique, we can no longer simply look at a balance of tokens — we have to look at each individual token created by the contract. The main contract keeps a running list of all the ERC721 tokens created in that contract in an array, so each token has its’ respective index within the context of all ERC721 tokens available from that particularity contract via the allTokens array.
uint256[] internal allTokensHowever, we also need to know which tokens we own, not just what the contract holds. So, in addition to the array of token indexes in the entire contract, each individual address has an array of token indexes or ids that is mapped to their address as ownership. We do not simply map an address to a token index because what if an individual owns more than one token? If we were only mapping individual indexes, say we owned token number 5 and that was mapped to our address. However, tomorrow, we buy token 6, then if we had only mapped individual values number 5 would be overwritten by number 6 in our mapping, and we would no longer have a recorded that we owned token 5 as well — hence the need for an array.
mapping (address => uint256[]) internal ownedTokensThis simple difference spurs many of the additional requirements of an ERC721 token. With an ERC20 token, we were checking against a balance, but now, we need to check ownership against a specific index of a token. To rearrange this array when we transfer tokens necessitates further requirements.
So do we iterate through our array of tokens each time we want to verify ownership of a certain token index? No, there is a much simpler and safer way. Rather, in addition to our array of token indexes that we own, we map each token index or id to an owner. In this way, every time we would like to know who owns a certain token index, we need only provide the token index to check the address to which it is mapped. (This variable is included in the ERC721BasicToken.sol, which is inherited into ERC721Token.sol.)
mapping (uint256 => address) internal tokenOwnerWhy do we do this in addition to the array? Can’t we just iterate through our array of tokens to ensure we own a specific token? Let’s first ask this question: If we transfer tokens, can’t we just add or delete tokens indexes to our array? Unfortunately, no. Recall that in Solidity, should we decide to delete an element in an array, the element is not actually, fully deleted but replaced with a zero. For example, let’s say that we have an array myarray = [2 5 47], which is of length 3. However, then we call a function that says to delete myarray[myarray.length.sub(1)]. Although we may expect that myarray = [2 5], we actually have the following array myarray = [2 5 0], and it would still be of length 3. We do not magically own the token of id 0, so this presents a problem. Recall that delete does not actually “delete” values in Ethereum but rather resets them to zero. Certainly, there are cases in which we would like to delete or remove a token from the ownership of an address. Rather than simply delete tokens from our array, we rather rearrange our array. We will see later on when we look at transferring (removing ownership) and burning tokens how this information comes into play. For this reason, we also keep track of the below. The ownedTokensIndex maps each token id to its respective index in its owner’s array. As stated below, we also map the token id to its index in the allTokens array as well.
// Mapping from token ID to index of the owner tokens list mapping(uint256 => uint256) internal ownedTokensIndex; //Mapping from token id to position in the allTokens array mapping(uint256 => uint256) internal allTokensIndex;
Another issue we may come across is if we want to check how many ERC721 tokens that we actually own. At this point, we introduce one more variable to keep track of ownership. (Again, this variable is in ERC721BasicToken.sol and inherited into ERC721Token.sol.)
mapping (address => uint256) internal ownedTokensCountNow, we map a number to keep track of how many tokens that we do own to our address. This ownedTokensCount is updated as we purchase, transfer or potentially burn tokens accordingly. Why do we need to keep track of how many ERC721 tokens we own? Verification. Let’s say we want to transfer all of our ERC721 tokens to a new address? Or just check that we own a certain amount?
At this point, we can see how introducing the ownership of a unique token adds new complexity to the ownership of a token. But what about the creation of these ERC721 tokens?
Token Creation
Recall that in the case of ERC20 tokens, we are mapping against a balance of tokens. Hence in order to create ERC20 tokens, we need only to set or increase the total tokens available. In the ERC20 design, we have a value that maintains our total available token supply, totalSupply_ below. In some cases, you may have seen a ERC20 token contract set the total supply through a value initialized in the constructor. Recall that a constructor function is run once to initialize a contract (but it is not required). The constructor must carry the exact same name as the contract — if it does not carry the same name as the contract, the EVM will register your intended constructor as a normal function and that means any one can call it after contract creation, which can present numerous security vulnerabilities depending on what you are doing. The constructor code is part of the transaction that creates the contract, but it is not part of the contract at the deployed location. The constructor may be used to set initial values, ownership, etc. In the below, MyToken is used to set the value of the totalSupply_ of tokens. With increased demand to allow for variability of the amount of ERC20 tokens within a contract, the ERC20 standard was expanded to also include a mint function in which a desired amount of tokens is added to the totalSupply_ and balances are mapped accordingly. Note that in the below Transfer is an event, not a function — am I the only one who has spent time trying to look for a function that turn out to be an event while reading Solidity? Anyways, again you can see from the mint function where our balance is updated.
uint256 totalSupply_- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - //Example of setting token supply via constructor
contract MyToken {
function MyToken(uint _setSupply)
{ totalSupply_ = _setSupply_ }
.....//Example of maintaining a variable token supply via minting
function mint(address _to, uint256 _amount)
onlyOwner
canMint
public
returns (bool)
{ totalSupply_ = totalSupply_.add(_amount);
balances[_to] = balances[_to].add(_amount);
Mint(_to, _amount);
Transfer(address(0), _to, _amount); return true; }
As to ERC721’s, we have learned that since each individual token is unique, we must create each individual token. With ERC20, we can easy create a batch of 100 more by adding to the totalSupply_. However, since we maintain an array of tokens in an ERC721 standard, we need to add each token to that array separately.
Here we see two functions that look to a ERC721 contract’s total supply addTokenTo() and _mint(). Let’s go over addTokenTo() first.
Here we call addTokenTo() from our full implementation contract, and then, the super.addTokenTo() allows us to first call the addTokenTo() function in our basic ERC721 contract. Essentially, over the the course of these two functions, we update all global ownership variables. The functions take in two parameters _to or the address to which the token will be owned and _tokenId or the unique id of the token — chosen by whomever you allow to call this function, you’d likely limit this call to the owner of the contract. In this case, the user can choose any unique number id. First, in the ERC721BasicToken contract, we check that the token id is not already owned. Then we set the token owner of the requested token id, and add one to the number of owned tokens of that individual account. Going back to the full implementation contract, we also update the array of the new owner’s (_to)tokens by adding this new token to the end of their ownedTokens array and saving the index of that new token.
From the above, we can see that addTokenTo() updates an address to an individual. However, what about the allTokens array? This is where _mint fills in the gap. Here we see when we call _mint() from our full implementation ERC721 contract that again we jump up to our basic implementation, which ensures that we are not minting to an address of zero and calls addTokenTo(), which as confusing as it is, will actually call back to our full implementation contract to kick off the addTokenTo() calls. (Again, Transfer() is an event, not a function.) After the _mint() function in the basic contract is completed, back in our full implementation, we add the _tokenId to our allTokensIndex’s mapping as well as our allTokens array.
From the above, you can see that although you can call addTokenTo() by itself, what you need to do to maintain all information in a full implementation ERC721 contract is use _mint() to create your new tokens.
But what about the metadata that ERC721’s can supposedly hold? We have created tokens and token ids, but they are not holding any data yet. Open Zeppelin gives us an example of how this could look with a mapping a token id to a string to of URI data.
// Optional mapping for token URIs
mapping(uint256 => string) internal tokenURIs;In order to set a token’s URI data, the following _setTokenURI() function is also included. Here using the token Id that you created via _mint() and your desired URI information, you can set the data that is mapped to a token ID in tokenURIs. Note the requirement in this function that we determine that a token Id exists (meaning someone owns it) prior to assigning data.
Although more complex and gas intensive, I find the ability to use a struct to store data instead of a mapping to an index far more interesting — at the least, creating a non-fungible token with a handful of variables is still far less expensive than creating a smart contract per each “asset” instead. In any case, if you are wondering how to include different data, these elements are what you would want to look at changing.
Transfer & Allowance
As before, let’s first review how transfers and allowance happens in the ERC20 standard. We can transfer an ERC20 token directly using the transfer() function in which we specify an address that we want to send to and how much, which is checked against our balances and then updated in the main ERC20 contract.
function transfer(address _to, uint256 _value) public returns (bool) { require(_to != address(0));
require(_value <= balances[msg.sender]);
balances[msg.sender] = balances[msg.sender].sub(_value); balances[_to] = balances[_to].add(_value);
Transfer(msg.sender, _to, _value);
return true; }But, what do we mean by allowance? When we want another contract or address to be able to transfer our ERC20 tokens, we need to allow the use the ERC20 contract address to do that for us — this need arises in many instances in distributed applications — escrows, games, auctions, etc. Hence, we need a way to approve other address to spend our tokens. Then, another transfer function requires the contract to check the allowance of who is allowed to spend them. I’ll start with how allowance is set up, and then show how that plays into transfers.
In the ERC20 standard, we have a global variable allowed in which an owners address is mapped to an approved spender’s address and then mapped an amount of tokens. In order to set this variable, there is an approve() function in which an individual is able to map an approval to their desired _spender and _value. Notice that here, we are not checking that actual amount of tokens owned by the sender — that comes later at transfer. Once more, Approval is an event not a function.
//Global variable
mapping (address => mapping (address => uint256)) internal allowed//Allowance of another address to spend your tokens
function approve(address _spender, uint256 _value)
public
returns (bool)
{ allowed[msg.sender][_spender] = _value;
Approval(msg.sender, _spender, _value);
return true; }
Now, once we have approved another address to transfer our tokens, how are our tokens actually transferred? Our approved spender would use the transferFrom() function below in which they would specify the _from or original owner’s address, the receiver’s address _to and the amount _value. Here we check that the original owner actually possesses the amount desired to be transferred with require(_value ≤ balances[_from]) , then we check that the msg.sender is allowed to transfer the balance through the allowed variable and ultimately we update all of our mapped balances as well as our allowed amounts. Again, Tranfer is an event. Note, there are two additional functions included to allow for increasing (increaseApproval()) and decreasing (decreaseApproval()) an approved spender’s allowance as well.
So, once more, we need to think that instead of approval and transfer of balances, in the case of ERC721’s, we need to approve and transfer token ids. The ERC721 standard offers the chance approve an address for token transfer by id or we can approve an address to transfer all owned tokens. To approve transfer by id, we use the approve() function as below. Here a global variable, tokenApprovals, maps a token index or id to an address that is approved to transfer it. In the approve() function, we first check for ownership or if the msg.sender isApprovedForAll() . Further below, you can see that you can use the setApprovalForAll() function to approve one address to transfer and handle all of the tokens that are owned by a particular address as we have a global variable operatorApprovals in which the owner’s address is mapped to a approved spender’s address and then mapped to a bool. This is set to 0 or false by default, but by using setApprovalForAll() we can set this mapping to true and allow an address to handle all of the ERC721’s are owned. Note, if a spender is approved for all tokens, then they can also assign additional address spend capabilities. Next, we use getApproved() to check that we are not setting approval for address(0) . At last, our tokenApprovals mapping is completed to the desired address. And like ERC20, Approval is event.
Now, we arrive to how we actually transfer ERC721’s. The full implementation actually offers a two ways in which to transfer. The first method is discouraged, but let’s go over it to understand. In transferFrom(), the sender and receiver addresses are specified along with the _tokenId to transfer, and we use a modifier, canTransfer() to ensure that the msg.sender is approved to transfer the token or owns it. After checking the sender and receiver addresses are valid, the clearApproval() function is used to remove the approval of the transfer from the original owner of the token, so that a previously approved spender may not continue to transfer the token. Next, the removeTokenFrom() is called in the ERC721 full implementation contract, and similar to the addTokenTo() function using super to call the removeTokenFrom() function in the ERC721 basic implementation. You can see that the token is removed from the ownedTokensCount mapping, the tokenOwner mapping, and one more twist is that we move the last token in the owner’s ownedTokens array to the index of the token that is being transferred and shorten the array by one (see lines 22–30). Lastly, we use the addTokenTo() function to add this token index to its new owner. And Transfer is an event.
Now, a question to ask, how do we ensure that we are sending our ERC721 to a contract that can handle additional transfers? We know an externally owned account (EOA) can use our ERC721 full implementation contract to trade tokens if desired; however, if we send our token to a contract that is does not have the appropriate functions to trade and transfer the token via our original ERC721 contract, then the token will be effectively lost as there is no way to get it out. This sentiment reflects much of the concern that was brought to light through the ERC223 proposal, which is a proposed modification to ERC20 to prevent these erroneous transfers.
In order to avoid issues and standardize, the ERC721 full implementation standard introduces the safeTransferFrom() function. Prior to diving into how that works, let’s look at some additional requirements in which we have an ERC721Holder.sol contract that implements the ERC721Receiver.sol interface. The ERC721Holder.sol contract is to be part of the wallet, auction or broker contract in which you would want to hold an ERC721 token. The reason this has been standardized goes back to EIP165 in which the goal is to create “a standard method to publish and detect what interfaces a smart contract implements.” How do we detect an interface? Below we will see a “magic value,” ERC721_RECEIVED, which is the function signature of the onERC721Received() function . A function signature is the first four bytes of the hash of the canonical signature string. In this case it is computed by bytes4(keccak256(“onERC721Received(address, uint256, bytes)”)) as described below. What is the function signature used for? To find the place in the bytecode that contains the code for the called function called. Each function in your contract will have it’s own signature, and when you make a call to your contract the EVM uses a series of switch cases to find the function signature that matches your call and executes your code accordingly. Consequently, in our ERCHolder contract we see that the onERCReceived() function signature will match the ERC721_RECEIVED variable in our ERC721Receiver interface.
Your ERC721Holder contract is not a complete contract for handling ERC721 tokens. This template is meant to provide you with a standardized interface to verify that the ERC721Receiver standard interface is used. You will need to extend or inherit the ERC721Holder contract to include functions in your wallet or auction contract to handle ERC721’s. Even to hold tokens in escrow you would need to add functionality so that this holder contract could make a call to transfer tokens out of the contract as needed.
Now, back to our original ERC721 contract, the safeTransferFrom() works in the following way — you can either transfer using Option 1 in which there is no additional data included with the safeTransferFrom() function or you can use Option 2 to include data in the form of bytes _data. As before the transferFrom() function is used to remove token ownership from the _from address and add token ownership to the _to address. However, we have an additional requirement that we run the checkAndCallSafeTransfer() function. First, we check that the_to address is an actual contract through the use of the AddressUtils.sol library — I included the function isContract() in the below so that you can quickly see what it is doing. As noted, there is currently research and development into allowing external owned accounts (EOAs) on Ethereum maintain their own code as well, so whenever that point comes, it would need to be noted for a check like this. After verifying that _to is a contract address, we check that calling the onERC721Received() function returns the same function signature that we are expecting from our standard interface. If the correct value is not returned, then the transferFrom() function is rolled back as we have determined that the _to does not implement the expected interface.
Whew, there we have it. Transferring ERC721 tokens. Now, burning tokens should look easy.
Burning
As to ERC20, since we are only manipulating a single mapped balance, we only need to burn or destroy tokens against a specific address, which can be a user or a contract. In the below burn() , we specify the number of tokens that we would like to burn via the _value variable. The address against which to burn is the msg.sender, so we update their respective balances, and then we reduce the totalSupply_ of tokens as well. Here Burn and Transfer are events.
For ERC721 tokens, we need to ensure that the specific token id or index is eliminated. Much like the addTokenTo() and _mint() function, our _burn() function uses super to call a function in our basic ERC721 implementation. First, we clearApproval(), then remove the token from ownership via removeTokenFrom() and use the Transfer event to alert this change on the front end. Next, we eliminate the metadata associated with that token by deleting what is mapped to that particular token index. Lastly, much like removing a token from ownership, we rearrange our allTokens array so that we replace the _tokenId index with the last token in the array.
If you made it to the end, thanks for reading! I imagine the biggest challenges will be adapting these standards around how to mint ERC721’s with the desired metadata and how to ensure transfer based on unique exchanges of values. At present, there are already a lot of examples — of course, the famous Cryptokitties, Cryptogs (from my team at EthDenver), Cryptocelebrities, Decentraland and if you visit OpenSea you can find a whole lot of digital assets and collectibles. I can imagine significantly more use cases for this standard — hope this write up helps to get you there…!
