Monitoring an Ethereum address with Web3.js
Retrieve transactions in real-time to an Ethereum address
I was recently working on a new project when I stumbled on a problem: I had to get all the transactions to a given account more or less in real-time. After looking through the Web3.js API docs as well as Stack Overflow, it was clear that there’s no well-defined way to do this, so I tried creating some programs on my own. This is what came out of it. Check out the full repo here.
After a lot of trial and error, I arrived at 2 scripts, both good for different things. The first one was pretty slow, but more extensible, while the other one was very lightweight, but not very customizable. Let’s explore them both.
Checking for transactions to a certain address might seem pretty straight forward, but it’s actually a lot harder than I thought at first. One would hope that we could somehow monitor an Ethereum address for incoming transactions by listening to some sort of network events, but that kind of functionality is not present yet.
Before we can start, there are a couple of things you’re going to need:
- A running Ethereum node like Geth or Infura (which I will be using)
- Node.js & npm
- A new directory initialized with npm init
Let’s first create the module that initializes our web3 client for us:
This module takes in the Web3 package and returns an initialized client. I used an Infura Ethereum node on the rinkeby testnet, and if you decide to do that as well (which I suggest), be sure to replace YOUR_INFURA_API_KEY with the correct key.
Next, we’ll create the actual transaction checker:
Our second module uses that web3 client to query the actual network. We have a private account variable, which you should replace with the address you’re interested in, and then we return the checkLastBlock function. First we retrieve the latest block, and log the number to the console. This is what such a block looks like (I’ve excluded some fields which are not useful to us):
You can see fields like number, nonce and hash, but what we’re really interested in right now is the transactions field. This is an array containing all the transaction hashes that have been included in that block.
On line 9 in transactionChecker.js, we check if the block and the block.transactions array are not null, and on line 10 we loop over said array. For every transaction hash in the array, we request the actual transaction. A transaction looks like this:
If we now find that the to field, which is the address of the receiving end of the transaction, equals our address (don’t forget the toLowerCase() function), we have found what we were looking for and can log some data to the console. (If a transaction does not contain a to field, it was a contract creation)
Finally, we need an index.js file to bring it all together:
The interval function at the bottom checks the current block every 7 seconds. I have chosen this number because the average Ethereum block time is 15 seconds, and we don’t want to miss any blocks. The problem with this program is that it doesn’t count on statistical outliers. For example, if a block gets mined in under 7 seconds, it’s possible that it will be missed completely. And if we try to mitigate this by decreasing the polling interval, we’ll find that we need a pretty fast internet connection to cope with all the asynchronous network I/O.
The plus side is that we can extend this script to, for example, check all transactions to said account between a range of blocks, like this:
Don’t forget to return this function as well.
The second program utilizes Ethereum pub/sub. Pub/sub is a system whereby a publisher constantly broadcasts events concerning certain topics to the network, to which the client (subscriber) can subscribe. This is way better and faster than having to constantly poll the network like we did in the first program. However, there are some aspects you have to take into consideration:
- Notifications are sent in real-time, for current events, and not for past events. The previous program can be tweaked to search for transactions between a range of blocks, but that won’t work with this program.
- Subscriptions require a full duplex connection. Luckily, both Infura and Geth provide such connections in the form of websockets.
Since we are monitoring an account in real-time, these points won’t bother us, so let’s move on. I’m working in a new npm directory right now, keep that in mind if you’re coding along.
First, we’ll have to create our client. For this program, we need both a normal http provider, as well as a websocket provider. In our code we’ll return both in an object — web3http will be the http client, web3 the websocket client:
And now for the second version of the transaction checker:
Let’s break this down. There are a couple topics one can subscribe to, like newBlockHeaders or logs. Logs would be ideal, but this subscription topic doesn’t work yet. So we’ll work with a pendingTransactions subscription. This happens on line 5. Our subscription is an event emitter, and the very moment someone sends a new transaction (thus, not yet being confirmed), it sends us the transaction hash of that transaction. This happens in the function watchTransactions, which we expose, on line 11.
Since these transactions haven’t been confirmed yet, we’ll use the setTimeout function to block further code execution for every event until a minute down the line, hoping that said transaction will have been mined by then. Because after that we actually just do the same as with the first program: retrieve the transaction, and check if our address was the address on the receiving end.
And then finally our index file to bring it all together:
This program is much more elegant, and not as resource intensive. Note, however, that both these programs have some reliability issues. The first one we discussed: a block could be missed if its block time was a lot lower than the average block time. With the second one, if someone includes a very small fee, it will definitely not have been mined after only a minute. In my project, I increased the setTimeout to 5 minutes, since that was still within bounds for my purposes. But be careful if you want to actually use this. Ideally, we would subscribe to logs, but that’s still really buggy and therefore a very bad idea.
That was it! If you have any questions, let me know in the comments. I’ll be happy to answer.