Why do my BIP32 wallets disagree?

You might face this question if using hierarchical deterministic crypto wallets (technically called BIP32 compatible wallets).

As you know these wallets generate your addresses from a common seed and each address corresponds to a given a path of calculation taken from that seed.

There are many implementations of the BIP32 specification across different languages and most wallets support importing and exporting the seed (in the form of a mnemonic). This should also ensure that the implementations remain compatible, because errors would be very apparent during transition between them.

This is not always the case though and bugs can remain hidden for a long while.

Recently some users of MetaMask and Ethereum TestRPC managed to find a case where two different BIP32 implementations disagree. MetaMask uses consensys/eth-lightwallet, while TestRPC uses ethereumjs/ethereumjs-wallet. These in turn depend on bitpay/bitcore-lib and cryptocoinjs/hdkey, respectively.

Don’t worry, there is no risk of losing keys or money as long as you use the same implementation.

Using a different implementation will not cause the loss of keys, only they will not be available on that wallet. Moving back to another one having the issue, will make those keys available again.

I would encourage wallet providers (using bitcore-lib or other affected implementations) to fix their implementation to match BIP32 and offer a way to their users to derive the non-compliant addresses as well.

Am I affected?

It is simple to test out. You’ll need to use the following mnemonic:

radar blur cabbage chef fix engine embark joy scheme fiction master release

And derive the following path:

m/44'/60'/0'/0/0

(This corresponds to the first account generated by most of the current Ethereum wallets.)

If you are affected, your Ethereum address will be:

0xe15d894becb0354c501ae69429b05143679f39e0

Otherwise:

0xac39b311dceb2a4b2f5d8461c1cdaf756f4f7ae9

How can this happen?

The problem lies in the hashing part of the derivation step. An SHA512-HMAC hash is created of:

[1 byte: zero][32 bytes: private key][4 bytes: derivation step]

The implementation in bitcore-lib has an issue with certain private keys. It drops the leading byte if it is a zero.

In the above case, the third step (m/44'/60'/0') takes in a private key with a leading zero:

0062b830267819c737684672a04f93cbad7774093d7c365d375d3a1dda12e182

The actual bug is in the serialisation method of private keys, which was reported earlier here.

For more information, check out the original MetaMask bug report and the one on bitcore-lib.

Note: bitcore-lib also doesn’t include the check for the very rare case, when a step results in a private key not valid on the curve. This is somewhat a grey are in the spec too.

How did we narrow it down?

We took a look at a third implementation in Javascript, which interestingly resulted in the same as bitcore-lib. However, the Trezor implementation matched that of hdkey and this was a trigger to dig deeper.

The next step was narrowing it down to the very first failing step of the derivation and finally comparing the inputs to that step.

Thanks to Dan Finlay for reporting this, to Christian Lundkvist narrowing down which path is breaking and to the Trezor team for their efforts on eliminating BIP32 edge cases.