Not your private keys, not your backup
This article is a wrap up of my learnings after a recent loss of a mining related wallet. In the spirit of TraceMayer’s [Jan/3➞₿ 🔑∎] ProofOfKeys https://www.proofofkeys.com/, i want to highlight the importance of proper backups at any point in time to avoid unnecessary losses.
People are aware of my work know that i do care a lot about security. All aspects of it. I am consulting customers, friends and sometimes even strangers that need help on twitter when they don’t know who to ask and time is key.
In my opinion, #ProofOfKeys is about more than just withdrawing your holdings from exchanges. Hodling any crypto means proper backup AND restore practices should be in place.
If you don’t care about the details of my specific incident, please follow best practices and export the private keys to all wallets now, test the restore/import actually works and store everything in a secure place.
My work environment
In the crazy world of crypto, there is always a new project or wallet to look into. Many of these projects must be compiled from source to try out new features or to get an edge via mining early on. Others are binary-only and should be evaluated carefully.
To make keep this mess of dozens of projects manageable, i have a Vagrant + Docker based setup to test/compile/evaluate new coins in an automated and secure but still convenient way.
My polished VBK test environment will also be open sourced, as most of the stuff i do and see useful enough for the crypto crowd.
I do virtual machine snapshots on a regular basis and before anything important like updates. Vagrant makes this really easy and so far this approach didn’t fail a single time — until 2 days ago.
I lost access to one of my VeriBlock PoP mining keys.
A VeriBlock primer
As some people have noticed, Veriblock is one of the implementations that is responsible for the steep rise of OP_RETURN outputs recently. VeriBlock utilises the BTC blockchain to store proofs.
VBK is comparable to dPoW by KMD, but more permission-less since there is no node election to participate. Anyone can PoP mine VBK by running the open source pop miner and spending mainnet BTC. KMD and VBK depend on bitcoin allowing OP_RETURN transactions. OP_RETURN outputs can’t be spent, but add to the size of the BTC blockchain.
You can find all details about the PoP transaction lifecycle details in the wiki.
VBK currently accounts for roughly 19% of all Bitcoin mainnet transactions. VBL can be mined via PoW, but PoP mining is what attracted me to the VeriBlock project back then. If you are curious about the details of the inner workings, i do highly recommend the VeriBlock wiki.
My backup incident
I PoP’ed VBK during the last 6 months. *rofl*
Yesterday night, i did my usual preparations to upgrade nodecore. I started by doing a VM snapshot of the running instance. With (with vagrant that’s one command “vagrant snapshot”) and downloaded the new release from github.
After installing / upgrading the new files and copying the old wallet files to the new location, i started syncing the chain.
WHILE syncing the chain, i created a new address to send some tVBK (Veriblock uses BTC mainnet, but is still in testnode mode itself) to.
Unfortunately, my VM hdd ran out of space while syncing about 2 hours later. Additionally, this lead to an out of memory error. Nothing serious, since i did a VM snapshot and also know my way around Linux. I re-sized the harddisk and started the new nodecore release instance but the existing “wallet.dat” file seemed to be corrupted and nodecore displayed a couple of sql related errors because the local chain db seemed to be in a corrupt state, too.
I overwrote the new (semi-corrupt?) wallet.dat and chain files with my older but working files to catch up syncing again. This lead to a donation of my funds to the VeriBlock network.
Where i fucked up
I knew VBK was different but still assumed backing up the wallet.dat file would be enough since this worked in the past. I failed to read about the actual inner workings of the wallet.dat format and assumed a similar behaviour as with btc/eth and it’s clones. Therefore, i assumed that my working nodecore backup would be sufficient for a restore of all wallet keys.
Obviously something went wrong, because my newly generated address didn’t show up after starting nodecore with my snapshot files even with a fully synced chain. This is because the VeriBlock wallet stores a list of public-private keypairs and generates a new keypair when generating a new address. This keypair is appended to the wallet.dat file.
I overwrote that file with my backup.
Learnings
Now i know that “dumpprivkey <address>” is the only way to backup a wallet (not only) for VBK and should be done right after generating a new address.
This is true for all cryptocurrencies, including Bitcoin. I also talked to members of the VBK team to discuss some ideas to improve their implementation to avoid such losses in the future. If it doesn’t happen to anyone else, i am already happy.
Create regular BACKUPS and test the RESTORE on a regular basis.
Thanks for reading that far ! I hope you found it inspiring that errors can happen to anyone, including me. Have a great one!
BTC donations welcome and never forget:
Have fun, this is crypto after all!
BTC 33ENWZ9RCYBG7nv6ac8KxBUSuQX64Hx3x3
