Jul 25, 2021
Cxxl Cxts & GHXSTS Airdrop — Dev blog
Harder than I expected.
Table of contents
Fair warning, this article gets code-heavy very fast.
Collaborating with GHXST is a big deal. He is such a massive name in the NFT sphere, we knew we had to do something special.
A week before the GHXST drop, we decided to do a fun, surprise summer-themed drop with a limited edition collection, totaling 300 tokens — it backfired, but why?
Why did our summertime drop backfire?
I think the simple answer is a mix of new NFTers being confused by airdrops and then there will always be people that are angry that they did not get 1 of the 300 items dropped.
From the perspective of the new NFTers, I fully understand the confusion. NFTs and crypto are littered with jargon. For this group of people, the confusion was our fault, as we failed to prepare them by explaining airdrops.
When you couple the new NFTers with people that understand airdrops but are essentially angry that they didn't get free money, it creates a self-perpetuating echo chamber.
We wanted a way to say sorry and give everyone a taste of the airdrop action.
GHXST airdrop plans
GHXST collectors will know that his collections are limited. He doesn't have a single collection that is anywhere the size of the 3,300 Cool Cats holders. For this reason, I was extremely shocked when GHXST suggested we drop a Cxxl Cxt -1 to every Cool Cat and GHXST holder.
Massive props to GHXST for this ❤
Just like that, we had a final plan — drop to everyone that holds a Cool Cat or an item from any GHXST collection.
Seems simple enough.
I was wrong.
Snapshotting Cool Cats
Taking a snapshot of the Cool Cats is easy enough. There are many well-documented ways to do this on eth. We opted for a very simple solution that takes a snapshot at the time of calling it:
Step 1 — Get the total population
We simply call our contract and ask it to return the totalSupply() which is the number of Cool Cats that have been minted.
const totalSupply = await this.contract.methods.totalSupply().call();Step 2 — Iterate over live tokens and log owners
We loop over the total population and for each token, we ask our contract to return the current owner.
The owner's address is then added to an object as a key and we keep track of how many cats a given owner has.
It's important to note that using the address as the key in the object is acting as the duplicate address filter. However, true as that is, the eagle-eyed among you have likely spotted a mistake I didn't spot for a long time (more on that later).
const holder = {};
for(let i = 0; i < totalSupply; i++){
const ownerAddress = await this.contract.methods.ownerOf(i).call();
if(!holder[ownerAddress]){
holder[ownerAddress] = 1;
} else {
holder[ownerAddress]++;
}
}Step 3 — Saving the snapshot
Not much to say here, standard fs usage to save JSON to file.
fs.writeFile('holders.json', JSON.stringify(holder),{ flag: 'wx' },function (err) {
if (err) return console.log(err);
});Snapshotting GHXST
Well, where to start?
Some aspects of Opensea are still alien to me and I naively assumed that their API would be all I required to collect a list of owners — turns out, not so much.
Gathering all item ids
The first hurdle was collecting a list of all the items in each GHXST collection:
Ghxsts: https://opensea.io/collection/ghxsts
PxinGxng: https://opensea.io/collection/pxin-gxng
Ghxst Culture: https://opensea.io/collection/ghxsts-cxlture
Cxllabs: https://opensea.io/collection/cxllabs
To achieve this we used part of the Opensea API
Step 1 — Pagination
Opensea only allows you to retrieve 50 results at a time, and some of the GHXST collections hold 250+ items, so we need some pagination to iterate over everything.
const totalTokens = 50;
const pages = Math.ceil(totalTokens/50);You can see that here we have dropped a magic number of 50. This number varies depending on the collection we are collecting data for. If we see one collection has 250+ pieces we change the value accordingly.
We then iterated over the pages and fired off calls to Opensea, requesting a list of all the items within the collection.
for(let i = 0; i < pages; i++) {
offSet = i * 50;
const response = await fetch(`https://api.opensea.io/api/v1/assets?order_direction=desc&offset=${offSet}&limit=50&collection=cxllabs`);
const listOfTokens = await response.json();Please note that in the above screenshot we are querying the Cxllabs collection. This value will need to change depending on the collection you wish to query.
Step 2 — Smashing my head on the desk
Now that we are able to grab all the items in a collection, you would think it would be easy to grab a list of holders for each item — nope, big fat nope.
Unless I was being totally stupid (very possible), I could not see anything in the Opensea API that allows us to query the owners for an item in a collection. If you know a way, please tell me. Sure I no longer need it, but I would love to know all the same.
To solve this issue we started sniffing around Etherscan, but collections are just part of the MASSIVE Openseas contract. We needed a better way.
Luckily, Lynqoid is great at finding useful APIs and he hunted down this little gem, https://moralis.io/. I will definitely be using this service for a lot of things ❤
const url = `https://deep-index.moralis.io/api/nft/contract/0x495f947276749ce646f68ac8c248420045cb7b5e/token/${asset.token_id}/owners?chain=eth&chain_name=mainnet&format=decimal&order=name.DESC`;
const header = { method: 'GET', headers: {
'X-API-Key': process.env.MORALIS_APIKEY
}}
const r2 = await fetch(url, header);
const l2 = await r2.json();I know, that code looks a little scary, but it's ok. We are simply asking the API service to return all owners of a given token id “${asset.token_id}” as found in the massive Opensea contract “0x495f947276749ce646f68ac8c248420045cb7b5e”.
To find the token of an item you simply click on the item and you can see it in the URL:
https://opensea.io/assets/0x495f947276749ce646f68ac8c248420045cb7b5e/80494307024529346018053650490912529916739680814770830097664395582100905918465
This breaks down into the following:
- https://openseas.io/assets = the site, we can ignore this
- 0x495f947276749ce646f68ac8c248420045cb7b5e = Opensea contract address
- 80494307024529346018053650490912529916739680814770830097664395582100905918465 = token id
We pass through our API key as a header and hey presto, we have a list of holders.
Step 3 — Filtering holders
GHXSTS has many items and we needed to snapshot all holders for all GHXST collections. As the plan is to drop on unique wallets and not per item we have to filter out duplicates.
// iterate through results
for(const result of l2.result){
if(!holderAddresses[result.owner_of]){
holderAddresses[result.owner_of] = 1;
}else{
holderAddresses[result.owner_of] ++;
}
}This code might look familiar. We used it when filtering Cool Cats too.
Step 4 — Saving and checking
Once all the owners have been filtered we need to save them.
console.log('total GY tokens:', totalGYTokens);
console.log('total owned:', totalOwned);
console.log('total holders', Object.keys(holderAddresses).length);
fs.writeFile('cxllabsHolders.json', JSON.stringify(holderAddresses),{ flag: 'wx' },function (err) {
if (err) return console.log(err);
});We also output a number of console checks. This allows us to compare our snapshot values with those shown on Opensea. Think of this as a safety check to ensure our count matches.
Step 5 — Repeat snapshots for all collections
We simply change some variables to target each collection in turn and fire off the snapshot.
The end result is a JSON file containing all the holders of each collection.
Distribution methodology
The original plan was to spend a whole week creating an automated airdrop system. Seven days would allow us enough time to build and test, test, test.
The timeline changed — for various reasons.
Now we have under two days to build it out — AHHHHHH!!!!
Sorry, we won't be sharing the airdropper code as it is still very WIP and currently getting a massive upgrade.
The original plan
In an ideal world, we would add the snapshot list, hit a big green “GO” button and all the tokens would be airdropped with no issues.
Harsh reality
Passing vast volumes of transactions through web3 or ether would obviously hit a few errors:
- timeouts
- 404s
- generic transaction failures
- who knows what else
I wanted to create a robust system that would track the success or failure of each transaction and store the data in a database of distinct addresses. Simple with very few moving parts, but it would need to be tested and error handling can sometimes take longer than you expect — queue plan B.
Plan B — Semi-auto drop system
Humans are pretty good at handling errors, sure we get tired and make mistakes but with such a short amount of development time, our inferior brains were the answer.
We created a simple React app that allows team members to connect to metamask and fill out a simple form where we drop in a block of 10 addresses, hit a big “GO” button and process those 10 addresses.
Yes, we really did process all 3500+ Cxxl Cxts in chunks of 10.
We did blocks of 10 addresses to make error handling easy. All the blocks were listed in a huge Google docs sheet and subdivided into tabs, one for each team member to work through. If an error occurred during a block we would have to manually check our personal activity on Opensea to validate our last successful transaction. This would allow us to trim that block, refresh the app and paste in the remaining addresses for said block.
Lowtech, I know, but it worked pretty well.
Why didn't we use existing services?
Most services we found, or that people shared with us, are orientated around ETH and we dropped on POLY (MATIC) to reduce fees.
A number of MATIC solutions do exist but even after talking to their owners, we were getting extremely inconsistent results. In fact, 1 in 3 test drops were failing, so no way we were going to use them.
A rather large company does run a dropping service but it is behind a “contact-us” wall and I believe we are still waiting to break through that wall.
Let the drop commence!
We were 5 minutes in, everything was going well, or so we thought. Then Adam dropped into my DMs and let me know that two people had just received a double drop.
We stopped the drop and returned to that code.
Dual coding an error fix
Tired and very aware that literally thousands of people were waiting for their -1 CXXL CXT to be dropped, the team slid off into the darkness.
We quickly realized the mistake (explained later) and began work on a solution.
Parallel coding in different languages
Knowing we were extremely tired and likely to make mistakes, we decided that Adam would code a solution in Python and Tom would code a solution in javascript. Independent of each other.
The plan being, each coder would produce a very different solution but the final results would need to be the same before we would feel confident in proceeding.
During the code fix, the rest of the team was verifying any errors and providing the coders with a list of all successful drops.
The list of successful drops was trimmed from the airdrop list and the resulting list was finally processed using the parallel coding method. After about an hour, Tom and Adam had created two separate methods that surprisingly resulted in two identical lists (even down to the address ordering) — a solution was found.
Drop continues…
Human error and mistakes
I would be doing you a disservice if I wasn’t completely honest and told you that mistakes were made — mostly by me.
Even after the code fix, I was extremely tired and accidentally sent cats to the same block of 10 addresses, twice.
Small mistakes were made by other people here and there, but all in all our error rate was extremely small.
Cleaning up duplicates — The hidden error
As you might expect there were duplicate addresses across all the collections, we needed to filter them out:
Step 1 — Gathering addresses
Grab all the singular snapshots and save them to a variable. All variables are then put into an array.
Step 2 — Combining and filtering
We iterate over the array and over each JSON within the array. Filtering all the addresses as we have done throughout this whole process
Step 3 — Save the final snapshot
Standard fs method to save a file.
The error
I'm sure some of you already know what happened, but for those that missed it, here is a short breakdown.
Using addresses as keys in an object is a very simple way to filter duplicates. That is true until different snapshotting methods return the same address in a slightly different format. We forget to account for the disparity.
Consider the following:
0xC5a8D47A2129d95f1dCF0f49a4B3dAEa50Fa78B4
0xc5a8d47a2129d95f1dcf0f49a4b3daea50fa78b4
Same address, both will work but one has capital letters and one has lowercase. Both will also create a unique key entry in the tracking object :(
The end result is multiple entries for the same address which means when the snapshot is handled for drops, the offending addresses will get a double drop.
The Solution
The solution is shockingly easy. In the above code, we simply change it to:
for(const obj of arr) {
const keys = Object.keys(obj);
for (let key of keys) {
key = key.toLowerCase();
if (!allHolders[key]) {
allHolders[key] = 1;
} else {
allHolders[key]++;
}
}
}Notice that we have simply added a method to normalize all keys to lowercase.
In an ideal situation, we would have done this at the point of snapshotting. But as each snapshot can take a while, especially the Cool Cats snapshot, we opted to normalize addresses during the combination phase.
Why did the fix take us an hour?
Once we identified the issue we needed to:
- filter out all duplicate addresses
- clean up blocks of addresses — once we noticed the error we stopped midway through. These needed to be cleaned up
- filter out the already successful airdropped addresses
- rebuild the combined snapshot list
- compare Tom’s result with Adam’s results
- check, double-check, triple-check EVERYTHING
We needed to be extremely thorough, we would hate for anyone to miss out on their -1 CXXL CXT.
References
Of course, we included some repos:
https://github.com/CoolCatsNFTPublic/CoolCatsSimpleSnapShot
https://github.com/CoolCatsNFTPublic/CoolCatsOpenseaCollectionSnapshotting
Where to find us:
Twitter: https://twitter.com/xtremetom
Cool Cats Team: https://twitter.com/coolcatsnft
Cool Cats Discord: discord.gg/coolcatsnft
Note: I feel silly saying this but I want to save people time. I am unable to help out on other projects. I am happy to talk to people, I just cannot devote any time to actually coding on your project — sorry.
