EvaluateNFT: helping digital artists and NFT investors to make pricing decisions
Digital artworks traded as Non-Fungible Tokens are notoriously difficult to price. EvaluateNFT aims at solving this problem with the help of Machine Learning
Please note that the EvaluateNFT project was discontinued in October 2023.
Art NFTs are a low-velocity asset class that is very difficult to price
The global market for Non-Fungible Tokens (NFTs) has exploded lately. The total NFT trading volume in 2021 reached $22B, up from $100M in 2020. This volume has been largely dominated by NFTs categorised as art and collectibles (Hays et al. 2021; Nadini et al. 2021).
However, a quick look at the Art section on OpenSea will reveal a significant oversupply of art NFTs, most of which never get sold. The reasons for that are numerous and include inadequate marketing, lack of the artist’s social media presence, lack of fandom, wrong pricing, etc. (CryptoMoogle 2021). And even if an artwork does get sold, it will rather unlikely change hands again any time soon (Franceschet & Read 2020; Nadini et al. 2021; Parker 2021). In this sense, art NFTs are similar to traditional artworks (DrFazal 2019).
Combined with their inherent uniqueness, this high illiquidity of art NFTs makes it very difficult to price them efficiently. In turn, this hampers the overall adoption of NFTs and impedes the development of new financial products built on top of this asset class (e.g., NFT indices, underwriting debt against NFTs, etc. — see Emmons 2020, 2021; Hays et al. 2021).
This article introduces EvaluateNFT, a Machine Learning-powered valuation platform for art NFTs. Our team at Next Game Solutions has developed EvaluateNFT to help digital artists and NFT investors make informed pricing decisions. We focus on a particularly challenging problem — predicting prices for artworks that have never been traded before.
Valuation of art NFTs can be done in different ways, each with its benefits and drawbacks
Online auctions
Online auctions on platforms such as OpenSea, Rarible, Foundation, LooksRare, etc. are the most common way to sell art NFTs. While using an online auction is a natural way to discover the price that the market participants are willing to pay for a piece of digital art, many artworks still do not get sold for the reasons outlined above.
One of these reasons is the wrong asking price, which can deter potential buyers. On the one hand, an overly high price may deter many retail investors with shallow pockets. On the other hand, an unnecessarily low price may be perceived by investors as a lack of interest in the lot (also implying a lower chance of reselling this lot in the future for a profit).
In addition, online auctions often suffer from price manipulations, such as wash trading (creation of inflated trade volumes) and shill bidding (artificial inflation of prices by colluding bidders). As a result of price manipulations, investors risk overpaying for the respective NFTs and having difficulties with making a secondary sale later on (Hays et al. 2021).
Appraisals-based valuation
Expert appraisals are a common valuation mechanism when it comes to traditional (physical) art, antiques, real estate, collectible wines, and other similar low-velocity assets.
Professional appraisers working for auction houses would typically provide a low and a high estimate for an artwork based on a multitude of factors, including the identity of the artist, age of the artwork, signature, materials, dimensions, rarity, subject, etc. They would also compare the artwork under evaluation with similar items recently sold at auctions and account for the current market demand (Aubry et al. 2019; Bailey 2020). Importantly, this approach allows one to obtain a reasonable price estimate even for artworks that are being put on the market for the first time.
A retrospective analysis of over 195,000 lots sold from Christie’s and Sotheby’s in 2016 and 2017 showed that actual hammer prices fell between the appraisers’ high and low estimates 41% and 37% of the time, respectively. This result suggests that appraisal-based valuations do provide accurate estimates in a substantial number of cases.
Several organisations (e.g., Appraisal Bureau, NonFungible.com, Tamoikin Art Fund) and individual experts now offer appraisal services for NFTs. However, appraisals can be heavily influenced by the expert’s experience and personal biases. Manual appraisals are also slow, expensive, and limited by the availability of experts (Aubry et al. 2019; Bailey 2020). There has been an attempt (the “Upshot One” application) to overcome these shortcomings by aggregating appraisals of NFTs from a set of decentralised, pseudonymous appraisers, who are financially incentivised to give honest valuations (Emmons 2020). Unfortunately, at the time of writing this article, Upshot One seemed to be still in development.
Machine Learning-based automated valuation
Given the large supply of art NFTs on the market, there is a clear need for automated and scalable appraisals of their value. Unsurprisingly, a number of such automated systems have been developed lately using Machine Learning. Examples include, but are not limited to, NFTBank, Upshot, DeepNFTValue, Zodiac NFT, NFTValuations, and Ginoa.
These systems differ in terms of their maturity and the underlying Machine Learning methodologies (see, for instance, Emmons 2021; Ryker 2021; Yakovenko 2021). Yet most of them are similar in that they currently focus on frequently traded NFT collections (e.g., Cryptopunks, Bored Ape Yacht Club, Meebits, etc.), whose individual items can be described by discrete traits (e.g., “background”, “clothes”, “eyes”, “fur”, “hat”, and “mouth” in the case of Bored Ape Yacht Club). Due to these clearly defined traits and the availability of historical prices, it is indeed often possible to quite accurately predict future prices for the respective items (Emmons 2021; Ryker 2021; Yakovenko 2021). This observation is in line with the results of employing Machine Learning for the valuation of physical artworks (Aubry et al. 2019; Bailey 2020).
While placing the initial focus on high-liquidity collections with identifiable traits makes perfect sense, a much larger proportion of NFT artworks available on the market today can neither be described using such discrete traits nor do they have any sales history (see, for example, the Art Blocks collections on OpenSea). This is where our platform, EvaluateNFT, comes into play.
EvaluateNFT mimics the traditional process of art appraisal
As mentioned above, an expert appraiser evaluating a physical artwork would typically
- assess various key characteristics of that artwork,
- compare it to similar items sold before, and
- account for the current market demand.
EvaluateNFT aims at mimicking this traditional process when predicting the price for NFTs that have no sales history. This is done as follows.
Extracting key characteristics of the query image
In its current implementation, EvaluateNFT does this by passing a query image (i.e. the NFT artwork under evaluation) through a convolutional neural network and getting a vector representation of that image. The assumption here is that the visual properties of the query image (encoded in its vector representation) are one of the main determinants of its market value. Of course, this is a rather strong assumption as there may be many other important drivers, such as the identity of the author, belonging of the image to a popular collection, the context within which the author is trying to sell that image, etc. Nevertheless, existing research suggests that the visual properties are indeed a significant factor contributing to the market value of an artwork (Aubry et al. 2019; Nadini et al. 2021).
Finding similar, recently sold artworks
Having a vector representation of the visual characteristics of the query image enables one to perform a similarity search. In EvaluateNFT, this search is done against an index of vector representations of thousands of NFTs that have been traded before and whose price history is therefore known. The data used to build this index are pulled from the OpenSea marketplace via its API.
The search operation returns up to three most similar images, along with the information on their levels of similarity, last sale date, last sale price, and the total number of sales recorded so far. Figure 1 illustrates what the search results look like. Importantly, EvaluateNFT performs a similarity search against NFTs that have been sold recently, i.e. within the last few months. This approach increases the relevance of comparisons as it accounts for the current market demand and price trends.
Estimating the price of the query image
Finally, we use a proprietary predictive model that estimates the price of the query image (expressed in ETH) based on the last sale price and other data points for similar NFTs returned by the similarity search.
Importantly, our model produces not only a point estimate (i.e. the most likely price) but also a 90% prediction interval (i.e. an interval that the true price is likely to be in with a probability of 90%; Figure 2). The lower and the upper limits of this interval are a proxy for the price range that a human appraiser would usually provide. Having such a prediction interval is of crucial importance in practice as it allows artists and NFT investors to understand the uncertainty around the reported valuation and thus make better-informed pricing decisions.
EvaluateNFT produces reasonable price estimates, but there is room for improvement
We used the following two metrics to assess the accuracy of predictions that EvaluateNFT produces for artworks with no previous sales history:
- median relative error (MRE) of the point price estimates;
- empirical coverage of prediction intervals, i.e. the percentage of observations from a holdout dataset that actually fell within their respective 90% prediction intervals.
To calculate these metrics, we used predictions for a holdout dataset that contained 330 art NFTs sold for the first time in early February of 2022.
The MRE on this holdout dataset made up ca. 40%. This means that the point price estimates generated by our model were usually off (in either direction) by no more than 40%. This value is substantially higher than the MREs reported for predictive models used in the Upshot platform (8%–19%). However, the Upshot models produce price estimates for high-liquidity collections and are thus not directly comparable to our result for NFTs with no sales history. Yet, the MREs reported by Upshot are still useful to compare against as they serve as an indicator of additional accuracy that one can potentially gain by including the sales history into pricing models for NFTs.
The prediction intervals of our model showed an empirical coverage of 92%, which is very close to the theoretical 90%. Arguably, this encouraging result is more important from the practical point of view than the model’s performance in terms of MRE. A statistically reliable prediction interval provides a more useful insight into the possible price of an artwork under evaluation than a single point estimate.
Figure 3 shows point estimates and their 90% prediction intervals produced by our pricing model for 50 NFTs randomly selected from the holdout dataset (notice the log Y scale). It can be seen that the majority of prediction intervals are reasonably narrow, meaning that the model is rather certain as to where the true price lies.
However, some intervals are very wide, reflecting the high uncertainty with regards to the true price. For example, the lower prediction limit for the 50th NFT in Figure 3 is 0.26 ETH, while the upper limit extends to 31.15 ETH (see also Figure 2). This high uncertainty around some predictions is the consequence of extremely high (yet natural) price variation among the artworks that the model was trained on. Additional research is required into factors that could help with explaining that variation and improving the model further. Our team is constantly working on such improvements, and we will keep announcing them here on Medium.
Conclusion
EvaluateNFT is a new, Machine Learning-powered valuation platform for art NFTs. It has been built to tackle a particularly difficult problem — predicting the price of artworks that have not been sold before. Many thousands of such dormant artworks exist currently on the market, and it is our hope that EvaluateNFT will help with activating these untapped assets and driving further adoption of the NFT and blockchain technologies.
Now, head to evaluatenft.io and give it a try! Our team at Next Game Solutions would love to hear your feedback and suggestions on how we could improve EvaluateNFT — just drop us a line or post your comments under this article.
