Had a Chat with ChatGPT about Provenance

A Story Based on True Events

With all the recent frenzy around ChatGPT, I thought it was time to have a chat with OpenAI’s chatbot myself.

As provenance is an area that I’ve been giving most of my attention to for the past year, why not have a conversation about that?

To be sure that we had the same understanding of what provenance meant — otherwise, the conversation could have gone awry — my first question to ChatGPT was: What does the term provenance mean?

The first answer I got was pretty much focused on the art world — unsurprisingly, as that’s where the term originally comes from — but as I asked the chatbot to continue (key word!), and posed follow-up questions such as How about other assets?, we finally got to the conversation I was really interested in having.

One thing I learned from my conversation with ChatGPT — which is something I have to keep in mind when I talk about digital provenance to clients and peers in the industry — is that people might associate this term with digital assets only, which is not how I think about it, and that’s not the message I want to convey. Obviously, unlike in the case of a digital asset, a physical asset needs to be digitized first, but a digital provenance solution as such can be applied to both digital and physical assets, at least that’s how I see it. (And, that’s what we mean when we say that BTP is a digital provenance company.)

I was pretty happy with the elaborate response I got to my question Why is provenance important? In summary, ChatGPT said that provenance was important for ensuring authenticity, integrity and trustworthiness of an object, artifact or digital asset; and that it can be used to establish the value of an object, ensure ethical sourcing, comply with regulations and traceability, and improve data governance and management.

One thing that I’ve spent considerable time researching is the size of the provenance market, and what I’ve found is that there is little to no data available about it. I found data about related issues such as the problem of counterfeiting. According to Statista, the global fake goods market was worth more than Ireland’s economy, in 2019. I guess that got only worse with the pandemic.

Market data about the size of the Blockchain in Supply Chain Market is also available. [Blockchain is important to me, well to all of us at BTP, because our provenance offering is backed by a distributed ledger.] According to Research Dive, the global blockchain in supply chain market is predicted to garner a revenue of US$14.9 billion in the 2020–2028 timeframe, growing from US$423 million in 2020, at a CAGR of 57.4%. Product traceability is anticipated to be the fastest growing area across a number of industries.

This is all good data, but I still couldn’t find a research report that would tell me how big the provenance market was. There is a PwC study from 2020 that predicts that blockchain’s ability to track provenance will contribute nearly one trillion USD to global GDP by 2030, which is a data point that I have been diligently using in conversations. This is great stuff as it talks about business value, and drives attention to the allure of blockchain-backed provenance. (Btw, this is a report that ChatGPT was not able to find. Just saying…)

However, some questions still remain. As someone responsible for strategy, I want to have a better sense of the number of potential customers that would buy a provenance product, and the revenue that these sales may generate. As I was struggling to find this kind of data, I asked ChatGPT about the size of the digital provenance market.

ChatGPT started by saying that the market size for this space is currently in a growth stage and there isn’t a specific number or range that defines it. It also said that it is considered to be a significant market with a high growth potential. Then, it went on talking about key factors that are driving this growth, which included the increasing demand for transparency and traceability in supply chains, increasing adoption of blockchain technology in various industries, and rising concerns about counterfeit products, among other things. So far so good, it sort of provided some confirmation and relief. (I’m a good analyst, ok?! :-))

However, there was one report that ChatGPT mentioned in its response. It said that according to a report by MarketsandMarkets, the global digital provenance market size was valued at USD 1.9 billion in 2020 and is expected to reach USD 5.2 billion by 2025, at a CAGR of 21.5% during the forecast period. It also said that other reports by different research firms suggest similar growth patterns and have a similar range of market size.

I was like…really?! How could I miss this?

So, I went to look for this report on the MarketsandMarkets website. I couldn’t find it, so I reached out to them. They said that they haven’t sized the digital provenance market yet (although they could do it in 4–6 weeks for $$$). Essentially, the report doesn’t exist. It turned out that ChatGPT confused the word provenance with the word evidence. There is a report on the global digital evidence market, however, that’s not what I was looking for. Although this sort of made my ego feel good (come on, I did a good job!), I was also disappointed. So, I asked ChatGPT to provide more sources, but ultimately, those turned out to be non-existent reports too.

In its defense, after being called out for its mistake — i.e. the MarketsandMarkets report — ChatGPT admitted that it provided incorrect information, and apologized. (It’s not such a common human behavior anymore, and I was impressed. ;-)) A bit later, I asked the chatbot again to find research reports on the market size of digital provenance. Interestingly, this time it told me that it wasn’t able to find any specific reports. I insisted with a few follow-up questions, and got a couple more elusive answers (you see, that’s human! ;-)), but in the end, I got my list of report citations. This was a shorter and different list from the one before — I guess, the chatbot did learn from its past mistakes — however, it was no good either. Essentially, ChatGPT seemed happy to make up citations or, to put it another way, fake it until it made it…

None of us is perfect. We all make mistakes. Fancy chatbots and humans alike. However, when it comes to specific data, mistakes like this are not only misleading, but can be very dangerous. In my specific case, I could relatively quickly and effortlessly correct the error; nonetheless, when you’re using AI-based tools, you definitely need to take into consideration its limitations and reliability, among other things. Don’t bet your life on it.