Lockdown, You decided to buy a T-shirt online. How can you trust a random seller on the portal/app?. You read reviews, count of total reviews, carefully read the bad reviews, check if the reviewer is a verified buyer, run through AI fake check scores, and still, Ughh! you are not convinced because fake reviews can (GAN) be spoofed like real reviews. It is inevitable in the future. This is the time your brain gives you a confidence score, based on that you decide the buy vs skip decision. You are smart, you know how to use a T-shirt, so you buy them carefully considering all the facts.
But did AI help you here?. How much? You are not sure, because you do not know the quality of the “fake review” AI model itself. Even if they share the details how do you trust them? The truth is there is no transparency in benchmarking the quality of the model and also what goes in and when was the model updated. Even if the model score is only 50% today we should be informed.
The world needs a consortium (or World AI Trust) for AI to audit the AI models (definitely not funded org like WHO). The independent organization can audit and provide a score that companies can disclose. Every model should be validated by the consortium and give a trustworthy score and every consumer of the product should be informed on what percentage they can trust. This is practically challenging unless there is a global evaluation platform provided by the AI consortium. Google initiated AI ethics a long time back but I believe there is no progress.
Facebook AI has recently released a Dynabench model evaluation platform, gives an option to dynamically create datasets and evaluate incrementally, and sets a new SOTA goal. It is definitely a good sign, a similar platform should come from an independent organization. I look for FDA / FSSI accreditation for buying a food product, similarly, I need accreditation for Trust. I come from the mobility industry where AI is playing a very advanced role as an auto-pilot, already making millions of decisions every second that involve human life.
There are few organizations that popped out as AI ethics organizations recently, but their future direction is unclear. An AI consortium should be a non-profit body, must have representatives from the people trusted organizations in each country such as Google, Microsoft, Apple, Facebook, Wikipedia, Tesla, Amazon, Tata, Siemens, Phillips, Samsung, Sony, Volvo, Lego Holdings, Commonwealth Bank, Barilla, Rolex and so on. Until one such organization exist AI cannot win trust from the end customer AI is going to take a few seconds of your time to make you explore or exploit your opinions, so you bet on your brain to buy your T-shirt.
