5 main challenges with datasets and AI ethics | Tooploox

With the growing impact of machine learning-based solutions on everyday human life, ensuring that AI ethics remains a part of its development is something we have to keep in mind.

The Artificial Intelligence (AI) markt is predicted to reach $190 billion in value by 2025, benefiting from a 36.62% CAGR between 2018 and 2025. The technology comes with various benefits for users and companies. According to the PwC data, the overall contribution of AI-based solutions towards the global economy is going to reach up to $15.7 trillion by 2030 and local economies will witness up to a 26% GDP boost by that year.

The AI-driven revolution brings multiple benefits — from automating tedious and repetitive tasks to digging deeper into big datasets to harvest greater insights. Companies enjoy more accurate predictions and the increased efficiency of the systems they use. Gartner predicts that up to 70% of all customer interactions will involve machine learning-powered solutions, like chatbots, by 2022. AI-powered systems are in charge of many decisions and the number of them will only increase due to investments and wider AI adoption

But these benefits come with multiple risks and challenges that AI development companies need to overcome. And most of them originate from the datasets that are a necessary part of building any machine learning-based solution.

Dataset problem — the source of AI’s ethical issues

The dataset used to train the machine learning-based solution is a core element, crucial to deliver the working effect. In the ideal world, the dataset would perfectly reflect the domain the solution is going to work for. But the real world is far from perfect, and so are datasets used by machine learning teams around the world.

The Dataset does not reflect the reality

As mentioned above, the ideal dataset would reflect reality in a perfect way, as such the AI-based image recognition solution would work seamlessly. But datasets used in machine learning are usually composed of data gathered by researchers taking advantage of any means available.

But “available” rarely means “sufficient.” For example, it is much easier to create an image of a tree or a cat than to collect a large enough number of CT scans of a particular disease. Even a more narrow category suffers from this — there are many more images of commonly used cars than unique oldtimers or exotic brands.

This situation directly impacts the AI model’s performance, reducing its accuracy when encountering the less represented data. As such, an autonomous car might get confused when approaching some rare vehicle. The real challenge comes when the AI’s decisions have a direct influence on the people’s life.

Gender imbalances in datasets

The overrepresentation of a particular type of data can lead to the solution working worse than predicted. Research done by the University of Washington shows that automatic captions are 13% more accurate when working with male voices than when working with female voices. The captioning gets even less accurate when the person has a strong accent or uses a dialect.

With reality being endlessly complex, there is little to no possibility of building a dataset that would fully reflect it — only a better or worse collection of data which the model considers canonical.

With the example above being not much more than an inconvenience, a more serious threat comes from gender imbalances in medical datasets. According to the PNAS journal, the AI-based solutions significantly drop in performance when dealing with an x-ray or other medical images of people of underrepresented genders. Thus, when it comes to AI systems used in healthcare, building an ethical dataset is crucial.

Also, one needs to remember that datasets are collected and evaluated by humans, with all the inherent dangers coming from this — from methodological flaws to suboptimal labeling due to various interpretations made by individuals.

The implicit bias problem

Researchers and data scientists working on AI-powered solutions are highly skilled, yet they are only people, with all of their weaknesses. Considering this, it is possible for the designed solution to suffer from the implicit biases that are projected by the designers.

Implicit bias is a deeply hidden bias that has an influence on the behavior of people who are otherwise conscious of their biases and try to avoid acting on stereotypes. A good example comes from a Stanford University Paper, where a theoretical figure of a consciously unbiased manager mistrusts the feedback of his female peers who advise him to hire a man instead of a woman.

This kind of bias can be seen in AI solutions as well, yet they are even harder to spot — researchers are working on removing explicit biases from datasets, yet implicit ones (like delivering images of females in a certain context or colors) can still remain and influence the solutions.

The concern is especially strong when it comes to facial recognition systems used to tackle crime. British police officers highlight their concerns about AI boosting popular stereotypes about particular ethnic groups when using automated systems to detect suspicious individuals.

The reason-effect problem

Machine learning solutions, especially used in predictive analytics, harness the reason-effect correlations hidden within data. But correlation does not imply causation. The graph below is a good example of the problem:

That which is obvious for a human due to his or her life experience and general knowledge is not obvious for a machine due to the lack thereof.

The ability to spot a hidden and unintuitive correlation within data is one of the greatest strengths of AI-based solutions, but on the other hand, there is a great danger to avoid.

The inability to properly identify the cause of a particular effect lies at the root of every stereotype and bias, from racism and xenophobia to anti-vaccination movements. Considering the power of the human mind when compared to that of machine learning, the danger of misattributing the cause and effect and its impact on the solution’s effectiveness raises the next ethical question about AI.

A good example of a failure in ethics and AI regarding the lack of reason-effect attribution is the, now shunned but previously used, AI application to manage work applicants. According to a Reuters article, the app was severely biased against female coders due to the underrepresentation of them among Amazon’s engineers. The machine reasoned that being a female makes one a bad engineer — a clearly absurd and excellent example of reason-effect misattribution.

The inhumanity of the artificial neural network

Another challenge is in the fact that a neural network is no way human in any form. The solution shouldn’t be considered a conscious being of any kind. It has no common sense, no previous experiences, and no general knowledge to reason from. Datasets that the company or research facility feeds it with is the only world the neural network knows, without any background knowledge, context, or frame of reference.

Considering that, it is possible for a neural network to give an unpredictable outcome or behave in an odd way — it is a machine after all and it will never be human. Perceiving its work as that of a human in any way is yet another cognitive bias in itself.

The inherent inhumanity of the artificial neural network is the source of the Black Box problem — the fact that even the creators can have problems describing the decision making process of their own AI-powered solutions as well as the basis of those decisions. Thus, for now, it is hard to imagine a machine learning-based judge that wouldn’t be able to justify their decisions and has no “common sense” or “life experience” to leverage while making those decisions.

No way to acquire new datasets

Last but not least, there are several commonly used datasets in the ML world, which are both large enough and publicly available. They are only as good as they are — and they are usually far from perfect. There can be hidden flaws or traps within commonly used datasets — the widely used dataset prepared by MIT researchers was revealed to have racial slurs and other disturbing names used in its labels of images of people. The dataset contained over 79,300,000 images and will be maligned due to being both too large and the images too small for manual inspection.

On the other hand, the dataset needs to be legally compliant, especially considering personal or medical data. The GDPR of the EU significantly limits one’s ability to build a dataset with images, names, or life histories.

Building the most sophisticated neural network relies on training them on large datasets that enable them to perform a task and further retraining, using more domain-oriented data to perform a more highly specified task. When combined with the dataset’s limited ability to fully reflect reality, this challenge gets massive.

Considering the issues mentioned above, the core of artificial intelligence ethics is based on the responsible building of datasets — and skipping this can be disastrous.

AI ethics policy and governance initiatives

The challenges related to AI ethics are widely commented upon and observed by international institutions. Many of them have raised initiatives to deliver a legal or philosophical framework of building ethical AI technologies:

• The European Union has released whitepapers about building ethical AI solutions regarding learning algorithms, datasets, and autonomous vehicles, among others. Also, the organization has published a separate whitepaper regarding gender equality in AI-powered solutions.
• Another initiative comes from the scientific community. The NeurIPS conference encourages researchers to predict the possible outcomes of technologies and solutions they show, be they positive or negative ones.
• An interesting initiative comes from the Vatican, which issued a pledge about ethical AI. The document has been signed by IBM and Microsoft.

Considering all of the above, an ethical approach toward AI-based solutions could be crucial to ensure a company will not suffer from a backlash. That’s why we at Tooploox put a strong emphasis on this aspect.

Summary — Ethics and bias in AI — the Tooploox perspective

As an AI software development company, we aim to deliver the highest quality of products possible. AI ethics is the key to make it even more flexible and functional.

Thus, there are two major policies we’ve implemented:

• We ensure our datasets are fair and diverse, thinking about the possible impact of particular data on the performance or the user
• We make sure the solution will serve a good purpose — we will never work for a company that wishes to wreak havoc or has harmful policies

If you wish to talk about AI ethics, our approach, and how it could support your business, don’t hesitate to contact us!

Originally published at https://www.tooploox.com on August 27, 2020.