Ethical Concerns in Speech Emotion Recognition, Part 3, Accuracy, Application, Responsibility, Accountability, and Monitoring.
This is the last article of the series on Ethics in Speech Emotion Recognition.
You can find the previous articles here:
So we talked about several ethical concerns about Speech Emotion Recognition. We covered privacy, consent, transparency, bias, and fairness.
This final post will review Accuracy, Application, Responsibility, Accountability, and Monitoring.
Wait! They seem rather technical than ethical concerns! That is correct. Professional ethics is an essential part of everybody’s work. And not following the best practices and highest standards and irresponsibly and haphazardly building a system, especially a piece of AI that can automate many decisions, in my mind, is highly unethical.
Technical standards have been created to ensure safety and reliability. They are created to ensure the system’s response is predictable and reproducible. By ignoring these standards, you’re, at minimum, compromising the quality of the service delivered and, possibly, in the case of SER, compromising the health and safety of the end users.
And also, there’s no need to mention that, generally, a significant part of technical standards promotes the security and safety of all the parties involved. However, those practices usually cost an overhead of resources to build and follow, and skipping them is cheating your customer base and the industry, as saving that cost in the short term creates an unfair resource advantage, but in the long run, endangers the end user. Think of a car manufacturer skipping following all the safety standards!
Now that we know how these are related let’s dive in and find out!
Accuracy
Accuracy is an important performance measure of a Machine Learning model. The model has to be accurate in predicting the emotions in the case of SER, and reporting the system’s accuracy should go beyond just calculating unweighted accuracy. First and foremost, most SER databases are heavily unbalanced. Therefore, calculating unweighted accuracy and reporting a model with 75% accuracy is lying to the customer when your weighted accuracy for the smallest class is 60%.
Consider a model used in a suicide hotline; emotions like despair and hopelessness are fundamental to gauging the situation. However, there are not many samples of these emotional classes in many databases. And they can easily be labeled as sadness or boredom. Now a model with 50% accuracy in despair and 80% accuracy in happiness, anger, sadness, and neutral emotions will definitely be tragic in the wild.
In the same context, there are many cases that we need to investigate Precision, Recall, and F1 score apart from accuracy. Based on a survey we published last year, and after reading and reviewing over 100 papers in SER, I can confidently say that they are not being reported in most of the research, the same as weighted accuracy.
A confusion matrix is another vital tool, especially for use cases with a few important classes and a group of non-important classes to focus on. Back to our suicide hotline example, high valance and high arousal emotions are not that important in alerting a case of possible suicide. And we know that we have a heavily unbalanced database. Therefore studying the confusion matrix and taking the risk of false negatives in target classes becomes more important than accuracy alone.
As we see, accurately reporting the system accuracy has a very ethical role in SERs, just think how devastating consequences would come from a system that misdiagnoses someone confidently, in case of psychiatric support, or the same suicide hotline we discussed.
Application
Another ethical consideration in SER is what the public is generally afraid about AI these days, the potential for it to be used for nefarious purposes. Think of SER becomes an aide in courts to decide the state of the mind of people and one individual being manipulated by tailoring messages to elicit specific emotions. That should be enough to understand how important and scary this can be.
Responsibility
Responsibility is another ethical consideration in SER. Developers and users of the technology must take responsibility for their work’s ethical implications and use. This means they have to be legally responsible for their code, consider their actions’ potential consequences, and take steps to mitigate any adverse effects.
Accountability
Accountability is also crucial in SER. If something goes wrong with the technology, there must be a way to hold those responsible accountable. This means there should be clear guidelines for using the technology, and those who violate these guidelines should be held accountable.
Monitoring
Finally, there is a need for ongoing monitoring and evaluation of SER. As technology continues to evolve, it is vital to evaluate its ethical implications and make changes as necessary. This means that there must be ongoing research into using SER and a willingness to implement auto-adaptive training concepts into the models and pipelines to ensure the system is dynamically getting updated by societal trends.
In conclusion, as we reviewed, ethics plays a significant role in Speech Emotion Recognition (SER), encompassing various aspects beyond technical concerns. Our exploration of ethical considerations, including privacy, consent, transparency, bias, fairness, accuracy, security, application, responsibility, accountability, and monitoring, highlights the importance of professional ethics in this field. Ignoring technical standards and best practices when developing SER systems is unethical. It compromises the quality of service and potentially endangers the health and safety of individuals relying on these systems. Technical standards ensure system responses’ safety, reliability, and predictability while promoting security for all parties involved. By adhering to these standards, we avoid unfair resource advantages and uphold the ethical responsibility towards customers and the industry. By embracing ethical principles, the field of SER can foster positive, responsible, and socially beneficial technological advancements.
Let’s hope, and try to build AI systems, that ethics, human rights, and values are embedded in every line of its code.
Thank you for taking the time to read this!
