The Perceived Usefulness of Ethical-Aligned AI Development Methods: Closing the Disconnect Between Developer Needs and Offered Solutions (Executive Summary)
Claiming the evidence-based nature and the economic benefits of artificial intelligence (AI), the world has seen an increasing adoption of AI-based systems to support, e.g., the decision-making process (De Fine Licht et al., 2020). However, the claim that AI systems are “evidence-based by no means ensures that it will lead to accurate, reliable, or fair decisions” (Barocas, Hardt, & Narayanan, 2019, p. 12). Recent practical developments have revealed that applying AI algorithms in the real world, with all its exceptions and edge cases, leads to significant social challenges, such as discrimination, power asymmetry, and opacity (Lepri et al., 2018).
Aiming to solve these challenges, the research field of Ethics & AI has seen a major shift from agreeing on ethical principles the AI system should adhere to, towards identifying approaches on how to implement those. One prominent mitigation strategy is to use procedural methodologies, which aim to adapt the current development processes to consider the ethical dimension (Morley, Kinsey, et al., 2021). However, this approach is characterized by a low adoption rate in the real-world, which is certainly due to a “clear disconnect between what is available to AI practitioners, and what they would find useful“ (Morley, Kinsey, et al., 2021, p. 6).
Aiming at closing this disconnect during my master thesis, I have analyzed, which elements of the currently available set of procedural methodologies (which can be used during the agile development of AI systems) are perceived as useful by developer teams and why, using a mixed method approach, consisting of semi-structured interviews and the system usability scale. Based on this, I was able to derive seven best practices on how future procedural methods should be designed in order to be perceived as more useful.
First: tensions and atomic aspects are perceived as a valuable tool to reduce the abstract nature of principles.
Imagine, I ask you to design a “fair” algorithm. I guess you would not exactly know what to do next, since “fairness” is an abstract concept and has different meanings in different contexts. This challenge is also faced by developer teams and complicates the implementation of ethical principles (Whittlestone et al., 2019).
However, my results indicate, that two concepts are perceived as useful to reduce the abstract nature of principles. Firstly, using tensions between principles, by identifying conflicts between requirements that various stakeholders have towards the system and the necessary attributes, that the AI system needs in order to work (Raji et al., 2020). Secondly, using atomic aspects, which are fine-grained sub-categories of one principle and thus decreases the number of variables, leading to a reduction of the inherent complexity (Vakkuri et al., 2021).
Considering the fact, that ethical principles are likely to be unavoidable as the first step in every ethical aligned AI project, these two tools provide a powerful and generally applicable way of reducing the inherent abstract nature. Thus I strongly promote the usage of tensions and atomic aspects for every AI project.
Second: positively change the working style through documentation
Most developers, and those who wrote code at some point, know about the special “love-hate relationship” of documentation. While documenting one’s work is certainly necessary to explain the underlying assumptions and to enable a knowledge transfer, some developers perceive documentation as a burden without concrete / measurable advantage (Aghajani et al., 2019).
However, my results show a positive consensus around the usage of documenting the ethical progress and the decision-making process. This is because documenting the ethical dimension leads to a data base of past decisions which increases the knowledge and performance for future projects and forces developers to explicitly think about ethics early on.
Interestingly, my results further indicate, that documentation and the associated act of arguing for a certain decision might also positively change the working style of developers. Consider a scenario, in which no quantitative metrics is used for the decision-making process, but only the quality of arguments, measured through the approval rating of a certain audience. This would firstly remove the need to reduce the ethical dimension into a quantifiable KPI, something that is certainly controversial in the academic literature (Saltelli et al., 2020), while encouraging (maybe even requiring) developers to extensively engage with the various ethical aspects in order to have the best argument. Consequently, the approval rate can be interpreted as a settle form of metrics and optimizing this rate requires an optimization of the argumentation logic and thus increases the level of ethical alignment. Thus, I strongly encourage the usage of logically arguing for ethical decisions by developers and documenting this process.
Third: enable actionable mitigation through technical tools
One emerging shortcoming of most processes was the limited help with regard to mitigating the identified ethical issues. However, I was not only able to identify this limitation but also to showcase the perceived usefulness of providing a mapping between principles and currently available technical toolkits. This increases not only the awareness about these tools, which is currently considered to be low (Siqueira de Cerqueira et al., 2022), but also provides developers with actionable tools to provide real-world change.
Generalizing these findings, I encourage scientists, who design new process, to ask “what’s next” after every process stage and to view everything from the perspective of AI practitioners. The goal should always be to make the process as tangible as possible and to outline a very clear path towards how to mitigate the ethical issues (e.g., through pointing developers to technical toolkits).
Fourth: combine divergent and convergent elements
On a meta-level, another prominent shortcoming was the usage of open-ended questions. While some processes use this to trigger an intra-team discussion to increase the ethical awareness, my findings indicate that this should only be used when combined with further convergent elements.
This is because discussions are seen as a divergent element, meaning it opens the solution space and triggers creativity. While this is an important element, it should not be the only one since this might lead to an intangible outcome and thus does not lead to a high perceived usefulness. This is also true when it is used as the last element of a process since it leaves the developer team without a concrete focus and plan on how to continue.
Consequently, divergent elements should be combined with convergent elements, meaning those elements which narrow down the solution space to a concrete subset, e.g., by helping developers to prioritize the ethical issues. However, while combining both elements is certainly critical, this thesis goes even further and advices to combine divergent and convergent elements, but to always end on a convergent one. This is important since otherwise, the result is not perceived as tangible and thus not as useful.
Fifth: guidelines on when to adopt the process
Another finding of mine is the expressed uncertainty by AI practitioners when to adopt a process. This is particularly important since the landscape of AI development is diverse in almost every aspect, including the team composition, the general organizational structure, and the nature of the AI projects. My fidings indicate, that these components have a significant influence on the perceived usefulness, which is why I strongly recommend to include clear guidelines as to when the concept should be implemented, or which aspects need to be in place in order for the process to work.
Sixth: best practices on how to adopt the process
Next to demanding guidelines on when to implement the concepts, I also identified a clear need for best practices on how to implement and execute the concepts in the best way. While this shortcoming can be traced back to the young age of the research field in general, this thesis argues that clearly stating best practices should play a more important role in the design and publishing process of the respective papers. Consequently, this thesis advices to either include an additional paragraph, which states important best practices on how to implement the process, or to use a case study methodology for every publication, that wishes for a high adoption rate in the real world.
Seventh: match work packages with developer roles
As also mentioned in the paper of Beckert (2021), developers have a certain tendency to view the implementation of ethical considerations as not an integral part of their responsibility. This is also one of my findings. However, my results indicate, that this could be solved by designing processes not only from a work-package perspective but by directly matching those with the roles of a development team, including strong arguments why this match makes sense. In doing so, the awareness of the diverse responsibilities of every role is affirmed and intra team discussions are prevented right from the beginning. This leads to the omission of one prominent reason stated by developers to not implement ethical principles despite the acknowledgment of their importance.
Concluding, translating between principles and practice is a difficult venture and this thesis clearly outlines how much work still needs to be done. However, I hope to have laid a cornerstone for future researchers to develop more useful procedural methods and thus to close the disconnect between what AI practitioners would find useful and what is currently offered.
Sources
Aghajani, E., Nagy, C., Vega-Márquez, O. L., Linares-Vásquez, M., Moreno, L., Bavota, G., & Lanza, M. (2019). Software documentation issues unveiled. IEEE (Ed.), 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) (pp. 1199–1210). IEEE. Retrieved from https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8811931
Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness in Machine Learning. fairmlbook.org. Retrieved from https://fairmlbook.org
De Fine Licht, K., & De Fine Licht, J. (2020). Artificial intelligence, transparency, and public decision-making. AI & SOCIETY, 35(4), 917–926. doi: 10.1007/s00146–020–00960-w
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018). Fair, Transparent, and Accountable Algorithmic Decision-making Processes: The Premise, the Proposed Solutions, and the Open Challenges. Philosophy & Technology, 31(4), 611–627. doi: 10.1007/s13347–017–0279-x
Morley, J., Kinsey, L., Elhalal, A., Garcia, F., Ziosi, M., & Floridi, L. (2021). Operationalising AI ethics: barriers, enablers and next steps. AI & SOCIETY, 1–13. doi: 10.1007/s00146–021–01308–8
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI Accountability Gap: Defining an End-to- End Framework for Internal Algorithmic Auditing. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 33–44. doi: 10.1145/3351095.3372873
Saltelli, A., & Di Fiore, M. (2020). From sociology of quantification to ethics of quantification. Humanities and Social Sciences Communications, 7(1), 1–8. doi: 10.1057/s41599–020- 00557–0
Siqueira de Cerqueira, J., Azevedo, A., Tives, H., & Canedo, E. D. (2022). Guide for Artificial Intelligence Ethical Requirements Elicitation — RE4AI Ethical Guide. 55th Hawaii International Conference on System Sciences, 1–10. doi: 10.24251/HICSS.2022.677
Vakkuri, V., Kemell, K.-K., & Abrahamsson, P. (2021). ECCOLA-a method for implementing ethically aligned AI systems. Journal of Systems and Software, 182, 195–204. doi: 0.1016/j.jss.2021.111067
Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019). The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions. 2019 AAAI/ACM Conference on AI, Ethics, and Society, 195–200. doi: 10.1145/3306618.3314289