On Recent Research Auditing Commercial Facial Analysis Technology

  1. There is an indirect or direct relationship between modern facial analysis and face recognition (depending on the approach). So in contrast to Dr. Wood’s claims, bias found in one system is cause for concern in the other, particularly in use cases that could severely impact people’s lives, such as law enforcement applications.
  2. Raji and Buolamwini’s study was conducted within the context of Rekognition’s use. This means using an API that was publicly available at the time of the study, considering the societal context under which it was being used (law enforcement), and the amount of documentation, standards and regulation in place at the time of use.
  3. The data used in the study can be obtained through a request to https://www.ajlunited.org/gender-shades for non commercial uses, and has been replicated by many companies based on the details provided in the paper available at http://gendershades.org/.
  4. There are no laws or required standards to ensure that Rekognition is used in a manner that does not infringe on civil liberties.

1. Facial analysis vs. recognition — Facial analysis and recognition have an important relationship and the racial and gender bias in the Raji & Buolamwini audit is cause for concern.

Dr. Wood writes:

Facial analysis and facial recognition are completely different in terms of the underlying technology and the data used to train them. Trying to use facial analysis to gauge the accuracy of facial recognition is ill-advised, as it’s not the intended algorithm for that‎ purpose.

Human-Centric Computer Vision Landscape for Single Images

Example human-centric vision tasks for single images.

2. Societal Context Matters — Raji and Buolamwini’s paper investigates Amazon’s products within the context and society they are used.

Dr. Wood and Mr. Punke repeatedly mention that the study did not test Amazon’s product in a manner it was intended to be used. E.g.,

3. Companies such as IBM and Microsoft have reproduced comparable data and results using the guidelines written in the Gender Shades work which reiterates that ethnicity cannot be used as a proxy for skin type while performing disaggregated testing.

Finally, Mr. Punke writes that

We call on Amazon to stop selling Rekognition to law enforcement as legislation and safeguards to prevent misuse are not in place.

Overall, we find Dr. Wood and Mr. Punke’s response to the peer-reviewed research findings disappointing. We hope that the company will instead thoroughly examine all of its products and question whether they should currently be used by police. Mr. Punke writes that the company supports legislation to ensure that its products are not used in a manner that infringes civil liberties. We call on Amazon to stop selling Rekognition to law enforcement as such legislation and safeguards are not in place.

Concerned Researchers

  1. Ali Alkhatib, Stanford University
  2. Noura Al Moubayed, Durham University
  3. Miguel Alonso Jr, Florida International University
  4. Anima Anandkumar, Caltech (formerly Principal Scientist at AWS)
  5. Akilesh Badrinaaraayanan, MILA/University of Montreal
  6. Esube Bekele, National Research Council fellow
  7. Yoshua Bengio, MILA/University of Montreal
  8. Alex Berg, UNC Chapel Hill
  9. Miles Brundage, OpenAI; Oxford; Axon AI Ethics Board
  10. Dan Calacci, Massachusetts Institute of Technology
  11. Pablo Samuel Castro, Google
  12. Stayce Cavanaugh, Google
  13. Abir Das, IIT Kharagpur
  14. Hal Daumé III, Microsoft Research and University of Maryland
  15. Maria De-Arteaga, Carnegie Mellon University
  16. Mostafa Dehghani, University of Amsterdam
  17. Emily Denton, Google
  18. Lucio Dery, Facebook AI Research
  19. Priya Donti, Carnegie Mellon University
  20. Hamid Eghbal-zadeh, Johannes Kepler University Linz
  21. El Mahdi El Mhamdi, Ecole Polytechnique Fédérale de Lausanne
  22. Paul Feigelfeld, IFK Vienna, Strelka Institute
  23. Jessica Finocchiaro, University of Colorado Boulder
  24. Andrea Frome, Google
  25. Field Garthwaite, IRIS.TV
  26. Timnit Gebru, Google
  27. Sebastian Gehrmann, Harvard University
  28. Oguzhan Gencoglu, Top Data Science
  29. Marzyeh Ghassemi, University of Toronto, Vector Institute
  30. Georgia Gkioxari, Facebook AI Research
  31. Alvin Grissom II, Ursinus College
  32. Sergio Guadarrama, Google
  33. Alex Hanna, Google
  34. Bernease Herman, University of Washington
  35. William Isaac, Deep Mind
  36. Phillip Isola, Massachusetts Institute of Technology
  37. Alexia Jolicoeur-Martineau, MILA/University of Montreal
  38. Yannis Kalantidis, Facebook AI
  39. Khimya Khetarpal, MILA/McGill University
  40. Michael Kim, Stanford University
  41. Morgan Klaus Scheuerman, University of Colorado Boulder
  42. Hugo Larochelle, Google/MILA
  43. Erik Learned-Miller, UMass Amherst
  44. Xing Han Lu, McGill University
  45. Kristian Lum, Human Rights Data Analysis Group
  46. Michael Madaio, Carnegie Mellon University
  47. Tegan Maharaj, Mila/École Polytechnique
  48. João Martins, Carnegie Mellon University
  49. El Mahdi El Mhamdi, Ecole Polytechnique Fédérale de Lausanne
  50. Vincent Michalski, MILA/University of Montreal
  51. Margaret Mitchell, Google
  52. Melanie Mitchell, Portland State University and Santa Fe Institute
  53. Ioannis Mitliagkas, MILA/University of Montreal
  54. Bhaskar Mitra, Microsoft and University College London
  55. Jamie Morgenstern, Georgia Institute of Technology
  56. Bikalpa Neupane, Pennsylvania State University, UP
  57. Ifeoma Nwogu, Rochester Institute of Technology
  58. Vicente Ordonez-Roman, University of Virginia
  59. Pedro O. Pinheiro
  60. Vinodkumar Prabhakaran, Google
  61. Parisa Rashidi, University of Florida
  62. Anna Rohrbach, UC Berkeley
  63. Daniel Roy, University of Toronto
  64. Negar Rostamzadeh
  65. Kate Saenko, Boston University
  66. Niloufar Salehi, UC Berkeley
  67. Anirban Santara, IIT Kharagpur (Google PhD Fellow)
  68. Brigit Schroeder, Intel AI Lab
  69. Laura Sevilla-Lara, University of Edinburgh
  70. Shagun Sodhani, MILA/University of Montreal
  71. Biplav Srivastava
  72. Luke Stark, Microsoft Research Montreal
  73. Rachel Thomas, fast.ai; University of San Francisco
  74. Briana Vecchione, Cornell University
  75. Toby Walsh, UNSW Sydney
  76. Serena Yeung, Harvard University
  77. Yassine Yousfi, Binghamton University
  78. Richard Zemel, Vector & University of Toronto

Click below to add your name and affiliation


[1] https://aws.amazon.com/blogs/machine-learning/thoughts-on-recent-research-paper-and-associated-article-on-amazon-rekognition/



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store