Facebook Innovating the World of Artificial Intelligence
In the world of social media, pictures play a large role in contributing to the uniqueness of each individual app. Each day people share more than 2 billion photos across Facebook, Facebook messenger, Instagram, and WhatsApp. Visual content provides a look into one’s current life in a fun and expressive way, this can then spark conversation among friends. For Facebook in particular, it is difficult to imagine the app without photos. The ability to upload an endless amount of photos targeted a global audience. Although pictures have a number of positive benefits throughout social media, they may pose as a challenge for those who are severely visually impaired or blind. With more than 39 million people who are blind and over 246 million who have severe visual impairment, many may feel excluded from the excitement or conversation a photo may uphold. With Facebook’s innovative updates they have decided to build the technology that helps the blind community experience Facebook the same way others do.
Throughout this technological process Facebook has been working with the American Foundation for the Blind (AFB). Facebook’s update is called “automatic alternative text,” the feature was created by their accessibility team led by Jeff Wieland, a former user researcher in Facebook’s product group. He has had much experience within closed captioning for videos as well as making an increase in font size available to users. Automatic alternative text is able to recognize objects in photos using machine learning, which uses algorithms to make predictions. For example, if you show a piece of software enough pictures of a tree, in time it will be able to identify a tree in a photo. Alongside the ability to recognize what is in the picture, automatic alternative text will then use the iPhone’s VoiceOver feature to describe what is in the photo out loud to the user. The technology is still in it’s early stages yet can identify a number of categories, for example in regards to transportation it may state “car,” “airplane,” boat.” While being able to identify objects it is also able to describe people’s features, for example if they were to be smiling.
Currently, the primary way blind people are able to access the internet’s capabilities is through a screen reader, which is a software that describes the elements displayed on a screen and makes it possible to interact with them. After reading much information on Facebook’s new technological feature, I was interested to see how the target audience was reacting to it’s new accessibility. Matt King, a Facebook engineer who is blind himself, represents Facebook on a World Wide Web conglomerate responsible for the technical specifications that make web pages accessible. I came across a statement made by King, “You used to hear file names and you didn’t know if they were clickable, it was a big Easter egg hunt and even when I found the eggs, a lot of the eggs were photos,” King states, “People talk in pictures, and talking in pictures is inherently out of reach for me.” He goes on to describe the approach Facebook took while dealing with the issue at hand, “We could probably require people when they upload a photo to please describe this for blind people, but it would drive people nuts.”
Interested in the details of the feature I did more research in regards to how King described his input on the technology. He provided a demonstration in which he pulled up a few stories on Facebook including photos, yet set the screen to black. King’s screen reader was in charge of every move he made on Facebook’s page while it read posts out load, identified links and exposed a various amount of buttons. Afterwards, King turned the screen back on to a photo of a pepperoni pizza with olives with the caption labeled, “Sunday night splurge.” King’s VoiceOver feature then read, “pizza, food.” In the final moments of his demonstration King states, “Sometimes it’s just really amazing what one word can do.”
Facebook is not the only one’s to begin using machine learning to caption photos. The level of sophistication artificial intelligence has reached is immense. Google Photos and Flickr have similar technology updates yet are still known to be prone to errors. For example, Google was forced to apologize after photos tagged two black people as “gorillas.” This error was especially dreadful for Google’s case. In regards to Facebook’s ability to identify race, it will only suggest a tag for a photo if it is very highly confident in what it is looking at. When the server is not confident it will not give the description. It seems as though no description is better than a bad or offending one.
Many companies may describe their newest update to be “just the beginning,” yet have much knowledge into the future of the update, this Facebook’s particular technology this is not the case. Automatic alternative text only works with one platform, Facebook, and only in English. There are still millions of objects that it can not recognize with 80 percent of it’s recommended confidence. Facebook’s team is leaning towards creating new tools in regards to recognizing objects in videos, and something it calls “visual Q&A,” which will allow users to ask questions about photos and receive an answer from the automatic alternative text. For example, one may ask who is in the photo and it could answer the names of the friends who appear in it.
With this innovative update Facebook has professionally introduced a captivating demonstration of technology is given. Automatic alternative text represents a large growth in opportunity to people with disabilities who in the past have been unable to use Facebook with the same pace as others for clear reasons. In regards to this King states powerfully, “Inclusion is really powerful and exclusion is really painful. The impact of doing something like this is really telling people who are blind, your ability to participate in the social conversation that is going on around the world is really important to us. It’s saying as a person, you matter, and we care about you. We want to include everybody — and we’ll do what it takes to include everybody.”
