Earlier this year, Facebook launched a new feature for iOS screen readers called automatic alt text. Powered by artificial intelligence and machine learning, it recognizes the objects in a photo and then describes them, which is incredibly handy for users who are blind. It arrived on Android soon after, and two weeks ago, the social network officially made the tool available in 20 other languages. These are just a few recent examples of how Facebook is attempting to make its products -- as well as the internet as a whole -- more accessible to those with disabilities. It all started five years ago when Jeff Wieland, who worked on the company's user research team at the time, discovered that there were people with disabilities who were having a rather terrible experience with Facebook.
"There's this concept of alt text for photos, which used to be provided by the site owner or web developer," said Wieland. "But now the internet is all photos ... The only way we could really provide the text for photos at our scale, would be to use computers to do this." Fortunately, Facebook's AI team had been working on computer vision for at least a year and a half already and had the technology and expertise to work on the project. According to Crista Earl, the director of Web Services for the American Foundation for the Blind, most sites have incorrect or poorly designed forms. "For example, when trying to book a flight on a travel site ... the manner of selecting choices poses an obstacle to users of assistive technology." Inadequately labeled videos and images are also a problem. Other common issues are low-contrast images and tiny fonts for those with low vision. "During my experience online with accessibility issues, I have struggled with a lot of different things," said Kevin Cao, a visually impaired tech support employee for the New York Institute for Special Education. Examples include the lack of voice descriptions for images and text. A few of his favorite sites are YouTube and Applevis.com, a community website for low-vision Apple users, which he said have "clear label buttons, links and heading recognition." As for how he thinks Facebook is doing, he believes it's good for now but could use improvement. "I would like to add more picture descriptions with faces, people," he said. That's the same feedback Wieland heard as well. "They want to know what a person's hair is like, et cetera.
There's an Empathy Lab located in the middle of the Facebook campus that's partially designed to simulate disabilities to engineers and developers. On display are assistive technologies such as a Braille display for phones, a computer that uses eye tracking for control as well as screen-reading software. "It allows them to play with some of the technologies that people use day in and day out," said Wieland. "It makes it feel real and close to home."