Earlier this year, Facebook launched a new feature for iOS screen readers called automatic alt text. Powered by artificial intelligence and machine learning, it recognizes the objects in a photo and then describes them, which is incredibly handy for users who are blind. It arrived on Android soon after, and two weeks ago, the social network officially made the tool available in 20 other languages. These are just a few recent examples of how Facebook is attempting to make its products -- as well as the internet as a whole -- more accessible to those with disabilities.
It all started five years ago when Jeff Wieland, who worked on the company's user research team at the time, discovered that there were people with disabilities who were having a rather terrible experience with Facebook.
For example, he found out that screen readers -- software employed by blind users to know what's happening on a page -- would interpret a button on a site as simply, well, "button." It would have no information on what the button does. "This is a really simple example," he said. "For an engineer, adding a label to a button is one line of code. But it changes everything."
The problem, he said, was that developers were not designing the web experience with screen readers in mind. It seems like a "No, duh" explanation, but as Wieland told me, accessibility is not a subject that's often taught in computer science classes. "It's not part of the core curriculum. You have people graduating from great programs, but who have no exposure to accessibility. It's a real tragedy." Indeed, he said that almost all incoming engineers have no prior exposure to the subject.
The community that needs this tech is a large one. According to the World Health Organization, about 285 million people worldwide are visually impaired, and of those, around 39 million are fully blind. And that's just those who suffer from vision loss. "Accessibility is about more than that; it's about serving those who have varying ability," said Wieland. "I discovered there was this vast community we were underserving."
In response to this, Wieland set about creating an accessibility team at Facebook with the company's blessing. "Philosophically, people got it," he said, adding that the goal aligns with Facebook's mission for connecting the world. "But the big question was, how do we do this?"
Over the years, the team has worked to raise awareness of accessibility issues and assistive technologies within the company.
It started out with two people, but now it's grown to more than 20. It's a cross-functional group made up of people in design, user research, data science, operations and more. Remember those unlabeled buttons? Now, Wieland said, that's a mostly solved issue (though, given the scale of Facebook, there are probably still a few unlabeled ones here and there). The key factor? Working with existing teams on Facebook to integrate accessibility across the board.
That automatic alt text photo captioning, for example, required the assistance of Facebook's artificial intelligence team.
"There's this concept of alt text for photos, which used to be provided by the site owner or web developer," said Wieland. "But now the internet is all photos ... The only way we could really provide the text for photos at our scale, would be to use computers to do this." Fortunately, Facebook's AI team had been working on computer vision for at least a year and a half already and had the technology and expertise to work on the project.
According to Crista Earl, the director of Web Services for the American Foundation for the Blind, most sites have incorrect or poorly designed forms. "For example, when trying to book a flight on a travel site ... the manner of selecting choices poses an obstacle to users of assistive technology." Inadequately labeled videos and images are also a problem. Other common issues are low-contrast images and tiny fonts for those with low vision.
"During my experience online with accessibility issues, I have struggled with a lot of different things," said Kevin Cao, a visually impaired tech support employee for the New York Institute for Special Education. Examples include the lack of voice descriptions for images and text. A few of his favorite sites are YouTube and Applevis.com, a community website for low-vision Apple users, which he said have "clear label buttons, links and heading recognition."
As for how he thinks Facebook is doing, he believes it's good for now but could use improvement. "I would like to add more picture descriptions with faces, people," he said. That's the same feedback Wieland heard as well. "They want to know what a person's hair is like, et cetera.
A human can describe a full context, but we're just returning a list of objects ... Hopefully in the future we can tell a richer narrative."
Facebook's work on accessibility encompasses more than just vision loss. Back in 2014, the company added the ability for users to provide closed captions to videos. It's currently working on a more automated captioning system with the help of Facebook's language-and-speech-technology group.
"The real lion's share work that we do is to prepare our teams to write and design applications that are accessible, to make it foundational," said Wieland. "We want to embed accessibility into all of the steps along the way, from ideation to actually shipping the product." To help this along, Facebook has now integrated accessibility to the QA process that Wieland says has been useful. "It helps push accessibility earlier in the process, without waiting to hear user feedback."
As part of the education process, the accessibility team has done a lot internally to raise awareness of the issue.
There's an Empathy Lab located in the middle of the Facebook campus that's partially designed to simulate disabilities to engineers and developers. On display are assistive technologies such as a Braille display for phones, a computer that uses eye tracking for control as well as screen-reading software. "It allows them to play with some of the technologies that people use day in and day out," said Wieland. "It makes it feel real and close to home."
Thanks..........
