Facebook is working on AI that can tell what is in your photos
Facebook is working on artificial intelligence (AI) feature that can understand whatโs going on in your photos, which in turn would help your Newsfeed perform better. This feature will allow it to answer questions about a photo, a feature aimed at helping blind people “see” images uploaded to the social network.
Speaking at Web Summit, Facebookโs CTO, Mike Schroepfer, detailed a new system the company has been developing to understand whatโs happening in photos and translate that into natural languageย elements.
At Web Summit conference in Dublin, Ireland, Facebook’s Chief Technology Officer (CTO) Mike Schroepfer, explained the new system which FB been developing to understand whatโs happening in photos and convey that into natural languageย elements.
“The beauty of AI systems is that we can get enough scale to deploy this to everyone on the planet,” said Schroepfer in a briefing for reporters.
Earlier this year, the social media giant unveiled a technology called “Memory Networks” which allows a machine to perform sophisticated questioning and answering. He showed an example where a computer was fed the synopsis of the “Lord of the Rings” and was able to answer questions about the story.
The goal is to help computers to “understand language more like a human would with context, instead of rote ones and zeroes memorization like a machine.”
One illustration displayed a photo in Newsfeed of a man skateboarding, which the computer deconstructed and explained in words to understand what was happening.
It broke it down into what it saw โa skateboard, a man, a trick, his skateboardโ and constructed a description of what was happening.
Schroepfer said that โwhen the machine understands attributes of an imageโ it is โeasy to understand the applications to Facebook.โ
The other illustration using the same technology showed a prototype application that was targeted at visually impaired users. A photo is fed into the app, and you can ask questions using speech, which the AI will describe in words.
Another effort uses “predictive learning” to study images and make informed predictions of what will happen next, such as whether a stack of blocks is stable enough to stand or fall over.
The social networking site, which has more than 1.5 billion global users, has also sponsored efforts to build drones and laser networks to help reach billions more people who currently do not have access to Internet. Schroepfer said Facebook can use artificial intelligence to understand the huge amount of information users upload and predict which messages they will want to see.
“We need systems that can help us understand the world and help us filter it better,” he added.
Facebook is pursuing a world where it only displays to you things that you are interested in. The technology would allow the company to โdo the best job of showing you what you want in your feed by understanding photosย based on the pixels.โ