Every image on Facebook and Instagram is complemented by an AI-generated caption. Alt text is a metadata field describing the image on the web page.

For example, “a man stands near the forest with a horse,” “a dog in a boat.” Such descriptions help people with poor eyesight to understand what is shown in the photo. Often the authors of posts add them manually, but most users simply ignore them. Therefore, social networks use automatic descriptions for pictures.

Facebook has been using Automatic Alt Text technology since 2016, which is a very long time, considering how rapidly machine learning is developing. Subsequently, the system has improved many times, in the latest update it became possible to create more detailed descriptions on demand.

Now this technology recognizes ten times more elements than before, about 1,200. And the description text itself will be more detailed. It used to be: “two people near the building”, now – “a selfie of two people near the Eiffel Tower.” “Perhaps” is added to the description to avoid misconceptions.

Of course, not everything is perfect, for example, in this image, the system simply reports the location of people and objects.

Obviously people are above the drums and hats are above people. But such information is not needed at all to understand what is shown in the photo. The system can give something like the following description: “house, some trees and a mountain.” From these words, it is difficult to understand what is depicted. House on the mountain or in front of it? Trees in front of or behind the house, or on a mountain in the background?

Without these details, it will not be possible to accurately describe the photo, even if the general idea is already clear. When a sighted person wants to examine a picture in detail, he brings it closer to the face, or zooms in. People deprived of this opportunity now have a command “create detailed description of the image”. On Android, it is activated with a long press, on iOS – with a special action.

Perhaps the new description will be clearer, for example, like this: “a house and several trees in front of a mountain with snow.”

The detailed description feature will appear on Facebook and Instagram as a test, but not in all countries at the same time. The description will be short to make it easier to translate into other languages.

Previously, Instagram started testing an option to hide the likes counter, and Facebook launched a content calendar for Instagram in Creator Studio.