Facebook introduced Rosetta - machine learning system, which in real time extracts text from more than a billion publicized images and videos in social networks in different languages, and then recognizes their context.
Rosetta performs simultaneously two independent processes: detection of areas that can contain text, and word recognition using the Faster R-CNN convolutional neural network on the ResNet18 architecture.
The algorithm recognizes English, Arabic, Hindi, German, Spanish and other languages, including those that have horizontal right-to-left writing, diacritics and other specific characters.
In the future, the corporation will try to teach the system to recognize more languages, types of text and image templates.
Facebook is going to use Rosetta to deliver a more personalized news feed, as well as tracking spam, offensive or banned content. Now it is sorted by operators and it takes a long time.
In June, 2018, researchers from Stanford talked about a model of machine learning that could create memes in the style of "advising animals." The authors noted that on average, an "artificial" meme is difficult to distinguish from "real" in the context of the quality of the joke in it.