Videos have long been the main content of social networks. However, not all people can use them due to physical limitations. Therefore, Facebook developed the VideoStory system for teaching artificial intelligence so that it “automatically tells stories”.
The basis is a system that analyzes the video and creates detailed descriptions of the video. Technically, this is a set of sentences that describe in some detail what is happening. For data processing, a recurrent neural network was used, and about 20 thousand videos and 123 thousand sentences with descriptions were made as “educational material”. To do this, select the popular video from social networks.
The neural network itself had to take into account the connection between past and future events on the video, for which the researchers added an understanding of the context.
At the first stage, the system generated descriptions, although the proposals were often not related. However, potentially VideoStory can serve as a good guide for creating future systems describing what is happening on the screen. So far, the project itself has not been declared as commercial, therefore it is not known when it will be released as a finished application.