Microsoft has trained XiaoIce's artificial intelligence system to read the image and generate Chinese poems describing what is depicted on it. This is reported by The Next Web.
The system consists of two neural networks. One of them recognizes the details in the picture and selects keywords, and then generates a poem. The second part evaluates the total. The algorithm received a set of instructions from the researchers and worked until the best result was achieved. If it did not suit the researchers, they changed the instruction set and restarted the system.
For example, for such an image, the algorithm generates a poem:
Example Image for XiaoIce
Wings hold rocks and water lightly
in the loneliness
Stroll the empty
The land becomes soft
According to scientists, modern Chinese poetry requires great imagination and creative use of language, which is a difficult task even for a person.
To determine the quality of the program, the researchers conducted experiments, where they offered people to choose between the poems of the Microsoft bot and other algorithms. In the overwhelming majority of participants chose the first option.