DeepMind to Develop Neural Arithmetic Logic Units

According to researchers, new architecture allows neural networks to perform number-related tasks more efficiently
16 August 2018   676

A team of researchers from DeepMind has developed a new architecture that allows neural networks to perform number-related tasks more efficiently. It involves the creation of a module with the basic mathematical operations described in it. The module was named Neural Arithmetic Logic Unit (NALU).

Scientists have noticed that neural networks are rarely able to successfully generalize concepts beyond the data set on which they were trained. For example, when working with numbers, models don't extrapolate the data to high-order quantities. After studying the problem, the researchers found that it also extends to other arithmetic functions.

When standard neural architectures are trained to count to a number, they often struggle to count to a higher one. We explored this limitation and found that it extends to other arithmetic functions as well, leading to our hypothesis that neural networks learn numbers similar to how they learn words, as a finite vocabulary. This prevents them from properly extrapolating functions requiring previously unseen (higher) numbers. Our objective was to propose a new architecture which could perform better extrapolation.

Andrew Trask

Lead researcher, NALU

The structure with NALU suggests predetermining a set of basic, potentially useful mathematical functions (addition, subtraction, division and multiplication). Subsequently, the neural network itself decides where these functions are best used, rather than figuring out from scratch what it is.

The tests showed that neural networks with a new architecture are capable of learning tasks such as tracking time periods, performing arithmetic operations on image numbers, counting objects on a picture, and executing computer code. 

In March 2018, DeepMind introduced a new paradigm for learning AI models. Unlike standard methods, it does not require a large set of input data: the algorithm learns to perform tasks independently, gradually mastering the necessary skills.

Microsoft to Use AI to Create Human Voice

Synthetic voice is nearly indistinguishable from recordings of people
27 September 2018   457

Researchers from Microsoft recorded computer voice, imitating human speech. To overcome the difficulties of the traditional model, they used neural networks for speech synthesis. Microsoft promises to provide support for 49 languages ​​and the ability to create unique voices for the needs of companies in the near future.

Synthesis of speech with the help of neural networks involves comparing the stress and length (so-called prosody) of the speaker's speech units, as well as their synthesis into a computer voice. In systems of traditional speech synthesis, prosody is divided into acoustic and linguistic analysis, controlled by various models. As a result, the speech is noisy and indistinct. Representatives of Microsoft argue that in the model of neural synthesis two stages are combined into one, so the voice sounds like a real one.

The developers are convinced that the synthesis of speech with the help of neural networks will make it more natural to communicate with virtual interlocutors and assistants. Moreover, it will enable you to convert e-books into audiobooks and will allow you to change the scoring of built-in navigators.

Microsoft Neural TTS
Microsoft Neural TTS

Azure computing power is available for real-time use, and Azure Kubernetes is responsible for this. Simultaneous application of neural synthesis of speech together with traditional speaks about expansion and increase of availability of service. At the moment, there are a female voice named Jessa and a man named Guy.

Microsoft is competing in speech recognition and synthesis technologies with Google, which updated its services in late August 2018. Google Cloud announced the release of a stable API for the synthesis of speech Cloud Text-to-Speech with the experimental function of audio profiles and support for several new languages.