Neural network Turing (neural Turing machines, NTM)

Recently, the Google deep Mind team put forward a machine learning model, and a particularly tall on the name: Neural network Turing machine, I translated this article for everyone, translation is not particularly good, some sentences did not read clearly, welcome everyone to criticize

Original paper Source: Http://arxiv.org/pdf/1410.5401v1.pdf.

All rights reserved, prohibited reprint.

Neural network Turing

Alex Graves [email protected]
Greg Wayne [email protected]
Ivo Danihelka [email protected]

Google DeepMind, London, UK

Abstract This paper enhances the ability of neural networks by introducing an external memory (external memory) that interacts with the attention program. The new system can be analogous to a Turing or von Neumann system, but each component is tiny and can be trained efficiently using gradient descent. Preliminary results show that neural network Turing can infer (infer) simple algorithms from input and output samples, such as copy, sort, and recall.

1. Introduction Computer programs use three basic mechanisms in the process of performing computational tasks (Von Neumann, 1945): Elementary operations (such as arithmetic operations), logical control flows (branching loops), and read-write memory. Although there has been widespread success in modeling complex data, modern machine learning theory has generally neglected the use of control flows and memory. Recursive neural networks stand out because of their ability to learn and complex transformations of data with time attributes. Further, RNN is also proven to be Turing's fully equivalent   (siegelmann and sontag, 1995), so as long as the rational modeling, it can simulate any computational process. But theoretically feasible does not mean that practice is easy to achieve. To this end, we have enhanced the ability of the standard recursive network to simplify the solution of algorithmic tasks. This enhancement is primarily dependent on a larger, addressable memory, and, similarly, the Turing machine is using an infinite storage to bring the enhanced and poor state machines, so we call this new device "neural network Turing." Unlike Turing, NTM is a micro computer that can be trained with gradient descent and is a useful mechanism for learning programs. In the human recognition ability, the process that shares the most similarity to algorithmic  operation is known as  "Working memory." In neurophysiology, the operating mechanism of working memory is unclear, and it can be understood as a short-term storage of information and a set of rules-based operations (baddeley et al., 2009), literally. In computer terminology, these rules are programs, and the stored information forms the parameters of these programs. Since NTM is designed to apply approximate rules to "fast-creating variables", it mimics a working memory system. Fast-creating variables (rapidly-created variables) (hadley, 2009) are data that can be quickly bound to a storage slot, just like a traditional computer where numbers 3 and 4 are placed in registers and then added together to get 7 (Minsky, 1967). Because the NTM architecture uses a note-taking process to selectively read and write memory, NTM uses another similar working memory model. AndMost working memory models, compared to our architecture, can learn to use his working memory without the need to introduce a fixed set of programs for symbolic data. In this paper, we begin with a brief review of work memory-related studies in the fields of psychology, linguistics and neuroscience, and artificial intelligence and neural networks. Then describe our main work, a storage architecture and attention controller, and we believe that this controller can meet the performance requirements of simple program induction (induction) and execution (execution) tasks like this. In order to test this structure, we design some problems and give the detailed test results. The final summary of this architecture is somewhat.

2. Basic research

More and better reading experience, go to the main station

Neural network Turing (neural Turing machines, NTM)