Neural network and deep learning article One: Using neural networks to recognize handwritten numbers

Source: Michael Nielsen's "Neural Network and Deep leraning"

This section translator: Hit Scir master Xu Zixiang (Https://github.com/endyul)

Disclaimer: We will not periodically serialize the Chinese translation of the book, if you need to reprint please contact [email protected], without authorization shall not be reproduced.

"This article is reproduced from" hit SCIR "public number, reprint has obtained consent. "

1. Using neural networks to recognize handwritten numbers

• Perception Machine

• Sigmoid neurons

• Structure of neural networks

• Using simple network structure to solve handwritten digit recognition

• Learning parameters by gradient descent method

• Implement our network to classify numbers

• About Deep Learning

2. How the inverse propagation algorithm works

3. Learning method of improving neural network

4. Neural network can calculate visual proof of arbitrary function

5. Why the training of deep neural networks is difficult

6. Deep learning

The human visual system is a miracle in the world. Consider the following handwritten numbers:

Most people can easily identify the figure on the graph is 504192. Behind this seemingly simple process is actually very complex. In each hemisphere of our brain, there is a part called the primary visual cortex (primary visual cortex), also known as V1. It has 140 million neurons and contains tens of billions of neurons connected. However, the human visual system relies not only on V1, but also on the concerted work of the entire visual cortex--v2, V3, V4 and V5 to achieve complex image processing tasks. Our brains are like a supercomputer, and after billions of years of evolution, we are able to understand the world of vision in a great way. Recognizing handwritten numbers is not a simple task, but humans are extremely good at understanding what the eye sees, and almost all of this work is done unconsciously, so we don't usually realize how difficult it is for our vision system to solve the task.

When you attempt to implement a computer program that identifies similar numbers, you will gradually become aware of the difficulty of visual pattern recognition. A thing that would have seemed so simple to humans suddenly became extremely difficult. "There is a circle in the upper part of the number 9, and a vertical line in the lower right." This kind of human intuition of shape recognition is difficult to represent in algorithms. When you try to define clear rules of recognition, you will quickly be plagued by a whole bunch of special cases. There seems to be no hope of resolution.

Neural networks (neural Networks) use a different approach to solve this problem. Its idea is to use a large number of handwritten figures, also known as training samples,

Learn from these training examples and build a system. In other words, neural networks use these examples to automatically learn the rules for recognizing handwritten numbers. Moreover, as the training sample increases, the neural network can learn more information and improve its accuracy. So, even though I've only given 100 training numbers above, maybe we can build a better handwriting recognizer using thousands, tens, billions of training samples.

In this chapter, we will implement a neural network of computer programs to learn and recognize handwritten numbers. Although this program is only 74 lines, and does not use any special neural network library, but it can be without any human intervention, to achieve more than 96 of the handwritten digit recognition accuracy rate. In the following chapters, we will further refine our approach to achieve an accuracy rate of more than 99. In fact, the best commercially available neural network is good enough to be used by banks to process cheques and be used by post offices to identify addresses.

We'll focus on the recognition of handwritten numerals, because it's basically the best model for learning neural networks, because it's a few pain points: it's challenging, it's not a simple project to recognize handwritten numbers, and it's not very difficult, and it doesn't need a very complicated approach, Or a lot of computing resources. And it is the foundation for higher-level technology, such as deep learning (learning). So, the problem of handwritten digital recognition runs through this book. In the latter part of this book, we will discuss how to apply these ideas to other issues of computational vision, even speech processing, natural language processing, and other areas.

Of course, the main thrust of this chapter is to implement a program to recognize handwritten numbers, so the content of this chapter will be much less! In fact, in this process, we produce many key ideas about neural networks, including two important artificial neurons (perceptron and sigmoid neurons), and standard learning algorithms for neural networks called Random gradient descent (stochastic Gradient descent). In this book, I focus on explaining why the problem can be solved and for you to build intuition about neural networks. Although this will take longer to discuss than just showing the basic theory process, it is valuable for your in-depth understanding. The receipt from this is, at the end of the chapter, a way to understand what deep learning is and why it can work.

• "Hit Scir" public number

• Editorial office: Guo Jiang, Li Jiaqi, Xu June, Li Zhongyang, Hulin Lin

• Editor of the issue: Xu June

Neural network and deep learning article One: Using neural networks to recognize handwritten numbers