Neural networks and deep learning (1): Neurons and neural networks

This paper summarizes some contents from the 1th chapter of Neural Networks and deep learning.

Catalogue

1. Perceptual device
2. S-type neurons
3. The architecture of the neural network
4. Using neural networks to recognize handwritten numbers
5. Towards Deep learning

Perceptron (perceptrons) 1. Fundamentals

Perceptron is an artificial neuron.

A perceptron accepts several binary inputs: X1,X2, ..., merged one binary output:

The mathematical model is as follows:

The perceptron can be thought of as a device that makes decisions based on weights.

 2. Sensors and non-gate The above simple weights and biases can be implemented with a non-gate. input 00, (−2) ∗0+ (−2) ∗0+3=3, Output 1input 11, (−2) ∗1+ (−2) ∗1+3=−1, output 0 because it is a general operation with non-gates, we can use the Perceptron network to calculate any logic function completely.  3. Depth of sensor features

Input: The value is 0 or 1.
Output: The value is 0 or 1.

S-type neuron, Sigmoid neurons (Sigmoid neurons)1. Reasons for introducing S-type neurons

S-type neurons are similar to perceptron, but there is a good feature in S-type neurons: minor changes to weights and biases only cause small variations in output. This feature is important for the self-learning process of neural networks.

Because for a network consisting of multiple perceptron, the following problems exist:

A slight change in weight or bias on a single perceptron in this network sometimes causes the output of that perceptron to be completely flipped, such as 0 to 1. That flip could then cause the rest of the network's behavior to change completely in an extremely complex way.

and S-type neurons can overcome the above problems.

 2. Similarities and differences with Perceptron

Input: You can take any value from 0 to 1, and only 0 or 1 is different from the sensor.
Output: Output is not 0 or 1, but any value between 0 and 1, which is determined by σ (wx+b), Σ is called the S-type function.

Neural Network Architecture (the architecture of neural networks)1. Terminology

Input layer, output layer, hidden layer

For historical reasons, although it is composed of s-type neurons rather than perceptron, this multilayer network is sometimes called a multilayer perceptron or MLP.

2. Design of the networkinput layer and output layer according to the specific problem of better design, hidden layer design needs a certain experience, rules.

For example, suppose we try to determine whether an image of a handwritten number is written as "9". Naturally, we can encode the intensity of the image pixel as the input neuron to design the network.
If the image is a grayscale image of 64 64, then we will need 4096 = 64*64 input neurons, each of which takes a suitable value between 0 and 1. The output layer only needs to contain 1 neurons, when the output value is less than 0.5 means "the input image is not a 9", the value greater than 0.5 means "the input image is a 9."

using neural networks to recognize handwritten numbers1. Network Architecture

Identify with the following neural networks:

Input layer: 28x28 = 784 neurons. Each neuron represents the value of one pixel: 0.0 full white, 1.0 full black.
Hidden layers: One-layer, n-neurons, examples of n=15
Output layer: 10 neurons representing the possible 0~9 10 digits of handwritten numerals. For example: The first neuron (representing 0) output value = 1, the other <1, the number is recognized as 0.

2. What is the hidden layer doing?

One possible explanation: Assume that the 1th neuron of the hidden layer is only used to detect the presence of the following image:

each neuron in the hidden layer learns different parts: Decision:   toward deep learning (toward Deepin learning)1. Deep Neural Networks

Networks that contain multilayer structures-two layers or more hidden layers-are called deep neural networks.

For a very complex problem (such as whether the image has a face), the deep neural network can be solved by a series of multi-layered structures, decomposing the original problem into individual sub-problems, and finally decomposing into a very simple question that can be answered on a single pixel level.

At the front of the network layer, it answers a very simple and unambiguous question about the input image, at the back of the network layer, it builds a more complex and abstract hierarchy.

2. Deep neural networks vs. Shallow networks

Deep neural networks are a bit like traditional programming languages that use modular design and abstract ideas to create complex computer programs.

The contrast between deep neural networks and shallow networks it's kind of like comparing a programming language with a function call to a thin language that can't be called.

TsianlgeoSource: http://www.cnblogs.com/tsiangleo/This article copyright belongs to the author and the blog Park, welcome reprint, without consent to retain this paragraph, and in the article page obvious location to the original link. Welcome to correct and communicate.

Neural networks and deep learning (1): Neurons and neural networks