From sensor to Neural Network

From sensor to Neural Network

Perception Machine

The sensor was invented by science and technology Frank Rosenblatt in and was influenced by Warren McCulloch and Walter Pitts's early work. Today, the use of other Artificial Neuron models is more common-in this book, and more modern neural networks work, primarily using a neuron model called S-type neurons.

How does the sensor work? A sensor receives several binary inputs and generates a binary output:

In the example, the sensor consists of three inputs ,. Generally, more or less input is allowed. Rosenblatt proposed a simple rule to calculate the output. He introducedWeight,The real number of the input importance to the output. The output of neurons, 0 or 1, is less than or greater than the sum after weight distribution.ThresholdDecide. Like the weight, the threshold is a real number, a neuron parameter. Use more accurate algebra:

S-type neuron

If small changes to the weight (or offset) can only cause small changes to the output, we can use this fact to modify the weight and offset, let our network behave as we want. The problem is that this does not happen when our network contains a sensor. In fact, a slight change in the weight or offset of a single sensor in the Network may cause the output of the sensor to be completely flipped, for example, 0 to 1. Such a flip may cause the behavior of other networks to change completely in a complicated way. This makes it difficult to gradually modify weights and offsets to bring the network closer to expected behavior.

We can introduce a new artificial neural network called S-type neurons to overcome this problem. The S-type neuron and sensor are similar, but minor changes that are modified to weights and offsets only cause minor changes in the output. This is critical to learning neural networks.

Let me describe the S-type neuron. We describe S-type neurons in the same way as the sensor:

Just like a sensor, S-type neurons have multiple inputs, but these inputs can take any value in 0 and 1, not just 0 or 1. For example, 0.638 ...... Is a valid input of an S-type neuron. Similarly, S-type neurons have a weight for each input, and a total offset, B. However, the output is not 0 or 1. Instead, it is now called an S-type function and is defined:

Put them together to make it clearer that the output of an S-type neuron with input weight and offset B is:

Initially, S-type neurons and sensors differ greatly. In fact, there are many similarities between sensor and S-type neurons. The comparison between S-type neurons and sensor is shown below:

A big difference between the sensor and S-type neurons is that S-type neurons not only output 0 or 1. It can output any real number between 0 and 1, so such as 0.173... And 0.689... Is a reasonable output.

For historical reasons, although it is composed of S-type neurons rather than sensors, this multilayer network is sometimes calledMulti-layer SensorOrMLP.

So far, the neural networks we have discussed are all input from the above layer as output from the next layer. This network is calledFeed-forwardNeural Networks.

Learning using gradient descent Algorithms

We hope there is an algorithm that allows us to find weights and offsets so that the output y (x) of the network can fit all the training input x. To quantify how we achieve this goal, we defineCost function:

Here, W indicates the set of weights in all networks, B indicates all offsets, N indicates the number of input data for training, and a indicates the vector output when input is X, the sum is performed on the input x of the total training.

From sensor to Neural Network