Neural networks used in machine learning (iv)

A simple example of machine learning

It's a very simple kind of neuralnet and it's gonna be learning to recognize digits and you gonna be able Weights evolved as we run a very simple learning algorithm.

A very simple learning algorithm for traing a very simple network to recognize handwritten shapes. The network has both layers of neurons. It has input neuron whose activities represent the intensity of pixels, and output neurons, whose activities represent the Class.

Consider a neural network with both layers of neurons.
–neurons in the top layer represent known shapes.
–neurons in the bottom layer represent pixel intensities.

What do we ' d like is so when we show a paritcular shape. The output neuron for that shape get active. If a pixel is active what it does are its votes for particular shapes. Namely the shapes that contain that pixel.
a pixel gets to vote if it had ink on it.
–each inked pixel can vote for several different shapes.
the shape that gets the most votes wins.

First step: We need to decide how to display weights, it seems natural to write the weights on the connection between input unit and output unit. But, we is never to is able to see what is going on if we get that. We need a display in which we can see the values of thousands of weghts. We make a little map with the. and int that map we show the strenght of connection coming from each input pixel at the location of the. And we show the strength of connection by using

A black or white blob, whose area represents the magnitude of the weight and the color representing the sign. So the initial weights it you see there is just small random weights

Now, we gonna do's show that network some data and get it to learn weights that's better than the random weights.

The the-the-gonna look was when we show it a image, we are going to increment the weights from the active pixels in the Image to the correct class.

If We just did then, the weights could get only bigger and eventually every class, so we need some it's the keeping the Wei Ghts under control.

What we gonna does is we'll also gonna decrement weights from the active pixels to whatever class the network guesses.

The weight of the initialization:

In training:

Eventually:

Look at the weights for each unit, sort of like a number template.

Why the simple learning algorithm is insufficient
a The layer network with a winner in the top layer are equivalent to have a rigid template for each shape.
, Haven Winner is the template, which has the biggest overlap with the ink.
the ways in which hand-written digits vary is much too complicated to being captured by simple template matches of whole s Hapes.
–to capture all the allowable variations of a digit we need to learn the features that it's composed of

　

Neural networks used in machine learning (iv)