Introduction to Neural networks (serial one)

  . The artificial intelligence technology in game programming
(serial one)

Introducing neural networks in normal language

(Neural Networks in Plain 中文版)

Because we do not have a good understanding of the brain, we often try to use the latest technology as a model to explain it. When I was a child, we all believed that the brain was a telephone switch. (Otherwise, what else could it be?) I also saw the famous British neurologist Sherrington the work of the brain as an interesting analogy to a telegraph. Earlier, Freud often likened the brain to a hydraulic dynamo, and Leibniz likened it to a mill. I also heard people say that the ancient Greeks imagined brain function as a slingshot.                          Clearly, the current metaphor for the brain is probably a digital computer. R.searle [1]

 

Introduction to Neural network (Introduction to Neural Networks)
For a long time, artificial neural network was a completely mysterious thing to me. Of course, I've read about them in the literature, and I can describe their structure and working mechanism, but I haven't been able to "aha." "A sound, as in the sense that a difficult concept in your mind is fortunate to suddenly be understood." My head seems to have been pounding with a hammer, or, like the movie Animal House (the Chinese film called "Animal Room"), the one screaming in agony, thank you, give me one more. "The poor fellow did that. I can't translate the mathematical concepts into practical applications. Sometimes I even wanted to grab the authors of all the neural network books I read, tie them to a tree, and shout to them loudly, "Don't give me any more math, just give me something real." ”。 But needless to say, it's never going to happen. I have to fill this void myself ... That's the only thing I can do under that condition. I'm starting to do it. < a smile >

a few weeks later, on a beautiful day, when I was on vacation at the seaside of Scotland, my mind was suddenly struck by a shock as I peered across a mist of narrow bays. Suddenly realized how the artificial neural network works. I got "aha." "Feeling out of it. But I have a tent and a sleeping bag and a half box of cornflakes, and no computer allows me to quickly write some code to confirm my intuition. Arghhhhh. It was then that I thought I should buy a laptop. Anyway, a few days later I came home, and I immediately let my fingers fly on the keyboard. A few hours later my first artificial neural network program was finally compiled and run, and worked very well. Naturally, the code is a bit messy and needs to be sorted out, but it does work, and, more importantly, I know why it works. I can tell you that I was a very, very proud person that day.

I hope that's what this book delivers to you, "aha." Feel When we finish the genetic algorithm, you may have a bit of a feeling, but you want this feeling to be wonderful, it is necessary to wait to complete the neural network part of the whole.


Biological Neural network-Brain (A biological Neural network–the Brain)

your brain is a gray, custard-like thing. It does not work as a CPU in a computer, using a single or few processing units. If you have a corpse that is freshly preserved in formalin, you can see the familiar brain wrinkles by using a saw to carefully cut its skull and remove the skull. The outer layers of the brain, like a big walnut, are all wrinkled [fig 0 left], and this layer of tissue is called the cortex (cortex). If you use your fingers to put your whole brain out of your skull again, to get a surgeon's scalpel and slice the brain, you'll see two layers of the brain [figure 0 right]: the gray outer layer (which is the source of the word "gray matter"), but the fresh brain without formalin fixation is actually pink. ) and white in the inner layer. The gray layer is only a few millimeters thick, which tightly compresses billions of tiny cells called neuron (nerve cells, neurons). The white layer, which occupies much of the cortex under the cortex, is made up of countless connections between nerve cells ( but no nerve cells themselves, as on the back of the printed circuit board, with only component lines, without the element itself ). The cortex is wrinkled like a walnut, which can plug a large surface area into a smaller space. This can accommodate more nerve cells than the smooth cortex. The human brain contains about 10G (10 billion) of such tiny processing units, and an ant has about 250,000 brains.

Table 1 below shows the number of nerve cells in humans and several animals.

table L person and number of neurons in several animals animal   species Number of nerve cells (order of magnitude) snail 10,000 (=104) Bee 100,000 (=105) Bee Sparrow 10,000,000 (=107) Mouse 100,000,000 (=108) Human 86,000,000,000 (=1011) Elephant 100,000,000,000 (=1011)

 

1 brain hemispheres like half a walnut. 2 Cortex is made up of gray matter. Fig. 0 The shape and slice of the brain

 

FIG. 1 Structure of nerve cells

Within the first 9 months of human life, these cells were created at an alarming rate of 25,000 per minute. Nerve cells are very different from any other type of cell in the human body, and each nerve cell has a wire-like axon (axon) that stretches sometimes to a few centimeters [2] to transmit signals to other nerve cells. The structure of the nerve cells is shown in Fig. 1. It consists of a cell body (soma), some dendrites (dendrite), and an axon that can be very long. The neuron body is a star-shaped sphere with a nucleus (nucleus). The dendrites are grown in all directions by the cell body and can be used to receive signals. Axons also have a number of branches. Axons contact the dendrites of the branches (terminal) and other nerve cells to form the so-called synapse (Synapse), a nerve cell sending signals to other nerve cells through axons and synaptic synapses.

each nerve cell is connected to about 10,000 other nerve cells through its dendrites. This makes it possible to connect all the nerve cells in your mind with a total of l00,000,000,000,000. This is more than a 100-gigabit modern telephone switch. So it's no wonder why we sometimes have headaches.

Interesting facts. It has been estimated that if the axons and dendrites of all neurons in a person's brain are connected in sequence and drawn into a straight line, they can be connected from the Earth to the moon and back to Earth from the moon. If the axons and dendrites of the neurons in the brain of all the people on Earth are connected, they can stretch to the nearest galaxy.

Nerve cells use electrical-chemical processes to exchange signals. The input signals come from other nerve cells. The synapses of these nerve cells (i.e. terminals) and the dendritic synapses of the neurons form a synapse (synapse), and the signals enter the cells from synapses on the dendrites. How the signal actually travels in the brain is a very complex process, but for us it is important to think of it as a modern computer, using a series of 0 and one to operate. That is, the brain's nerve cells also have only two states: excitement (fire) and not excited (ie inhibition). The intensity of the transmit signal is constant, and the change is only frequency. Nerve cells use a method we do not know to add all the signals that come in from the tree, and if the sum of all the signals exceeds a certain threshold, the neurons will be stimulated into the excited (fire) state, and an electrical signal will be sent out to the other nerve cells via an axial burst. If the sum of the signals does not reach the threshold, the nerve cells will not get excited. Such an explanation is a bit simplistic, but it has been able to satisfy our purpose.

It is because of the huge number of connections that make the brain incredibly capable. Although each nerve cell works only about 100Hz of frequency, because each nerve cell works in parallel in the form of an independent processing unit, the human brain has the following very obvious characteristics:

To achieve unsupervised learning. One of the incredible truths about our brains is that they can learn by themselves without the supervision of a mentor. If a nerve cell is stimulated at a high frequency for a period of time, the intensity of the connection between it and the neurons in the input signal changes in a certain way, making the nerve cell more susceptible to excitement the next time it is stimulated. This mechanism was elaborated more than 50 years ago by Donard Hebb in his book Organination of Behavior. He wrote:

"When a axon of nerve cell a repeats or permanently stimulates another nerve cell B, one or both of the two neurons will have a growth process or metabolic change that will increase the efficiency of a cell that stimulates B cells."

 

Conversely, if a nerve cell is not stimulated for a period of time, the validity of its connection will slowly decay. This phenomenon is called plasticity (plasticity).

The damage is redundant (tolerance). Even if a large part of the brain is damaged, it can still perform complex work. A famous experiment is to train mice to walk in a maze. Then, the scientists removed part of the brain, and more and more of it. They found that even if a large part of the brain of a mouse is removed, they can still find a path in the maze. This fact proves that in the brain, knowledge is not kept in a local place. Other tests have shown that if a small part of the brain is damaged, the nerve cells can recreate the damaged connection. "I think we can see this in humans too: cerebral infarction patients with large brain necrosis due to cardiovascular disease or other causes can recover after a period of rehabilitation, especially when memory is not impaired," he said. 】

Processing information is highly efficient. The transmission of electrical-chemical signals between nerve cells is very slow compared to the data transmission of a CPU in a digital computer, but because neurons use parallel working methods, the brain can process large amounts of data at the same time. For example, the brain's visual cortex can be completed in about 100ms of time as it processes an image signal that is entered through our retina. Considering that the average working frequency of your nerve cells is only 100hz,100ms time means that only 10 steps per second are completed. Think about how much data we have on our eyes and you can see that it's an incredibly big project.

Good at induction and push wide . The brain and the digital computer, one of the things it is very good at is pattern recognition, and can be based on the familiar information to generalize (generlize). For example, we can read the text written by others, even though we have never seen anything written by him before.

It is conscious. Consciousness (consciousness) is a topic widely and enthusiastically debated by neuroscientists and AI researchers. A great deal of literature has been published on this subject, but there has not yet been a substantial unified view of what consciousness actually is. We cannot even agree that only humans have consciousness, or that the relative of human beings in the animal kingdom is conscious. is a gorilla conscious. Is your cat conscious? Was the fish you ate in the dinner last week conscious?

 

Therefore, an artificial neural network (Artificial Neural Network, Ann) referred to as Neural Network (NN) is to simulate this large amount of parallelism under the existing scale of modern digital computers, and when this is done, So that it can show many traits similar to those of a human or animal brain. Let's take a look at their performance below.

 

"1" quoted John R.searle's "Minds,brain and Science", P44. John R.searle, a contemporary American philosophy-psychologist, has written a wealth of books on the nature of the brain and consciousness.

" 2" This refers to all the nerve cells in the brain, otherwise, if it is the dominant body parts of the nerve cells, it can be far longer, such as more than one meter.

" 3" A reader pointed out: the number of neurons in the brain of the original book said 10G (= 10 billion) should be wrong, should be 86 billion, that is, and the elephant 100G has the same order of magnitude.