From the bottom up--the new cortex data's reading the third part Nn-ann 70 years ago Reverse deduction

Part III Nn-ann 70 years ago Reverse deduction      from this part, adjust the angle of view the main learning neural network algorithm, and the biological neural network for horizontal comparison, to spy one or two.   Now NN-based AI applications are almost everywhere, the effect is good, this seemingly magical thing how to think out. The individual can not help but curious, then a disorderly find. Want to figure out a thread, who knows a dig found this line behind the story is too rich, here deliberately shared out.      In fact, the cognition of neurons is relatively early, in terms of mechanisms and functions are more inclined to biological interpretation, in this field is the most natural neuroscience and biologists. In the biological framework of the results of the rapid advance of the Second World War (Germany and Britain and the United States during World War II investment is particularly large, this is a war welfare), and gradually clear a series of micro-brain facts. Until 1943, neuroscientists McCulloch and logic scientists (he didn't even have a high school diploma when the article was published) Walter Pitts a joint paper that pulls a bunch of other academics into the big pits of brain research.     This article is called "a Logical calculus of Ideas immanent in nervous Activity" (the first time to learn the neural network algorithm of the students must SEE), One hypothesis is that the ability of a cell to connect through a network can be calculated, that is to say, "The brain is a computer calculator." It shows that a set of Turing-heads can be built on the basis of a cell, and this working pattern may be the essence of the brain's work. Further, Pitts has designed several basic connection patterns for neurons, forming a complete set of logic (similar to Russell's   and/or not) to further explain a possible functioning mechanism of the brain. Which means he found a set of frameworks for building complex computations from simple logical units of neurons. In this framework, the functional interpretation of biology is assembled by a logical framework. In the next few decades, the most important neural network algorithms are adjusted to improve on this basis. Even to go Master Alphago still have to recognize it as a pro mom. Subsequently Pitts by MIT and Norbert Wienner Fancy, recruit for postgraduate. Yes, this wienner is the author of the [cybernetics] who tortured thousands of automation students (the book is well worth reading, thin and small), the founder of the cybernetic discipline. Speaking of cybernetic, the study group that we call "cybernetics" is actually not just a matter of automatic control, in fact it is a miscellaneous group, the goal is to use mathematical logic to explain all the mechanisms of dynamic things, especially the living. According to the wienner of his ladyship, this studyThe purpose of the section is one: Unified communication, control, statistical mechanics of the core issues. (cybernetic itself is a Latin translation, the original meaning is "the helm", listen to see the people this spirit.) The important thing is that this is a bunch of sub-practitioners who are not content with purely academic research and are always trying to make these living things in other ways. So the people in it seem particularly magical, a subject filled with, mathematicians, biologists, sociologists, physicists, and even psychologists and all kinds of engineers, but none of them is called control scientists. In this kind of a miscellaneous discipline, in today's colleges and universities with a single skill specialization teaching, Automation department has become the second major of electronic department, is the teaching method Nieyibahan, Wienner if Upper nether know whether will vomit blood and die once again. Pitts naturally took part in the cybernetic organization under the leadership of Wienner. According to Wienner in the book, Pitts know Claude Shannon, at this time the Shannon has not started to engage in information theory, the main work is followed by Vannevar Bush (this cargo to do the Manhattan Project) to write papers and so on, when his graduation study is to engage in switching circuit (It's actually a digital circuit). At that time wienner his two rub together is to let Pitts to find the "brain brand Computer" with digital circuit, let Shannon give him to spread the knowledge of digital circuit. He then took Pittes to a small gathering, during which John Von neumann (the inventor of the modern computer was also a member of the organization), during which Pitts mainly gave von Neumann and Goldstone's team a "brain brand computer" The story, while also doing neuroscience science. This cross-organizational, interdisciplinary effect is good, von Neumann "Pitts's loops " used in the edvac of improvement (that is, one of the results of the article, this loops is specifically through the closed-loop network to implement memory), The basic storage System of the instruction and data of the computer is basically complete. After reading this generalist came to understand that the computer and brain structure in many aspects of similarity is not accidental, these people are mixed together. The production of cybernetic and computer actually has a fundamental influence on the basic method of studying the real problem, this time we will talk about it.      According to the theory of cybernetics, McCulloch and Pitts1947 also designed a blind reading machine, which can imitate the fourth layer of the visual cortex, but unfortunately the age can not be seen, even the paper can not be found. Later Pitts joined a team trying to figure out how neural networks work using statistics, information theory, mathematics, and biology. At the time, he felt neurons knotProcess modeling can be done by statistical mechanics in the learning formation process of the structure.  1959 year they came up with a paper on Frog Vision, which reveals the process of information transmission from visual to the brain and the pretreatment mechanism of the eyeball itself to the visual information. The data-entry information for the neuron network has been accurately observed from biological experiments, and today appears to be fruitful. With his understanding of the details of biology, Pitts to the road gradually feel despair and wienner relationship also produced a fierce contradiction, rage the doctoral thesis to burn. In the follow-up study, Pitts did not make a greater breakthrough in modeling the process of the network structure itself. Not many years, this man hung up, it is a great pity. Pitts This story line is basically here. Later in the study of NN did not follow the use of mathematical logic + biological reverse engineering basic logic goes on, and gradually separated from the biology. The neuroscience study goes back to the microscopic details.           and then nn really became Ann (artificial neural network), more in the algorithm and logic directly developed, it is not difficult to understand that today's neural network algorithm does not need to learn biology.  

Throw away the wonderful and magical stories, and take a closer look at what the legacy of this Pitts is.        1: The neuron model is given, and the Mcculloch-pitts neurons which is used by NN is often said to be MP/MCP neurons. 2: Found a complete set of basic computational logic connections.        Each specific connection form in the diagram. 3: Explains how to use these operators in open-loop structure to set up the function of calculation with these basic logical structure.        That is the Turing of the biological logic 4: Mathematically explains the recursive effect (the effect of Loop Network computing itself) if a ring-mounted connection in a network is generated. 5: The absence of time in the ring-mounted structure is revealed.            It sounds weird, and once you get into the ring logic it's just the order but the time is erased, and this view is recognized in many theories later on, but intuitively it's not easy to understand. This is sufficient for the 40 's, which can be said to be a logical "modern computer" model (though not created), and you can enter "formulas" and "data" through the design to help you complete the calculation process. In fact, this is our daily "programming", "Input" the basic mode. But one step forward, this is in comparison with the human brain, there is an essential difference in the working paradigm---the ability to grow "formulas" from "input" information. This ability is "learning", with "learning" ability of the system, only need to input data to it, it can form a matching calculation mode itself. Also in the 40 'sDonald Hebb out a set of hypotheses, called Hebbian Learning, to the effect that when two neurons are activated repeatedly, the connection will be strengthened. The Hebbian is a psychologist who does not have a complex mathematical reasoning and modeling, and Hebbian learning has been used as a tool or method in subsequent processes until the LTP hypothesis is found to be verified. This first put down not the table. Basically, the most important legacy of the 40 's NN is the basic idea of MP neurons and Hebbian learning, which has been used today.

From the bottom up--the new cortex data's reading the third part Nn-ann 70 years ago Reverse deduction