Notes on the beauty of mathematics: Natural Language Processing-from rules to statistics

The natural language processing, mainly realizes the human and the computer uses the natural language to carry on the effective communication the method and the theory , it experienced from the rule to the statistical stage, the so-called rule, refers to according to the definition grammar carries on the language processing, so-called statistics, Refers to the method of natural language processing proposed by IBM for solving speech recognition problems, based on statistics.

In the 1946, when modern computers were born, humans began to consider using computers to deal with natural languages, mainly involving two cognitive aspects: first, whether computers can handle natural languages, and second, if so, whether it handles natural language in the same way as humans. The rapid development of modern natural language processing shows that the answers to these two questions are positive.

Father of computer science Alan Turing first proposed the idea of machine intelligence, but also proposed a way to verify that the machine is intelligent: let the person and machine to communicate, if people can not judge the object of their communication is a person or machine, it means that the machine has intelligence. This is the famous Turing Test (Turing tests).

In the summer of 1956, 28-year-old John McCarthy, as well as the same age Marvin Minsky, 37-year-old Rochester and 40-year-old Shannon, 4 proposed a brainstorming session at Dartmouth College, which they call "Dartmouth Summer AI research Conference." The meeting was attended by 6 young scientists, including 40-year-old Herbert Simon and 28-year-old Allen Neuville.

    at the symposium, the 10 people discussed the unresolved problems of computer science at the time, including artificial intelligence, admission to language processing and neural networks. The reference to artificial intelligence was raised at this meeting. Of these 10 people, 4 Turing Award winners (McCarthy, Minsky, Simon and Neuville) and inventor Shannon of information theory were later developed.

    Dartmouth Conference has more than 10 Turing awards. Unfortunately, the 10 most intelligent minds in the world were struck by a one-month spark that was limited by history, and did not produce any great ideas. This is because at the time, the world's research into natural language processing has fallen into a misunderstanding.

    Rules-based natural language processing refers to the grammatical rules of natural language, part of speech and word-formation, etc., which are described using computer language. For semantic research and analysis, semantics is more difficult to express in computer than grammar. Scientists have designed a simple syntax parser for natural language sentences, hoping to solve the problem of natural language comprehension through a comprehensive generalization of natural linguistic grammar.

    But soon there was a problem, and some statements had different semantics under different usage environments. This requires a constant addition of new grammatical rules, and even if the set of grammatical rules that cover all natural language phenomena is written, it is very difficult to parse by computer. In the the 1970s, rules-based natural language processing was a bottleneck, and so many years of effort were considered a failure.

    The advent of statistical linguistics after 1970 has enabled natural language processing to regain its new life. There is a key history, IBM in order to solve the problem of speech recognition, improve the speech recognition rate at that time, the use of a statistical-based approach, which makes speech recognition from the laboratory to practical applications. After the advent of statistics-based language processing, rules-based and statistical-based debates have been around for about 15 years, and with the advent of web search and data mining techniques, it has greatly accelerated the transformation of natural language processing from a rule-based approach to a statistical-based approach., culminating in a rule-based natural language processing that won。

Notes on the beauty of mathematics: Natural Language Processing-from rules to statistics