Mid-year records and digging pits

Recording

Hangyuan Li The big Guy said (original link)

It is also very difficult to achieve the same ability to speak on a computer. Today's technology, typically data-driven, is based on machine learning.

A single-wheel dialog has an analysis-based, retrieval-based, generation-based approach.

Well, I also feel that based on the search is low, but now the smart speakers do not seem to break through this threshold. Everyone hears the b-box of Siri, behind a boring tease programmer; Little love classmate's amazing quotations, writing corpus may be a bunch of pull foot han.

This half-year work focuses on the development of a search-based semantic understanding platform, which has just completed the third edition of Evolution in recent days.

First-generation target

At the end of last year, just came here to take over the job, the boss gave me the structure is simple pure retrieval, based on ES, the pre-compiled template written into the search engine. Every time you receive a front desk request, the ASR obtained text thrown into ES search, at the beginning of the template less than the time, the classification effect is very good, and even a little generalization ability, but when the sample amount up, the retrieval effect becomes very poor, the accuracy rate is very bad. At that time, the reason for the analysis is that ES retrieval principle is to put the text into the word segmentation, and then the TF-IDF score for each word, the formation of a vector, the matching phase may be the highest one-to-return score.

During the use of the main encountered two problems, the first is the wrong identification, because the retrieval is a template, and then refer to the template named entity extraction, there is a problem. For example, what is the weather like in Beijing today, ES usually classifies it into templates: What's the weather like today @city? In fact, the correct template is @city What is the weather today, using the wrong template to go through the sequence of detection than the algorithm needle, there will be an error. Next is the word segmentation problem, put the text into ES, if the use of the word breaker , it will be a separate Chinese, that is, the word is only English, and then to Es configured IK plug-ins basically solve the problem. But fundamentally, with third-party tools, the problems that arise are always hard to control and can only be fixed every time a vulnerability is made, and then a hole may be poked elsewhere.

Second-generation Mesh

The main problem of the first generation, we think that is based on the TF-IDF classifier, can only be recognized words, and do not have the semantics. Later thought of the previous job to do the ad text classification, and then began to resort to the almighty Word2vec , so there are two generations.

In the second generation, we have adopted the simplest algorithm: KNN, the corpus of the Word2vec, because of the protection of the body, at the same time, the matching process and NER combination, and finally completely avoid the first generation of problems; At the same time I added a context configuration in the system, so that the simple cross-examine can be achieved, such as the weather in Beijing, and then cross-examine Nanjing, will also be successfully classified into the weather forecast, and the correct extraction slot.

The second generation system has really succeeded in helping us get rid of ES, basically do to fill what can identify what, not every day people ask me you this again recognized wrong, but the problem is also very obvious.

First, the matching efficiency is low, in the early only about a little more artificial template, a single record of the classification time actually reached 600ms, the efficiency is simply miserable, and then look at others aiui, basically around 50ms. I said I was helpless.

Secondly, it is affected by the word breaker greatly. For example, when the company developed a voice remote control, added a lot of such as, @index, @index plus @num. etc. These things correspond to the words of the first one, the first plus three. But in the conversion word vector before the word segmentation step, the first point out of this is the first, the template is divided by the index, each time there is such a situation, you must adjust the word thesaurus corresponding to the egg, very painful. The more the egg hurts is that these kinds of problems, easy to solve, difficult to prevent, which is not the demonstration of the hanging up.

Thirdly, there is no generalization ability to investigate the reason, we training the word vector corpus is very small, so that each field corresponding to the threshold is very high, which also limits the generalization ability.

Three-generation Mesh

For the second edition of the three problems, the first solution is the efficiency of the problem, that is, in the process of solving the problem of efficiency, found that their operating system knowledge of the weak, and thus a relatively thin children of Tsinghua textbooks. To understand the thread scheduling, and then simply learn the next Golang inside the Goroutine, wrote a note, the public is ready to book with the simple synchronization. Details can not be mentioned, anyway, I have to optimize the interface response time from 600ms to 40ms.

Next is the recent third edition of the development, this version of the cycle is relatively short, the result is that several projects in the use of the second version of the system, encountered a series of problems. In the process of pits, I found a pit to fill one is not what, so thinking for a few days began to develop three generations of purpose. I first separated the NER as a micro-service from the system, reducing the coupling of the various parts (in fact, I was too much to start loading things, debugging, such as half a day); Then we have a user-defined threshold function, and into the matching process, it is amazing to find that my threshold with the latest in the recent similar function of the recommended range of a hair, I really did not copy ....

And then there's the point. A while ago I watched a game (SMP2018, a vertical field of short this classification contest) the title of the PPT, found their idea is CNN + match. One of the most important things is to do the NER before going to the algorithm. This train of thought in fact I also considered a few months ago, but at that time the NER over needle algorithm, now has a new NER, this idea also can practice. So I overhauled the entire matching process while refactoring the code, and the original 700-line core code was reduced to 300 lines, which is really cool. The additional benefit is that the second disadvantage in the second generation has been solved.

Digging pits

The full text of the black and bold Italic is the back of the content to be well written, that is, pit, of course, there are other pits.

Four-generation mesh development

In this reason with the number of fire shadow, also corresponds to the characteristics of the system, the first generation of simple rough, the second generation found black technology, three generations of perfect but old. So our four-generation development goal is.

It's very yellow and violent.

Not right, in fact, is fast quasi-ruthless.

Tentative plan September began to try to abandon the previously blind CNN model, cottage A classic CNN text classification paper, and then try Transformer model and attention mechanism . Some time ago. The code for migrating learning is also ready to try and see if we can combine the saniaoqu with the generation of a dialogue system that coexists.

Model Derivation and code implementation

In fact, I am also the feeling of learning things are not strong, want to review the statistical learning methods, a good look at the watermelon book, and the previous purchase of the dead expensive flower book, in the process of the basis of all the algorithm implementation once, write to just can also standardize code.

Continue to Brush Leetcode

Have the interest to brush the problem of my Gayhub library, the code is not good, welcome to guide. The goal is mid and above in this year to brush to more than 100 questions.