The story of Me and NLP (reproduced)

Positive ACL Employment results released, the domestic teachers and students are a great harvest, here again congratulations to all the papers are employed teachers and students! My character broke out and I also harvested my second ACL thesis on my Master's stage. Originally just want to share the joy of their own paper, but did not Chengxiang received so many teachers and classmates congratulations and encouragement, is really flattered, here also once again thank you teachers and classmates, look forward to working with you in the ACL face-to-head communication. After the release of My Weibo, Dragon Star Biaoju sent to invite, I hope I can write a small article about their own research. But as a small graduate student certainly did not take the results of the shot, reasoning, talk about the past few years to do NLP research experience and feel good, hope to be able to study in the domestic, and I also confused classmates have some help.

First Contact Natural language processing is in the third grade, when I was in Harbin University of Technology Network Intelligent Laboratory with Guan Yi teacher do wi Input method, I was responsible for the pinyin input method of intelligent error correction function. During this period, the concepts of language model and Hidden Markov model were learned. Although there was no deep understanding of the underlying principles behind it, I was immediately fascinated by the wisdom behind it and decided that this was the direction I was going to study. To the senior year, received the Baidu NLP Department internship offer, this internship on my life planning has had a huge impact. Although it was already insured to Peking University, but like most of his peers, began to doubt whether to continue reading good or direct work. With the thought of trying, decided to go to internship experience. At that time internship in the Department of Daniel gathered, including the top of the NLP Wang Haifeng teacher, Wu Hua teacher, and so on, a variety of PhD, the great God is even more numerous. With the internship, I further feel the charm of NLP, I hope that one day I can also design their own model, I hope that my model can be online to change the lives of millions of people. But the ideal is very plump, the reality is very bony, I began to realize how insufficient the knowledge reserves. So I decided to make my own decision: Graduate School

Now that we have chosen to study, we must prepare for the research. After graduating from the undergraduate summer vacation, I did not choose to travel around, but chose to put themselves in the home to add a variety of knowledge. At that time, I watched Andrew Ng's public class and Coursera's Open class, and read the "Speech Recognition and Language processing" in the English original, and read the Pattern recognition and machine learning ". Of course, the two books I read the first time after reading the 30% of the content, so follow-up after reading and reading many times. Up to now, PRML I have read about a total of 4 times, but to tell the truth, this Bible may I can now only really understand the content of 60%--70%. Of course, this is enough for me, because I don't want to be an expert in the field of ML, so a lot of the content is not studied carefully.

Because of the "stimulation" during the internship, began to study at the beginning of the complacent must make use of a good graduate student three years to do the most top research, so in the first day of school I sent an email to the tutor, determined to publish a top-level conference papers during the master's degree. It was a dream for me at the time because there were few graduate students in the history of the lab who were able to publish top papers during the study. Here I have to mention my mentor Baobao Chang teacher, if without him my goal is impossible to achieve. After I sent the impassioned e-mail, the tutor expressed great encouragement. Unlike most of the mentors in China, teachers have never made mandatory provisions for our research, and have never asked us to do outside projects, instead, he encourages us to look for our own research direction and to do what we are interested in. At the same time, his academic level is very high, can give very detailed guidance when I encounter difficulties. These excellent external conditions pave the way for follow-up research.

In the study, because of limited knowledge reserves, so with the tutor discussed the next research content, the tutor decided to do topic model related content. Because the topic model is relatively good for getting started. After several discussions, the first idea in life was formed and the code was written quickly, and the results were very good. So in the study period I ushered in the first time in my life to contribute--acl2013. My English writing ability is not really good, at that time after writing the paper, the tutor almost all overturned rewrite again. At that time also coincided with the new year, I deeply remember the morning of the evening, I and the tutor on the phone remote discussion on how to modify the paper. After countless times of carving, my first paper "inducing Word sense with automatically learned Hidden concepts" was born. I was interested in dash the draft, expecting the result, but the end result is: Reject. The rejection of the thesis is naturally frustrating, and I have always thought that the reviewer did not understand the core idea of our thesis and brawl the reviewer in mind. But teachers often teach me that no matter what the outcome, we must face the peace of mind, and actively revise the paper. In this way, my research ended with a paper that was reject.

Research on the time to continue to revise the paper, and with class peace often read the accumulation of paper, the knowledge volume has been further improved, also put forward an idea of their own, the experimental results are also good. Because of the last refusal of the shadow, decided to first from the domestic meeting to try, the tutor also expressed support. So I wrote a CCL cast. At the same time with the senior of the senior security research, further improved the rejected paper, added a new model, cast a EMNLP. The result is CCL hired, EMNLP was rejected again. Ushered in the first article of his life is very happy, but the goal of the top will be far from their own, the natural heart is unwilling. At this time the mood is still very depressed, the thesis two consecutive times refused to let me doubt whether the idea is really a problem, so I began to lose confidence in this paper, so there is no further to change the paper. At the end of the study, I have not had a unique research direction, and started a wide range of reading literature. Just then I saw some of the content of deep learning. At that time, the content of deep learning has not been so comprehensive today, I think this should be a once-in-a-lifetime opportunity, so began to focus on and deep learning literature reading.

Second semester is a happy semester. My brother's paper was IJCNLP hired, but senior has graduated can not meet, so the teacher decided to send me to attend, so I ushered in my first international conference in life and the first time in my life oral presentation. Meeting is open in Japan, all kinds of beer and skittles naturally needless to say, but give me the greatest pleasure is and peer teachers, students exchange. I felt that it was a great thing to have a meeting abroad and to exchange ideas with the field's Daniel, so this meeting was even more determined by my determination to send a paper of my own. After the meeting came back to the excitement of the first idea in depth learning, based on deep learning of the word segmentation model. At that time, the use of deep learning in structured models was still rare, including now. The so-called structured model is that the output of the model is not a simple classification, but a concrete structure. For example, the sequence labeling model and syntactic analysis are all within the framework of a structured model. At that time, reading a large number of this literature, that since the maximum entropy can be extended to CRF, the normal Perceptron model can be extended to a structured perceptron model, the neural network can also be extended to a structured neural network model. The word segmentation is of course the simplest sequence labeling model, so I propose the Max Margin Tensor neural Network for the characteristics of the sequence model, which is also inspired by the deep learning community, Daniel Richard Socher. With the experience of writing an article, this article was soon completed, of course, after several changes, and finally in deadline before and before the article repeatedly rejected and voted ACL2014.

Research two the next semester is a harvest of a semester. I ushered in my first ACL paper "Max Margin Tensor neural Network for Chinese Word Segmentation", the paper's hard-won may only cast the person can realize it .... From writing papers, revising papers, contributing, review, to speculation reviewer thoughts, to response, and then to the anxiety of waiting. When I opened the Mail and saw delighted's moment, I shouted "I x!" It's in! "When it was really tears came out, and then ran to the tutor's office to the good news." As for another article, it is still rejected .... The tutor and I basically have given up that article, so change also did not change and cast coling, did not expect also in (perhaps really character save enough). This semester also with the younger brother with a unsupervised word-breaker model, later cast to EMNLP also was hired. The first time to participate in the ACL, coling I was frightened by the great gods are afraid to speak, but then summon up the courage to go forward and before only in the paper and open class to see the Daniel exchanges, found that they are very kind, and will very much encourage our young researchers. Ijcnlp,acl,coling these three meetings have opened my eyes and greatly improved my ability to communicate, which has played a very important role in my interview with Facebook, which is, of course, an off-topic.

Happiness comes too fast, a kind of feeling of being on the go. At this time the tutor to talk to me, hope to be able to pick up again, not proud, continue in the field of deep learning. Three last semester began to try a lot of idea, but most of them died, and later decided to dependency parsing field to try the effect of deep learning model, this is my first choice for the direction of a significant branch, that is, tree-based structure of the structure of the model. At that time, dependency parsing only the basic concept of understanding, to many predecessors of the work have no in-depth research. So find a lot of literature, daily put themselves in the literature, look at the previous methods, look at various algorithms, to achieve the previous model. Later also formed their own initial idea, and then realized that the results are really good, so my second ACL, is also this ACL was hired article "an effective netural Network Model for graph-based Dependency parsing "was born. As in previous years, is still in the eve of the night to revise the paper repeatedly, until the recent paper was hired.

Looking back on a few years of graduate school, there are two indispensable reasons to accomplish my goal: 1, the tutor selfless support 2, the luck. I do not want to pour too much chicken soup, say some through their own efforts to finally achieve the ideal and so on. Trying to achieve their goals is an indispensable condition, and it is almost impossible to succeed without effort. However, efforts are not sufficient conditions, many times whether the paper is hired, whether to find a job can get an offer depends on the "right and right people and". So don't be discouraged even if we get rejected, it's probably just a little bit of luck. Of course, there is a sentence I have always liked, that is, "the more efforts, the more fortunate," the so-called thick, thin hair, I believe that tireless efforts of people will not be too unlucky. Aside from these subjective factors, this short three years of research has also accumulated a number of objective contributions to the experience, written here to share to everyone, hope to be the same as me in the domestic study of students can help it:

1. The article is not the more advanced the more easily, on the contrary, can be employed in the paper is usually easy to understand. At the beginning of the submission, I pursued some tall words, listed the formula appears tall, but it turns out that the reviewer is unlikely to have time to look at the formula carefully, speculation model. So we must write our paper in the most popular language, explain our motivation, explain why our model can work
2. For those of us who have a shallow experience, there are two sources of idea: Old method new task, old task new method. I used the old task of the new method of strategy, very fortunate to be able to catch up with deep learning this climax, and in time to do some improvement and application.
3. Never think your idea is too water to do it. In fact, most of the work is increamental, can produce subversive impact of the model is very few, as long as there is a certain improvement ideas, we can try to do, and write articles. Most of the time, we feel that our idea water is because we know so much about our field of study that we feel that we are aware of it. But in fact, for outsiders, it is possible that our idea has its own value.
4. It's a great way to read a lot when there's no idea, and maybe an article will give us a lot of inspiration.
5. To read more reference materials in English, on the one hand, most of the information is directly written in English, look at the translation of the content will inevitably have the loss of information. On the other hand, a lot of reading English information can have a great help to our sense of language, in writing a paper can be done with God, the table meaning clear.
6. The level of the reviewer is naturally uneven, and many times the reviewer does ask some more bizarre questions. Even so, in the response time also do not talk intense, think oneself is God, reviewer is xx. Reply must be decently, flat down the heart to answer the questions raised by the reviewer, the recognition of the wrong to admit. Generally speaking, the reviewer is a knife mouth tofu heart, the final result may not be as bad as we actually think

The above is my research experience as an ordinary graduate student in China, hoping to be able to help the same classmates as me. The dream of the meeting is not so far away, I believe that our domestic young researchers can produce very good papers as long as they are willing to work hard. Or that sentence "the more efforts, the more fortunate", I wish all good luck!

The story of Me and NLP (reproduced)