Papers to be tasted | Leveraging Knowledge Bases in Lstms

Yang, B., Mitchell, T., 2017. Leveraging knowledge Bases in LSTMS for improving machine Reading. Association for Computational Linguistics, pp. 1436–1446.

Links: http://www.aclweb.org/anthology/P/P17/P17-1132.pdf

This paper is an article published in the ACL this year, from the work of CMU, which proposes to solve machine reading problems by making better use of the external knowledge base. Because the knowledge of knowledge base of discrete features in traditional methods has the disadvantage of poor feature generation and partial task of feature engineering, this paper chooses the continuous vector representation method to represent the knowledge base. The end-to-end model of the traditional neural network makes most of the background knowledge neglected, and the paper proposes the extended network kblstm based on the BILSTM network, combining the attention mechanism to effectively integrate knowledge in the knowledge base when doing the task.

To answer the question of how to join background knowledge, and what information to add to the two-part content-oriented, and with the following two examples to illustrate the importance of the two parts. "Maigretleft viewers in tears." Using background knowledge and context we can know that Maigret refers to a TV show, "Santiago is charged Withmurder." If you rely too much on the knowledge base to mistakenly see it as a city, it is also important to judge what knowledge is relevant in the context.

Kblstm (Knowledge-aware bidirectional LSTMS) has three main points:

(1) Retrieving concepts associated with the current Word V (x_t)

(2) Semantic correlation of attention dynamic modeling

(3) Sentinel Vector s_t decided not to join background knowledge.

The main process is divided into two lines:

(1) Consider the Knowledge module when considering background knowledge.

(2) If the concept associated with the current word is not found, then set m_t to 0, directly LSTM hidden state vector as the final output.

The latter is straightforward and shows the structure of the former. Knowledge module modules s_t, h_t, V (x_t) as input, each candidate knowledge base concept relative to the h_t weight α_t, s_t and h_t β_t as the weight of s_t, the last weighted sum obtained m_t and h_t together as the output The final output is determined by finding the relevant concepts and related weights to determine what knowledge is added to the knowledge base.

This paper uses WordNet and NELL Knowledge Base to do entity extraction and event extraction task on ACE2005 and ontonotes data sets. Both have an improved effect relative to the previous model, and using two repositories at the same time is better than either one.

Note-taking: Li Juan, Zhejiang University in Reading PhD, research direction for the knowledge map, common sense reasoning, knowledge base distributed representation and learning.

openkg.cn

The Chinese Open Knowledge Atlas (openkg.cn) aims to promote the openness and interconnection of Chinese Knowledge Atlas data, and promote the popularization and wide application of Knowledge atlas and semantic technology.

Reprint NOTICE: Reprint need to indicate the source "openkg.cn", the author and the original link. If you want to change the title, please specify the original title.

Click to read the original and go to openkg blog.