Dnn deep Neural Network alignment

Source: Internet
Author: User
Tags dnn

1. In the paper [1], the idea of dnn is used for alignment. The training data required for alignment comes from HMM and IBM model4. The input layer consists of four layers. For details, see:

  

The effect is better than that of the original hmm And ibm4. For details, see:

  

This idea can be used in many places, such as segment similarity, sentence similarity, translation probability, word vector, and other calculations. However, in practice, dnn cannot beat other methods completely. In principle, it is similar to constructing a machine learning model that contains contextual features, of course, some people may say that word vectors imply more features in them, but from the perspective of actual use and observation by bloggers, word vectors still have limited functions, the problem may be that, in terms of the rich meaning and flexible combination of languages, language is a living thing, while mathematical representation is dead. For example, a word vector is equivalent to a word that is regarded as a fixed industrial part. It has its own specification parameters and is clearly displayed at what position, but unfortunately, language is not that simple. A word is more like a liquid metal. It not only has the current shape and size, but can also be combined with other metal blocks, the formation of a new shape is given a new way of use. For example, the word "big" has a meaning of "big", but if I say big is very high, it means "forced, A fixed dimension cannot represent a living word. To put it bluntly, words are active and vectors are dead. This is why I think word vectors are useful but not practical. I have some immature ideas that need to be verified. 2. knowledge points:

To be supplemented...

  

3. Approximate code implementation (for python, use IBM model1 + nn ):

Follow-up supplements...

 

Reference:

[1] ACL '13, Word Alignment modeling with context dependent deep Neural Network

 

Reprinted, please indicate the reference from:

Http://www.cnblogs.com/breakthings/p/4049854.html

 

Dnn deep Neural Network alignment

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.