Network Representation learning/embedding

Reprinted from:

http://blog.csdn.net/u013527419/article/details/74853633
http://blog.csdn.net/u013527419/article/details/76017528

Network Representation Learning (representation LEARNING,NRL), also known as graph embedding method (graph embedding Method,gem), is a work that has arisen over the past two years and is now very hot, Many direct research networks represent the work of learning and simultaneously optimizing network presentation + downstream tasks in progress.

2. A study group of Tsinghua University's computer department Sina Weibo @ the paper list of ultra-finishing: Https://github.com/thunlp/NRLpapers, and has been continuously updated, the detailed list of the last few years about network expression learning Representation Learning/network Embedding) compares a representative list of papers with their code.

4. [(MLSS2017) network means learning] "representation learning with Networks" by Jure Leskovec [Stanford University] Part1: Web link Part2: web link PART3: Web link Part4: web link????

6. A review article (University of Southern California (USC)) and its code:
   (1) Article:
   Goyal P, Ferrara E. Graph embedding techniques, applications, and Performance:a Survey[j]. ArXiv preprint arxiv:1705.02801, 2017.
   (2) Code:
   Https://github.com/palash1992/GEM

8. A blog post:
   Http://blog.csdn.net/Dark_Scope/article/details/74279582#0-tsina-1-3919-397232819ff9a47a7b7e80a40613cfe1

10. A GitHub profile with part of the paper +code (mostly Python implementations, Matlab followed):
    Https://github.com/chihming/awesome-network-embedding

12. Three x Slides:
    (1) Https://pan.baidu.com/s/1nuB5Rex
    (2) Https://pan.baidu.com/s/1geUHeQB
    (3) https://pan.baidu.com/s/1cwB7pc

Network Representation Learning (DEEPWALK,LINE,NODE2VEC,SDNE)Original July 24, 2017 12:49:01 detailed information can be consulted: network means learning related information
1. Tradition: Graph-based representations (also known as symbol-based representations)

As shown on the left, G =(V,E), the different nodes are named with different symbols, the storage structure of the two-dimensional array (adjacency matrix) indicates whether there is a connecting edge between the two nodes, there is 1, otherwise 0.

Disadvantage: There is no relationship between the nodes in the long tail distribution, so the adjacency matrix is very sparse, which is not conducive to the storage computation.

2. Network Expression Learning (representation learning,NRL), also known as graph embedding method (Graph embedding Method, GEM): using low-dimensional, dense, real-valued vectors to represent nodes in the network (with semantic relationships, to facilitate the calculation of storage, no more manual feature (adaptive), and heterogeneous information can be projected into the same low-dimensional space to facilitate downstream computing).

Deepwalk"1":

Implement 1:https://github.com/phanein/deepwalk

  skipgram skipgramcontext,< Span lang= "ZH-CN" > neighborhood Nlpneighborhoodword The word around, This article uses random walk to get graph or networkneighborhood

• (1) random walk randomly and evenly select the network node, and generate a fixed-length random walk sequence, the sequence is analogous to the natural language of the sentence (node sequence = sentence, the sequence of nodes = words in the sentence), Applied Skip-gram Model Learning node distributed representation,Skip-gram model see: http://blog.csdn.net/u013527419/article/details/74129996
• (2) If a node of a network obeys a power-law distribution, the number of occurrences of a node in a random walk sequence is also subject to a power-law distribution, and the occurrence frequency of the word in NLP is also subject to the power law distribution.

• (3) General steps:

Network/graph---------Random walk--------- get the node sequence (representation mapping)--------put in the Skip-gram Model (intermediate node Prediction context node)---------output:representation

Line "2":

(1) distinguish two concepts first:

first-order similarity: direct connections between nodes, such as 6 and 7.

Defines the joint probability between the node VI and VJ

V represents the node, andu represents the embedding of the node . The above expression means that the more similar two nodes, the larger the inner product, the greater the value of the sigmoid map, that is, the greater the weight of the two nodes connected, that is, the probability of the occurrence of the two nodes is greater???

Second degree similarity: between nodes that are connected through other mediation nodes such as 5 and 6.

with a conditional probability.

(2) The goal is to make the NRL before and after the similarity between the nodes are unchanged, and the node indicates that if two nodes are similar before learning, then the two nodes after embedding will be similar to each other. -- This article uses KL divergence to measure the distance between two probability distributions. For more information about KL divergence see: http://blog.csdn.net/u013527419/article/details/51776786

To ensure the first-order similarity of the example:

Embedding : The empirical joint probability between node VI and VJ is

So, minimize:

Node2vec "3":

Thesis + realization and other: http://snap.stanford.edu/node2vec/

Similar to Deepwalk, the main innovation is to improve the strategy of random walk, defined two parameters p and Q, in the BFS and DFS to achieve a balance, At the same time, it takes local and macroscopic information into account and has high adaptability.

(1)

(2) parameter control jump probability of random walk, before completely random,p=q=1.

     -- return probability parameters ( return parameter) pbfs,p control the probability of returning to the original node, from t jump to v1/p< The probability of span lang= "ZH-CN" is in the node vt

-- departure probability parameter (in outparameter)q, correspondingto DFS,Q controls the probability of jumping to other nodes.

, just fromEdge（T，V) come over, now in the nodeV to decide next step (v,x ) How to go. where dtx means node tx< The shortest path between the span lang= "en-us", dtx=0t itself, dtx=1 means node t and the node x is directly connected, but in the previous step chose the node vdtx=2 means node t Not with x< Span lang= "en-us" > is directly connected, but the node vx

(3) in the field of computational advertising and recommendation, There are two interesting applications around Node2nec:

Facebook:http://geek.csdn.net/news/detail/200138

Tencent:http://www.sohu.com/a/124091440_355140

Sdne[4] ::

one of the major contributions of this paper is to propose a new semi-supervised learning model , which combines the advantages of first- order estimation with the second estimation to represent the global structure and local structure properties of the network.

The description eigenvector of the node (such as the "adjacency vector" of the point) uses Autoencoder encoding, which is represented by the Autoencoder middle layer as a vector, so that the 2ndproximity (similar neighbor's point similarity is higher because the two-node " Adjacency vectors are similar, indicating that they share a lot of neighbors, and the last vector that is mapped to Y will be closer. I always feel that the above figure in the local and global write the reverse.

Target function:

"1" Perozzi B, Al-rfou R, Skiena s.deepwalk:online learning of social Representations[c],KDD 2014:701-710.

"2" LINE< Span lang= "ZH-CN" >:large-scaleinformation Network embeddingwww2015 ,jiantang, Meng Qu, Mingzhe Wang, Ming Zhang, June Yan, Qiaozhu mei,microsoftresearch Asia; Peking University,china; University of Michigan

"3"node2vec:scalable Feature learning fornetworks,A Grover, J Leskovec [stanforduniversity] ( KDD2016)

"4" Structural deep Network embedding,KDD

Above are I more interested in a bit, detailed can refer to: https://github.com/thunlp/NRLpapers

Network Representation learning/embedding