GRAPHX Implementing N-degree relationships

Background

This paper presents a simple algorithm for calculating the N-degree relation points of each point in the graph, that is, the N-hop relationship.

Prior to the official document to learn and understand the Graphx computing interface.

N-degree relationship

Goal:
In the N-round, find the point set of the N-degree relationship of a point.

Implementation ideas:
1. Prepare the Edge dataset, which is a point relationship such as "1 3", "4, 1". Using the Graphloader interface load into Graph
2. Initialize each Vertice property to an empty map
3. Use the aggregatemessages to spread the Verticeid and the totalrounds to the point, the out-of-the-point to synthesize a large map of the collected information
4. Updated vertice "Join" with the original, updating the changed point properties in the diagram
5. Repeat steps 3 and 4, and the final output updates the relational Vertice after the N-wheel

Code to execute under Spark-shell:

ImportOrg.apache.spark._ImportOrg.apache.spark.graphx._ImportOrg.apache.spark.rdd.RDDValFriendsgraph = Graphloader.edgelistfile (SC,"Data/friends.txt")ValTotalrounds:int =3 //Total N roundvarTargetverticeid:long =6 //target Vertice//Round OnevarRoundgraph = friendsgraph.mapvertices (id, VD) = Map ())varRoundvertices = Roundgraph.aggregatemessages[map[long, Integer]] (CTX = {if(Targetverticeid = = ctx.srcid) {//Only the edge has target Vertice should send msgCTX.SENDTODST (Map (Ctx.srcid-Totalrounds)}}, _ + + _) for(I <-2To Totalrounds) {ValThisroundgraph = Roundgraph.outerjoinvertices (roundvertices) {(vid, data, opt) = Opt.getorelse (Map[long, Integer] ( )} roundvertices = Thisroundgraph.aggregatemessages[map[long, Integer]] (CTX = {Valiterator = Ctx.srcAttr.iterator while(Iterator.hasnext) {Val(k, v) = Iterator.nextif(V >1) {ValNEWV = v-1Ctx.sendtodst (Map (K-NEWV)) ctx.srcAttr.updated (k, NEWV)}Else{//Do output and remove this entry}}, (Newattr, oldattr) = {if(Oldattr.contains (newattr.head._1)) {//Optimization to reduce MSGOldattr.updated (Newattr.head._1,1)//Stop sending this ever}Else{oldattr + + newattr}} )}Valresult = Roundvertices.map (_._1). Collect

Data and output

2 14 11 26 37 37 66 73 74 31 66 1

Array(6137)

Summarize

The implementation of the comparison naive, there are many areas that can be optimized.

Complete the full text:)

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

GRAPHX Implementing N-degree relationships