Giraph Basic Introduction

Source: Internet
Author: User

Giraph Introduction:

Apache Giraph is a iterative graph processing system built for high scalability. For example, it's currently used at Facebook to analyze the social graph formed by users and their. Giraph originated as the Open-source counterpart to Pregel, the graph processing architecture developed at Google and DESC Ribed in a paper. Both systems are inspired by the Bulk synchronous model of Parallel distributed computation by introduced Leslie. Giraph adds several features beyond the basic Pregel model, including master computation, sharded aggregators, edge-orient Ed input, Out-of-core computation, and more. With a steady development cycle and a growing community of users worldwide, giraph are a natural choice for unleashing the Potential of structured datasets at a massive scale.

Principle: Giraph based on Hadoop, will be mapreduce in mapper encapsulation, not using reducer. Multiple iterations are performed in mapper, each iteration equivalent to the superstep in the BSP model. A Hadoop job is equivalent to a BSP job. The underlying structure is shown in the following illustration.

Infrastructure for Giraph

The functions of each section are as follows:

(1) Zookeeper:responsible for computation State

–partition/worker Mapping

–global State: #superstep

–checkpoint paths, aggregator values, statistics

(2) Master:responsible for coordination

–assigns partitions to workers

–coordinates Synchronization

–requests checkpoints

–aggregates Aggregator Values

–collects Health Statuses

(3) Worker:responsible for vertices

–invokes active vertices compute () function

–sends, receives and assigns messages

–computes Local aggregation values

Source.

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.