Alluxio Introduction and role

I. Introduction of ALLUXIO

Tachyon formally renamed Alluxio, and released v1.0.0 version, Alluxio is a high-speed virtual distributed storage system memory.

Alluxio is a memory-centric, virtual distributed storage system that unifies data access and bridge computing frameworks and underlying storage

System. The application requires only Alluxio to access the data connections stored on any underlying storage system. In addition, within Alluxio

A centralized architecture enables data access to an order of magnitude faster than an existing solution.

In the big data ecosystem, Alluxio lies between the computational framework or jobs, such as Apache's Spark,apache

MapReduce, or Apache Flink, and a variety of storage systems, such as Amazon S3,openstack Swift,glusterfs,

Hdfs,ceph, or OSS. Alluxio brings significant improvements in ecosystem performance; For example, Baidu uses Alluxio to improve the speed of data

30 times times the throughput of the pipeline.

Second, the existing characteristics

(1), flexible file API

Alluxio's native API is similar to the Java.io.file class InputStream and OutputStream interfaces, providing efficient memory-mapped I/O, and we recommend using this API for the best performance Alluxio. In addition, Alluxio provides a Hadoop-compliant file system interface that allows Hadoop mapreduce and spark to use Alluxio instead of HDFs.

(2), support pluggable storage

(3), tiered storage

(4), unified naming space,

(5), descent

(6), WebUI and command line

This article from "Set Sail Loki" blog, reproduced please contact the author!

Alluxio Introduction and role