Scality Object Storage Add Hadoop, OpenStack plug-ins

Object Storage Startups scality add their storage to Hadoop, allowing users to avoid loading data through Hadoop's own file system. They also launched a plug-in for cinder--'s block storage layer in the OpenStack project.

The ring is an object storage infrastructure based on a set of X86 server nodes that stores objects instead of files or blocks, and can operate in parallel.

Scality provides a "production-level Hadoop storage Implementation" Using the cloud storage standards for cloud data management, developed and promoted by the SNIa cdmi--. CDMI began slowly to be supported by suppliers but at a pick-up pace.

Scality has replaced the Hadoop named node server with its own metadata schema, eliminating the single point of failure in the Hadoop architecture. The company says its Hadoop implementations can handle and compute appropriately on the storage node itself, and significantly reduce the need for data transmission by tracking shared data with the job.

Scality says its ring erasure code means that it eliminates any hadoop hardware overhead that is generated by replication. In addition "users can write and read files through a standard file system, and use the Hadoop process at the same time without the need to load files through the HDFs (Hadoop Distributed File system)."

"We have contributed our Hadoop solution to the CDMI community to make sure it can be used with any CDMI compatible storage," said Jerome Lecat, chief executive of Scality. ... Our CDMI framework (the framework) can read data directly from our outward extension (scale-out) file system, and there is no need to do HDFS acquisition before performing a mapreduce job. “

Scality products are compatible and have been tested with Hortonworks HDP 1.0 and Cloudera CDH4-not showing that scality is seeking alternative or competing with the existing Hadoop release. By adding a ring back end, to some extent, Scality says it offers a more cost effective, easier to use, more resilient, and more high-performance Hadoop base?? Infrastructure, while users benefit from the Scality sofs (scale-out file system).

"Our perspective is that we think people want to do hadoop work on" normal "data, not just for Hadoop, Lecat says. In my impression, this is very valuable for Hadoop, but it is stifled by the fact that people need to do a hdfs intake before any mapreduce work. Because we don't need that anymore. ”

One implication is this, says Lecat, "Imagine what you can do if you're using MapReduce now--this is working on a storage node--To do data conversion, like the new code, to produce results as a new version, which saves a lot of processing time." It previously needed to move data from storage to the server, convert it, and write it back to the store. ”

OpenStack Object Storage

OpenStack is a cloud or infrastructure as a service (IaaS), based on free, Open-source software to control computing, storage and network resource pools in a data center, where users are allocated through a portal, and managers manage the entire group through the dashboard (dashboard). Rackspace and many other vendors are actively and loudly supporting OpenStack. Now scality to join in the fun.

Cinder is the code name for a block storage layer in OpenStack that enables virtual machines (VMS) to discover and use persistent block-level volumes, while Scality provides a ring plug-in for it. "This contribution allows OpenStack's followers to catch up with Amazon's EBS persistence volume for virtual machines," Lecat said. With the release of Grizzly (release), OpenStack computing will have a storage partner deployed in a highly demanding cloud computing environment, which will increase the OpenStack adoption rate on the market. ”

Grizzly is the next version of OpenStack, scheduled for release in April.

Scality is not alone. Coraid Company also provides ata-over-Ethernet (AOE) and Coraid ethercloud drivers to OpenStack cinder block storage for open source projects, so openstackers can use its storage arrays as block storage. Full-flash "cloud" storage array start-up Solidfire has done the same thing, and it has been involved in the cinder project for several years now. Coraid claims that traditional storage vendors such as NETAPP, EMC, Hewlett-Packard and Dell have done only part of their openstack drivers, and they have joined the OpenStack community as a corporate sponsor.

The ring for OpenStack provides a POSIX file interface through an outward extended file system (SOFS) encapsulation. Scality narrative:

Cinder integration is based on the scality ... Distributed sparse file technology embedded in Sofs. Each cinder volume is actually a file inside the Scality scale-out store. This ensures easy management, seamless scalability, and advanced virtualization capabilities such as real-time migration of virtual machines and instant failover, in the context of computing node hardware failures.

"This block storage interface completes our unified storage strategy," says Philippe Nicolas, Scality's product strategy director. Scality is the actual delivery commitment to the real and complete unified storage access, including objects, files and blocks of one of the first manufacturers. ”

Scality Cinder consolidation will be available when OpenStack Grizzly is released.