VIPR and DJ lead software-defined storage

 for software-defined storage (SDS), each vendor has its own definition and understanding of the advantages and technology accumulation in a given domain, similar to cloud computing concepts, controversial. But personally, no matter how the manufacturer proposes to differentiate the definition should be close to SDS programmable, policy-driven, resource pool virtualization, data surface and control surface separation, service automation and other SDS requirements. In fact, these requirements are also IDC's definition of SDS.

IDC's definition of SDS coincides with Onf's idea of software-defined Networking Sdn (OpenFlow). The core idea of snd is the separation of data plane and control plane, which supports heterogeneous, business orchestration and service automation. This enables storage automation, templating, and data intelligence through software layer service catalogs and policies.

last month I was in the article Consideration of data re-deletion in the era of big data did a technical interest in the questionnaire, from the results of the only feedback, we are still more concerned about storage and cloud computing, thank you for your feedback and suggestions, I will adjust accordingly.         

Write an article about storage for everyone today. This article takes VMware SDDC as lead, and compares the two SDS pure software storage products that have commercial delivery experience in real sense to realize the separation of control data, EMC VIPR and Huawei Oceanstor DJ.

SDDC Reference Architecture

The SDDC Reference Architecture is also a division of similar control and data planes, and the SDDC is now integrated with VIPR and NSX products. The management layer, as a separate plane in charge of the entire data center, similar to the storage network management software. The infrastructure layer contains a variety of physical infrastructures that host data business flows.

The control plane is similar to the SDN control surface, which is virtualized (virtualized framework) for compute, network and storage, and provides a unified abstraction layer to develop control strategy. The application layer includes VMware's various applications, other services, and users ' applications.

2013 EMC released ViPR, followed by ViPR2.0 into Scaleio and providing OpenStack cinder support for heterogeneous storage, packaged software VIPR and ECS with commercial hardware deployment; this year at EMC World Congress released VIPR3.0 enhanced cloud integration and third-party storage takeover capabilities. In contrast, the Huawei DJ released later, but the current DJ in finance, operators and automotive industry has commercial applications, combined with enterprise network management software esight to achieve unified management and service automation capabilities. Below we analyze these two products from the aspects of system architecture, ecological cooperation, technical advantages and solutions.

System Logical Architecture

VIPR integrates all storage (including third-party) into one storage pool, and the virtual storage array as a system management object while preserving the underlying physical storage characteristics. It can span multiple locations and data center distributed deployments, providing unified storage services for block storage, file storage, and objects, while VIPR provides unified metadata services and self-service deployment, metering, and monitoring services based on multi-tenancy.

The control plane is responsible for discovering, setting up, and managing storage devices, including multi-tenancy, self-service portals, metering and provisioning, and more. VIPR divides the discovered physical storage or storage pool into multiple virtual storage arrays, providing storage services on demand. Based on virtual storage pools, it implements volumes, Nfs/cifs file systems, and advanced protection services such as snapshots, clones, and replication disaster recovery, and provides interfaces for integration with cloud platforms.

The data plane is responsible for reading and writing the actual data. The control plane provides only service catalog views and management, and true traffic interacts directly with the underlying storage hardware through the data plane. So VIPR does not become a performance bottleneck, currently supports the conversion protocol for NAS and object (object data Services in files, Isilon objects and file interoperability), support for Amazon S3, OpenStack Swift, Atmos object API, EMC VNX, Isilon , NetApp file storage access, HDFS data Services, and more.

Based on the X86 hardware architecture, OceanStor DJ supports storage services such as files, blocks, objects (planned), and integrates with third-party data protection, disaster recovery and other software and value-added features. The architecture is similar to VIPR, which enables scheduling, orchestration, and automation based on policy and customizable templates. Resource pooling, unified management of storage, and SAN switches through the heterogeneous takeover of enterprise-class storage, which is presented as a storage volume for Huawei, and the standard interface for OpenStack storage.

EMC VIPR and Huawei Oceanstor DJ products are much more similar in architecture, using data plane and control plane separation architectures. Based on virtual storage pools, it implements volumes, Nfs/cifs file systems and advanced protection services such as snapshots, clones, replication disaster recovery and backup, and provides an OPENAPI interface for integration with cloud platforms.

DJ supports Huawei Storage, fusionstorage and third-party storage devices (HDS, EMC, HP, etc.). Compared to VIPR, the main difference is that DJs are fully compliant with the OpenStack architecture, including the use of cinder, Malina, and other interface implementations for third-party storage management, without a private interface like VIPR.

VIPR Architecture and Key Features

VIPR is a VIPR instance vapp that is deployed on a VMware virtual machine pure software PRODUCT and runs on 3-5 server virtual machines and consists of multiple virtual machines that comprise an ESX cluster. The user creates virtual Storage pools (VSP) in VIPR based on a dummy array, specifying storage attributes (such as software features, RAID, number of paths, etc.) for each VSP, and VIPR aggregates all physical storage arrays based on the matching of the VSP attributes. Depending on the tenant's request, the VSP finds the correct array and pool to satisfy the resource request.

VIPR automates the discovery of physical storage arrays and the physical storage pools and ports in them, which divide these physical storage pools into individual virtual storage pools, depending on their capabilities. Multiple physical storage pools on the same physical storage array product can be provisioned to multiple virtual storage pools.

One VIPR instance can extend the management of multiple physical data centers located in different cities, geographically and by the same VIPR-managed physical storage as a virtual storage center.

VIPR can provide automated storage dr, Emcvipr (virtual) block controllers combined with VPLEX and RecoverPoint (physical) block data nodes, and virtual storage arrays in different locations can be continuousremote Replication ( CRR) technology can provide data disaster recovery capabilities.

DJ Architecture and Key Features

OceanStor DJ is also a pure software product that supports physical deployment and virtual machine deployment, single node and multi-node HA deployments (3 nodes and more). The DJ architecture is divided into storage function virtualization (SFV) architecture and SDS architecture.

The SFV architecture supports resources such as generic physical hardware, X86 servers, and hypervisor virtualized servers. Oceanstor storage platforms and features can be run in virtual machines, providing customized and modular storage services (HDFS, NAS, SAN, etc.) based on virtual machines. The SDS architecture is the core of the DJ, which manages the underlying physical storage and SFV-based virtualization storage, and ultimately pooling, automating, and templating is automatically assigned to users on demand.

Storage function Virtualization (SFV) is similar to NFV functionality, OceanStor OS storage platform software similar to the NFV pipeline OS, which provides hardware abstraction and resource pooling, storage basic functions and value-added features (such as Nas, object, replication, backup, disaster tolerance, etc.) like network elements in NFV EPC, IMS, DPI, DJ SDS platform is like Mano in NFV or marketplace in Amazon public cloud.

The DJ provides a data disaster recovery feature that requires at least two primary storage arrays to be deployed to two sites, but two site storage can be managed by the same DJ Cloud platform.

the recovery volume for primary and standby replication is created by the DJ extension OpenStack Cinder module, which creates a corresponding disaster recovery volume based on the protected LUN size, capability, AZ, and so on. DPS Call OM service to create, start, Cancel and other interface transformation, the command issued a corresponding cinder interface, the corresponding cinder driver to find the corresponding array and perform the disaster preparedness task.

OceanStor DJ provides a robust Data Protection Service Module DPS, each tenant landing system to submit data protection (backup and disaster recovery) requirements, the DPS engine automatically matches the required resources to complete user requests, completely shielding the tenant of the underlying data interaction details.

The tenant submits the backup service request through the Manageone Service Center, which completes the matching and task scheduling (selecting matching backup storage, backup policy, network, etc.), and the backup drive invokes the backup software to implement the snapshot-based backup.

Business Automation Release Process

VIPR employs a service directory and policy-based storage allocation method, when a user logs in to the VIPR portal and requests the relevant storage, first the user submits a service request (such as high performance), and then VIPR looks at the policy associated with the VSP to find the most appropriate storage array and array pool.

The storage port is then automatically selected based on performance, a zone is established as required, a device is placed in a storage group and a fast policy is specified, an output workflow is performed on the storage array, and the host to the physical storage connection is confirmed.

The hosts treat these devices as high performance devices and directly read and write to these devices, and when users want to do operations such as data protection and data disaster recovery, expansion or deletion, they can directly log in to VIPR portal to perform these tasks based on user options.

The OceanStor DJ uses the service catalog implementation to define various storage services or features (such as resource pools, whether thin allocations, whether to use tiered storage, tiered storage migration strategies, QoS settings, and so on), and users can customize this information.

When a tenant submits a storage resource request, the dispatch tier looks for the appropriate storage device to service according to the requested service type. The scheduling algorithm automatically matches these requirements based on the capabilities of the underlying storage device and finds the most appropriate storage device.

The system administrator plans the storage pool based on the type, purpose, capacity, and other attributes of the storage. The service catalog is designed and configured as well. Tenants view the storage services directory in the Oceanstor DJ system, and tenants Select and book the required storage services through the catalog. The OCEANSTORDJ system automates the provisioning and distribution of storage resources based on the type of service a tenant chooses, enabling storage mounts to hosts.

Interface opening capability

VIPR provides a south-to-north interface for RESTful APIs that can be well integrated with third-party applications such as Appsync and management software. Discover and manage physical storage through the OpenStack interface, SMI-S, XML, storage takeover private interface, rest, and other APIs to enable automatic connection, discovery of arrays, and host and San switches for third-party arrays.

VIPR can be integrated with VMware, Hyper-V, and OpenStack. In addition, VIPR provides good support for both Vasa,vcenterorchestrator and vcenter operations. For example, vcenter administrators can gain end-to-end visibility from virtual machines to physical storage.

The DJ is fully integrated into the OpenStack architecture to seamlessly connect to the OpenStack cloud platform, enhancing the original storage capabilities. It is technically possible to replace OpenStack's storage modules and provide more storage, business automation, and data protection capabilities.

The DJ provides the Restful/open API northbound interface implementation with OpenStack, VMware VASA, Manageone, Esight, and third-party management software (CA, vcenter, etc.) docking, providing restful ( OpenStack API), SMI-S south-to-interface implementations, and third-party storage docking.

The DJ, in collaboration with major backup vendors such as Veritas and CV, implements the data Protection Engine (SMAUG) Protection Service module, which integrates with third-party backup software through restful interfaces to enable data self-service backup and recovery for tenants.

Visual management Capabilities

The VIPR version integrates the SRM (Storage Resource Management) and SAS (Serviceassurance Suites) software to manage the entire VIPR data center.

SRM provides features such as resource utilization and capacity planning, change management, and performance analysis and reporting. Supports storage management such as VNXE3200, XtremIO, HDS, HP, IBM XIV, and NetApp. SAS provides end-to-end service level assurance, and VIPR manages virtual storage.

DJ is mainly used for storage resource management, scheduling, it is in the data center is the data center Manageone Service center scheduling. Esight is integrated as part of the operations center. Manageone is a cloud data center management software that includes service Automation SC components, service assurance OC parts, and Device management esight components.

Esight Complete data Center unified management and monitoring, compatible with heterogeneous equipment, provide unified operation and maintenance. Integrated as a monitoring component by Manageone. OCEANSTORDJ is focused on storage virtualization and service, can be sold independently, and can be integrated into the Fusionsphere cloud computing platform by Huawei's cloud management platform.

Simple summary:

At present, although many manufacturers to the storage hardware and software decoupling, platform X86, new hardware, storage heterogeneous takeover capacity equivalent to SDS, but really based on the control plane and data plane of SDS closer to the idea of software definition, direct access without changing the current network structure, based on the non-gateway form to avoid becoming a performance bottleneck. Up-to-north interface open storage capabilities, preserving the advanced features of heterogeneous storage in the overall SDS scenario. Here's a quick summary of VIPR and DJ.

Architecture comparison

Both VIPR and DJ are data and control plane separation architectures that enable service catalogs and business automation, but the deployment approach and architecture implementation are very much different. VIPR uses a general virtualization deployment that uses OpenStack (Cinder, Malina, Swift, and so on) and private interfaces to manage third-party storage; DJs support single-node and HA physical and virtualized deployments, and are flexible to deploy virtualized platforms. All of the third-party storage management uses OpenStack interface, so the main heterogeneous storage support OpenStack interface can be achieved and DJ docking, but the degree of support depends on the third-party storage to the interface implementation degree.

Third Array Management

VIPR and NetApp collaborate on a private interface for FAS storage Management, and private interfaces have the advantage of communication efficiency and feature display, but product upgrades and replacements need to be customized and cost-effective. DJ relies on OpenStack and array heterogeneous takeover of third-party storage (as Huawei Device Management), the technology is more open, but the currently supported heterogeneous array is not VIPR.

Data protection capabilities

VIPR integrates Vplex,appsync, RecoveryPoint, and storage-value-added features, which are significantly more advantageous in dual-live and DCP. DJ's basic data protection suite, such as Ebackup, storage replication, third-party backup software (CV), and storage-value-added features, has some advantages in data backup, especially the integration and enhancement of Fusionsphere OpenStack version storage capabilities, Can achieve the cloud Platform data backup and disaster recovery capabilities.

Automation capabilities

VIPR and DJ have almost the same capabilities in business templates, service catalogs, and resource management, but VIPR has the advantage of automatic resource mounts and currently supports automatic mounting of AIX, Aix VIO, Linux, Windows, ESXi, and other servers. But DJs are currently limited to Linux and ESXi servers.

Open capacity

VIPR and DJ are more open platforms, open interfaces support the third Fang Yunvi, management platform. However, VIPR is currently working with VMware to provide both programmatic and functional enhancements to support Vasa, VCenter, Microsoft Systemcenter, and more. DJ based on OpenStack development, natural compatibility OpenStack, in addition to support Vasa, vCenter, manageone, etc., so in the open ability DJ has some advantages.

Service and operational capabilities

VIPR primarily provides NAS, SAN, HDFS, Object Services, and disaster recovery services. The DJ mainly provides SAN, NAS and data recovery service capabilities, the object should be in the development phase. With regard to management and operations, VIPR integrates SRM and SAS for business origination, resource management and quality assurance, while DJs can flexibly integrate OpenStack, Manageone, esight for business automation, cloud Backup, Yunjong disaster, and resource monitoring and management.

Warm tips:

Please sweep the following QR code to follow the public number for more exciting content.

VIPR and DJ lead software-defined storage