Gradual evolution of cloud storage and data center Storage

Gradual evolution of cloud storage and data center Storage

For any enterprise, it is unlikely that 100% of their data will be migrated to cloud services. However, most enterprises are willing to use the advantages of cloud storage to store data. Using cloud storage as a hybrid cloud means that the data stored in the data storage can be seamlessly integrated with the data stored in the cloud. Integration with cloud storage can be implemented using embedded software, cloud-based applications, or embedded into storage systems or cloud gateway products.

Why cloud storage?

This year, we may have seen with our own eyes that the popularity of public cloud has surpassed development projects and Web 2.0 companies and has become the mainstream IT business. Cloud Service Providers gradually show great advantages in terms of elasticity, flexibility, capacity scalability, and cost. Naturally, unavoidable security, competitiveness, long-term costs, and performance concerns remain. In addition, not all applications or loads are suitable for migration to the cloud, and most enterprises still cannot deploy the IT architecture completely on the public cloud. However, these concerns have actually guided us to the hybrid cloud solution and tried to integrate the two best solutions. Research Reports from Taneja support this view, with only about 10% of corporate IT departments considering migrating the entire environment to the public cloud. Most IT departments are willing to deploy the future IT architecture on the hybrid cloud solution, and add the cloud to the local architecture, at least in the next 3-5 years. However, in these same IT departments, storage integration, virtualization, and cloud service creation have become the main tasks of the next 18 months. These main plans tend to use available public cloud services to support web applications and mobile users, and implement sharing, deep archiving, and offline backup functions, disaster recovery can be used as a service or in another sense as the primary storage layer.

For many IT departments, IT is estimated that the data to be stored, managed, backed up, or processed will increase at twice the annual rate. However, the capacity of the data center is actually visible, such as management personnel and budget. With the development of time, it will become increasingly difficult for data centers to manage these increasing data year by year.

Make good use of cloud storage

Many may ask why cloud storage is used, however, the convincing answer of many enterprises is that they use cloud storage as their data storage space as their storage capacity needs increase. In this way, you do not need to completely migrate the entire data center or application to the cloud, but you can achieve a hybrid cloud at the storage level. For most enterprises, the most appropriate method in the next few years is hybrid use of self-built data centers and cloud architectures, that is, hybrid cloud.

There are actually some different ways to build a hybrid cloud storage service. One of them is to simply migrate some methods, such as user file synchronization and sharing, to the cloud, and put those applications with performance requirements locally. Some applications can use similar EBS provided by AWS or their brand new EFS, or use SoftNAS services that are managed by their IT staff and stored on the cloud.

The most popular cloud storage is based on object storage. Nowadays, many modern applications support applications deployed on the cloud or locally. Currently, HTTP storage protocols, such as restful APIs, can be used to directly access object storage on the cloud.

Why is cloud storage used?

There are many convincing reasons for the IT department to use cloud storage services. The following highlights the problem:

Low capacity cost per GB

Provide unlimited capacity services as needed

Cost-based Elastic subscription

Few or almost no Capex Investment

Distributed access

Distributed replication by region

Low operating costs

Cloud-based data recovery for Disaster Tolerance

Almost all cloud platforms use object storage products, whether they are private or public clouds. Many OSS products can be deployed in a hybrid manner, either in a self-built data center or in the cloud. AWS S3 storage is an industry-leading oss api, And the OpenStack SWIFT interface is also in its development direction for OpenStack cloud creators.

Similar to cloud storage, and able to implement distributed object storage solutions, it is a software-defined storage product that can be distributed from a local data center to cloud storage, and is based on storage nodes of applications and virtual machines. Based on the type of hybrid cloud architecture to be built and the storage service level to be provided, a solution similar to Maxta, Nexenta, Qumulo, or Tarmin can be used as an option.

The door to the cloud

If the business application cannot directly manage the cloud architecture as the cloud develops rapidly, cloud gateway is a good choice. The cloud storage network looks like a traditional array, but it exists between the cloud environment and local storage as a high-performance local cache. CSG can be physical or virtual. In many scenarios, traditional data block-based and file storage products can be seamlessly replaced.

The gateway can be designed as a local cache or hierarchical algorithm. Some are based on the Operation habits of recent users, some are based on application design, and others can be linked with snapshots stored on backend cloud storage. Most of the data deduplication and compression technologies are embedded to support more optimized data transmission services. Although the effect of such operations remains to be discussed, it may even be somewhat different from the WAN optimization technology. Some cloud gateways even use advanced I/O log and cache functions, I/O optimization, and non-peak transmission plan functions.

These differences actually depend on the gateway storage method. For example, Gateway storage can be used as a large-capacity backend storage to provide Object Storage with a capacity of hundreds of TB, such as SteelStore like MetApp. Alternatively, it can be used as a front-line array to replace local primary storage on a remote office site. In the following ways, the IT department can reduce the number of problems, because the data stored on the remote site is not only cached locally to improve performance, but also can be automatically synchronized back to the cloud, data protection and access at any time are both quite good.

For larger data centers, long-term cloud storage gateway providers such as Nasuni and Panzura reflect their advantages with high cost performance and high availability NAS to replace the disadvantages of a single global name. For this solution, the core issue is how to deal with regional distribution, cache, document version management, or file lock rules.

Array backend connection to cloud storage

Another hybrid cloud approach is Microsoft's StorSimple, which uses a local block storage array and can be integrated with the back-end cloud storage.

Most traditional storage vendors may gradually integrate cloud storage layers into traditional arrays. According to a recent Teneja Group survey, most enterprises expect to use the cloud as a supplement to the existing storage architecture in the next three to five years. Similarly, both EMC and NetApp have acquired ready-made cloud storage gateway products, namely the SteelStore of TwinStrata and Riverbed.

Is Cloud storage suitable for you?

Before using cloud storage, the IT team should first consider what they actually need to achieve. When constructing a hybrid cloud solution, the following points need to be considered at the core.

Network connectivity and bandwidth. As the connection between the data center and the cloud environment, the network link is the most influential factor to the performance, availability and cost. In essence, the network is not reliable. When considering which and how much data can be stored in the cloud, you need to evaluate the end-to-end availability, such as the front-end cache and WAN optimization functions implemented by deduplication and data compression technologies. These features can be integrated into a specific network device or embedded into the cloud storage gateway, but it does not mean that you both need them. For large cloud environment data replication and migration requirements, you can consider the CloudBeam provided by Attunity, which is mainly used to reduce large data stream scenarios.

Real-world cloud storage

Cloud storage scenarios include:

Backup storage target of the active/standby site

Implement archive access for data that requires on-demand access

Provides data storage services for cloud-based analysis processes

Provides primary data services for web and mobile applications

Distributed file synchronization and sharing

The Data Warehousing service for video and image files usually uses the coding/decoding and processing services in the cloud environment.

Cloud storage as the back-end level of the main site storage

Security. Find the integrated authentication function with the user-created environment. Most cloud storage vendors and products can implement data encryption services during transmission and after storage. However, you need to know where the key is stored and how the key is managed and protected. Are there policies that can automatically strengthen security measures and comply with regulations?

Data access perspective. Starting from the accessible angle and width. Does your data only need to be accessed from the data center or do you need to enable cloud-based processes to ensure global file sharing? Which of the following data can be copied to mobile and distributed users through encryption and decryption?

Cost and data crawler control. Cloud storage may be used on a large scale accidentally, but it also means that the cost may become very high in the long term. Let's take a look at the cost allocation and data crawler management features provided by cloud products.

Performance. Can cloud storage performance be allocated based on user and load requirements? The performance requirements of end users for file sharing and the performance requirements for database Request responses on sales terminals are completely different.

Migration. Consider whether your hybrid cloud solution needs to point out multiple public cloud service providers. If there are two at a time, what should you do?

Finally, I believe that in the near future, cloud storage will become part of the data center level and there will be more and more cases. Therefore, the question may not be "What if this happens", but "how to better use cloud storage ". A cloud storage gateway is a good start to use public cloud storage, which needs to be combined with the long-term IT strategy of enterprises, especially the need to be combined with the final use of globally Scalable NAS.