Automation tools for large-scale public cloud

Cloud computing is not just a fast self-service service for virtual infrastructure. Developers and administrators are looking for ways to supply and manage large-scale cloud computing. This Infoq article is one of a series of automated tools and ideas for dynamic computing resource pool maintenance. You can subscribe to a new article in this series to post a notice here.

The typical deployment of early cloud computing was that one or two employees used several servers to build a small private cloud for a particular requirement. However, as more and more employees across the enterprise use a large number of functions in various cloud service models (IAAS,PAAS,SAAS), we have seen more and more public cloud adoption cases.

As more organizations expand their use of public cloud services, they are smaller to start-ups, to the world's largest companies and governments. At the same time, the problems of large-scale cloud computing are beginning to emerge.

Potential problems with large public clouds

There is no doubt that all kinds of enterprises benefit from the adoption of public cloud, but the large-scale adoption of public cloud is also accompanied by many challenges and risks. The main things are as follows:

Cost

When the public cloud is initially used, only a limited number of people are allowed to access it, which is relatively straightforward to track costs. However, as more and more people in more (usually independent) departments gain access, you may experience repetitive features, excessive provisioning, unauthorized sourcing, unused zombie instances, extra bandwidth and storage costs, and other unnecessary factors that erode the expected cost savings.

Unauthorized access

The management of small-scale public cloud service access is relatively simple, but as the size of the public cloud is gradually increasing, management will quickly get out of control. The company's former employees may still have access after the separation, the employee's role changes, there is no corresponding update access rights, new employees have difficulty accessing the resources they need. Since most cloud service providers are unable to provide enterprise-class security, you will soon be victimized by unauthorized access as the scale of the public cloud is progressively expanded.

Malicious intrusion

More serious than an employee's access control problem is an external malicious intrusion into the cloud service. Lost passwords, shared user IDs, data leaks, simple passwords, social engineering, phishing, and malware all have the potential to expose public cloud services to data loss, tampering, attacks, denial of service, and other malicious intrusions.

Human error

Public cloud services can be easily managed manually when they are small in size, but as the scale continues to grow, it is not possible to continuously increase human resources to maintain their manageability. This means that fewer people have more work to do, and the law of equilibrium tells us that there will be some mistakes in the end. This could lead to large-scale failures, although this is not a unique problem for cloud services.

Visibility of

When there are only a few services, management can be meticulous, as long as one or two people can understand the deployment location of these services, configuration, cost, usage, affiliation, problem reasons, solutions, service shutdown time, recovery methods, and so on. However, in larger systems, cloud usage will become increasingly unclear as the size of the public cloud deployments expands and more use-case access is released.

Classification Diagnostics

One of the consequences of poor visibility is that the classification of problems has become more difficult. For example, if you do not know where the system is running or how it is connected to other services, it is basically impossible to determine the cause of the slow flow of transactions. Expert in system thinking W. Edwards Deming once said, "an immeasurable person is not manageable," perhaps more appropriately, the invisible person is not manageable.

auditable

Another side effect of poor visibility is that as more and more systems and services are abstracted into cloud services, it becomes more and more difficult to track who is accessing what, when, how and why, and the critical issues associated with auditable are followed. Without automated tools, tracking, documenting and reviewing access, changes, failures, exposures, utilization, and so on in a large cloud environment can be difficult.

Recoverability

Although serious downtime failures are not unique to the cloud, almost every week we hear reports of new and disturbing public cloud failures. However, most cloud service providers, especially commercialization services, do not have built-in recovery capabilities, and even more robust services may not provide timely recovery services or prioritize your business needs. If no system is available for backup, failover, and recovery, downtime can lead to catastrophic consequences.