The image of American cloud computing

March 19, 2012-26th, we took part in the "Cloud America Tour", organized by the China Electronics Society Cloud Computing Experts Committee, to visit a number of US cloud computing companies in Seattle and San Francisco, including Amazon, Microsoft, Google, Facebook, Rackspace such as the industry giants, Heroku, Engine Yard, puppet and other emerging cloud computing companies, but also including Cloudcamp as the representative of the U.S. cloud computing community. Although in a hurry, it is difficult to carry out very in-depth communication, but a wide range of contacts, the U.S. cloud computing industry and technology development has more intuitive understanding.

Microsoft

The first stop we visited was at Microsoft headquarters in Seattle. Enterprise IT to cloud computing has become a trend, so Microsoft and software and hardware giants IBM, HP, Oracle, Dell, are actively layout cloud computing technology, products and platforms, or acquisitions, or actively research and development.

Why is cloud computing so important now? Rolf Harms, who is in charge of Microsoft's corporate cloud strategy, gives the answer from an economic point of view:

After moving to the Azure platform, Microsoft's own official web site, microsoft.com, reduced costs by 90%, availability from 99.1% to 99.997%, and deployment time from weeks down to 45 minutes, improving dramatically.

Harms's introduction contains a picture of the different uses of the private cloud and the public cloud, very clear, as shown in Figure 1.

Figure 1 Private cloud and public cloud application scenario

In the long run, the public cloud has an obvious price advantage over the private cloud, as shown in Figure 2.

Figure 2 Comparison of private and public cloud costs

One of the focuses of the cloud computing war is how to reduce the cost of building and operating data centers and environmental issues. Facebook's Open-source Open Compute project has exacerbated competition in data center technology. Microsoft has also invested heavily in cloud computing data centers to develop a new generation of data centers. This time we had the honor to visit Microsoft in Redmond Ridge Research and Development Data center, all aspects are very advanced, although the equipment density is extremely (52u/rack), the Pue value still can reach 1.17. It should be said that it is not backward in pure equipment, but this thousands of square meters of data center only 10 people in charge of operation, while the main Quincy data center supporting the azure platform, covers an area of 470,000 square feet, operating personnel also only 35 people, the degree of automation is astonishing. In addition, because the equipment density is too large, Cisco existing switches can not support, we must develop a new generation.

Figure 3 Gartner's IaaS Magic Quadrant

Amazon

The concept of cloud computing has been in fact developed for many years, and as a term has been popular since 2006, but there is still a lot of controversy, which is similar to the domestic and international situation. Cloud Management Platform Enterprise Enstraus Vice President Bernard Golden in the United States CIO Circle in the well-known, he talked about a very interesting phenomenon: many large companies in the CIO although one aspect of cloud computing is not safe, can not be used lightly, On the other hand, SaaS software, such as Salesforce, Workday, is widely used to manage the most sensitive information such as customers and employees. But he admits that while cloud computing has become mainstream in the US, there is still a lot of disagreement and discord, because there are so many different vendors and people who say they are doing cloud computing.

Figure 4 Services issued by AWS over the years (from Jeff-barr's presentation)

But there is a consensus that Amazon has built its own strengths in the public cloud. December 2011 Gartner Research shows that AWS has maintained a high rate of product development since its launch in 2006. The AWS Cloud Platform is a leader in IaaS providers, both in terms of execution capabilities and forward-looking. And it's expanding to PAAs.

This corresponds to the alarming exponential growth rate of AWS. As can be seen from Figure 5, Amazon S3 has stored 762 billion objects by the end of 2011, with an annual growth rate of 192% and accelerating. In addition, AWS has accounted for 59% of the US market share of IaaS, according to a report by the US research firm 451Group.

Figure 5 The growth of the number of objects stored by Amazon S3 (from Jeff Barr's presentation)

Although Amazon rarely publishes the exact size figures of Figure 5, we can make some estimates from a variety of sources. Previously Accenture's Huan Liu had estimated that AWS's servers might have 445,000, but it was generally thought that the figure might be high. Last June, James Hamilton revealed that AWS's increased capacity every day was enough to support Amazon's first five years of global operations, when its revenues were close to 3 billion dollars. According to data from Deepfield in mid-April, 1/3 of American internet users now visit AWS at least once a day, and it accounts for 1% of total Internet traffic and is already the fourth-largest CDN in the world.

When visiting Amazon's cloud computing department, it was told that it now has about 2000 people (a significant proportion of whom are salespeople) and is ready to recruit around 500 people this year. Those who are familiar with the workings of American companies have roughly calculated that Amazon's revenue target this year should be at least 2 billion dollars. The 2011 Gartner estimate for this figure was 1 billion. The pace of development is indeed staggering.

The initial users of cloud computing were mainly start-ups, but now big internet companies and many large companies have turned to AWS. One of the video service providers Netfilix is a classic case, this huge traffic site completely abandoned its own data center. Using AWS to do mass storage, large data analysis and disaster-recovery backup is more: SAP, Adobe, Samsung, Ericsson, Hitachi, Trends ...

Notably, in addition to Web applications and massive data, AWS has also been exploiting enterprise cloud services in recent years, with all of IBM, Microsoft, Oracle, SAP's full range of commercial products and Open-source products deployed on the AWS Cloud service. At the end of April, Amazon also launched the App Store AWS Marketplace, which is responsible for unified billing and more user-friendly.

Rackspace and OpenStack

We also visited Rackspace's office in San Francisco. Rackspace was founded in 1998 by Richard Yoo, a Korean American, and was initially an ordinary ISP. Among traditional hosting vendors, the early transformation of cloud computing has also become another leading manufacturer of IaaS, with a market capitalisation of close to $8 billion trillion and a 2011 revenue of more than 1 billion dollars. Recently, they revealed that the total number of servers has more than 80,000 units, cloud computing has accounted for more than 20% of the total business, it is estimated that the size of the Amazon cloud platform of One-fifth.

Fig. 6 Photo of Robert Scoble, head of Rackspace entrepreneur relationship

Another important reason for Rackspace's leadership in cloud computing is that it is the main initiator and real leader of OpenStack, one of the most focused open source cloud computing projects at the moment. Some of OpenStack's core developers are in the San Francisco office. On the day they appeared very busy, not reasearchers, later learned that shortly after the fifth OpenStack version Essex released.

Technically speaking, OpenStack does not have an absolute advantage over other open source cloud technology projects. But as early as the embrace of open source, the roadmap is clear, quickly gaining the support of many vendors in need of IaaS and private cloud platform technology. To a large extent, OpenStack has become the backbone of the camp to counterbalance Amazon's dominance. Of course, OpenStack also has many successful experiences for domestic enterprises to use in community operation and ecosystem construction. In addition to Rackspace, Nebula, there are red Hat, Nicira, HP, Canonical, DreamHost, Sina, Wikipedia, Cisco and even Citrix.

In the United States, we found that the VMware-led open source PAAs Project Cloud Foundry also has a high level of concern, in addition to what everyone will say is OpenStack. Interestingly, both have set their goals to be Linux in the cloud era. In contrast, Citrix's recent donation to Apache's Cloudstack (the main founder, the author of the Sun JVM, Sheng Liang) and the veteran eucalyptus are much less popular, but they are all compatible with AWS and can be included in the Amazon camp. How the three ecosystems compete together will be one of the big points of cloud computing.

Cloud Computing Drives Entrepreneurship

More important than the cloud platforms themselves, such as AWS, they have become the cornerstone of US cloud computing and entrepreneurial ecosystems. Thanks to the presence of the cloud platform and the great opportunities of the mobile Internet, a large number of startups have sprung up, and there is a boom in Silicon Valley, New York, Boston and elsewhere. In San Francisco, for example, in a humble four-storey building we visited, Wired magazine, Wikia and engine yard. AWS's List of customers (100,000) includes many of the most popular U.S. companies (many of which are advertising platforms): Pinterest, Dropbox, Instagram, Reddit, Zynga ...

Figure 7 The most visited customer per day on AWS (from Deepfield's presentation)

The network services of these start-ups are built on the AWS platform, eliminating the "80% of time and cost to spend on it for the lights to Shine" (Gartner), which can focus on developing applications and meeting user needs. For example, Instgram, which has recently been bought by Facebook for 1 billion dollars (most of which are in real value), has adopted a large number of AWS (the host chooses Amazon EC2, and the image database uses Amazon S3,CDN to choose Amazon CloudFront, etc.). So while the Instgram has only 13 employees (the engineering team is only 3), it has built the most powerful mobile-side picture-sharing platform and even made Facebook feel threatened.

In addition to consumer-facing internet and mobile apps, cloud computing also creates opportunities for technology providers. During this visit, we bumped into puppet founder Luke Kanies in Cloudcenter and had a brief conversation. He was a senior system administrator, and after the development of 2003, writing puppet in Ruby, which was not learned for long, was one of the first drivers of DevOps. After this resignation business, because the use of open source model, the first three years the company basically no income, is struggling to survive. Today, Puppet has been used for thousands of companies such as Google, Twitter, and Zynga, with a cumulative financing of $25 million at the end of 2011. Puppet's rival, the chef of Amazon's disaster-management master Jesse Robbins, is now the Opscode's chief community officer. Alex Polvi, head of Rackspace's San Francisco office, was also a successful entrepreneur who was a system administrator, and his cloud management and monitoring service Cloudkick was rackspace last year.

Figure 8 AWS is more than just IaaS

Notably, Amazon is now not limited to the infrastructure layer. It is also clear from figure 8 that it is also evolving to the platform layer, adding a lot of management and monitoring services. It's no surprise that the IaaS is admittedly not profitable (albeit higher than Amazon's old line), and PAAs and SaaS are fertile ground. And from a user's point of view, one-stop service is always attractive.

But the various Third-party PAAs services based on AWS are still thriving, and our visit to engine yard and Heroku are excellent representatives.

Engine Yard is the oldest and most modular PAAs platform with Amazon investment. There are more than 2,400 paid subscribers, including Apple's itunes platform in Europe. Currently 90% of applications are Ruby, followed by PHP and Node.js.

Heroku, a PAAs company renowned for its user experience and expertise, initially supported Ruby (Ruby founder Matz as its chief architect) and now covers Python, Java, Scala, and Node.js. Sold to Salesforce for 212 million dollars in 2010, it is one of the biggest acquisitions in the cloud computing platform. But interestingly, the takeover has been going on for more than a year, and Heroku is still operating independently, moving from its original address to a bigger building, completely keeping up with startups, and the platform has no intention of moving from AWS to its own Salesforce infrastructure. It is also interesting that Facebook is not a self-built platform, but Heroku in September 2011 to provide PAAs services to Facebook's social application developers. For this collaboration, Heroku specifically added PHP support. and cooperation also brings huge rewards for Heroku: The number of users on the platform, more than 1.4 million applications.

There is competition, but cooperation and win is the mainstream, we put more energy in products and innovation, to provide users with a better experience. This kind of harmonious coexistence phenomenon, especially makes us those who have been accustomed to the disorderly view the Chinese people to be impressive.

Today, a common technician can use the cloud computing platform for a short time, have the same computing resources as the Giants ' rivals, and realize the dream, which is the real value of cloud computing. We need to work together.