Big data and cloud computing are inseparable coins on the opposite side

"Big data also needs to be a platform for cloud computing, which is the opposite of a coin," he said. Aliyun President Wang Wenbin (nickname: the Philippines) and the media, said. The IT industry has changed dramatically over the years, until now people are still talking about cloud computing. The cloud concept emerged, the industry has been the voice of doubt, with the efforts of the major cloud service providers, now all walks of life have begun to accept cloud computing services. July 2014, Aliyun ODPs Project was officially opened to the outside.

Along with the Internet and mobile Internet related technologies are maturing, cloud computing began to be accepted by the market, a huge tide of data, manufacturers and companies have seen the prospect of large data, we now live in a data era.

Traditional it has been overturned

Aliyun has been studying cloud computing services for five years since 2009, and Aliyun has broken through a variety of technical challenges in the five decades, from last year's 5K cluster to ODPs this year, Aliyun has become a leader in the cloud services industry. In Aliyun's view, cloud computing and large data are inseparable, just like the two sides of a coin, accurate large data analysis relies on a strong cloud computing platform.

Aliyun President Wang Wenbin said: "The arrival of large data and cloud era subvert the traditional IT architecture, enterprise it into the cloud services + data era." Traditional It also exists on the basis of purchasing hardware and software, while cloud services transforms enterprise it into on-demand purchases, from the purchase model for hardware and software to a unified purchase of services. With the advent of the mobile Internet plus the Internet, the entire model is now a sweeping change.

The changes that big data brings

Big data has become civilian, spending a hundreds of of dollars, everyone can use large data services, playing big data is no longer the exclusive skills of large enterprises. After five years of accumulation, Ali Cloud computing issued a nuclear-level large data products--odps. Through the ODPs online service, small companies and small teams can quickly analyze and excavate the vast amounts of data they have. Aliyun President Wang Wenbin said that ODPs would change the whole of China, regardless of cloud computing, or the entire big data milestone, so we have always felt this thing.

What is ODPs?

In short, ODPs (Open Data 處理 Service) is a Web services that can analyze massive amounts of data without spending a lot of money building data centers. We have measured that the 100PB data task can run out within 6 hours. This amount of data is equivalent to 100 million high-definition movies.

How do you make data valuable? First you have to have a large processing capacity, then the mining, algorithms and analysis. The traditional approach is this: rent a computer room, buy a bunch of expensive equipment to build a data warehouse, and then ask a bunch of technicians to maintain the operation. Once a bug is triggered, or when the amount of data exceeds 100TB, your engineers may be devastated by these problems.

Hadoop is a great open source system that lowers costs considerably. Ali is one of the best companies in China playing Hadoop, and Hadoop supports the rapid development of Taobao and Alipay early business. However, since building a decent hadoop cluster, millions of start-up funds, professional hadoop talent is scarce. The threshold is still too high.

What can ODPs do?

In the country, say better than to have a good case. ODPs has done a lot of use and testing inside Ali. Inside Ali, the first use of ODPs is Ali small loans, Ali small loans to deal with 30PB of data every day, including shop level, collection, evaluation and other 80 billion information items, operations more than 100 data models, even to assess the small business owner of the hypothetical situation to hide and lie to the extent. After a long period of testing, ODPs to help small loans to complete a large number of data calculation and analysis of the work, so as to grasp the correct reputation of the user.

In the future, when people apply for loans, the machine will be able to complete the process, according to the large data analysis of the personal credibility of the automatic completion of the relevant credit evaluation.

Data mining in the public domain, can you use ODPs? Of course! With the China Meteorological Bureau's accurate forecast data, the gold map will soon tell you: "The road ahead has been seriously stagnant water, your vehicle may be flooded, suggest a detour." "Nowadays, every box of drugs has an electronic ID card, from production, circulation, storage, distribution, sales to use, the whole process of data are running on the ODPs, once the problem of drug discovery, the regulatory authorities can take immediate action." We look forward to the future of every barrel of oil, every dish of data are running on the ODPs, food safety problems need to be solved by innovative means.

Explore Aliyun's "Royal Dining Room"

Based on ODPs, Ali for Third-party software providers and brands to provide large data computing, mining, storage cloud environment development platform to build Ali data ecology. Through the royal Dining Room data market, data consumers and data providers can securely trade and use large amounts of data to achieve data value.

1. Rich Taobao data. At present, the Royal dining Room has opened goods, businesses, customer service performance, brand four themes of data, the follow-up will continue to open the industry, competing products and other thematic data; Developers can also upload personal data to use.

2. The indicator standard is reliable. The open construction of the Poly Stone Tower Data Standards Alliance, Royal house provides perfect and accurate basic indicator definition, calculation caliber, inspection tools, etc. to ensure data standards, the only reliable.

3. Self-help calculation. Strong and stable computing ability, provide off-line distributed computing platform and powerful algorithm environment, submit computing task autonomously, develop model mining data value independently, large data compute fast response.

4. Safe Operation platform. Royal Boarding House supports isolated data storage, independent data task deployment, and ensures that data transactions and data development are safe and leak-free in the royal dining Room.

5. Flexible data interface. The developer can customize the API according to the requirement flexibly, and the data output conforms to the top API specification.

6. Large data mining tools. Royal House provides R, Python, MySQL, the mainstream of large data mining tools, support model research and rapid iterative testing, improve data value.

Ali's internal moon landing program

October 2013, in order to fuse Ali small loans and Alipay data, Alipay hope ODPs team to help them move, will pay a few warehouse business from the Hadoop group to move to ODPs, the lunar 1th project launched. In May 2014, the 1th project on the moon was successful, and all the data services of microfinance began to be based on ODPs development.

At the end of 2013, inspired by the 1th project on the moon, Ali data Platform team joint technical Support and Group division, began a series of ambitious moon landing program, is committed to search, advertising, logistics and other BU data unification, the future ODPs will become the bearer of Ali group all the data of the unified processing platform. "Moon landing Program" a total of more than 20 projects, involving Alibaba and small micro-gold clothing all the division, covering the entire group of data personnel, its involved personnel, resources, the group within the rare.

From Oracle to Hadoop, Aliyun solves the problem of how massive data is stored and analyzed, and Ali's data business is no longer constrained by the bottleneck of scale; from Hadoop to ODPs, it is a qualitative leap, which clears the way for the subsequent development of large data business.