Cloud Applications and Cloud Talent

Column message:

Cloud computing has opened the door for us. Enhancing cloud technology, adding cloud applications and highlighting the value of cloud, "China Computer News" and Microsoft (China) Co., Ltd. jointly organized the Windows Server cloud Chuangyi Competition, which lasted more than five months attracted a total of nearly ten thousand players to sign up, 103 teams participated in the enterprise level competition. The official website of the high volume of visits as much as 1.8 million times, the official microblogging interaction reached more than 10,000. Competition for the IT people to understand a cloud, cloud applications, experience the exchange of cloud, interactive, sharing platform. You think, cloud to achieve. Through the competition, more cloud talent showed their ability to use more cloud applications. This column will serial contest, the final award-winning 10 enterprise-class programs, used to demonstrate benchmarking applications, promote industrial development for a better future.

"The data is the foundation.If the data is not efficiently carried, the application is no better, will not achieve good results." Qingdao Customs cloud architect Ren Lin believes.

"Data processing is a very important capability because before the customs data were dispersed across departments, we wanted to find a solution that could be put together to build a system that would enhance the overall data processing power of the customs." The program at Linkou is the work of his team (XRZ team) participating in Microsoft's Windows Server 2012 Yun Chuang Yi Competition team race - the dynamic data warehouse high-performance cloud computing platform (referred to as the data warehouse cloud platform).

In fact, the dynamic data warehouse project has been operating in the customs for many years, but using a distributed deployment. There are currently 46 directly under the customs units nationwide, 600 under the customs and offices, customs clearance nearly 4,000 points. Decentralized data management is obviously not convenient for unified control, the optimal allocation of resources and other shortcomings. As cloud computing technology matures today, the customs technical team including Ren Lin begins to think about whether cloud computing can be used to provide data services to users in a cloud-based manner to achieve centralized data services, reduce costs and increase efficiency and genuine The meaning of the dynamic control.

"In the past, we had real needs, but there was no effective way." Ren Lin told the "China Computer News" reporter that "the introduction of Windows Server 2012 provides us with ideas for solving real-world needs and provides achievable means."

Data warehouse cloud

Cloud is not only floating in the sky, it can be appreciated by you and touched. Especially with Windows Server 2012 this is the system software released by Microsoft as the kernel of the cloud operating system, Ren Lin et al put the customs data warehouse cloud on the agenda.

Ren Lin introduction, they first cloud infrastructure, and then based on Microsoft Parallel Data Warehouse (Parallel Data Warehouse, referred to as PDW) for high-performance data storage and management, the data processing execution from the original node to the cloud engine and The application has been redesigned to service to the user. In this way, the design of the CTS cloud platform based on multiple cloud data centers including Beijing, Shanghai and Guangzhou came into being. It also includes the dynamic data warehouse cloud portal, the dynamic data warehouse cloud engine and the top-level web application cloud deployment center.

"We've done a lot of experimentation with cloud computing in dynamic data warehousing and now we do some work based on Microsoft Windows Server 2012 and Azure architecture and have set up a data-centric management system to provide our users with a high level of efficiency Platform to support our entire customs application. "Ren Lin said.

Provide data as a service

"Big data is the hot spot in the industry right now, but our plan is by no means a follow-up and hype, all of which are our actual needs and our real applications," Ren Lin said.

Ren Lin regarded data processing capabilities as one of the most important capabilities in the information system. It can not be separated from the practical application of the Customs. The national customs system generates a large amount of structured and unstructured data every day. Customs terminals throughout the country should use these data for business handling, analysis and inquiry. "At present, the overall data volume of the platform has reached the scale of hundreds of terabytes, and the next step is to migrate the unstructured data based on the Hadoop architecture to the platform. According to our plan, the platform can support petabytes of data." Ren Lin said.

According to Lin's team test, in the customs data warehouse cloud platform, the 4TB data amount of the two tables combined query only takes 29 seconds, and the 10TB data amount three tables combined query takes only 1 minute and 30 seconds. Relying on the cloud computing platform provided by Windows Server 2012 and the high-performance data warehouse provided by PDW, the integrated management service platform of the Customs provides various data processing functions such as ad hoc query, fixed query, multi-dimensional analysis and report indicators. Cloud computing and high performance, but also allow all departments of the Customs efficient linkage approval possible. For example, if a business person finds a risk point in a business through an indicator analysis, he can immediately contact the disposal order and ask the department to handle it.

In fact, the concept of "all services are provided by customs cloud" is not only reflected in the infrastructure level, but also fully reflected in the application level. "Our goal is to use cloud computing to allow users to submit requests and receive results without having to worry about the specific operations involved." Lin said that "Windows Server 2012 and other Microsoft products helped us to achieve this A concept of our own to provide a visual interface, users do not have to write the code to complete the operation of the integrated business management platform.For example, in the generation of SQL queries, the user can completely use the drag and drop to select the various tables and fields, Very easy to complete a variety of custom queries. "

Unified management of various platforms

Ren Lin believes that the greatest value of cloud computing is to maintain business continuity. "Windows Server 2012 basically has the characteristics of the cloud operating system, it can provide effective management of the underlying, providing sustained support for the upper-layer services."

"Because of the complexity of the customs IT system, we introduced a gradual approach to Windows Server 2012 and put a lot of energy into compatibility testing." To ensure the high availability of data warehouse cloud platform services, Ren's team experimented , Ultimately based on Windows Server 2012, System Center 2012 established a more compatible resource management system. "Through Windows Server 2012 and System Center 2012, the platform can support the software and hardware of all major vendors and the basic environment, for example, we can manage systems such as Linux and VMware and basically meet the demand." Ren Lin introduced to reporters.

Windows Server 2012 enables real-time task management of an entire data center with one node, a management approach that allows customs operations staff to easily manage large-scale server clusters in one location. "In the past, server cluster management required from this console to that console, even from that desktop to that desktop, and now one desktop can manage thousands of devices in the data center." Ren Lin said excitedly, "Right This is a cool feature and useful for O & M engineers. "

It is worth mentioning that, the current platform is the use of Microsoft System Center 2012 Virtual Machine Manager (Virtual Machine Manager, referred to as SCVMM) for unified management. No doubt, SCVMM heavyweight tool in the new version has been a new upgrade. "It allows us to manage multiple vendors' products at a lower cost, enabling operators to dramatically increase their productivity without having to face multiple different management interfaces every day," said Ren Lin.

Achieve the optimal allocation of resources

Efficiency comes not only from high-performance equipment and management tools, but also from the design of data warehouse resource allocation. According to the business requirements, Ren Lin's team divides the application of the dynamic data warehouse of customs into three levels: the highest-level application is based on PDW, the middle-level application is based on the database cluster constructed by Qingdao Customs, and the low-level application is based on SQL Server.

"There are a lot of businesses in the customs that need ad hoc queries, business terminals need immediate inquiry and immediate return of results, which requires rapid implementation through PDW." Ren Lin told reporters, "and there are some fixed queries, such as some The report is generated once a month, so it can be arranged at a lower application level to complete, or even ahead of schedule. "

In such a solution to data management at the level of business needs, it not only improves the data management efficiency, but also improves the utilization of hardware resources. Ren Lin introduced in the data warehouse cloud platform project, they gradually introduce new systems and equipment, and the original equipment has been integrated transformation, arrangements for relatively low-level work, so that IT resources are better configured.

The essence of cloud computing is to make more efficient use of resources. In terms of resource utilization, Ren Lin's team actually has a greater vision. "If we can fully allocate IT resources through cloud computing as the public management and service department, we hope to sort out the internal IT capabilities and eventually release the capabilities to provide the public with a wider range of services," said Ren Lin.

Program highlights

(1) The program allows the value of Windows Server 2012 in a large-scale application scenarios to fully reflect the high availability of the platform.

(2) The scheme is also suitable for medium and large scale applications, combined with PDW, to achieve the system scalability and high performance.

(3) to achieve safe and efficient unified management: multi-center, unified network management outside the network, multi-server remote management.