Microsoft Big Data: SQL Server as main Azure and Hadoop integration assistance

Data such as video, music and text have been expanding indefinitely, driven by surging networking devices and ubiquitous Internet connectivity. A recent report by Gartner, a research institute, says "Big data" will become a new conventional industry in the next 10 years. At present, including Google, IBM, Microsoft, EMC, Hewlett-Packard and many other giants, has begun to layout large data, in the coming Big Data era to do a good job in the foreshadowing.

How to use massive data to bring value to the enterprise is the main focus of Microsoft's current, but also the core elements of large data. Through data collection, storage, transmission, processing and analysis, to help enterprises to develop more effective policy and policies, which in the global enterprise demand is very strong, the global 50% of corporate leaders have such awareness. Microsoft has spared no effort in this regard, said Feng, director of the China Cloud Computing Innovation Center at Microsoft Asia-Pacific Research and Development Group.

"Many companies choose to use public clouds to handle complex data through the extranet in order to save management and operating costs, but this is bound to bring security problems," he said. But if the enterprise adopts the private cloud under the net monitor, it can not only guarantee the data security, but also improve the efficiency of the data center. "So it seems that the hybrid cloud is probably the best way to do it," Feng said. It can be based on enterprise business characteristics, which applications and data need to be handled through the private cloud, which needs to be handled through the public cloud, and the final data management storage is implemented in private cloud mode. ”

This has created Microsoft's Big Data mindset: A SQL Server platform, integrated with the Windows Azure Public cloud and Hadoop system, to form a complete solution covering the entire industry chain.

Data acquisition, data storage, processing analysis by Microsoft as a large data three life cycle, but also large data critical three links. Especially in the storage link, large data requirements are very high, its security, not discontinuous, sustainable and scalability must be guaranteed. According to Feng, Microsoft's large data layout is mainly based on SQL Server platform. In May this year, Microsoft has just released a 2012 new version of the platform, which is excellent at ensuring data continuity.

SQL Server can handle more than just structured data, and other types of data can be easily handled and can even customize data types. For relational data, queries and indexes can be made using a variety of techniques, such as full-text queries, unstructured attribute queries, and later managed and stored through a variety of technologies to make better use of the data. Different from the complexity of relational data, the non relational data will pay more attention to the reading of data and simple attribute query, and do not need too complicated relational operation. "SQL Server currently has the advantages of performance, efficiency, and ease of operation in storing data that is not relational." ”

When the data is stored in place, it needs to be packaged into a systematic data report to form a valuable reference document for the enterprise. Wang Feng thinks, similar to the concept of application store, large data also need "data store". On this platform, no matter suppliers, integrators, collectors, developers, can share data with other enterprises, and enjoy the platform of other kinds of data resources. On Windows Azure platform, Microsoft, in addition to its own data store, has enriched its data store platform with the integration of SQL Server and Azure and with Hadoop. The public cloud domain, Microsoft has successfully occupied some market space by Windows Azure. Today, the domestic pplive global video push is to work with Microsoft to provide computing resources, storage resources and CDN services.

In addition, Microsoft, in addition to having its own independently platform, integrates with today's world's leading distributed system Hadoop to create a more open and open-source platform for the industry. Microsoft, for example, integrates the Active Directory of SQL Server with Hadoop to further improve the security of the data. The integration of the main management tools, such as cloud computing, with Hadoop makes it more sophisticated and flexible in its management style. In addition, with the integration of SQL Server storage and business intelligence and Hadoop, Microsoft is able to better present the data that Hadoop has processed to the user. "The integration of Microsoft and Hadoop has been largely played out. Feng so talked.

Microsoft's large data program has been able to penetrate all aspects of the industry chain, which can be embedded in each device through windows, the various data services generated for it. In recent years, Microsoft's big data has many success stories all over the world. China as a very important market for Microsoft, the success of large data cases are not a few. For example, Chengdu is currently operating in the Cloud Computing Center, the use of large data platforms, virtualization, BI Business intelligence analysis, such as a series of technical means to achieve the Chengdu pork from feeding to the sale of real-time monitoring. According to Feng, the cloud computing system can be deployed within a week and reduce deployment costs by two-thirds.

(Responsible editor: The good of the Legacy)