Big Data virtualization starts from scratch-1

Virtualization of big data: enterprise IT Development Trend

Virtualization of big data is a development trend of big data and the Hadoop community. Gartner mentioned at the Hadoop Summit conference held in June 2013 that in order to let Hadoop and other big data technologies truly fall into the enterprise, we need to start from the specific business needs, drive the further development of big data-related technologies and products, and gradually say goodbye to technology-driven innovation. Gartner also pointed out that virtualization is an important trend in this new phase. As the proportion of global IT virtualization exceeds 2/3, virtualization-based software-defined data centers become increasingly popular and important for enterprises, in this wave, how big data influences and integrates the infrastructure of existing enterprise data centers has become a real challenge. This blog will focus on this topic from the O & M, technology, and solution perspectives.

650) this. length = 650; "src =" http://www.bkjia.com/uploads/allimg/131228/005QUN8-0.png "title =" Picture1.png "width =" 600 "height =" 335 "border =" 0 "hspace =" 0 "vspace =" 0 "style =" width: 600px; height: 335px; "/>

What is Big Data virtualization?

To answer this question, we must first review why enterprise IT needs to be virtualized? I think the reasons are as follows:

1. virtualization can significantly improve server utilization and achieve better utilization by integrating server resources.

2. The cost of ownership of virtualization, represented by x86 servers, is more cost-effective compared with minicomputers and integrated hardware and software devices. In addition, the performance is not inferior at all, and horizontal scaling is a huge advantage.

3. virtualization plays an important role in both public and private cloud computing. Without virtualization technology, the elasticity of cloud computing and the real implementation of multi-tenancy are often difficult.

4. virtualization can already support key enterprise applications such as ERP, email servers, and business production databases. This proves that there is no need to choose between virtualization and performance stability. In addition, numerous success stories and technical whitepaper can help more customers enhance their confidence. The sign that virtualization is becoming fully mature has been established.

Obviously, the process of Enterprise Virtualization will not stop. leading vendors including VMware are currently expanding to virtualization 2.0. Virtualization, including storage and network, has seen the most cutting-edge innovation in the previous isolated islands that are relatively difficult to directly use for virtualization, for example, "software-defined data center", "Storage Virtualization", "Network virtualization" and other hot spots have already seen specific products and solutions.

Big Data virtualization is to run or migrate Big Data workloads to a virtualized basic environment. In addition to the general advantages of virtualization mentioned above, it is worth mentioning several special advantages:

1. Since the big data infrastructure is often difficult to determine how many computing and data nodes are needed at the beginning, these nodes need to be heap one by one with physical servers. Without the support of an expert team, it will be very time-consuming and labor-intensive, and will be inconvenient to expand in the future, with extremely low utilization and outstanding management efficiency issues. Virtualization can not only quickly deploy clusters, but also flexibly manage them and significantly improve utilization.

2. Big Data uses both shared storage and local storage to improve performance. Virtualization can fully meet these needs and allow us to flexibly expand and design policies.

3. virtualization can form a multi-tenant and Data Analysis Service from the underlying level of big data, effectively isolating the computing environment and laying the foundation for promoting big data as a service.

4. virtualization also facilitates the integration and integration of other data applications on a unified virtualization platform, greatly reducing the complexity of the IT infrastructure and O & M costs.

I think the above not only explains what is Big Data virtualization, but also shows the existence value of this market. So what else do we need? Knowledge and skills. The biggest problem enterprises face is not their real needs, but they do not have professional talents to discover and handle it. 57% of enterprises think they urgently need talents to master specific technologies and knowledge. At the same time, management and security are also a major challenge, accounting for 37%. These figures confirm the necessity and value of virtualization. Data from Microsoft's report on Hadoop Summit 2013)

On the new topic of big data virtualization, I think there may not be many people on the market who can understand how to implement it and what technologies and products are needed. If you don't understand it, it will easily lead to subjective speculation. You feel that big data and virtualization are in conflict, and even feel that it is "unreliable" to combine the two. I will elaborate on how to implement big data on Virtualization through a series of blog posts to be released, so that readers can understand the relationship between the two and help them solve their doubts. Therefore, the next series of blog articles are technical "dry goods" and will guide readers or enterprises interested in further understanding and trying to use this field.

If you have any questions, you can send an email to the bigdata_apac@vmware.com.

About vSphere Big Data Extensions:

VMwarevSphere Big Data Extensions (BDE) supports Big Data and ApacheHadoop jobs based on the vSphere platform. Based on the open-source Serengeti project, BDE provides enterprise users with a series of integrated management tools. By virtualizing ApacheHadoop on vSphere, it helps users flexibly, elastically, securely, and quickly deploy, run, and manage big data on their infrastructure. Understanding