Xie: The visible Future--10 talking about the age of large data

I. Welcome to the Big Data age

If your site has tens of millions of active users, if the site has hundreds of product services, if these services in the form of multimedia on multiple network terminals run, If you are or are going to try to move towards cloud computing and WEB2.0 architecture, congratulations, because you have unwittingly become the forerunner or victim of the big data age.

According to the Wikipedia definition, the so-called "big data" in today's internet industry refers to the phenomenon: the day-to-day operation of a network company generated and accumulated user network behavior data "growth so fast that it is difficult to use the existing database management tools to control, the difficulty exists in the acquisition of data, Storage, search, sharing, analysis and visualization. "The volume of these data is so large that it is not measured in terms of how many g and t we know, but rather in terms of P (1000 t), E (1 million t) or Z (1 billion t), which is called large data."

Large data phenomena in physics, biology, environmental ecology, automatic control and other fields of science and military, communications, finance and other industries have been around for some time, in the Internet industry has emerged in recent years has been gradually noticeable. This can be attributed to three basic reasons. First, the rapid growth of network users and the continuous extension of user average network usage time, this makes the user network behavior data increase greatly; second, the Network Service from a single text form to the image, voice and image multimedia forms, resulting in a large increase in data; third, the network terminal from the past single desktop into a desktop, tablet computer, Books and periodicals readers, mobile phones and television and other terminals, greatly expanded the content and scope of the network services, greatly improve the user's dependence on the Internet, also greatly increased the amount of data.

The emergence of large data brings both opportunities and challenges to the network industry. From the potential opportunity to see, the increase in data for network companies to provide a precise grasp of user groups and individual network behavior patterns of the foundation, if fully utilized, you can explore personalized, individualized, accurate and intelligent advertising push and service promotion services, Create a new business model that is several times or even dozens of times times more cost-effective than existing advertising and product promotion formats. At the same time, network companies can also through the grasp of large data, looking for more and better to increase user stickiness, the development of new products and services, reduce operating costs of methods and ways. From the realistic challenge, it focuses on the following three aspects.

First, big data challenges the ability of network companies to make strategic decisions. The sharp increase in data volumes requires not only an increase in spending on infrastructure such as bandwidth and storage, but also a dilemma for internet companies. If the adoption of inaction, adhere to the original ostrich policy, it may lose the opportunity for future development, the loss of competition in the industry, sooner or later the industry will be eliminated or live in the downstream; if the times, the transition to adapt to the arrival of large data era, it needs to the company's existing products and operating such as the rebuilding of the website architecture, the generalization, standardization and modularization of the products, the innovation of the business model, etc. For the vast majority of network companies, it is a dilemma to maintain the existing business, maintain the stability and growth of performance, and increase investment and rapid transformation.

Second, the large data challenge the network company's technology development and data processing capabilities. The emergence of large data and the potential business value not only require network companies to use specialized database technology and dedicated data storage equipment, but also requires specialized data analysis methods and usage systems. At present, general data mining methods and general business databases are not able to meet the challenge of large data age. Moreover, network companies need a lot of high-end professionals, not just the average programmer and database engineer, but also astrophysicists, ecologists, math and statisticians, social networking scientists, social behavioral psychologists, and so on. The analysis of mass data can not only be confined to the level of general data law and model, but also have the comprehensive and deep ability of theoretical thinking and overall grasping.

Third, large data challenges the organization and operational capabilities of the network company. Generally small and medium-sized network companies do not have specialized data management and analysis experts, even large network companies, data management and analysis departments are decentralized, passive, auxiliary status, is the company's Dragon tail rather than a leader. The basic unit of data analysis in a large data age is the individual user, looking for the individual's comprehensive, complete, dynamic, real-time network behavior patterns and the group behavior patterns summed up on this basis, rather than the fragmented static statistical analysis based on a single product, service, and channel in the past. Therefore, the overall grasp of large data is the network company product development, operation settings, business model of the basis and starting point, is the leader rather than the dragon tail. This requires restructuring the existing corporate structure, organizational system, resource allocation and power structure, so that the data management and analysis departments in the overall upstream position of the company. One of the hallmarks of the success of the reorganization is the position of the company to set up the chief Data Officer (figuratively data officer, CDO), together with the CEO, COO, CFO, CTO, etc. to form the company's core decision-making.

Big data is an important part of the entire WEB2.0 revolution, and leading companies in the world's network industry, such as Google,facebook, Apple and Amazon, are already in the first place. The Chinese network industry which company can catch up, who is the forerunner, otherwise, can only play the role of the victim.

(Responsible editor: The good of the Legacy)