"To charm" for large data--a reflection on "The age of large data"

-Once in

&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; Go shopping at some big shopping malls, and sometimes you'll find that there's diapers next to the beer. The approach comes first from Wal-Mart, which has analyzed a lot of data from supermarket shoppers and found that men often buy diapers when they buy beer. So supermarkets put diapers and beer together for sale, raising profits.

The wonderful connection between diapers and beer is the digging and analysis of a lot of data. This small case reflects a large data thinking. Victor Maire Schoenberg The three principles of large data thinking in the age of large data: first, not causation, but relevance; second, "Sample = All"-not immediately sample, but all data; third, not precision, but mixed. That is to say, "Big data" can be easily "understood" by digging out as many of the data as possible into the secret links that we are not aware of at all, even if we don't know why.

Big data depicts an exciting future, and it's no wonder that "big data" has been one of the hottest concepts for a long time. People on the big data of the fans and good imagination, on the one hand, we live in the world is "data": "Internet of Things", the data of the purchase behavior, navigation, location of the data, micro-credit, communication data ... This provides the possibility for a large data age, and on the other hand, modern society still faces many unresolved problems, many obstacles, and people are hoping that big data can "tide over" and help the modern man out of trouble.

In this context, large data is being constantly "deified". David, New York Times columnist, can't do big data? "It's hard to make a different sound. He pointed out several flaws in the big data. First, big data is good at analyzing the number of relationships rather than quality, so it ignores a lot of important information. For example, social networking data can tell 6 of your co-workers that you meet them 76% times a day, but it's hard to find a childhood partner who sees you two times a year. Second, large data do not understand the background. Whether we say a word is serious or joking, is to express anger or goodwill, these should be placed in the context of specific analysis, data analysis is difficult to understand these. For example, big data can bring a lot of meaningless artifacts; data favors the trend, ignoring innovation; The original data is not original, the original data is often distorted, and so on.

In addition, some people think that the biggest problem with large data is that it exaggerates the role of data, the more data the better. In fact, our biggest problem will never be how to get the data, but how to find the link between the data, the scale of the application of probability models has been expanding over the last decade, but the accuracy has stagnated-the lesson should not be forgotten. Beer and diapers are just the most superficial data mining, and true data processing is tens of thousands of times more complex than Google translation, but even if Google translation is so "advanced", you don't expect it to be "fidelity". "A Pacific is water, plus the Atlantic Ocean is the same water," the scale of the data to a certain extent, continue to expand the significance of no longer, no association, no more data are no use, "promiscuous" is actually "pseudo-related."

Everyone calls for big data, just like everyone calls for innovation and reform. However, the difficulty of the problem always lies in: How to innovate, how to reform. We need big data thinking for us to light up the spark of thought, but at the same time we must face up to the great difficulty of finding data association. Otherwise, large data can easily become a hollow in situ swirling words, in vain to give a lot of people dozen chicken blood-like recklessness and enthusiasm, put a lot of human and financial resources, thought dug a "Jinshan", actually is a bunch of useless data.