What's the big data?

Not so long ago I dined with the http://www.aliyun.com/zixun/aggregation/32086.html "> CEO of a large bank." He is considering whether to withdraw from the Italian market because of the sluggish economy and the prospect of a euro crisis in the future. The CEO's economist paints a bleak picture and calculates what the downturn means for the company. But in the end, he made his decision under the guidance of his own values.

The bank has a history of several decades in Italy. He did not want the Italians to feel that his bank could only share the pain. He does not want the bank's employees to think they will abandon their armour when times are tough. He decided to stay in Italy, no matter what the future crisis will stick to, even pay a short-term price.

He did not forget the data at the time of making the decision, but in the end he adopted a different way of thinking. Of course, he was right. Business is built on trust. Trust is a kind of reciprocity that wears an emotional coat. People and institutions who make the right decisions in difficult situations can gain self-esteem and respect from others, something that is invaluable, even if it is not captured and reflected in the data.

The story reflects the strengths and limitations of data analysis. The biggest innovation of the present history is that our lives are now regulated by computers that collect data. In this era of complex situations where the mind cannot understand, the data can help us interpret the meaning. Data can make up for our overconfidence in intuition, and data can reduce the degree to which desire is distorted by perception.

However, there are still some things that "big data" is not good at, I will follow the way.

Data doesn't socialize. The brain is poor at maths (don't believe, please do your math quickly, 437 square root is the number), but the brain understands social cognition. People are good at reflecting the emotional state of each other, good at detecting the act of not cooperating, and being good at giving value to things with emotions.

Computer data analysis is good at measuring "quantity" rather than "quality" of social interaction. Web scientists can measure the social interaction you have with 6 colleagues in 76% of your time, but they can't capture the feelings of your childhood playmates that have only been seen 2 times a year, not to mention Dante's Beatris feelings for only two sides. Therefore, in the decision-making of social relations, do not be foolish enough to give up the magical machine in your mind and trust the machine in your work.

The data does not understand the background. Human decisions are not discrete events but embedded in time series and backgrounds. After millions of years of evolution, the human brain has become adept at dealing with such realities. People are good at telling stories that are intertwined with multiple causes and multiple backgrounds. The data analysis does not know how to narrate, also does not understand the thought the emergence process. Even an ordinary novel, data analysis can not explain the idea.

The data will create a bigger "haystack". This view was made by Nasim Taleb (Nassim Taleb, a famous business thinker, who had "Black Swan: How to deal with the unknowable future"). As we have more and more data available, statistically significant correlations can be found. Many of these relationships have no practical significance and are likely to lead people astray in the real solution of the problem. This deception increases exponentially as data increases. In this huge haystack, the needle we're looking for is buried deeper. One feature of the big Data age is that the number of "significant" discoveries is overwhelmed by the noise of data expansion.

Big data doesn't solve big problems. If you only want to analyze which emails can bring the most campaign funding sponsorship, you can do a randomized control experiment. But assuming the goal is to stimulate the economic situation in the recession, you will not be able to find a parallel world of society as a control group. What is the best way to stimulate the economy? People are arguing about it, and although the data is coming in like waves, as far as I know, no major "debater" in this debate has changed its stance because of the reference to data analysis.

Data favors the trend, ignoring masterpieces. Data analysis can detect this trend acutely when a large number of individuals are interested in a cultural product quickly. However, some important (and profitable) products were discarded at the outset simply because their peculiarities were not well known.

The data masks the values. I recently read an academic monograph with a great title-the ' raw data ' is just a rhetorical act. One of the main points of the book is that the data is never "raw", and the data is always built according to someone's inclinations and values. The results of data analysis seem to be objective and impartial, but in fact, the value selection runs through the whole process from construction to interpretation.

This article is not about criticizing big data as a great tool. Just like any tool, big data has good strengths and areas that are not. As Professor Edward Touffett of Yale University (Edward Tufte) said, "The world is more interesting than any subject." ”

(This article is published in the New York Times.) )