The Big data explosion era, from the details to foresee the future

&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp;

Illustrations _roger.

In the great data age of information explosion, with the pace of life becoming more and more rapid, from the health of the global economy to the rout of terrorism, we rely on the quality of the forecast. The speed and number of predictions we have to make are increasing.

Philippe Tetlock, the author of the psychologist and the expert's political judgment, has done nearly 20 years of research on the accuracy of expert predictions in the field of social science. However, he found that the experts ' predictions were very unreliable. "Those experts predicted almost as much as the chimps threw darts at random to hit a possible outcome." Tetlock said humorously.

Starting in 2011, Tetlock and his wife Barbara Meiras and friends Tang Moore started a research project entitled "Good Judgment", sponsored by the U.S. Intelligence Advanced Research project. They recruited thousands of participants from all walks of life in the form of a contest, using a series of questions to test their ability to predict, such as "is Robert Mugabe on September 30, 2011 The president of Zimbabwe?" or "June 1, 2012, will Greece be a member of the European Union?" Participants are asked to estimate the probability of each event occurring. One of the main purposes of the experiment was to observe the capabilities of these forecasters and to sum up the rules in order to find new ways of predicting political or economic trends.

A year later, Tetlock elected 2% of them-the so-called "super forecasters"-to form a team to complete the rest of the experiment. At the end of the second year, they were 4 times times more accurate than professional analysts. and "good judgment" participants to provide a different forecast results, many experts to predict the actual only give a vague trend analysis, they are familiar with the truth of Chatterbox, but also by their own professional knowledge imprisoned, but can not more objectively look at the data, analysis problems.

"We find that amateur ' super forecasters ' have the characteristics of openness, sensibility, self-criticism, and nature." And those who are not accurate to predict the characteristics of the academic experts are logical, but lack of flexibility. "Truly successful forecasters are not afraid to change their minds, they are brave enough to accept new evidence and discard their old ideas," Tetlock said. ”

Statisticians, known as the "Wizards of the Big Data Age", were Natte Silver by the success of the 2008 U.S. election in predicting the results of 50 states in the 49 states and 2012 elections, leaving all current politicians, political party advisers and political commentators speechless. His column on the New York Times website allows many companies to see the authenticity of large data applications and the promise of large data visualization and industry.

"The data itself cannot speak for itself, we often carry out data analysis with biased thoughts and personal interests, and we must know that there is a huge gap between the information you receive and the information you know," said Silver. But that is the fact that the experts are unwilling to admit. ”

In contrast, Silver believes the modelling data obtained from the popular survey is more reliable, because without the media's rendering, the public opinion information directly to his hands is truer.

In the era of big data explosion, how to use the wisdom of folk collective to predict is very important. Truly good forecasters think in terms of probability, they are humble and diligent, they can clearly distinguish what is unpredictable and what is predictable, they focus on the hundreds of small details that lead them closer to the truth, identifying what is noise and what is a signal. Ultimately, the Tetlock team hopes these findings will help change the government's decision. "It's not about prophecy, it's about precision." "We want to improve the authenticity of the prophetic information," he said. ”

Become a Super Predictor

1. The comparison is very important, using the relative parameters as a starting point.

2. It will be helpful to look back on past trends, unless you feel strongly that this trend will reverse the historical rule.

3. The opinions of the experts are often confrontational, diverging, or even contradictory, to see what they say and then choose a neutral position.

4. Do not reject mathematical models when modeling data is needed to make predictions.

5. Don't let your inner desires influence your predictive judgment, don't stick to the new evidence and discarded the predictive bias.