Big data is expected to revolutionize lifestyles

Lead: The New York Times Printing Paper published an article on the 30th that big data will evolve into a new chapter in the history of human commerce, most likely to replace the paradigm of ideas, examples, organizations, and even human thinking of the world. But at the same time, experience and intuition are as indispensable.

The following is the summary of the article:

"Big data is important and intuitive as well." This was the subject of an industry conference held at the Massachusetts Institute of Technology earlier this month.

Andrew McAfee, chief scientist at the Massachusetts Institute of Technology Digital Commerce Center, said big data will be a new chapter in the history of human commerce. Erik Brynjolfsson, another professor at the center, said big data will replace ideas, paradigms, organizations, and the way people think about the world.

These avant-garde predictions are based on the premise that data such as Web browsing history, sensor signals, GPS tracking, and social networking information can open the door to measuring and monitoring human and device behavior to an unprecedented extent. Through computer algorithms, many things can be predicted by human beings, such as shopping, dating or voting.

Industry experts predict that the end result is that the world is becoming smarter, businesses are getting more productive, the quality of service consumers receive is getting higher and higher, and people's decisions are made more and more reasonable.

I've written quite a few articles on Big Data before, but at this particular moment in late 2012, I think it's time to reflect, ask, and question big data.

Mining practical insights from business assessments is not new. More than 100 years ago, Frederick Winslow Taylor's famous book "Principles of Scientific Management" was the predecessor of big data. Taylor's assessment tool is a stopwatch that regularly monitors and monitors every employee's actions. Taylor and his aides use this "time and action" research paradigm to redesign the most effective way of working.

But if this method is overstated, it becomes the ironic object of Chaplin's Modern Times. Since then, people's enthusiasm for this quantitative method also began to ups and downs.

Often, the Internet is a big data advocate for successful data services, exemplified by Google. Today, many big data technologies, such as mathematical models, predictive algorithms, and artificial intelligence software, are widely used by Wall Street.

At this month's MIT conference, when asked about some of the major failures in big data, few have been able to say such a failure. Later, Roberto Rigobon, a professor at the Massachusetts Institute of Technology Sloan School of Management, said the financial crisis undoubtedly affected the data business. He said: "Hedge funds fail all over the world."

The problem is that the mathematical model is a simplification. This model is derived from natural science and according to the laws of physics, the behavior of particles in a fluid can be predicted.

With so many big data applications, a mathematical model often comes with accurate data about human behavior, interests and preferences. The dangers of this approach in the financial arena are also obvious to all, as Emanuel Derman, director of the Department of Financial Engineering at Columbia University in the United States, elaborated on the dangers in his book Models. Behaving Badly.

Claudia Perlich, chief scientist at New York startup Media6Degrees, said: "You can fool yourself with data, and I'm worried about the big data bubble." Perlich worried that many would call themselves "data scientists," but did not do enough homework, Smear the area.

Perlich believes big data appears to be facing a labor bottleneck. "Our skills are not improving enough," she said. "A report released last year by the McKinsey Global Institute shows that the United States needs 140,000 to 190,000 workers with" deep analysis "experience and 1.5 million more proficient managers, both retired and employed.

Thomas H. Davenport, a visiting professor at Harvard Business School, is writing a new book titled "Keeping Up With the Quants," designed to help managers cope with big data challenges. Davenport believes that an important part of managing big data projects is asking the right questions: how to define the problem, what data do you need, where do you come from, and so on.

Rachel Schutt, a senior statistic at Google Research, said that if modelers were able to think about ethical dimensions and other issues, they would be better served by society. Schutt said: "Models are not just predictions, they can really make things happen."

The model can create what the data scientist calls the "behavioral loop," which guides one's behavior if one is provided with enough data.

To Facebook as an example, personal data uploaded to your Facebook page, Facebook's software will track your clicks and searches. Use algorithms to evaluate the data before providing your friend's suggestions.

But this behavior of tracking users through software has caused privacy concerns, is it big data will usher in the arrival of digital surveillance?

My personal biggest concern is that the algorithms currently defining our personal digital world are too simple and not smart enough. This is one of the issues discussed by Eli Pariser's The Filter Bubble: What the Internet Is Hiding From You.

Encouragingly, thoughtful data scientists like Perlich and Schutt are aware of the limitations and deficiencies of big data technology. They think listening to the data is important, but experience and intuition are just as important.

At the Massachusetts Institute of Technology conference, Chat was asked how he could become a good data scientist, saying that computer science and math skills were needed, curiosity, innovation, and data and experience guidelines. She said: "I will not demonize the machine."