Live big data combat - crowd tagging and tagging mining (1)

In early 2013, the 85th Academy Awards was held in Hollywood, USA. Prior to the ceremony, David Rothschild, an economist at Microsoft Research in New York City, predicted the winners of the Oscars through big data analysis. The results show that, in addition to the best director award discrepancy, all other awards hit. This is not the first time David has accurately predicted that in the 2012 US presidential election he had accurately predicted the election results in 50 of the 51 constituencies with an accuracy of over 98%.

The advent of "big data" era has played a crucial role in predicting, analyzing, and optimizing the use of data by various industries. And how to make big data play its fundamental value, really for our use, is the world's data algorithms scientists struggle for technical problems.

Find out the relationship between the data -

In 1980, Toffler had predicted in "The Third Wave": "If IBM's host opened the curtain of the information revolution, then Big Data is the third wave of the colorful" .

At a time when data is growing at a terabyte of ZB levels, how to capture and filter valuable relational information from massive amounts of data is a major challenge for all data practitioners. How to establish the relationship between data, but also how to make big data "live" the only way up.

In our daily life, we often find such a situation that when keywords such as "mascara", "non-blooming", "thick" and "slender" are searched after a search engine such as Google and Baidu search for keywords, Search results page often see mascara advertising. It seems that these search engines have a clear idea of ​​what we want to do and what we are interested in.

All this is not really magical, it is only algorithms scientists through the data collection, modeling, analysis, users, search words, search-related ads these types of data were related. So when we search, it's not hard to understand matching ads.

Recently, the Prism program of the United States has drawn worldwide attention. Topics such as personal privacy are constantly mentioned. In a series of controversies, with the IT giants have been Snowden into the water, "Big Data" is a pioneering technology concept was again pulled to the spotlight.

Someone even solicited the follow-up scholars who went to the United States to follow suit. When talking with family or friends, they even mentioned more than the sensitive words such as "how to make a bomb with a pressure cooker" and "how to make TNT explosives" The workload of the major U.S. intelligence agencies. However, this method really effective? I think not always.

In fact, there is no point in having no regular and structured data, and it is clear to the American data analyst that this has long been realized. Simply getting data such as telephone recordings and surfing the internet is not enough to bring these data together, and that's just "big data." And the real value of the data, only these fragmented data analysis and comparison, the people's true identity, personality, spending habits, needs and other personal information restored, the data will be able to "live" up.

According to U.S. data analysts, the possibility of a terrorist attack on the call can be judged only by the time of a call, the duration of the call, and the location of the call. And this is through the establishment of massive user calls and terrorist links between the conclusions reached before.