The big data that we misread in those years

Now, the industry and academia have been hot on the big data, whether it is academia or it circle, as long as can talk about some big data is very tall. However, big Data mining, big data analysis, big Data marketing and so on are just the beginning of course, there are many people directly criticize the big data or large data marketing to create privacy threats to us, what is the big data in the end? What is the value of it?

Today, the industry and academia have been talking about a word, which is big data. Whether it's academia or it, it's big enough to talk about a little bit of data. However, big Data mining, big data analysis, big Data marketing and so on are just the beginning, for most companies, large data still has a strong mystery. So, when we have not fully understood how to use large data for mining, all kinds of too large data of the public opinion has been heard. Of course, there are also a lot of people who directly criticize the privacy threats posed by large data or large data marketing. There are also a lot of people who don't know what big data is and what the value is.

So, from an objective point of view, around the following issues to share with you on the big data of several views, but also grilled big data of those things:

1. Is there any causal and logical relationship between large data marketing and personal privacy disclosure?

2, the big Data marketing in the end can bring to the enterprise what value? What value can it bring to the user? Do users reject or resent large data marketing?

3. How to treat large data correctly? How do you see the relationship between large data and traditional investigative methods or statistics?

4. What are the challenges of big data marketing?

One, the rapid development of large data and data privacy concerns accompanied by

The emergence of social media, so that the number of user data sharing reached an immeasurable degree. Today, the growing variety of social media, the wider popularity of smartphones, and the transfer of more users to the mobile internet have contributed to more data and content. Such data increases have made global social media revenues soar, according to Gartner2012, a consultancy, which estimates that global social media revenues are estimated at $16.9 billion trillion in 2012.

One side is social media because of the large number of pots and pans, on the other hand, users are constantly leaving their personal information to the Internet, which includes age, gender, geography, life status, attitude, whereabouts, hobbies, consumption behavior, health status and even sexual orientation. For a time, large-scale user information for large data mining, large data analysis, accurate marketing of large data, advertising precision and so quickly by the major companies to put on the agenda.

For example, a true story in America tells us how to use data mining to master our whereabouts. An American family received a promotional coupon for maternity supplies from a shopping mall, and the promotional coupons were apparently given to the 16-year-old girl at home. The girl's father was very angry and looked for the market to claim. But a few days later, the father found out that the 16-year-old daughter was really pregnant. And the market is the most predictable, it is through a number of commodity consumption data to predict the customer's pregnancy.

Similar big data mining and marketing events have taken place today, especially after social media generated large amounts of data. As a result, many people began to worry about personal data, and began to criticize the fact that big data and precision marketing violated privacy, worried that we were in the era of runaway data, and attributed it to social media.

Second, large data marketing and personal privacy disclosure can not be completely equated! Logical relationship is not tenable!

If the objective analysis of the above problems will find that this is a difficult to perlocution chicken eggs or chicken eggs. It is not objective to criticize large data analysis to disclose or misuse personal user data.

Because the essence of social media is to share and spread, the emergence of social media does satisfy people's desire to share personal information and bask in all kinds of data, so that people suddenly move to a platform where the world can see themselves in the silent life of the past. People thus achieve a sense of inner satisfaction and existence. Therefore, the social media is beneficial to them simply from the psychology of the individual, they do not think they contribute to the hidden secrets, since the sharing, it must be hope or allow others to see. Therefore, this is an intangible tacit acquiescence of the transaction, the user is willing to expose their various trivial details of social media, and the social media messy mass of user data on the orderly classification and analysis is nothing wrong.

Of course, if the social media platform arbitrarily abuses or leaks the user's background data, such as personal contact, home address, bank and other extremely secretive information, it is indeed a naked violation of privacy, extremely immoral, must be condemned and legal sanctions.

But at present, many large data accurate marketing premise is the user on the Internet left on the public information in the algorithm classification and content analysis, so that the mass users of the Population Division, or to a small group of further fine differentiation, or even to a certain extent, for individual personalized customization, Finally to achieve accurate push advertising or targeted marketing activities.

Therefore, from this perspective, there is no contradiction between the large data precision marketing and the individual's active sharing and dissemination of the information data on the network. People may be surprised at first: why do they know what I want to buy? Why do they know what I need? But as the "mind-thinking" push off to make people's lives more convenient, such as eliminating the amount of time to search, find, and compare products or services, they can be very accustomed to and rely on this precision, and do not care about how they are randomly sharing the information on the network and how to exploit it.

As a result, the information that the user publishes and shares is private and is carefully considered and screened before the user shares the information. This is very important, it is a violation of privacy limits. Information that is chosen by the user to be unsuitable for publication or not to be known is the privacy that the user deems, and the information that has been publicly published to social media or the network is considered by the user to be propagated.

Therefore, the common to mass Public information analysis, mining, classification, so that accurate marketing of large data behavior can not blindly be scolded as the damage to the interests of users. Information that is stored in certain locations and does not want to be understood by others (privately stored information) is a privacy violation if it is leaked or exploited by someone with ulterior motives. But this should not be blamed on large data, but the security of the storage platform should be questioned.

Therefore, we can not read too much data precision marketing. In fact, the essence of the problem is, do people really care about the whereabouts of messy information (involving the psychology and motivation behind the sharing of information)? And does big data marketing really touch people's dirty secrets or the bottom line (need to redefine the secret and the bottom line)? Because, if people default to share is public, then the concept of invasion of privacy is not tenable. If people have information that they do not want others to know, they will not be able to share and spread it on the internet.

Third, the big data marketing to the enterprise and users what value?

After discussing the above questions, should we be honest with the big data precision marketing? So what is the value of big data marketing for both businesses and users?

1, for the value of the enterprise

Let's look at a foreign case first:

We all know that the greatest credit for the success of the "card House" in the United States is the big data analysis. As a result, the card house is almost a classic case of big data marketing and a successful attempt by Netflix to determine content production based on user information mining.

Netflix's subscribers have reached around 30 million, and most of the users ' views are related to the precision recommendation system. Netflix regularly collects and analyzes the behavior of users watching movies or TV dramas, for example, according to the user's score on the film, the user's sharing behavior, the user's visual record and other information to analyze the user's watching habits, so as to infer what kind of TV drama users like, what kind of style, like what kind of director and actor. On the basis of this, we use the algorithm to recommend and sort the videos that users are interested in, until they find their favorite movies and TV dramas. The director and protagonist of the card House are the projections of Netflix's mining of user information.

Let's look at a domestic case:

We all know Alibaba and Sina Weibo cooperation thing, Alibaba spend 586 million stake in Sina Weibo. In addition to the network of major media analysis, that Alibaba hopes to build the ecological circle, enhance the flow of the entrance, challenge Tencent and so on, there is another important reason may be the Big Data marketing strategy.

Now the big Internet big guys are in happy enclosure, circle users, who can circle users, so that users in their platform active, who has mastered the user's large amount of information (including the foreground information and hidden background information). Sina Weibo has hundreds of millions of users in China, which is a huge amount, but if Sina cannot use the information that these users produce properly, these resources are a huge waste. We look at Alibaba, China's largest electric platform, it has products, but there is no complete user life behavior information, only purchase information, but these purchase information is not enough to understand the characteristics and preferences of the crowd. Therefore, only with Sina Weibo cooperation, master a large number of users of the behavior of information, so that their classification, to find different groups of people and even different preferences, preferences, interests, hobbies, habits, communication habits, share paths and so on, then can achieve precision marketing, and even through different users of information dissemination laws, and to develop the best brand of product transmission channels. This is a huge gold mine.

Sina Weibo and Alibaba cooperation, micro-Blog appeared some product recommendation information, and Sina Weibo has launched the payment function. As you can imagine: In the future, you will see the recommended products on Weibo, just the product you like, then you can realize payment and purchase directly on Weibo. Sina Weibo and Alibaba to share the benefits. Of course, this is my personal observation and analysis, but Alibaba's big data strategy is also very obvious.

2, for the user's value

The above two examples are all large data brought to the enterprise value, then, large data marketing for users, in the end there is no value? Are users very averse to precision marketing? Let's take a look at a new survey data:

The National Advertising Institute of China University of Communication has just released a "2014 Sino-US Mobile Internet Development Report", which contrasts the use of mobile internet usage between Chinese and American users and the attitude of mobile users to mobile advertising.

Surveys show that The most likely ad content to be answered by the smart end user is: (1) with the user to buy items related to the Ads (2) and to buy items related coupons (3) Funny Ads (4) and the user favorite brand-related advertising ((5) with the user online access to the site or use of the application-related ads (6) and the most recent online shopping-related ads (7) and the user's location related to the ads (8) Advertising related to radio/television which has been tuned in recently. (accounted for >=20%)

From these data we can see that of the 8 results, 6 are related to the large data precision marketing. For example, with the user to buy items related to advertising, more can cause users to respond or interact. How to understand? The premise of large data marketing is to calculate and speculate the real needs of users, to see what the user needs to buy related products, and then give users directly to push users want, like, do a precise arrival. What about the user? Users are happy to respond to such promotional ads or products because these ads are less intrusive to users, and allow users to struggle to reduce the decision process after the comparison or around, saving time and allowing users to directly find the products or services they really need.

So, this result shows that the big data precision marketing is not completely will let the user disgusted, but see you read the user's mind level. So, if the content you push is related to the item that the user wants to buy, it's related to the user's favorite brand and so on. Then this kind of precision excavation will not be disgusted by users, but will bring convenience to users.

Don't be too superstitious about big data; What is the essence of big data?

Looking at the above analysis, you might think that big data analysis is really omnipotent. But we can't be too superstitious about big data, so the next question arises.

1. What is the relationship between large data analysis and traditional statistical methods?

The big data is based on a large amount of data, and even all of the data, and then using algorithms to compute the analysis, so that the correlation between the various factors (not causality) can be more accurately found to find the rules between the data.

So let's take a look at the traditional statistical approach, which is how statistical analysis is done by selecting a small number of samples, analyzing the samples, and then inferring the overall trend and regularity. So, it's a probability. It is generally stated that the maximum of the total is inferred from the confidence level (accuracy) of 90%, 95%, or 98%. If the purpose is clear, the sample is properly selected and the operation is scientific, then the law can be analyzed without a large amount of data, thus inferring the general rule and discovering the causal relationship between the different factors. For example, after the sampling method is determined, the sample quantity can be determined, and if properly sampled, there is not much direct relationship between the number of samples and the total quantity.

Give an improper example to understand: Suppose 1000 samples are selected, the inference rule is a, select 2000 samples, also show a law, choose 3000 is almost the same. In fact, we can achieve our goal by scientifically selecting more than 1000 samples. Therefore, the traditional sampling and statistical methods to the greatest extent to solve the cost problem, although there will be errors, but can still be found in the law.

Therefore, from this perspective, the results of large data analysis are likely to be similar to the results of traditional statistical methods, but the original small sample into a large sample analysis. Although large data analysis is theoretically more accurate and can make up for flaws in traditional errors, accuracy may not be as much improved as we think (because large data analysis is heavily influenced by data sources). In addition, it is not necessarily possible to find more new laws. If that is the case, we cannot help asking why the big data exists.

In addition, in the traditional statistical analysis, such as the analysis of market conditions, we need the actual environment and background to interpret data and analysis of data, we do not regard the data as the only and omnipotent guidance. Therefore, there are people based on experience and the actual situation of the process of data analysis, and the ability of people to participate in the analysis is very important.

2, what kind of thing is large data can not do, and the traditional method of investigation and analysis could do?

The premise of large data marketing is large data analysis, and large data analysis is based on algorithm, computer curing mode. In other words, the original part of the data analysis by people, now we put it into the algorithm. Moreover, the large data precision marketing is to analyze the user's network browsing data, sharing data, searching data and so on, so as to classify the crowd or things, and then speculate the preference and interest of the people.

However, the preference does not equal to the real demand, the Click does not represent must like. A man in social media today said: "This product is good," that he must like or must need this product?

Machines can classify behavior, but they can't really detect people's psychological and real needs. So, how do we do to detect the true psychology and needs of people? At this time, the traditional market research and analysis methods are irreplaceable. For example, in-depth interviews, such as focus group interviews, projection methods and so on. These methods can be to the greatest extent, from the psychological point of view to analyze and discover that people's real desires and essential needs. So, today many big advertising companies, marketing companies, they still use such a traditional way to understand the story behind the surface data and reasons. And these stories and reasons, is the algorithm can not do at present, must be done by people. The communication between people can detect the heart of man.

From this point of view, the big data is not omnipotent, can not be blindly myth, we must clearly understand its essence, it can be used to do, not to do. We can understand that the calculation and analysis of human data is now likely to be replaced by machines, but the other part of human work (the ability to probe the human heart) cannot be replaced by an algorithm.

For example, the first two years I have reported that "write books can be automated by the algorithm, what to save the publication of" Such a new technology, it is said that the current Amazon a large number of books are written by the algorithm, the algorithm will write a book based on the logical ideas to organize the language. However, these books can not make up for the lack of human emotion, can not express the social background and the author's environment caused by emotional fluctuations, and so on.

What are the real challenges facing big data analysis or big data marketing?

1, data redundancy problem, is there any need to use so much data?

Data source problem, data quality is not secure, is it really needed?

The advantage of large data analysis has been praised: the use of massive data. But is the more data the better? How do I filter this data? How to find valuable and useful data? What is the impact of large data and redundancy on data analysis?

For large data, huge data sources are the fundamental guarantee of accuracy analysis. However, the amount of data to a certain extent is also facing a big problem: to ensure that the accuracy of the difficult to change. This makes it difficult to guarantee the accuracy of the analysis results. There are also many examples of large data analysis and prediction failures. One of the most typical and famous, for example, is Google's failure to predict flu trends.

According to the report, Google is based on search engine data analysis, the analysis of the United States Centers for Disease Control and Prevention of surveillance data nearly twice times. Although Google has been tweaking the algorithm, it still does not guarantee the accuracy of the results. This illustrates an important issue: data source issues. Google is based on search terms on search engines to analyze, many search terms are ineffective, meaningless, so they can not really represent the trend of influenza, but they are also counted. This leads to a serious deviation of the result.

So, how do you get these data, and how do you guarantee that they are really what you need? Is it really important? If there is a serious deviation in your data source, then your analysis is more accurate and futile. For example, you spend a lot of time gathering information from Internet users, and you analyze all of their information, and they predict several consumer trends. However, these sharing information has a lot of redundant information, the accuracy of the data is very poor, many are not related to consumption, then the analysis result is likely to be inaccurate. You follow the result of the next marketing strategy of course may be a failure.

2, big guy platform game, common enterprise difficult to master a lot of data; difficult to test credibility

The platform of each big Internet company grasps the user resources, the information that the user produces is of course also gather in each platform. However, data on companies or platforms will not be fully open to the public. We can only use some tools to crawl the information scattered on the network, but not accurate grasp of the actual value and significance of the background data and information.

And the vast amount of information, for a big internet company like Google, is the treasure. Big data may just be the game of these big guy platforms, the common enterprise is more difficult to participate in.

And, these platforms are not interoperable and open, their analysis of data results without third-party verification and testing, we can not know their large data analysis results of the validity and credibility. Of course, they are valuable in analyzing the user's own product development and development. So the desire of ordinary people or ordinary businesses for big data may be wishful thinking. In the future, it is possible that Internet big platform companies will sell large data Analysis Services. And, in the future, the field of personal data management innovation and entrepreneurship will increase, the application will be increased.

In addition, the current algorithms for large data analysis are not standard, and there is no universally recognized and effective tool.

So, from these aspects, big data analysis and big Data marketing still have a long way to go. We need to look at big data correctly and rationally.

(Responsible editor: Mengyishan)