In the big data age, I embrace with great trepidation

This is the best of times and the worst of times, let us embrace the era of big data. ----Preface

These days read Victor Maire's "Big Data times", feeling a lot, technology leads us into the data age. Data storage, analysis capabilities greatly changed, based on big data analysis results have completely changed our lives, such as convenient translation, convenient input, new product recommendations. Big data has become the concern of the academia, the industry is a matter of keen concern, the big data era of the technology model, management model is not yet known. On the basis of combing the academic research of big data, this paper analyzes the relationship between big data and cloud computing, the connotation of big data and its relationship with the development of information technology, and tries to forecast the development of big data in academia and industry.

         Labs this year's application of the self-fund, more or less related to big data, although from a technical point of view to deliberately weaken the connotation of big data, but from the review of the view that big data as a bright spot is pointed out, the academic attention to big data. Although big data in the Internet industry is being discussed in full swing, based on the author's knowledge, Big data was originally described in the biological field for gene sequence detection of the large amount of high-speed data, "Nature" in 08 published a special issue on big data to discuss, And Lynch from the university scientific research data management to put forward the scientific data is one of big data, put forward the data value of time dissipation phenomenon. From this book, Big data from the biological science, and Hing in the Internet. Whether it's Google or Amazon, it's a record of the daily behavior of a large number of netizens, forming large amounts of data. Articles for Big Data applications,< Harvard Business Review > Article <strategy & competition Big data:the Management revolution> in-depth analysis of big data business changes, It is believed that big data will push the personalized business service of the Internet to individuation, and the enterprise will be able to portray user's preference more effectively, and provide better data support for customer management management and product recommendation. However, so far, the academic research on big data is more in the qualitative analysis, still lack of empirical and computational science research, the key attributes of big data is not yet known, the author believes that the academic community is satisfied with the study of big data for two reasons: first, the data set to meet the characteristics of big data is difficult to obtain, Both quantitative studies are based on sample data, however, the computational power of academic experiments is not enough, algorithms for big data and applications are more for the enterprise groping stage. Many of the cases given in the big Data era attest to the availability of big data, but these cases are also using big data to explain business behavior and the results obtained, directly based on the concept of big data applications are not yet visible, the root cause is the satisfaction of defining what is big data. I agree with Victor in the "Big Data Age" definition, big data is not the big data, but in the overall data, its essence is to use the data as a whole rather than data samples to analyze the data. Big Data brings three changes: the pursuit of aggregate data, the pursuit of relationships rather than causality, the pursuit of mixed lines rather than precision. For the analysis of Big data features, I very much agree with Professor Chen Tsinghua University, the 4V, the scale is huge (volumn), various forms (vary), high-speed production (V??), and the potential value. Big Data provides two levels of data description from a personal level to a macro level.

I think big data is the extension of cloud computing, cloud computing is big Data simple application, a few days ago about cloud computing is more common in cloud storage and cloud music and other simple applications, but cloud computing for big Data ready technology, data unification, data sharing, and big data is a further extension of cloud computing, More attention to data analysis technology and data application thinking. The author agrees that big data is the era of the Three Kingdoms of data, technology and thinking, while the Big Data era has spawned new business models, data owners and data brokers have become new business darlings, the huge impact on business models, the use of big data to predict consumer behavior is the business connotation of big data, Companies have the opportunity to serve consumers more accurately. Although the big data era put forward the importance of correlation in big data, but will be related to the big data connotation still need in-depth analysis, I believe that causality is still at the core of data analysis, big data is not the case, there are two reasons: the essence of correlation relationship is causality chain; The logical reasoning ability of human is based on the accumulation of causality, not the discovery of related relations. Of course, the idea of what is in the book is understandable, the problem is only to know what it is, and for why it can be put aside for efficiency, but it is not appropriate to abandon the causal relationship.

The Big Data era, the key to the development of big data in two areas: first is the acquisition of data, how reasonable, efficient, fast, flexible access to support now and the future of big data analysis of the data set is still a problem, including the legitimacy of data acquisition and data design issues, After acquiring storage that is believed to be falling in storage costs, the development of non-relational storage technologies will not be a problem. Then there is the big data analysis problem, effective analysis technology is still the key to big Data application, although Google uses search terms to predict influenza outbreaks is a successful application of big data, but the general analysis technology, reduce the demand for analysis hardware is still worth discussing, SaaS may be a possible channel to solve this problem, However, how to achieve data analysis is still a problem, the existing mapreduce,hadoop and other systems are essentially palliative, large data fragmentation of the process of data is necessarily a large number of data association. I think the key of big data analysis technology is how to apply the existing analysis technology in big Data set, so that it can deal with big data. In the big data times, the simple algorithm presented in the big data set shows a better effect than the complex algorithm, I think that the complex algorithm can not be effectively calculated under the existing hardware conditions, the development of information technology must solve this problem.

Finally, recommend the "Black Mirror" for everyone to see, here is the big data on the moral deep thinking, not to repeat.

In the big data age, I embrace with great trepidation