Big data ....

With regard to big data, there is this passage:

"Big data is like teenage sex,everyone talks about It,nobody really knows what to do It,everyone thinks everyone else is do ing it,so everyone claims they is doing it. "

After reading this sentence, what is "big data" a bit of a concept? At present, most people's concept of big data still stay in: is massive data, PB (1PB=1024TB) level, even EB, ZB above the data, through the deep analysis of these data, can draw very valuable conclusion, guide the enterprise to make the best decision.

Big data is the kind of thing that everyone has heard, or read, but don't know.

In fact, today's big data is not just a huge amount of data, but more accurately a way to analyze big data. The traditional data analysis is to validate the hypothesis by making assumptions and then obtaining the corresponding data, and finally through data analysis. Big data is not the case, the big data is collected from the vast amount of data, through the algorithm of these from different channels, the format of the data directly analyzed, to find the correlation between the data. In simple terms, big data is more focused on discovery, as well as guessing/verifying the loop approximation process.

The value of big data is embodied in its analysis and utilization. All along, the bottleneck of big data is not the storage, operation and other problems caused by the huge data size, but the way of collecting the data in front-end, and the structured processing of the data to guide the model and algorithm problem in the later business decision.

Data is being generated in every industry, and the volume of data in modern society continues to grow at an unprecedented rate. These different types of data and data type are extremely complex, including structured, semi-structured, and unstructured data. Businesses need to consolidate and analyze data from complex traditional and non-traditional sources, including internal and external data. With the explosive growth of sensors, smart devices, and social collaboration technologies, the type of data becomes difficult to count, including text, Weibo, sensor data, audio, video, and more.

And now the big hot data analysts are doing the job of gathering information, structuring it, and finally, the magic power of big data we can see. But the problem is that the amount of data that is being processed is too large. According to interviews and expert estimates, the data analyst's 50%~80% time is spent on processing data.

Monica Rogati, who is responsible for data work at the Smart bracelet company Jawbone, says:

Processing data is a huge part of the whole work. But sometimes we feel depressed because it seems like we're doing everything we do with data.

It sounds a bit like the iceberg theory that the big data we can see is just a small corner of the iceberg, and where we can't see it, such as the early work of big data, is the bigger part of the ocean.

But McKinsey, the consulting firm, said in a 2011 report:

"Data, which has penetrated into every industry and business function area today, has become an important production factor. The excavation and application of massive data indicates a new wave of productivity growth and consumer surplus. ”

Yes, there are opportunities lurking in the wrong places. The format and source of raw data cannot be counted, for example, if an enterprise in a food industry needs to collect and analyze big data, it can collect data including production, location information, weather reports, retailer daily sales, social media reviews, etc. According to this information, enterprises can gain insight into the direction of the market and the changes in demand, and then make the corresponding product plan.

Indeed, the more information you get, the better it is for your business to make informed decisions. But this decision is based on a different set of data, all of the data from various sensors, documents, Web pages, and databases, all of which are in different formats and must be converted to a uniform format so that the software can understand them and analyze them.

Formatting all kinds of data is a daunting challenge, because data is as vague as human language, some data people know what it means, but computers don't recognize it, so we need to manually repeat this work over and over again.

Many start-ups are now trying to develop technologies to mitigate this, such as clearstory data, a start-up company in Palo Alto, which develops software that identifies different sources of data, integrates them, and renders the results visually, in tables, graphs, or data maps. Another example is Paxata, a California start-up company focused on data automation-discovering, cleaning up, provisioning data, and paxata processed data that can be fed into various analytical or visual software tools.

The current situation of big data is somewhat similar to the trajectory of computer development. An advanced technology that is often mastered by only a few elites, but as time passes, the technology, or the tools, will become more and more effective through constant technological innovation and investment. Especially when it is integrated into the commercial field, this tool can be widely used and become the mainstream of society.

So now we are the witness of history, looking at how big data can be perfected, we all need to master or choose the best analytical method to better dig out the value of big data.

Go ahead and explore.

Copyright NOTICE: Welcome reprint, Hope in your reprint at the same time, add the original address, thank you with

Big data ....