Let's get a new understanding of the big data.

The concept of "big data" is really not new.

The big data age has come, but not everyone is accepting it. More precisely, what we now call "big data" it's what the internet bosses of Silicon Valley did a few years ago, and it's a new and interesting thing to think about today, because the technology that drives big data now is completely open source and popular with most companies and companies.

In conversations with many of Europe's traditional businesses, it becomes clear that big data is nothing new except that it allows development applications to grow rapidly in non-Silicon Valley. Europe is mentioned because it lags behind the United States in the development of IT technology. Whether it's cloud computing or big data calculations, Europe lags behind America for one or two years. So when we see that companies in Europe are serious about big data projects, it means that the concept of big data is really starting to come to the hearts of all.

The concept of "big data" involves more scope than we think.

Gartner reports: 42% of IT business leaders have gone deep into the development of large data projects. In other words, it has a great development space. But I suspect the figure is undervalued, which involves defining the concept of "big data". For example, when I ask an expert on an IT enterprise to develop a large data project, the general answer is NO. But when I further clarify what I mean, the project you are working on is not the kind of data that involves megabytes or even larger amounts, but rather a project product that can pull data from a dispersed port and then be able to analyze it in real time. When replaced by such a question, often her answer is "YES"! Such projects are, of course, in the context of "big data". But the word "big data" makes people tend to pay more attention to "big" rather than data, so people go into the wrong area.

The conclusion is clearer in the Newvantage survey, where only 15% per cent of respondents were dealing with large scale data. And from the remaining 85% of respondents, we can see that the enterprise's most concerned goal is to have the ability to continuously manage the increasingly diverse and expanding data resources, rather than simply processing large data. So it's not surprising that even Hadoop, the company known for storing and processing oversized data, is more involved in the ETL process. (ETL: That is, the construction of data Warehouse is an important link, the data source to extract the required data, after the data cleaning, and finally according to a predefined data warehouse model, the data into the Data Warehouse)

In the concept of big data, scale really doesn't matter.

Google and Facebook have already developed databases such as MapReduce and NoSQL to cope with the need for real-time data analysis processing by applications, which are primarily data-driven. Now this technology is open source, available and used everywhere, so that now the internet bosses are looking to the "larger data" technology development and utilization, while others will enjoy the convenience of these technologies in a few years. Perhaps the Blaine Prost of the read-write web will be more revealing about the value of the Hadoop Web site. "Hadoo is just making the data stores that are supposed to be expensive cheaper," he said. Gigaom's Derrick Harris, Derrick Harris, commented on NoSQL: "It is not replacing the role of other databases in managing complex transactions." NoSQL, on the other hand, spawned a series of applications that are more responsive to the processing of semi-structured data. "So it seems to me that the best way to define big data is to stand in the perspective of the data you're dealing with and have nothing to do with the size of the data you're dealing with."

I recently met a European IT business owner who says he has now transformed his work team from a "waterfall" approach to a more agile development approach. The team feeds real-time customer feedback on the market, reads from 3,000 servers, generates 500G of information a day, and only takes 24 minutes from development to configuration. The company is undoubtedly transforming itself into a data-driven model. This means that they have the ability to sweep the old rigid data base system and encounter a lot of resistance and frustration in the process of transformation, but in the end I believe I can reach their goal.

But the most important thing to look forward to is whether more mainstream companies will develop their own big data technology to meet some of the needs that the internet giants have never had, or that innovation will "return" to Silicon Valley.

Time will tell us everything