Research finds that most companies invest in large data items with blindness

ReadWrite, a famous blog site, published an analysis today, saying that while big data are now being fired up, most of the companies that have invested in the field have yet to get a reasonable return, even if they haven't seen the return. The article reads as follows:

As the investment progresses, the big data is now performing significantly below market expectations. In the media, many companies have not fully understood how to dig out the value of large data before they plunge into the market. According to the latest analysis, most companies derive much less value from the big data than they expect, and even investment is not recoverable.

Every dollar invested in big data gets a few cents.

So how low are the benefits for those companies? According to Wikibon studies, the benefits they receive are not very low, but rather low.

The Wikibon study found that 46% of big data investors barely managed to sustain their investment projects, and even 2% had failed altogether, with all the investment drain.

However, this is not to say that big data is deceptive, failure is doomed. This is not the case.

As Wikibon points out, one of the biggest causes of the failure of large data projects for those companies is the lack of specific and measurable commercial applications when investing in large data technologies such as Hadoop. When they hear that a project has something to do with big data, they invest money directly in it, and they don't really know what they're looking for. Gartner, a market research firm, has come to the same conclusion through analysis.

Got Hadoop, but lost the data.

For many companies, big data is equivalent to what an executive in the hadoop,it industry calls "unsupervised digital landfills". "The big data is big, but there is no data," said Giulian Simon Julien, Criteo's vice president of engineering and technology. ”

We save that data, but we don't know why we need them, and the last data we collect becomes some obscure, discordant signal in our entire database. As the famous statistician Nettershire (Nate Silver) said:

"If the amount of information in the world is increasing at a rate of 2.5 trillion bytes a day, the number of useful messages must not be that fast," he said. Most of the added information is spam, and they grow faster than useful information. There are too many hypotheses to be validated, so much information is useless, which is a relatively stable objective. ”

In other words, adding more data will not solve the problem of big data. In general, it is the cause of the problem, and it is one of the most important reasons why many companies cannot derive significant value from large data investments.

What is the success of big data?

According to Wikibon's research, the best big data projects are usually not initiated by the IT industry, but driven by lines of business, such as marketing, and most small applications that are focused on large data projects rather than strategic solutions. "They make full use of their in-house expertise and are well aware of what they can do with their existing resources." ”

Such projects are usually small at the outset and will be scaled up only after the initial results have been achieved.

To avoid investing blindly in big data projects, we recommend that you do the same. Since all the best big data technologies are open source, you can try and find the big data technology that works best for you before you invest in real money.