Inapplicability of big data

In 2012, "big data" has been greatly developed. However, it is not hard to see from the perspective of the enterprises themselves, service prices and the limitation of "big data". Not everyone needs big data.

From social media startups to New York's Central Park, big data analytics are spread across almost every company. This is an era of "big data" flooding.

This is also evidenced by a recent report from Gartner, a well-known data analyst firm that shows that big data will drive $ 28 billion in global IT spending in 2012 and more than $ 230 billion in 2016. The 230 billion U.S. dollars is almost Portugal's annual gross domestic product.

However, you need to spend big money to deploy big data technology solutions. Most companies do not have so much IT budget, can not afford data scientists or data analysis team.

If companies that provide big data services want to serve businesses of all sizes, then a few issues must be recognized and addressed.

Big data is too expensive!

You may have heard of brilliant examples of using big data: Facebook stores about 100TB of user data every day; NASA processes about 24TB of data daily. These figures are indeed impressive.

So what is the cost of processing the data? According to Amazon Redshift, NASA needs to pay over $ 1 million for its 45-day data storage service.

According to a recent survey, most corporate CIOs say their budget can not afford the cost of big data deployments. The cost of data storage and processing is so high that we need to find other solutions that keep smaller companies from being blocked by Big Data.

The key to big data is not "big"

At present, the world's largest technology companies need to deal with PB-level data. However, SAP's research shows that 95% of organizations typically only need 0.5TB to 40TB of data.

The example of Facebook and NASA is an exception, not a normal one. The fact is that processing data is not a patent for large companies. If you look at the size of a U.S. company, you'll find that there are over 20,500 employees in more than 50,000 companies, most of whom have a need to solve data problems. Therefore, the biggest demand for the big data market is not from those big Fortune 50 big companies, but from Fortune 500000. Why do we focus on those few exceptions and ignore those companies that have most of the data processing needs but are neither "Fortune" nor PB-rated?

Sometimes I wonder if what happens if we change the definition of Big Data? Usually people describe big data with 3V (velocity, volume, variety), let's change the saying: "big data is a subjective state that Describes the situation when a company's infrastructure can not meet its data processing needs. "

This definition may not be so bright, but it will certainly be closer to the reality of today.