Break the rumors! 10 powerful facts about Big data

Regardless of how they are defined, large numbers have been controversial since the day they were born--both faulty and derogatory. Big data is important to many people, especially scientists and retailers. But the advent of the technology also raises a number of related privacy issues and security threats.

Is it a messiah, a scam or both? In any case, the big data remains extremely hot and popular among technologists, trend analysts, marketing people and security practitioners. In fact, as of today, there is still no universally accepted official definition of the big data. So what's the big data? Wikipedia's description can be said to be a good way to establish the concept of Big data: "Any dataset that is difficult to process through existing database management tools or traditional data processing applications because of its size and complexity." ”

Although the management of this large, data sets with varying forms and high speed requirements (these three points, the classic 3V definition) are challenging, but the current number of data-sharing devices for such tasks is showing exponential growth, which in turn brings more variation to big data challenges. This type of hardware is collectively known as the IoT, which includes machine sensors and devices for ordinary consumers, such as networked thermostats, light bulbs, refrigerators and wearable health monitoring tools. IDC expects the market to grow rapidly over the next few years – the number of units installed will grow from 9.1 billion at the end of 2013 to 28.1 billion in 2020.

Companies view the feasibility analysis of large data as potentially positive news, not only because such conclusions can help businesses sell more tools and services, but also better deal with medical matters, prevent shoddy drugs from circulating, track terrorists and even monitor the content of specific targets. Therefore, the big data itself does not have good and evil, the real decisive role is our actual use of the way.

Ironically, while large data contains potential possibilities for improving human experience, such valuable information is often difficult to collect, filter, analyze, and interpret. Today's article focuses on challenges and opportunities in large data areas, and these facts and arguments are likely to surprise you. What is worth looking forward to? Well, as a leader in a large data platform, Hadoop has a bright future. and data scientists and large data-related technology people will be rewarded handsomely in the coming years.

Insiders make predictions that "big data" will disappear as a popular word. "Everything will eventually come down to data, that's all." Big data and all this predictive behavior will be the work of data management by analysts and many ' big ' technology suppliers, "Hortonworks's president Herb Cunitz wrote in a December 2012 blog post.

Cunitz's "Big data" concept could be premature, and he made an important conclusion that everything would eventually come down to data. Only the tools necessary to manage this information will usher in change. Now please follow our footsteps, together through the graphic understanding with large data closely related statistics and research results.

How much data has been overlooked?

Most companies estimate that they have only analyzed about 12% of their own data, according to a recent survey by Forrester Research. Is this good news or bad news? Well, the 88% of data they ignore is likely to contain valuable information that will bring data-driven conclusions. But from another point of view, they may have wisely avoided the huge resource depletion of the so-called "boiling ocean" strategy. There are two reasons why companies ignore the vast majority of their own data: the first is the lack of correlation tools and "controllable" data warehouses, and the second is that it is difficult for them to know exactly what information is achievable and which is best overlooked, Forrester said in its report.

Second, large data-related jobs continued to grow

The frenzy of big data is a boon to practitioners with specific skills. According to the Dice website (a job site dedicated to technical and engineering professionals), the industry's demand for data specialists is continuing to soar. Compared with the previous year, the current number of recruitment positions for NoSQL technicians has increased by 54%, while that for "big data talent" has risen by 46%, the website said in its April report this year. While impressive, it is still dwarfed by the job requirements of cyber security experts, which grew by as much as 162% per cent year-on-year.

Third, the big data will eventually grow to what size?

Over the next six years, data problems in the digital domain will grow from the current 3.2 ZB (i.e., Chak bytes) to ZB. (1 ZB is basically equivalent to 1 billion TB.) "It's really exciting when we look at the volume of data that's going to be swept," Hortonworks CEO Rob Bearden said at the 2014 Hadoop Summit held this year at California State San Jose. "From now until 2020, the number of companies holding the problem will be 50 times times per year of the rate of increase." I think the most important task at the moment is to be aware that 85% of the data comes from emerging network data sources. "These emerging data sources, including mobile, social media, and web and machine-generated data, will pose major challenges and Bearden to global businesses," he said.