Big Data Management Driven by Internet of Things

Introduction: The value of the Internet of Things lies in its data, and the unprecedented data size of the Internet of Things will drive fundamental changes in today's data services businesses, requiring businesses to adapt their big data strategy.

iDoNews Insiders say 369 July 9 (WeChat iLoveDonews)

Viktor Meyer, who is described as a "big data age prophet," once predicted in his famous treatise "The Big Data Age," that the development of Internet of Things (IoT) technology will dramatically change traditional data storage analytics. It is no wonder that so many companies are trying to join the Internet of Things tide. According to the latest report of the McKinsey Global Institute, by 2025, the total revenue of the Internet of Things industry will reach 6.2 trillion U.S. dollars.

But as Victor is worried - are these trendsetters really ready for the Big Data era that the Internet of Things will create?

Of course, in addition to technical considerations, security issues can not be ignored. However, what the author is more concerned about is that most data centers around the world (including those specializing in nonprofit data storage and analysis) appear to be unprepared for the massive data that will soon be brought by the Internet of Things.

Of course, some technology companies still insist that they are fully capable of managing their own data center, but when the amount of data in PB or EB as the unit increase, do not know if these companies still think I phlegmatic? If they still persist, they will have to make substantial investments in the appropriate infrastructure. The relatively smart companies will choose the industry-leading cloud storage company as their strategic alliance. Therefore, the trend of big data caused by the Internet of Things will boost the development of cloud storage and cloud computing.

The Big Data Processing IoT can be summarized in three basic steps: data collection, data storage and data analysis. Data acquisition and storage are basic functions, and the real value of the Big Data era is embedded in data analysis. For the advent of the big data era, some experts once estimated that more than half of big data companies may die because they have not mastered the technology of data acquisition. Of course, not to say that the difficulties of data acquisition can do anything, then the data storage is still a series of challenges. For example, companies must master advanced storage computing methods such as distributed computing and parallel computing.

In 2009, the H1N1 influenza virus raged around the world. Compared with the speed of transmission of influenza virus, the U.S. government's system of reporting cases of influenza appears to be inefficient. Only then did people re-pay attention to an article published by Nature in a few weeks prior to the outbreak of the flu virus. In the article, Google compared the seasonal influenza transmission data released by the 50 million most-searched entries across the United States with those released by the CDC and found that a massive influenza outbreak is likely to erupt in the near future, and Clearly predicted specific regions and states. At the end of the outbreak, CDC was surprised to find out that Google's prediction actually matched the outbreak precisely. So, for the big data era, what really matters is data analysis.

The challenge of data analysis also lies in the integration of new IoT data with existing databases. iDoNews believes that the two most troubling aspects. First of all, in terms of software, the storage method used between the original database and the IoT database is different. At this time, the company has to rely on massive manual redefinition of the original mass data. Second, the hardware aspect - the difference between the hardware media used in the two databases (storage infrastructure such as servers, disks, and infrastructure) will result in companies needing more infrastructure.

At this time if there are companies who want to rely on their own efforts to manage the company's data, it is undoubtedly to die.

To this end, the real vision of the enterprise can adopt the following three programs:

The first and most popular way - to leverage proven third-party database services (DBaaS) such as Amazon's Redshift, has the advantage that client companies do not need the experience to install, manage, and run any large database technology.

The second, the use of big data hosting services. The Managed Service Provider (MSP) will be responsible for data collection, database management, and services for analyzing and extracting data sets. This model not only allows enterprises to focus on the data analysis of their business value and outsources some difficult things, but also enables enterprises to quickly enter the marketization stage of big data applications without a large amount of up-front investment and also solves Many companies in this area of ​​technology shortage.

Third, based on cloud computing database matrix solution. This model is targeted at companies that have many different types, even non-relational databases. These companies often require that data be stored in multiple data centers and exist on both the public and private cloud. Companies not only require different types of database solutions, but also have their own big data applications have different needs. The GoGrid cloud computing platform from ServePath, a US hosting provider, is working on this kind of database management service.

The value of the Internet of Things lies in its data, and the unprecedented scale of data brought by the Internet of Things will drive fundamental changes in today's data services businesses, requiring businesses to adapt their big data strategy.

Wait and see, the internet of things is bound to spawn big winners in big data management.

(Author Jordan Chan, Debut to iDoNews column, please specify source and source.)