IOT database requirements and current technical analysis
Iot is currently the most promising technology trend. By 2020, the world will have 20 billion-200 billion Iot devices (Gartner predicted 26 billion, ABI predicted 30 billion, Oracle predicted 50 billion, and Intel predicted 200 billion ). Manage the (dynamic) data generated by a large number of connected devices, compared with the management of traditional files (static) data is much more complex-the most important difficulty is the significant increase in "data volume" and the significant enhancement in "real-time processing" requirements.
- Data volume: the increase in data volume includes two aspects: total data volume and data generation speed. During the evolution of Iot application systems, the number of sensors is constantly increasing, the data sampling frequency is constantly increasing, and the data accumulation time is also getting longer and longer, as a result, the data volume generated is very large (I .e. billions, tens of billions, and hundreds of billions of storage), and the data generation speed is also very fast (I .e., 100,000 million records per second ).
- Real-time: Sensor time series data is often used for exception warning, trend prediction, and other purposes, requiring immediate response based on data. Therefore, data must be queried and analyzed in real time.
The simple requirements for IOT-oriented databases are refined on the premise that data volume changes and real-time requirements are greatly improved:
Read/write features
- Write operations are more than read operations, but read and write operations require high speed.
- Primary append, but a small number of update operations should be allowed
- Sequential append-based, but should allow warehouse receiving in disorder
- Records can be deleted by time period, but records should be allowed to be deleted
- Supports (optimized) queries for a given period of time, and queries for a specified field, such as exact, fuzzy, and prefix;
- High read/write concurrency requirements, especially read concurrency
- Mass Storage support (T-P level)
Real-time
- You can use (ingest realtime) to import data into the input database, which can be immediately used by third-party applications.
- Data Adhoc query/Analysis
Search and analysis features
- Supports filtering projection.
- Support for aggregate analysis
- Support Join Analysis)
- Support Data Mining
High scalability: On-demand online horizontal scalability High Availability: Advantages and Disadvantages of current IOT database alternatives
- Relational Database: Provides complex retrieval capabilities, but is limited by the data storage scale and input speed.
- NOSQL Database: It has scalability and high performance, but is limited by insufficient retrieval and analysis capabilities.
- HADOOP: provides data analysis capabilities and scalability, but is limited by insufficient real-time data processing capabilities.
