First of the previous picture:
Explanation: The image above is a tree-shaped data structure, thought for the acquisition of 50000 machine performance data, every 5 minutes collection, storage for two years (two years after the data can be deleted, so there are 730 days), each machine has an IP address (50,000 machines have 50,000 IP address), Multiple ports, such as switches, are: eth0,eth1,... ), each port collects 16 performance data (the number of packets, the number of contracts, the number of error packets ...). And so on, that is, the key in the diagram, each key corresponds to 288 value (because every 5 minutes collection, one key a day has 288 value.) )
My initial idea was to use NoSQL to store the data and to design the date (the first layer of the tree) as a collection. The IP address (the second level) is designed as a document. A subdocument is designed using MongoDB, ports, and subsequent (third and subsequent layers). This way when a data is collected, the tree is searched, the data is inserted, and no branch bar is added to the road.
For example: A piece of data came: 2013-5-8. 192.168.10.1. eth0. Key1. Value4 inserts the value4 underneath the value3.
Here comes another piece of data: 2013-5-8. 192.168.10.1.eth1. Key1. value1 a new eth1 branch just below Eth0 and continues to add data.
But MongoDB's subdocument insertion (first querying the subdocument to be inserted) is somewhat repetitive.
For the above requirements how to choose a reasonable database (optional database) and design storage format.
IP address is can be added (may be a few days and a new IP address needs to be collected), the port is also uncertain (some machine port, some port less)
I hope we can discuss ideas ...