Asp + sqlserver problems in massive data operations and solutions

Existing problems:
1. All records are stored in one table. No category
2. During development, this large amount of data is not considered. The query statement is executed in the program, resulting in a slow speed.
3. According to the principle of the insert process of a relational database, an index query is created every time an insert operation is performed, which will consume a large amount of memory and CPU resources and greatly reduce the speed. The speed and efficiency of inserting 100 records in a table are completely different from those of 10000 records!
4. Insert and query are in the same table. The number of concurrent processes may peak at over 1000.
5. According to the query principle of relational databases, if someone wants to query a record table, it will be such a mathematical expression.
One record <= 1 K
A total of 100,000 records, 20 thousand growth rate per day
If you do not know the conditions, any query will be like this:
(1 K * 100,000)/1024 = 10 M
1 person is 10 M. If 200 people check at the same time, it will be like this:
200*10 M = 2000 M (about 2G)
Such big data is retrieved from the database. And download it to the local machine to view it, which is huge.
6. The network speed of each input outlet is still "cat", and the speed will definitely not keep up.
7. Other data is stored in the server.
7. The server bandwidth is only 8 MB for the leased line. Even if the server's CPU can be computed, the data cannot be sent out, and it will be congested!
Due to the above problems, the following problems occur:
1. The hard disk of the website server is physically burned down.
2. The website bandwidth is fully occupied and is basically difficult to access.
3. The website page speed is extremely slow and the data transmission efficiency is low.
4. Some input units cannot respond to their operations due to the network speed, and the sent data packets cannot be returned and the input records cannot be completed.
Solution (idea ):
1. Server update. (On hardware)
2. The network bandwidth is increased. (On hardware)
3. Store the query in the database and use the stored procedure. However, at a speed of Mbit/s, there is no significant difference between the application of the stored procedure and the program query. (Software ).
4. When inserting a record, the buffer table is used to reverse the data of the buffer table to the primary record table every 10 minutes. This can relieve the pressure on the primary record table. Allow the primary record table to respond specifically to the query action (on software)
5. When querying, use text to read Records. Because the system moves the data based on the underlying metric, the query efficiency will be improved by 100 times. However, the FileObjectSystem component is required. Security considerations. (Software)
If no action is taken, the following problems may occur:
The database is overwhelmed and the hard disk will be burned again.
The server CPU has been running at 100% full load.
The program system crashes completely.
Data cannot be instantly inserted and cannot be instantly responded.
Statistics and tracing are not supported.
Websites cannot run normally.