[Image Search] Speed debugging and optimization of BT hypervisor Database

DHT capture program Open Source Address: github. comh31h31H31DHTDEMO: github. comh31h31H31DHTMgr thank you for your support. You have found a VPS for testing. Foreign servers: You can give some comments... the server is crawling and processing at the same time, so the slow access speed is

DHT capture program Open Source Address: https://github.com/h31h31/H31DHTDEMO data processing program Open Source Address: https://github.com/h31h31/H31DHTMgr Thank you garden friends support, has found a VPS for testing, foreign servers: You can give some advice... the server is crawling and processing at the same time, so the slow access speed is

DHT capture program Open Source Address: https://github.com/h31h31/H31DHTDEMO

Data processing program Open Source Address: https://github.com/h31h31/H31DHTMgr

Thank you for your support. You have found a VPS for testing. Foreign servers: you can give me some comments...

The server is crawling and processing at the same time, so the access speed is slow. In particular, the search speed is slow through SQL like query and is being improved through word segmentation ..

Bytes ---------------------------------------------------------------------------------------------------

When the data in many tables in the database is about 3 million and the total file size is 8 GB, it is found that the new data is much slower than the original speed, some performance and speed optimization problems must be considered.

Because the server also needs to run the website, the query speed and program insertion speed will lead to slower and slower queries.

You can view the number of data records stored in the database table structure;

1. Currently, the data operation process is to query whether there is a database in the HASH value. If there is no HASH value, insert it directly. If yes, update the Count of this record directly.

2. Because the record table uses ID to associate with the file list, and sets ID as the primary key value, only the unique key constraint index design is performed on the hashkey. The table structure is designed:

3. Before processing experience, you only need to add log output information.

: 2: 2: thread >>>> 67F8DAC16B2ACB5CC79BDD02F7478457E99C5966 update to database 1010 successful 1 TIME: 0-78-0-140: 2: 2: thread >>>> 57D55712F097DFDA3F3204C3E35B59461CCFE851 update to database 1011 Success 1 TIME: 140-109-0-219: 2: thread >>> success update to database 4011 success 4 TIME: 0-422-0-16: 2: 2: thread >>>> 7B534EAFF508F861B8B1E5A5D79D9C11F1655B43 update to database 1011 successful 1 TIME: 0-78-16-31: 2: 2: thread >>>> 9AAF76DE08F2ACA7DEDD11B139EE76798591D30F update to database 1011 Success 1 TIME: 0-485-0-31: 2: thread >>> timeout update to database 1011 Success 1 TIME: 140-485-0-15: 2: 2: thread >>>> 6a424827cd07a9fb725aa9df317de180b342a4a4 update to database 1011 successful 1 TIME: 407-94-15-78: 2: 2: thread >>>> 95A6DAF234532E10012169372448096544D58D68 update to database 1011 successful 1 TIME: 0-94-0-109: 2: thread >>> timeout update to database 3011 successful 3 TIME: 0-62-0-47: 2: 2: thread >>>> dc82ddb68f6f5f0cb3101_cd9a3a382a8549802 update to database 2010 successful 2 TIME: 0-31-0-16: 2: 2: thread >>>> 3CBB82952AA59A020388415B299AA79B46CCF7DF updated to database 1011 Success 1 TIME: 140-62-0-32: 2: thread >>> timeout updated to database 1010 Success 1 TIME: 0-47-0-110: 2: 2: thread >>>> 3A804B13102E6C5B427F7E1F0F472A88F64A225C update to database 1011 Success 1 TIME: 140-78-0-16: 2: 2: thread >>>> history updated to database 1010 succeeded 1 TIME: 0-94-0-93: 2: 2: thread >>>> 70CC8C9B5EDE1F5673B7A4B684219F39E75DB660 updated to database 1011 succeeded 1 TIME: 0-31-0-32: 2: 2: thread >>>> caeaf85ed0d3a9dea7da2ec75445565108f51e63 update to database 6011 success 6 TIME: 407-328-0-47