Some developers, especially those who have contact with mysql, may encounter slow mysql queries. Of course, I am referring to a large data volume of millions or tens of millions, not dozens of times, next, let's take a look at the solution to slow queries.
Some developers, especially those who have contact with mysql, may encounter slow mysql queries. Of course, I am referring to a large data volume of millions or tens of millions, not dozens of times, next, let's take a look at the solution to slow queries.
Developers often find that they do not use indexed or limit n statements, which will have a great impact on the database, for example, a large table with tens of millions of records needs to be fully scanned, or filesort is constantly performed, causing io impact on the database and server. This is the case above the image library.
In addition to the non-index statements in the online database, the limit statement is not used. This is another case where the number of mysql connections is too large. Here, let's take a look at our previous monitoring practices.
1. Deploy zabbix and other open-source distributed monitoring systems to obtain the daily database I/O, cpu, and connections
2. Weekly Performance Statistics for deployment, including data increase, iostat, vmstat, and datasize
3. Mysql slowlog collection, listing top 10
I used to think that the monitoring has been perfect. Now, after the mysql node process monitoring is deployed, many drawbacks are discovered.
Disadvantages of the first approach: zabbix is too large and is not used for internal monitoring in mysql. Many data is not very prepared and is usually used to view historical data.
The disadvantage of the second approach: Because it only runs once a week, it is impossible to detect and trigger alarms in many cases.
Disadvantages of the third approach: when there are many node slowlogs, top10 becomes meaningless, and many times you will be given the regular task statements that must be run .. Little value for Reference
How can we solve and query these problems?
For troubleshooting and identifying performance bottlenecks, the most common problems are slow queries in MYSQL and queries that do not have to use indexes.
OK. Start to find the SQL statement that is not "refreshing" in mysql.
========================================================== ========================
Method 1: I am using this method.
The Code is as follows:
MySQL and later versions Support recording slow SQL statements.
Mysql> show variables like 'long % '; Note: This long_query_time is used to define how many seconds is slower to calculate as "Slow query"
+ ----------------- + ----------- +
| Variable_name | Value |
+ ----------------- + ----------- +
| Long_query_time | 10.000000 |
+ ----------------- + ----------- +
1 row in set (0.00 sec)
Mysql> set long_query_time = 1; Note: I set 1, that is, if the execution time exceeds 1 second, the query is slow.
Query OK, 0 rows affected (0.00 sec)
Mysql> show variables like 'slow % ';
+ --------------------- + --------------- +
| Variable_name | Value |
+ --------------------- + --------------- +
| Slow_launch_time | 2 |
| Slow_query_log | ON | Note: whether to Enable Logging
| Slow_query_log_file |/tmp/slow. log | Note: where to set
+ --------------------- + --------------- +
3 rows in set (0.00 sec)
Mysql> set global slow_query_log = 'on' Note: Enable Logging
Once the slow_query_log variable is set to ON, mysql starts recording immediately.
In/etc/my. cnf, you can set the initial values of the above MYSQL global variables.
Long_query_time = 1
Slow_query_log_file =/tmp/slow. log
Method 2: mysqldumpslow command
The Code is as follows:
/Path/mysqldumpslow-s c-t 10/tmp/slow-log
This will output 10 SQL statements with the maximum number of records, of which:
-S indicates the sorting method. c, t, l, and r are sorted by the number of records, time, query time, and number of returned records, ac, at, al, and ar indicate reverse descriptions;
-T indicates the top n, that is, the number of previous data records returned;
-G, followed by a regular expression matching mode, which is case insensitive;
For example
/Path/mysqldumpslow-s r-t 10/tmp/slow-log
You can obtain up to 10 queries from the returned record set.
/Path/mysqldumpslow-s t-t 10-g "left join"/tmp/slow-log
Obtain the query statements containing the left join in the first 10 results sorted by time.
Summary of the benefits of node monitoring
1. Lightweight monitoring, real-time, and can be customized and modified according to the actual situation
2. Set the filter program to filter the statements that must be run.
3. promptly discover those queries that do not use indexes or are illegal. Although it takes a lot of time to process those slow statements, it is worthwhile to avoid database crashes.
4. When there are too many connections in the database, the program will automatically save the processlist of the current database. This is a powerful tool when DBA finds the cause.
5. When using mysqlbinlog for analysis, you can get a clear period of time when the database status is abnormal.
Some people will make mysql Configuration File Settings.
Some other parameters are found when tmp_table_size is adjusted.
Qcache_queries_in_cache number of queries registered in the cache
Qcache_inserts number of queries added to the cache
Qcache_hits cache sample count
Qcache_lowmem_prunes number of queries deleted from the cache due to lack of memory
Qcache_not_cached does not have the number of queries cached (cannot be cached, or because of QUERY_CACHE_TYPE)
Qcache_free_memory
Qcache_free_blocks query the number of idle memory blocks in the cache
Qcache_total_blocks
Qcache_free_memory can cache some common queries. If it is a common SQL statement, it will be loaded into the memory. This will increase the database access speed.
