mysql in 子查詢 效率慢 最佳化

現在的CMS系統、部落格系統、BBS等都喜歡使用標籤tag作交叉連結，因此我也嘗鮮用了下。但用了後發現我想查詢某個tag的文章列表時速度很慢，達到5秒之久！百思不解(後來終於解決)，我的表結構是下面這樣的，文章只有690篇。

文章表article(id,title,content)
標籤表tag(tid,tag_name)
標籤文章中間表article_tag(id,tag_id,article_id)
其中有個標籤的tid是135，我幫查詢標籤tid是135的文章列表
用以下語句時發現速度好慢,我文章才690篇
select id,title from article where id in(
select article_id from article_tag where tag_id=135
)
其中這條速度很快：select article_id from article_tag where tag_id=135
查詢結果是五篇文章，id為428,429,430,431,432
我用寫死的方式用下面sql來查文章也很快
select id,title from article where id in(
428,429,430,431,432
)
我在SqlServer中好像不會這樣慢，不知MySQL怎樣寫好點，也想不出慢在哪裡。

後來我找到瞭解決方法：

select id,title from article where id in(
select article_id from (select article_id from article_tag where tag_id=135) as tbt
)

 

 

其它解決方案：（舉例）

mysql> select * from abc_number_prop where number_id in (select number_id from abc_number_phone where phone = '82306839');

為了節省篇幅，省略了輸出內容，下同。

67 rows in set (12.00 sec)

只有67行資料返回，卻花了12秒，而系統中可能同時會有很多這樣的查詢，系統肯定扛不住。用desc看一下(註：explain也可)

mysql> desc select * from abc_number_prop where number_id in (select number_id from abc_number_phone where phone = '82306839');
+----+--------------------+------------------+--------+-----------------+-------+---------+------------+---------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------------+--------+-----------------+-------+---------+------------+---------+--------------------------+
| 1 | PRIMARY | abc_number_prop | ALL | NULL | NULL | NULL | NULL | 2679838 | Using where |
| 2 | DEPENDENT SUBQUERY | abc_number_phone | eq_ref | phone,number_id | phone | 70 | const,func | 1 | Using where; Using index |
+----+--------------------+------------------+--------+-----------------+-------+---------+------------+---------+--------------------------+
2 rows in set (0.00 sec)

從上面的資訊可以看出，在執行此查詢時會掃描兩百多萬行，難道是沒有建立索引嗎，看一下

mysql>show index from abc_number_phone;
+------------------+------------+-------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------------+------------+-------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| abc_number_phone | 0 | PRIMARY | 1 | number_phone_id | A | 36879 | NULL | NULL | | BTREE | | |
| abc_number_phone | 0 | phone | 1 | phone | A | 36879 | NULL | NULL | | BTREE | | |
| abc_number_phone | 0 | phone | 2 | number_id | A | 36879 | NULL | NULL | | BTREE | | |
| abc_number_phone | 1 | number_id | 1 | number_id | A | 36879 | NULL | NULL | | BTREE | | |
| abc_number_phone | 1 | created_by | 1 | created_by | A | 36879 | NULL | NULL | | BTREE | | |
| abc_number_phone | 1 | modified_by | 1 | modified_by | A | 36879 | NULL | NULL | YES | BTREE | | |
+------------------+------------+-------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
6 rows in set (0.06 sec)

mysql>show index from abc_number_prop;
+-----------------+------------+-------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+-------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| abc_number_prop | 0 | PRIMARY | 1 | number_prop_id | A | 311268 | NULL | NULL | | BTREE | | |
| abc_number_prop | 1 | number_id | 1 | number_id | A | 311268 | NULL | NULL | | BTREE | | |
| abc_number_prop | 1 | created_by | 1 | created_by | A | 311268 | NULL | NULL | | BTREE | | |
| abc_number_prop | 1 | modified_by | 1 | modified_by | A | 311268 | NULL | NULL | YES | BTREE | | |
+-----------------+------------+-------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
4 rows in set (0.15 sec)

從上面的輸出可以看出，這兩張表在number_id欄位上建立了索引的。

看看子查詢本身有沒有問題。

mysql> desc select number_id from abc_number_phone where phone = '82306839';
+----+-------------+------------------+------+---------------+-------+---------+-------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------------+------+---------------+-------+---------+-------+------+--------------------------+
| 1 | SIMPLE | abc_number_phone | ref | phone | phone | 66 | const | 6 | Using where; Using index |
+----+-------------+------------------+------+---------------+-------+---------+-------+------+--------------------------+
1 row in set (0.00 sec)

沒有問題，只需要掃描幾行資料，索引起作用了。查詢出來看看

mysql> select number_id from abc_number_phone where phone = '82306839';
+-----------+
| number_id |
+-----------+
| 8585 |
| 10720 |
| 148644 |
| 151307 |
| 170691 |
| 221897 |
+-----------+
6 rows in set (0.00 sec)

直接把子查詢得到的資料放到上面的查詢中

mysql> select * from abc_number_prop where number_id in (8585, 10720, 148644, 151307, 170691, 221897);

67 rows in set (0.03 sec)

速度也快，看來MySQL在處理子查詢的時候是不夠好。我在MySQL 5.1.42 和 MySQL 5.5.19 都進行了嘗試，都有這個問題。

搜尋了一下網路，發現很多人都遇到過這個問題：

參考資料1：使用串連（JOIN）來代替子查詢(Sub-Queries) mysql最佳化系列記錄
http://blog.csdn.net/hongsejiaozhu/article/details/1876181
參考資料2：網站開發日記(14)-MYSQL子查詢和巢狀查詢最佳化
http://dodomail.iteye.com/blog/250199

根據網上這些資料的建議，改用join來試試。

修改前：select * from abc_number_prop where number_id in (select number_id from abc_number_phone where phone = '82306839');

修改後：select a.* from abc_number_prop a inner join abc_number_phone b on a.number_id = b.number_id where phone = '82306839';

mysql> select a.* from abc_number_prop a inner join abc_number_phone b on a.number_id = b.number_id where phone = '82306839';

67 rows in set (0.00 sec)

效果不錯，查詢所用時間幾乎為0。看一下MySQL是怎麼執行這個查詢的

mysql>desc select a.* from abc_number_prop a inner join abc_number_phone b on a.number_id = b.number_id where phone = '82306839';
+----+-------------+-------+------+-----------------+-----------+---------+-----------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+-----------------+-----------+---------+-----------------+------+--------------------------+
| 1 | SIMPLE | b | ref | phone,number_id | phone | 66 | const | 6 | Using where; Using index |
| 1 | SIMPLE | a | ref | number_id | number_id | 4 | eap.b.number_id | 3 | |
+----+-------------+-------+------+-----------------+-----------+---------+-----------------+------+--------------------------+
2 rows in set (0.00 sec)

小結：當子查詢速度慢時，可用JOIN來改寫一下該查詢來進行最佳化。

網上也有文章說，使用JOIN語句的查詢不一定總比使用子查詢的語句快。

參考資料3：改變了對Mysql子查詢的看法
http://hi.baidu.com/yzx110/blog/item/e694f536f92075360b55a92b.html

 

 

mysql手冊也提到過，具體的原文在mysql文檔的這個章節：

I.3. Restrictions on Subqueries

13.2.8. Subquery Syntax

摘抄：

1）關於使用IN的子查詢：

Subquery optimization for IN is not as effective as for the = operator or for IN(value_list) constructs.

A typical case for poor IN subquery performance is when the subquery returns a small number of rows but the outer query returns a large number of rows to be compared to the subquery result.

The problem is that, for a statement that uses an IN subquery, the optimizer rewrites it as a correlated subquery. Consider the following statement that uses an uncorrelated subquery:

SELECT ... FROM t1 WHERE t1.a IN (SELECT b FROM t2);

The optimizer rewrites the statement to a correlated subquery:

SELECT ... FROM t1 WHERE EXISTS (SELECT 1 FROM t2 WHERE t2.b = t1.a);

If the inner and outer queries return M and N rows, respectively, the execution time becomes on the order of O(M×N), rather than O(M+N) as it would be for an uncorrelated subquery.

An implication is that an IN subquery can be much slower than a query written using an IN(value_list) construct that lists the same values that the subquery would return.

2）關於把子查詢轉換成join的：

The optimizer is more mature for joins than for subqueries, so in many cases a statement that uses a subquery can be executed more efficiently if you rewrite it as a join.

An exception occurs for the case where an IN subquery can be rewritten as a SELECT DISTINCT join. Example:

SELECT col FROM t1 WHERE id_col IN (SELECT id_col2 FROM t2 WHERE condition);

That statement can be rewritten as follows:

SELECT DISTINCT col FROM t1, t2 WHERE t1.id_col = t2.id_col AND condition;

But in this case, the join requires an extra DISTINCT operation and is not more efficient than the subquery