Scheduling and locking of MySQL query optimization Series Lectures

The previous section focuses on how to make separate queries faster. MySQL also allows you to change the statement scheduling priority so that queries from multiple clients can better collaborate, so that a single client will not wait for a long time due to locking. Changing the priority also ensures that queries of specific types are processed faster. This section describes the default Scheduling Policies of MySQL and the options that can be used to influence these policies. It also talked about the use of concurrent insert operations and the impact of the storage engine lock level on the concurrency of the client. For the convenience of the discussion, we call the client that executes the search (SELECT) as the "Reader" and perform the modification operations (DELETE, INSERT, REPLACE, or UPDATE) the client is called "Writer ".

The default Scheduling Policies of MySQL can be summarized as follows:

· Write operations take precedence over read operations.

· Write operations on a data table can only occur once at a time point, and write requests are processed in the order they arrive.

· Multiple read operations on a data table can be performed simultaneously.

The MyISAM and MEMORY storage engines use data table locks to implement such scheduling policies. When the client accesses a table, it must first obtain its lock. When the client completes the table operation, the lock is released. It is feasible to explicitly obtain or release the LOCK using the lock tables and unlock tables statements, but in general, the server's LOCK manager will automatically obtain the LOCK as needed, release the lock when it is no longer needed. The type of the obtained lock depends on whether the client writes or reads the lock.

The client that writes data to a table must have exclusive (exclusive) Access lock. During the operation, the data table is in an inconsistent (inconsistent) state, because when the data records are deleted, added, or modified, indexes on a data table may also need to be updated to match each other. If other clients are allowed to access this data table during the change process, problems may occur. Obviously, it is unfavorable to allow two clients to write data to a table at the same time, because such an operation will quickly make the information in the table a pile of useless garbage. However, it is not good to allow the client to read the changed data table, because the data in the reading location may be changing (modifying), and the read result may not be real.

The client that reads a table must obtain a lock to prevent other clients from writing or changing the table during the reading process. However, this lock does not require exclusive access. The read operation does not change the data, so there is no reason for a reader to prevent other readers from accessing the table. Therefore, the read lock allows other clients to read the table at the same time.

MySQL provides several statement modifiers that allow you to modify its scheduling policy:

· The LOW_PRIORITY keyword is applied to DELETE, INSERT, load data, REPLACE, and UPDATE.

· The HIGH_PRIORITY keyword is applied to SELECT and INSERT statements.

· The DELAYED keyword is applied to INSERT and REPLACE statements.

LOW_PRIORITY and HIGH_PRIORITY regulators affect storage engines that use data table locks (such as MyISAM and MEMORY ). The DELAYED modifier acts on the MyISAM and MEMORY data tables.

Change statement scheduling priority

The LOW_PRIORITY keyword affects the execution scheduling of DELETE, INSERT, load data, REPLACE, and UPDATE statements. Generally, when a data table is being read, if a write operation arrives, the writer waits until the reader completes the operation (the query cannot be interrupted after the query starts, therefore, the reader is allowed to complete the operation ). If the writer is waiting and another read operation reaches, the read operation will be blocked because the default scheduling policy is that the writer takes precedence over the reader. When the first reader completes the operation, the writer starts the operation and starts the operation until the writer completes the operation.

If the write operation is a LOW_PRIORITY (low priority) Request, the system will not consider it as having a higher priority than the read operation. In this case, if the second reader arrives while the writer is waiting, the second reader is allowed to be inserted before the writer. The writer is allowed to start operations only when there are no other readers. Theoretically, this scheduling modification implies that the write operation of LOW_PRIORITY may be blocked forever. If other read operations have been completed during the previous read operation, new requests will be inserted before the LOW_PRIORITY write operation.

The HIGH_PRIORITY (high priority) keyword of the SELECT query is similar. It allows SELECT to insert a pending write operation, even if the write operation has a higher priority under normal circumstances. Another effect is that a high-priority SELECT statement is executed before a normal SELECT statement, because these statements are blocked by the write operation.

If you want all statements that support the LOW_PRIORITY option to be processed by default at a low priority, use the -- low-priority-updates option to start the server. INSERT HIGH_PRIORITY is used to increase the INSERT statement to a normal write priority, which can eliminate the impact of this option on a single INSERT statement.

Use delayed insert operations

The DELAYED modifier is applied to INSERT and REPLACE statements. When the DELAYED insert operation arrives, the server puts the data row into a queue and immediately returns a status message to the client, in this way, the client can continue the operation before the data table is actually inserted into the record. If the reader reads data from the data table, the data in the queue will be kept until there is no reader. Then, the server inserts data rows in the delayed-row queue. During the insert operation, the server also checks whether new read requests arrive and wait. If yes, the delayed data row queue is suspended and the reader is allowed to continue the operation. When no reader is available, the server inserts delayed data rows again. This process continues until the queue is empty.

It seems that LOW_PRIORITY and DELAYED are similar. Both allow the data row insertion Operation to be DELAYED, but their impact on client operations is quite different. LOW _ PRIORITY forces the client to wait until the data rows can be inserted into the data table. DELAYED allows the client to continue operations. The Server caches those data rows in the memory until it has time to process them.

If other clients may run a long SELECT statement and you do not want to block it, insert delayed is very useful when the INSERT operation is complete. The client may process the insert delayed statement very quickly because the server simply queues the data rows to be inserted.

However, you must also know some other differences between normal INSERT and insert delayed behaviors. If the insert delayed statement contains a syntax error, the client will get an error, but cannot obtain other information that can be used normally. For example, when a statement returns, you cannot depend on (get) AUTO_INCREMENT (automatically increase. Similarly, you cannot obtain the number of unique index copies. The reason for this is that the insert operation has returned status information before it is actually executed. Another possible scenario is that because the data rows of the insert delayed statement are all queued in the memory, when the server crashes or exits using kill-9, data rows may be lost (normally, the kill-TERM termination command does not cause this situation because the server inserts data rows into the table before exiting ).

Use concurrent insert operations

The MyISAM storage engine has an exception rule that allows the reader to block the writer. This occurs when the MyISAM data table does not contain "holes" (possibly the result of deleting or updating data rows. When the data table does not have "holes", any INSERT statement must add data rows at the end rather than the center. In this case, MySQL allows other clients to add data rows to the data table while reading data. This is "concurrent insert operations" because they occur at the same time and the search is not blocked.

If you want to use concurrent insert operations, note the following:

· Do not use the LOW_PRIORITY modifier in the INSERT statement. It causes the INSERT operation to be blocked frequently by the reader, and thus hinders the execution of the concurrent INSERT operation.

· If the reader needs to explicitly LOCK the data table for Concurrent Insertion, use lock tables... read local instead of lock tables... READ. The LOCAL keyword will get a lock and allow concurrent operations to continue, because it can only be applied to existing data rows in the data table and will not block new data rows added to the end.

· The load data operation should use the CONCURRENT modifier to allow the SELECT statement on the DATA table to be executed simultaneously.

· MyISAM data tables with "holes" in the middle cannot be inserted in concurrency. However, you can use the optimize table statement to segment the data TABLE.

Lock levels and concurrency

The scheduling modifier discussed earlier allows you to change the default scheduling policy. Most of the content is about using these modifiers to solve the problems caused by table-level locks. These are all problems that the MyISAM and MEMORY storage engines are used to manage data table contention.

The BDB and InnoDB Storage engines implement different levels of locks, so their performance characteristics and Management of contention are different. The BDB engine uses a page-level Lock. The InnoDB engine uses a row-level Lock, but is used only when necessary (in many cases, such as when the read operation is complete, innoDB may not use locks at all ).

The lock layers used by the storage engine have a great impact on the concurrent operations on the client. Assume that both clients want to update a row in a data table. Each client needs a write lock to perform updates. For MyISAM data tables, the engine assigns a lock to the first client, which causes the second client to block until the first client completes the operation. For BDB data tables, it can achieve greater concurrency: Two update operations will be performed simultaneously, unless both data rows are located on the same page. In the InnoDB data table, the concurrency is higher. As long as the two clients do not update the same row, the two update operations can occur simultaneously.

The general rule is that the lower the lock level, the better the concurrency, because as long as the client uses different data tables, the client that uses the table can have more. It actually implies that different storage engines are suitable for mixing different statements (mixes ):

· MyISAM retrieval is very fast. However, the use of table-level locks may be a problem in the mixed search and update environments, especially when the search tends to run for a long time. Under these conditions, the update may take a long time.

· BDB and InnoDB data tables provide better performance when there are many update operations. Because the table is locked at the page or data row level, the scope of the table to be locked is small. This reduces lock contention and improves concurrency.

Table-level locks are more advantageous than sub-level locks in preventing deadlocks. A deadlock does not occur when table-level locks are used. The server can view the statements to detect the required data tables and lock them in advance. The InnoDB and BDB data tables are deadlocked because these storage engines do not