MySQL (8) ---------- MySQL thread pool summary (2), mysql Thread Pool

MySQL (8) ---------- MySQL thread pool summary (2), mysql Thread Pool
This article is a supplement to the previous article. It focuses on the following two aspects: one-connection-per-thread implementation and epoll usage in the thread pool. One-connection-per-thread

Based on the scheduler_functions template, we can also list several key functions in one-connection-per-thread mode.

static scheduler_functions con_per_functions={ max_connection+1, // max_threadsNULL,NULL,NULL, // initInit_new_connection_handler_thread, // init_new_connection_threadcreate_thread_to_handle_connection, // add_connectionNULL, // thd_wait_beginNULL, // thd_wait_endNULL, // post_kill_notificationone_thread_per_connection_end, // end_threadNULL // end};

1. init_new_connection_handler_thread

This interface is relatively simple. It mainly calls pthread_detach and sets the thread to the detach state. After the thread ends, all resources are automatically released.

2. create_thread_to_handle_connection

This interface is used to process new connections. For the thread pool, a thread is obtained from the group corresponding to thread_id % group_size for processing, the one-connection-per-thread method determines whether thread_cache can be used. If not, a new thread is created for processing. The specific logic is as follows:

(1) Determine whether the number of cached threads is used up (compare the blocked_pthread_count and wake_pthread sizes)

(2) If a cache thread exists, add thd to the waiting_thd_list queue and wake up a thread waiting for COND_thread_cache.

(3) If no, create a new thread for processing. The thread entry function is do_handle_one_connection.

(4). Call add_global_thread to add the thd array.

3. do_handle_one_connection

This interface is called by create_thread_to_handle_connection to process the main implementation interface of the request.

(1). cyclically call do_command to read the network package from the socket and perform the parsing;

(2). Exit the loop when the remote client sends a closed connection COMMAND (such as COM_QUIT and COM_SHUTDOWN)

(3). Call close_connection to close the connection (thd-> disconnect ());

(4) Call the one_thread_per_connection_end function to check whether the thread can be reused.

(5) determine whether to exit the working thread or continue executing the command cyclically Based on the returned result.

4. one_thread_per_connection_end

Determine whether the main function of thread_cache can be reused. The logic is as follows:

(1) Call remove_global_thread to remove the thd instance corresponding to the thread.

(2) Call block_until_new_connection to determine whether thread can be reused.

(3) Determine whether the cached thread exceeds the threshold. If not, blocked_pthread_count ++;

(4). Blocking the waiting condition variable COND_thread_cache

(5) After being awakened, it indicates that a new thd needs to be reused. Remove thd from waiting_thd_list and use thd to initialize thd-> thread_stack of the thread.

(6) Call add_global_thread to add the thd array.

(7) if it can be reused, false is returned; otherwise, true is returned.

Thread Pool and epoll

Before the thread pool is introduced, the server layer has only one listening thread to listen to mysql port and local unixsocket requests. An independent thread is allocated for each new connection for processing, therefore, it is easier to listen to threads. mysql achieves IO multiplexing through poll or select. After the thread pool is introduced, each group has a listening thread to listen to all connection socket requests in the group, except for the listening thread at the server layer. The working thread is not responsible for listening, only process requests. For thread pool setting with overscribe 1000, each listening thread needs to listen to requests of 1000 sockets, and the listening thread uses the epoll method to implement listening.

Select, poll, and epoll are all IO multiplexing mechanisms. IO multiplexing supports listening to multiple fd (descriptors), such as socket, once a fd is ready (read or write), it can notify the program to perform corresponding read/write operations. Epoll has greatly improved compared with select and poll. First, epoll is registered through the epoll_ctl function. During registration, all fd is copied to the kernel, and only one copy does not need to be repeated, each time poll or select is called, The fd set needs to be copied from the user space to the kernel space (epoll waits through epoll_wait). Secondly, epoll specifies a callback function for each descriptor, when the device is ready, wake up the waiting person and add the descriptor to the ready linked list through the callback function. The poll method does not need to be like select, And the poll method uses the polling method. Finally, the select method supports only 1024 fd requests, epoll has no restrictions. For more information, see cat/proc/sys/fs/file-max settings. Epoll runs through the process of using the thread pool. The following describes how epoll is used to create, use, and destroy epoll lifecycles.

Note:

1. register the fd in epoll. If the request is ready, put the corresponding event into the events array and clear the transaction type of the fd. Therefore, for old connection requests, you still need to call epoll_ctl (pollfd, EPOLL_CTL_MOD, fd, & ev) for registration.

Thread Pool function call relationship

(1) Create epoll

Tp_init-> thread_group_init-> tp_set_threadpool_size-> io_poll_create-> epoll_create

(2) Disable epoll

Tp_end-> thread_group_close-> thread_group_destroy-> close (pollfd)

(3) associate the socket Descriptor

Handle_event-> start_io-> io_poll_associate_fd-> io_poll_start_read-> epoll_ctl

(4) Handling connection requests

Handle_event-> threadpool_process_request-> do_command-> dispatch_command-> mysql_parse-> mysql_execute_command

(5) When the worker thread is idle

Worker_main-> get_event-> pthread_cond_timedwait

Wait for thread_pool_idle_timeout and exit.

(6) Listening to epoll

Worker_main-> get_event-> listener-> io_poll_wait-> epoll_wait

(7) Port listening thread

Main-> mysqld_main-> handle_connections_sockets-> poll

One-connection-per-thread function call relationship

(1) The worker thread waits for the request

Handle_one_connection-> do_handle_one_connection-> do_command->

My_net_read-> net_read_packet-> net_read_packet_header-> net_read_raw_loop->

Viow_read-> viow_socket_io_wait-> viow_io_wait-> poll

Note: different from the thread pool worker threads that have monitoring threads to help them listen on requests, one-connection-per-thread worker threads are idle, will call poll to block and wait for the network package to come;

The worker threads in the thread pool only need to concentrate on processing requests, so they are more fully used.

(2) Port listening thread

Same as thread pool (7)

References

Http://www.cnblogs.com/Anker/p/3265058.html

Http://blog.csdn.net/zhanglu5227/article/details/7960677

Copyright Disclaimer: you are welcome to reprint it. I hope to add the original article address at the same time. Thank you for your cooperation and learning!