MySQL detailed (8)----------MySQL thread pool summary (ii)

This article is a supplement to the previous article, mainly around the following two points, the implementation of One-connection-per-thread and the use of epoll in the thread pool. One-connection-per-thread

Depending on the scheduler_functions template, we can also list several key functions of the one-connection-per-thread approach.

Static Scheduler_functions con_per_functions={max_connection+1,//Max_threadsnull,null,null,//InitInit_new_ Connection_handler_thread,//Init_new_connection_threadcreate_thread_to_handle_connection,//Add_connectionNULL, Thd_wait_beginnull,//Thd_wait_endnull,//Post_kill_notificationone_thread_per_connection_end,//End_threadNULL END};

1.init_new_connection_handler_thread

This interface is relatively simple, mainly called Pthread_detach, the thread is set to the detach state, the thread ends automatically releasing all resources.

2.create_thread_to_handle_connection

This interface is the interface that handles the new connection, and for the thread pool, a thread is fetched from the thread_id%group_size corresponding group, and the One-connection-per-thread method will determine if there is thread_ The cache can be used if there is no new thread to process. The specific logic is as follows:

(1). Determine if the number of threads cached is exhausted (compare blocked_pthread_count and Wake_pthread sizes)

(2). If there is a cache thread, add the THD to the Waiting_thd_list queue and wake up a thread waiting for Cond_thread_cache

(3). If not, create a new threading process, and the thread's entry function is Do_handle_one_connection

(4). Call Add_global_thread to join the THD array.

3.do_handle_one_connection

This interface is called by Create_thread_to_handle_connection to handle the main implementation interface of the request.

(1). Loop call Do_command, read the network packet from the socket, and parse execution;

(2). Exit the loop when the remote client sends a close connection command (such as Com_quit,com_shutdown)

(3). Call Close_connection to close the connection (Thd->disconnect ());

(4). Call the One_thread_per_connection_end function to confirm that the thread can be reused

(5). Depending on the results returned, determine whether to exit the worker thread or continue the Loop execution command.

4.one_thread_per_connection_end

The main function that determines whether the thread (Thread_cache) can be reused is as follows:

(1). Call Remove_global_thread to remove the corresponding THD instance of the thread

(2). Call block_until_new_connection to determine if thread can be reused

(3). Determine if the cached thread exceeds the threshold, and if not, then blocked_pthread_count++;

(4). Blocking Wait condition variable Cond_thread_cache

(5). After being awakened, a new THD is required to reuse the thread, remove the THD from the waiting_thd_list, and use the THD to initialize the thread's Thd->thread_stack

(6). Call Add_global_thread to join the THD array.

(7). Returns False if it can be reused, otherwise returns ture

Thread pool and Epoll

Before the thread pool was introduced, there was only one listener thread in the server layer, which was responsible for listening to MySQL ports and local unixsocket requests, and for each new connection, a separate thread was allocated for processing, so the task of listening to threads was relatively easy, MySQL uses the poll or select mode to implement IO multiplexing. After the thread pool is introduced, each group has a listener thread that listens for all connection requests to the socket in the group, in addition to the listener threads in the server layer, and the worker thread is not responsible for listening, only processing requests. For Overscribe 1000 thread pool settings, each listener thread needs to listen for 1000 socket requests, and the listener thread uses Epoll mode for monitoring.

Select,poll,epoll are all IO multiplexing mechanisms, IO multiplexing through a mechanism to listen to multiple FD (descriptors), such as the socket, once an FD ready (read ready or write Ready), can notify the program to read and write operations. Epoll compared to select and poll has a great improvement, first epoll through the EPOLL_CTL function registration, registration, all FD copy into the kernel, copy only once do not need duplicate copies, and each call poll or SELECT, The FD collection needs to be copied from the user space to the kernel space (Epoll through epoll_wait); second, the epoll specifies a callback function for each descriptor, and when the device is ready, the wake-up waiting person, through the callback function, adds the descriptor to the ready list, without the need for a SELECT, Poll mode adopts polling method; The last select supports only 1024 fd,epoll, and there is no limit, and the number can be referenced by the Cat/proc/sys/fs/file-max settings. Epoll throughout the process of using the thread pool, I epoll the creation, use, and destruction lifecycle to describe how epoll is used in the thread.

1. Thread pool initialization, epoll through Epoll_create function to create Epoll file descriptor, the implementation function is thread_group_init;
2. Port listening line Cheng after hearing the request, create the socket and create the THD and connection objects, which are placed in the corresponding group queue;
3. When the worker thread obtains the connection object, the login verification is performed if it is not yet logged in
4. If the socket is not yet registered to Epoll, call Epoll_ctl to register, Epoll_ctl_add, and put the connection object in the Epoll_event structure
5. If the old connection request, still need to call EPOLL_CTL registration, the registration method is Epoll_ctl_mod
6. The listener thread within the group calls Epoll_wait to listen for registered fd,epoll is a synchronous io, so it waits
7. When the request arrives, get the connection in the epoll_event struct and put it into the queue in group
8. When the thread pool is destroyed, call Thread_group_close to close epoll.

Note:

1. In Epoll, if the request is ready, the corresponding event is placed in the events array and the transaction type of the FD is emptied, so for the old connection request, the Epoll_ctl (POLLFD, Epoll_ctl_mod, FD) still needs to be called. &ev) to register.

Thread pool function call relationship

(1) Create Epoll

Tp_init->thread_group_init->tp_set_threadpool_size->io_poll_create->epoll_create

(2) Close Epoll

Tp_end->thread_group_close->thread_group_destroy->close (POLLFD)

(3) Associated socket descriptor

Handle_event->start_io->io_poll_associate_fd->io_poll_start_read->epoll_ctl

(4) Processing connection requests

Handle_event->threadpool_process_request->do_command->dispatch_command->mysql_parse->mysql_ Execute_command

(5) When the worker thread is idle

Worker_main->get_event->pthread_cond_timedwait

After waiting for Thread_pool_idle_timeout, exit.

(6) Monitor Epoll

Worker_main->get_event->listener->io_poll_wait->epoll_wait

(7) Port listener thread

Main->mysqld_main->handle_connections_sockets->poll

One-connection-per-thread function Call Relationship

(1) Worker thread waits for request

Handle_one_connection->do_handle_one_connection->do_command->

My_net_read->net_read_packet->net_read_packet_header->net_read_raw_loop->

Vio_read->vio_socket_io_wait->vio_io_wait->poll

Note: The worker thread with the thread pool has a listener thread to help its listening requests, and the one-connection-per-thread way of working threads is called poll blocking waits for a network packet when idle;

The worker thread of the thread pool only needs to concentrate on the request, so it is more fully used.

(2) Port listener thread

Same as the thread pool (7)

Reference documents

Http://www.cnblogs.com/Anker/p/3265058.html

http://blog.csdn.net/zhanglu5227/article/details/7960677

Copyright NOTICE: Welcome to reprint, hope to reprint the same time add the original address, thank you for your cooperation, learning happy!

MySQL detailed (8)----------MySQL thread pool summary (ii)