MySQL specific explanation (8)----------MySQL thread pool summary (ii)

This article is a supplement to the previous article, mainly around the following two points unfold. The implementation of One-connection-per-thread and the use of epoll in the thread pool.

One-connection-per-thread

Based on the scheduler_functions template, we can also list several key functions of the one-connection-per-thread approach.

Static Scheduler_functions con_per_functions={max_connection+1,//Max_threadsnull,null,null,//InitInit_new_ Connection_handler_thread,//Init_new_connection_threadcreate_thread_to_handle_connection,//Add_connectionNULL, Thd_wait_beginnull,//Thd_wait_endnull,//Post_kill_notificationone_thread_per_connection_end,//End_threadNULL END};

1.init_new_connection_handler_thread

This interface is relatively simple, mainly called Pthread_detach. The thread is set to the detach state, and the thread ends up releasing all resources on its own initiative.

2.create_thread_to_handle_connection

This interface is the interface that handles the new connection, and for the thread pool, a thread is fetched from the thread_id%group_size corresponding group, and the One-connection-per-thread method infers whether there is thread_ The cache can be used, assuming no new thread is being processed. Detailed logic such as the following:

(1). Infer if the number of threads in the cache is exhausted (Blocked_pthread_count and wake_pthread size)

(2). If there is also a cache thread, increase the THD waiting_thd_list queue to wake up a thread waiting for Cond_thread_cache

(3). If not. To create a new thread handler, the thread's entry function is Do_handle_one_connection

(4). Call Add_global_thread to increase the THD array.

3.do_handle_one_connection

This interface is called by Create_thread_to_handle_connection to handle the main implementation interface of the request.

(1). Loop call Do_command. Reads the network packet from the socket and resolves the run.

(2). Exit the loop when the remote client sends the close connection command (for example, Com_quit,com_shutdown)

(3). Call Close_connection to close the connection (Thd->disconnect ());

(4). Call the One_thread_per_connection_end function to confirm that the thread can be reused

(5). Depending on the return result, determine whether to exit the worker thread or continue the Loop Run command.

4.one_thread_per_connection_end

Infer whether the main function of the thread (Thread_cache) can be reused, such as the following logic:

(1). Call Remove_global_thread. Remove the corresponding THD instance of the thread

(2). Call Block_until_new_connection to infer whether the thread is reusable enough

(3). Infer if the cached thread exceeds the threshold, and if not, then blocked_pthread_count++;

(4). Blocking Wait condition variable Cond_thread_cache

(5). When awakened, indicates that a new THD is required to reuse the thread, remove the THD from the Waiting_thd_list, and initialize the thread's thd->thread_stack with the THD

(6). Call Add_global_thread to increase the THD array.

(7). Assume that you can reuse, return false, or return ture

Thread pool and Epoll

The server layer has only one listener thread before the thread pool is introduced. Responsible for monitoring Mysqlport and local unixsocket requests. For each new connection, a separate thread is allocated for processing, so the task of listening to threads is easier. MySQL uses the poll or select mode to implement IO multiplexing. After the thread pool is introduced. In addition to the listener thread at the server level, each group has a listener thread that listens for all connection requests to the socket within the group, and the worker thread is not responsible for listening, just processing the request. Set the thread pool for Overscribe to 1000. Each listener thread needs to listen for 1000 socket requests. The listener thread uses the Epoll method to implement monitoring.

Select,poll,epoll are all IO multiplexing mechanisms. Io multiplexing is done through a mechanism. Ability to listen to multiple FD (descriptive descriptors). For example, socket. Once an FD is ready (read-Ready or write-ready), the program can be notified of the corresponding read and write operations. Epoll compared to select and poll has a very big improvement, first epoll through the EPOLL_CTL function register, the register, all FD copy into the kernel, only copy once do not need to repeat the copy, and each call poll or SELECT, It is necessary to copy the FD collection from the user space to the kernel space (Epoll through epoll_wait); second, epoll specifies a callback function for each descriptive descriptor that, when the device is ready, wakes the waiting person, adding the descriptive descriptor to the ready list through a callback function, No need like SELECT. Poll methods are used for polling. The last select only supports 1024 fd,epoll and there is no limit, the detailed number can be the Cat/proc/sys/fs/file-max setting.

Epoll through the process of using the thread pool, I will create the epoll below. Use and destroy the life cycle to describe how the narrative epoll is used in the thread.

1. Thread pool initialization. Epoll creates a Epoll file descriptive descriptor through the Epoll_create function, which is the thread_group_init of the implementation function.
2. Port Listener Cheng After hearing the request, create the socket and create the THD and connection objects, which are placed in the corresponding group queue.
3. When the worker thread gets the connection object. Login verification If you are not logged in
4. If the socket has not been registered to Epoll, then call Epoll_ctl to register, the registration method is Epoll_ctl_add, and the connection object into the epoll_event structure
5. If the old connection request, still need to call the EPOLL_CTL register, the registration method is Epoll_ctl_mod
6. The listener thread within the group calls Epoll_wait to listen to the fd,epoll is a synchronous IO mode, so it waits
7. When the request arrives. Gets the connection in the epoll_event struct, and puts it into the queue in group
8. When the thread pool is destroyed. Call Thread_group_close to close epoll.

Note:

1. Register the FD in Epoll if the request is ready. The corresponding event is placed into the events array and the transaction type of the FD is emptied, so for the old connection request. Still need to call Epoll_ctl (POLLFD, Epoll_ctl_mod, FD, &ev) to register.

Thread pool function call relationship

(1) Create Epoll

Tp_init->thread_group_init->tp_set_threadpool_size->io_poll_create->epoll_create

(2) Close Epoll

Tp_end->thread_group_close->thread_group_destroy->close (POLLFD)

(3) Associative socket descriptive descriptor

Handle_event->start_io->io_poll_associate_fd->io_poll_start_read->epoll_ctl

(4) Processing connection requests

Handle_event->threadpool_process_request->do_command->dispatch_command->mysql_parse->mysql_ Execute_command

(5) When working thread spare

Worker_main->get_event->pthread_cond_timedwait

After waiting for Thread_pool_idle_timeout. Exit.

(6) Monitor Epoll

Worker_main->get_event->listener->io_poll_wait->epoll_wait

(7) Port listener thread

Main->mysqld_main->handle_connections_sockets->poll

One-connection-per-thread function Call Relationship

(1) Worker thread waits for request

Handle_one_connection->do_handle_one_connection->do_command->

My_net_read->net_read_packet->net_read_packet_header->net_read_raw_loop->

Vio_read->vio_socket_io_wait->vio_io_wait->poll

Note: A worker thread with a thread pool has a listener thread that helps its listening requests differ, one-connection-per-thread the way the worker thread is spare. Will call poll blocked waiting for network packets to come over;

The thread pool's worker threads simply need to concentrate on the request, so they are more fully used.

(2) Port listener thread

Same as the thread pool (7)

Documentation for references

Http://www.cnblogs.com/Anker/p/3265058.html

http://blog.csdn.net/zhanglu5227/article/details/7960677

MySQL specific explanation (8)----------MySQL thread pool summary (ii)