Rpc-server-Side IO service model implementation: Epoll thread pool

(a) function

RPC (remote function call) provides services for client-side calls to functions on the remote server side through the network.

The Rpc-server side needs to provide high throughput capability and support large concurrent connections.

(ii) Design ideas

• Epoll Monitor multiple connection fd for IO multiplexing

1) Epoll does not have the maximum concurrent connection limit, the maximum can open the number of files, generally far more than 2048

2) Epoll efficient, active connection, regardless of the total number of connections

3) Use shared memory, eliminating the memory copy

• Producer & Consumer models manage request tasks

A Io-thread thread maintains connections in the connection pool and is connected in 3 states: No_used, Ready, BUSY.

1) no_used status, FD location idle, connection bit not yet used

2) Ready state, FD already connected to client side, waiting for request event

3) busy status, has received the request event, the FD into the task queue, waiting for work-thread processing

• Placing a queue for a task unit with a single request event

When the connection is placed in the task queue and Work-thread finishes processing the request, the FD continues to become ready to be placed in the connection pool, waiting for a new request to arrive. Therefore, a request on a connection may be handled by multiple work-thread threads.

• Managing connections in connection pooling, setting time-outs

The server actively disconnects when there is a connection more than one time without an event, and the Nlastactive field for each connection records the time when the last event came to the Orbusy state.

(iii) process

(iv) Data structure

• pendingpool Connection pool

sitem*

ordinal	field	type	meaning
1	m_listen_fd	int	Listen for client-requested FD
2	m_aysocket	holds a collection of connection objects that have been established with the client
3	m_socketnum	int	Connection Pool size
4	m_ayevent	epoll_event*	record active events heard by Epoll
5	m_eventnum	int	epoll active event Monitored size
6	M_ayready	int*	task queue, FD with readable events
7	m_queuelen	int	Queue Length
8	m_mutex	pthread_mutex_t	Mutex, control team out
9	m_condition	Pthread_con d_t	conditional lock, control queue, queue out

• Sitem Connection Object

Serial number	Field	Type	Meaning	Value
1	Nlastactive	Int	Last active time of the connection	Number of seconds to record
2	Status	Int	Connection Status	0: Not used; 1: Connection established; 2: readable
3	Epoll_status	Int	Whether to join Epoll Monitor	0: not listening; 1: Listening
4	Processor	shared_ptr<tprotocol>*	Thrift Protocol Object

(v) achieving

• Mask

+ = Add a ready connection in the connection pool to the Epoll listener

• Pool_epoll_wait

= = Wait for the Epoll to monitor the FD has event occurred

• Deal_work

= = Handle all fd in turn

1) listen_fd=> has a new connection request, accept to create a new socket, and put in the connection pool

2) Other socket=> have a readable request, remove the listener from the Epoll & put the socket into the task queue; other events, close the connection

• Fetch_item

= = Remove FD from task queue

(vi) Use

• Io-thread Threads

That is, the thread that receives the request is processed. It uses Epoll to monitor LISTEN_FD and multiple client-side FD, and to handle all events heard by Epoll.

• Work-thread Threads

Continuously remove the readable FD task from the task queue, call the corresponding function to process the request, and then continue to put FD into the connection pool for epoll monitoring.

(vii) Question

• Epoll_wait Time-out setting

The io-thread thread undergoes a mask->wait->deal->checktimeout loop that removes the requested connection FD from the Epoll listener when deal

If the request does not complete when mask is completed, the FD does not join the listener when mask

If at this point 1) there is no new connection 2) The other connection does not have a new request, the Io-thread thread waits for the epoll time-out to re-listen for the FD

If the FD's execution consumes 1ms& next request at 2ms, and the Epoll timeout is set to 1000ms, the FD's next request waits for 999ms to be heard by Epoll

Therefore, when the number of connections or requests are small, the client's time-consuming approximation is max (execution time, epoll_wait), and epoll_wait is set to server-side request average processing time is more reasonable

• Deal processing a readable connection (request) Why is the FD removed from the Epoll listener

If Epoll continues to listen on the FD and there is a new request on the FD, then the FD is placed in the task queue and the task queue two FD

There may be multiple threads dealing with this connection FD, and there is no guarantee of sequential

Rpc-server-Side IO service model implementation: Epoll thread pool