Simple implementation of connection-oriented Socket Server

I. Basic Principles

Sometimes we need to implement a public module and provide services to multiple other modules. The most common method is to implement a Socket Server, accept customer requests, and return results to the customer.

This often involves the problem of managing multiple connections and providing services with multiple threads. The common method is the connection pool and thread pool. The basic process is as follows:

First, the server has a listening thread that constantly monitors connections from clients.

After a client connects to the listening thread, a new connection is established.

The listening thread puts the new connection into the connection pool for management, and then continues to listen to the new connection.

There are multiple service threads in the thread pool. Each thread listens to a task queue. A established connection corresponds to a service task. When a service thread discovers a new task, you can use this connection to provide services to the client.

The number of connections that a Socket Server can provide can be configured. If the number of connections exceeds the configured number, the new connection is rejected.

When the service thread completes the service, the client closes the connection, the service thread closes the connection, idle, and waiting for processing new tasks.

The monitoring thread of the Connection Pool clears the closed connection object to create a new connection.

Ii. Socket Encapsulation

The Socket call mainly includes the following steps:

The call is complicated. We first distinguish between two types of sockets: Listening Socket and Connected Socket.

The Listening Socket is the responsibility of MySocketServer. Once accept is enabled, a Connected Socket is generated, and MySocket is the responsibility.

The main implementation of MySocket is as follows:

Int MySocket: write (const char * buf, int length) { Int ret = 0; Int left = length; Int index = 0; While (left> 0) { Ret = send (m_socket, buf + index, left, 0 ); If (ret = 0) Break; Else if (ret =-1) { Break; } Left-= ret; Index + = ret; } If (left> 0) Return-1; Return 0; }

Int MySocket: read (char * buf, int length) { Int ret = 0; Int left = length; Int index = 0; While (left> 0) { Ret = recv (m_socket, buf + index, left, 0 ); If (ret = 0) Break; Else if (ret =-1) Return-1; Left-= ret; Index + = ret; } Return index; }

Int MySocket: status () { Int status; Int ret; Fd_set checkset; Struct timeval timeout; FD_ZERO (& checkset ); FD_SET (m_socket, & checkset ); Timeout. TV _sec = 10; Timeout. TV _usec = 0; Status = select (int) m_socket + 1, & checkset, 0, 0, & timeout ); If (status <0) Ret =-1; Else if (status = 0) Ret = 0; Else Ret = 0; Return ret; }

Int MySocket: close () { Struct linger lin; Lin. l_onoff = 1; Lin. l_linger = 0; Setsockopt (m_socket, SOL_SOCKET, SO_LINGER, (const char *) & lin, sizeof (lin )); : Close (m_socket ); Return 0; }

The main implementation of MySocketServer is as follows:

Int MySocketServer: init (int port) { If (m_socket = socket (AF_INET, SOCK_STREAM, 0) =-1) { Return-1; } Struct sockaddr_in serverAddr; Memset (& serverAddr, 0, sizeof (struct sockaddr_in )); ServerAddr. sin_addr.s_addr = htonl (INADDR_ANY ); ServerAddr. sin_family = AF_INET; ServerAddr. sin_port = htons (port ); If (bind (m_socket, (struct sockaddr *) & serverAddr, sizeof (serverAddr) =-1) { : Close (m_socket ); Return-1; } If (listen (m_socket, SOMAXCONN) =-1) { : Close (m_socket ); Return-1; } Struct linger lin; Lin. l_onoff = 1; Lin. l_linger = 0; Setsockopt (m_socket, SOL_SOCKET, SO_LINGER, (const char *) & lin, sizeof (lin )); M_port = port; M_inited = true; Return 0; }

MySocket * MySocketServer: accept () { Int sock; Struct sockaddr_in clientAddr; Socklen_t clientAddrSize = sizeof (clientAddr ); If (sock =: accept (m_socket, (struct sockaddr *) & clientAddr, & clientAddrSize) =-1) { Return NULL; } MySocket * socket = new MySocket (sock ); Return socket; }

MySocket * MySocketServer: accept (int timeout) { Struct timeval timeout; Timeout. TV _sec = timeout; Timeout. TV _usec = 0; Fd_set checkset; FD_ZERO (& checkset ); FD_SET (m_socket, & checkset ); Int status = (int) select (int) (m_socket + 1), & checkset, NULL, NULL, & timeout ); If (status <0) Return NULL; Else if (status = 0) Return NULL; If (FD_ISSET (m_socket, & checkset )) { Return accept (); } }

Iii. Implementation of Thread Pool

A thread pool generally has a task queue. Each thread that is started competes for a task from the task queue. The resulting thread is processed: list <MyTask *> m_taskQueue;

Task queue is protected by a lock to ensure thread security: pthread_mutex_t m_queueMutex

The task queue requires a condition variable to support the producer consumer mode: pthread_cond_t m_cond

If the task list is empty, the thread waits. The number of waiting threads is m_numWaitThreads.

A list is required to maintain the thread in the thread pool: vector <MyThread *> m_threads

Each thread needs a thread to run the function:

Void * _ thread_new_proc (void * p) { (MyThread *) p)-> run (); Return 0; }

The MyThread class is responsible for each thread. The main functions are as follows:

Int MyThread: start () { Pthread_attr_t attr; Pthread_attr_init (& attr ); Pthread_attr_setschedpolicy (& attr, SCHED_FIFO ); Int ret = pthread_create (& m_thread, & attr, thread_func, args ); Pthread_attr_destroy (& attr ); If (ret! = 0) Return-1; }

Int MyThread: stop () { Int ret = pthread_kill (m_thread, SIGINT ); If (ret! = 0) Return-1; }

Int MyThread: join () { Int ret = pthread_join (m_thread, NULL ); If (ret! = 0) Return-1; }

Void MyThread: run () { While (false = m_bStop) { MyTask * pTask = m_threadPool-> getNextTask (); If (NULL! = PTask) { PTask-> process (); } } }

The thread pool is the responsibility of MyThreadPool. The main functions are as follows:

Int MyThreadPool: init () { Pthread_condattr_t cond_attr; Pthread_condattr_init (& cond_attr ); Pthread_condattr_setpshared (& cond_attr, PTHREAD_PROCESS_SHARED ); Int ret = pthread_cond_init (& m_cond, & cond_attr ); Pthread_condattr_destroy (& cond_attr ); If (ret_val! = 0) Return-1; Pthread_mutexattr_t attr; Pthread_mutexattr_init (& attr ); Pthread_mutexattr_setpshared (& attr, PTHREAD_PROCESS_SHARED ); Ret = pthread_mutex_init (& m_queueMutex, & attr ); Pthread_mutexattr_destroy (& attr ); If (ret_val! = 0) Return-1; For (int I = 0; I <m_poolSize; ++ I) { MyThread * thread = new MyThread (I + 1, this ); M_threads.push_back (thread ); } Return 0; }

Int MyThreadPool: start () { Int ret; For (int I = 0; I <m_poolSize; ++ I) { Ret = m_threads [I]-> start (); If (ret! = 0) Break; } Ret = pthread_cond_broadcast (& m_cond ); If (ret! = 0) Return-1; Return 0; }

Void MyThreadPool: addTask (MyTask * ptask) { If (NULL = ptask) Return; Pthread_mutex_lock (& m_queueMutex ); M_taskQueue.push_back (ptask ); If (m_waitingThreadCount> 0) Pthread_cond_signal (& m_cond ); Pthread_mutex_unlock (& m_queueMutex ); }

MyTask * MyThreadPool: getNextTask () { MyTask * pTask = NULL; Pthread_mutex_lock (& m_queueMutex ); While (m_taskQueue.begin () = m_taskQueue.end ()) { ++ M_waitingThreadCount; Pthread_cond_wait (& n_cond, & m_queueMutex ); -- M_waitingThreadCount; } PTask = m_taskQueue.front (); M_taskQueue.pop_front (); Pthread_mutex_unlock (& m_queueMutex ); Return pTask; }

MyTask is responsible for executing each task. The main method is as follows:

Void MyTask: process () { // Read the command from the client with read // Process the command // Write the result to the client with write }

 

IV. Implementation of Connection Pool

Each connection pool saves a linked list to save the established connection: list <MyConnection *> * m_connections

Of course, this linked list also needs to be locked for multi-thread protection: pthread_mutex_t m_connectionMutex;

Here, a MyConnection is also a MyTask, and a thread is responsible for it.

The thread pool is also a member variable of the Connection Pool: MyThreadPool * m_threadPool

The connection pool is the responsibility of the MyConnectionPool class. Its main functions are as follows:

Void MyConnectionPool: addConnection (MyConnection * pConn) { Pthread_mutex_lock (& m_connectionMutex ); M_connections-> push_back (pConn ); Pthread_mutex_unlock (& m_connectionMutex ); M_threadPool-> addTask (pConn ); }

MyConnectionPool also needs to start a thread behind it to manage these connections and remove the closed connection and the wrong connection.

Void MyConnectionPool: managePool () { Pthread_mutex_lock (& m_connectionMutex ); For (list <MyConnection * >:: iterator itr = m_connections-> begin (); itr! = M_connections-> end ();) { MyConnection * conn = * itr; If (conn-> isFinish ()) { Delete conn; Conn = NULL; List <MyConnection *>: iterator pos = itr ++; M_connections-> erase (pos ); } Else if (conn-> isError ()) { // Handle the wrong connection ++ Itr; } Else { ++ Itr; } } Pthread_mutex_unlock (& m_connectionMutex ); }

 

5. Implementation of listening threads

The listening thread needs to have a MySocketServer to listen to the client connection. Each time a new connection is formed, check whether the maximum number of connections has been exceeded. If the number of connections has exceeded, close the connection, if the maximum number of connections is not exceeded, a new MyConnection is formed and added to the connection pool and thread pool.

MySocketServer * pServer = new MySocketServer (port ); MyConnectionPool * pPool = new MyConnectionPool (); While (! StopFlag) { MySocket * sock = pServer-> acceptConnection (5 ); If (sock! = Null) { If (m_connections.size> maxConnectionSize) { Sock. close (); } MyTask * pTask = new MyConnection (); PPool-> addConnection (pTask ); } }