Research and Analysis on spserver open source server framework

There are still few open-source C/C ++ server frameworks on the Internet. Recently I have studied spserver, which uses many design patterns. The purpose of the design pattern is to separate and encapsulate unchanged things from mutable things, so as to avoid code modification in the future, the so-called "Close modification and open expansion", but abuse of design patterns will complicate the simple problem.

Spserver has a small amount of code and is easy to understand. If it is just a simple echo server, spserver can quickly build a server framework for beginners. After testing with VLD, we found that spserver has many memory leaks. In addition, the msgqueue_destroy function will cause thread blocking and the program cannot end normally.

1. The sp_handler class is responsible for processing business logic. A connection corresponds to an sp_handler object instance.

2. Create the sp_handler object when sp_msfactory has a new link.

3. ms_decoder is responsible for parsing data packets. Note: Do not return Eok when the buffer is empty. Otherwise, the CPU usage will be too high. Returning Eok triggers the sp_handler: handle event.

4. sp_handler indicates timeout or error. Finally, sp_handler: Close will be called.

5. The data type that identifies an online user is sp_sid_t, which can be obtained through response-> getfromsid.

6. To send a message to an online user, it is a new sp_message, and then it is thrown to the response object using the sp_response: addmessage method. Of course, use MSG-> gettolist ()-> Add () to add sp_sid_t ..

The semi-synchronous semi-asynchronous mode in spserver is equivalent to the master thread responsible for all data sending and receiving, and the worker thread is responsible for logic. The main thread communicates with the worker thread through the message queue. After the main thread receives the data, it notifies the worker thread that there is data. The worker thread parses and processes the packets, puts the data to be sent into the queue, and then notifies the main thread that there is data, you can send a response. This model is suitable for scenarios where the logic is complex, the processing time is long, and a small amount of data needs to be sent and received. However, if you need to process a file server like this, there is a lot of Io data, but the logic is very simple, it is not appropriate,
The main thread will become a bottleneck, that is, Io will become a bottleneck. In addition, spserver does not implement code to connect to other servers as a client. You need to connect to other servers by yourself.

For spserver, I am still interested in its thread pool implementation. It may be because of the relationship between application scenarios. spserver's implementation of the thread pool is relatively simple and there are no complicated patterns.

The thread pool encapsulation mainly provides the dispatch function, which assigns a function to be called to the thread in the thread pool.

class SP_ThreadPool {public:    typedef void ( * DispatchFunc_t )( void * );    SP_ThreadPool( int maxThreads, const char * tag = 0 );    ~SP_ThreadPool();    /// @return 0 : OK, -1 : cannot create thread    int dispatch( DispatchFunc_t dispatchFunc, void *arg );    int getMaxThreads();private:    char * mTag;    int mMaxThreads;    int mIndex;    int mTotal;    int mIsShutdown;

The following is the dispatch function code:

int SP_ThreadPool :: dispatch( DispatchFunc_t dispatchFunc, void *arg ){    int ret = 0;    pthread_attr_t attr;    SP_Thread_t * thread = NULL;    pthread_mutex_lock( &mMainMutex );    if( mIndex <= 0 && mTotal >= mMaxThreads ) {        pthread_cond_wait( &mIdleCond, &mMainMutex );    }    if( mIndex <= 0 ) {        SP_Thread_t * thread = ( SP_Thread_t * )malloc( sizeof( SP_Thread_t ) );        thread->mId = 0;        pthread_mutex_init( &thread->mMutex, NULL );        pthread_cond_init( &thread->mCond, NULL );        thread->mFunc = dispatchFunc;        thread->mArg = arg;        thread->mParent = this;        pthread_attr_init( &attr );        pthread_attr_setdetachstate( &attr,PTHREAD_CREATE_DETACHED );        if( 0 == pthread_create( &( thread->mId ), &attr, wrapperFunc, thread ) ) {            mTotal++;            syslog( LOG_NOTICE, "[tp@%s] create thread#%ld\n", mTag, thread->mId );        } else {            ret = -1;            syslog( LOG_WARNING, "[tp@%s] cannot create thread\n", mTag );            pthread_mutex_destroy( &thread->mMutex );            pthread_cond_destroy( &thread->mCond );            free( thread );        }    } else {        mIndex--;        thread = mThreadList[ mIndex ];        mThreadList[ mIndex ] = NULL;        thread->mFunc = dispatchFunc;        thread->mArg = arg;        thread->mParent = this;        pthread_mutex_lock( &thread->mMutex );        pthread_cond_signal( &thread->mCond ) ;        pthread_mutex_unlock ( &thread->mMutex );    }    pthread_mutex_unlock( &mMainMutex );    return ret;}

If no idle thread exists, create a new thread to process the transaction. Each thread runs a function package.

void * SP_ThreadPool :: wrapperFunc( void * arg ){    SP_Thread_t * thread = ( SP_Thread_t * )arg;    for( ; 0 == thread->mParent->mIsShutdown; ) {        thread->mFunc( thread->mArg );        pthread_mutex_lock( &thread->mMutex );        if( 0 == thread->mParent->saveThread( thread ) ) {            pthread_cond_wait( &thread->mCond, &thread->mMutex );            pthread_mutex_unlock( &thread->mMutex );        } else {            pthread_mutex_unlock( &thread->mMutex );            pthread_cond_destroy( &thread->mCond );            pthread_mutex_destroy( &thread->mMutex );            free( thread );            break;        }    }    pthread_mutex_lock( &thread->mParent->mMainMutex );    thread->mParent->mTotal--;    if( thread->mParent->mTotal <= 0 ) {        pthread_cond_signal( &thread->mParent->mEmptyCond );    }    pthread_mutex_unlock( &thread->mParent->mMainMutex );    return NULL;}

In this function, if the transaction is completed, call pthread_cond_wait to wait for the new task.
From the above implementation, from the thread pool perspective, itNo buffered tasks to be processedThis is because in spserver, a thread is used to receive messages in the msgquene queue. For each received message, the dispatch function in threadpool is called to process the task, if there are no Idle threads in the thread pool and the number of threads reaches the maximum number, the current thread will wait. (Here it may be called the leader/followers mode application ).
Because the incoming task queue and the receiving queue from the queue are in different threads, the addition of the task thread and the receiving queue thread worker communication mainly use Unix domain sockets. When a task is added to the task queue, the read end is notified by writing a file, and a task is added to the task queue.