High Performance Server C Development Library-minac working notes

Background
Gleasy some high-performance middleware (such as distributed NoSQL cluster Cloudredis, using Java), after a line of time verification, found that the CPU is always high, and the NoSQL cluster itself does not have complex business logic, Most of the cases are direct proxy to the back end of the Redis. Due to the stringent performance requirements, the decision to use C to refactor Cloudredis. In the process of refactoring, because the Java version of Cloudredis is based on MINA2, especially the codec links rely heavily on the Iobuffer and iosession of these two classes, indeed very useful. Therefore, the derivation of the use of C language to achieve a complete MINA2 interface idea, for reference MINA2 itself excellent design, avoid a lot of detours, and finally realize a complete set of development library, in order to pay tribute to the Mina2, named Minac.

Minac Introduction
Minac based on Epoll, using the event-loop-per-thread structure, the number of IO thread defaults to the CPU's kernel number.
Familiar with the MINA2 students basically can be used directly, without any learning costs.
Here are some of the basic class libraries provided
1. Iobuffer
The full implementation of the Java Iobuffer class, all of which are compatible with the Java version (all the basic types written to BUF will be converted to the big endian, which will be converted back to the system-supported endian mode) provides the following methods:

extern iobuffer_t *iobuffer_create (int32_t);
extern void Iobuffer_destroy (iobuffer_t *);
extern void Iobuffer_set_auto_expend (iobuffer_t *,int flag);
extern void Iobuffer_set_auto_shrink (iobuffer_t *,int flag);
extern void Iobuffer_set_increment (iobuffer_t *,int32_t);

extern void Iobuffer_flip (iobuffer_t *);
extern void Iobuffer_compact (iobuffer_t *);
extern void Iobuffer_clear (iobuffer_t *);
extern int32_t iobuffer_set_position (iobuffer_t *,int32_t);
extern int32_t iobuffer_get_position (iobuffer_t *);
extern int32_t iobuffer_set_limit (iobuffer_t *,int32_t);
extern int32_t iobuffer_get_limit (iobuffer_t *);
extern void Iobuffer_skip (iobuffer_t *,int32_t);

extern int32_t iobuffer_remaining (iobuffer_t *);
extern int32_t Iobuffer_put_int (iobuffer_t *,int32_t);
extern int32_t Iobuffer_get_int (iobuffer_t *,int32_t *);
extern int32_t iobuffer_put_double (iobuffer_t *,double);
extern int32_t iobuffer_get_double (iobuffer_t *,double *);
extern int32_t Iobuffer_put_long (iobuffer_t *,int64_t);
extern int32_t Iobuffer_get_long (iobuffer_t *,int64_t *);
extern int32_t iobuffer_put (iobuffer_t *,char *,int32_t);
extern int32_t Iobuffer_get (iobuffer_t *,char *,int32_t);
extern int32_t Iobuffer_put_by_read (iobuffer_t *,int fd,int32_t);
extern int32_t Iobuffer_get_and_write (iobuffer_t *,int fd,int32_t);
/**
* Get content from src write to another www.111cn.net dest
* Note: This function does not support SRC and dest for the same iobuffer case
*/
extern int32_t Iobuffer_put_iobuffer (iobuffer_t *dest,iobuffer_t *src);

2. Iosession
Realize the basic function of MINA2 iosession, the main interface is as follows:

extern iosession_t *iosession_create (int connfd, mina_session_config_t *sessionconfig);
extern void Iosession_destroy (iosession_t *session);
extern int Iosession_write (iosession_t *session, iobuffer_t *buf);
extern int iosession_write_wait (iosession_t *session, iobuffer_t *buf,int seconds);
extern int Iosession_close (iosession_t *session);
extern int Iosession_flush (iosession_t *session);

3. Socket_acceptor
Implements the Mina2 Iosocketacceptor class, the main interface is as follows

extern mina_socket_acceptor_t* mina_socket_acceptor_create ();
extern void Mina_socket_acceptor_destroy (mina_socket_acceptor_t *acceptor);
extern int Mina_socket_acceptor_bind (mina_socket_acceptor_t *acceptor,struct sockaddr_in *addr);
extern void Mina_socket_acceptor_unbind (mina_socket_acceptor_t *acceptor);

4. Socket_connector
Implementation of the Mina2 Iosockectconnector class, the main interface is as follows:

extern mina_socket_connector_t* mina_socket_connector_create ();
extern void Mina_socket_connector_destroy (mina_socket_connector_t *acceptor);
extern iosession_t *mina_socket_connector_connect (mina_socket_connector_t *acceptor,struct sockaddr *addr);
extern void Mina_socket_connector_stop (mina_socket_connector_t *acceptor);

5. Socket_handler
The iosockethandleradaptor of the MINA2 is implemented, defined as follows:

typedef void (*_messagereceived) (iosession_t *,iobuffer_t *);
typedef void (*_messagesent) (iosession_t *,iobuffer_t *);
typedef void (*_sessioncreated) (iosession_t *);
typedef void (*_sessionopened) (iosession_t *);
typedef void (*_sessionclosed) (iosession_t *);
typedef void (*_exceptioncaught) (iosession_t *,mina_exception_t *);
typedef void (*_sessionidle) (iosession_t *,mina_iosession_idle_t);

typedef struct mina_socket_handler_t{
_messagereceived messagereceived;
_messagesent messagesent;
_sessioncreated sessioncreated;
_sessionopened sessionopened;
_sessionclosed sessionclosed;
_exceptioncaught Exceptioncaught;
_sessionidle Sessionidle;
}mina_socket_handler_t;

6. Session_config
MINA2 Sessionconfig is implemented, the main interface is as follows:

typedef struct mina_session_config_t{
int readbuffersize;
int minreadbuffersize;
int maxreadbuffersize;
int writebuffersize;
int minwritebuffersize;
int maxwritebuffersize;

int tcpnodelay;
int sendbufsize;
int revbufsize;

int Readeridletime;//time in SEC
int Writeridletime;//time in SEC
}mina_session_config_t;

extern mina_session_config_t *mina_session_config_create ();
extern void Mina_session_config_destroy (mina_session_config_t *);

7. Examples of use:
Use a few lines of code to easily implement the server program:

#include "minac.h"
#include "Common.h"
#include <zlog.h>

static void Messagereceived (iosession_t *session,iobuffer_t *buf) {
Dzlog_info ("received a row, size:%d", Iobuffer_remaining (BUF));
int32_t remaining = iobuffer_remaining (BUF);
if (remaining>0) {
iobuffer_t *rbuf = iobuffer_create (remaining);
Iobuffer_compact (RBUF);
Iobuffer_put_iobuffer (RBUF,BUF);
Iobuffer_flip (RBUF);
Iosession_write (SESSION,RBUF);
Iobuffer_destroy (RBUF);
}
}

static void Messagesent (iosession_t *session,iobuffer_t *buf) {
Dzlog_info ("Written www.111Cn.net data, size:%d", Iobuffer_remaining (BUF));
}
static void sessionclosed (Iosession_t *session) {
Dzlog_info ("Session is closed.%d", SESSION-&GT;CONNFD);
}
static void sessionopened (Iosession_t *session) {
Dzlog_info ("Session is opened.%d", SESSION-&GT;CONNFD);
}
static void Exceptioncaught (iosession_t *session,mina_exception_t *exception) {
Dzlog_info ("Exception met, code:%d, messag:%s", exception->errorcode,exception->errormessage);
Iosession_close (session);
}
static void Sessionidle (iosession_t *session,mina_iosession_idle_t idletype) {
Dzlog_info ("Session idle, fd:%d, idletype:%d", Session->connfd,idletype);
}
static void Exit_handler (void *arg) {
mina_socket_acceptor_t *acceptor = (mina_socket_acceptor_t *) arg;
Mina_socket_acceptor_unbind (acceptor);
}
extern void Test_acceptor () {
mina_socket_acceptor_t *acceptor = Mina_socket_acceptor_create ();
Acceptor->sessionconfig->tcpnodelay = 1;
Acceptor->sessionconfig->readeridletime = 2;
Acceptor->sessionconfig->writeridletime = 2;

acceptor->sockethandler->messagereceived = messagereceived;
Acceptor->sockethandler->messagesent = messagesent;
acceptor->sockethandler->sessionopened = sessionopened;
acceptor->sockethandler->sessionclosed = sessionclosed;
Acceptor->sockethandler->exceptioncaught = Exceptioncaught;
Acceptor->sockethandler->sessionidle = Sessionidle;

Register_exit_handler (Exit_handler, (void *) acceptor);

struct sockaddr_in servaddr;
servaddr.sin_family = af_inet;
Servaddr.sin_port = htons (6666);
SERVADDR.SIN_ADDR.S_ADDR = htonl (Inaddr_any);

Mina_socket_acceptor_bind (ACCEPTOR,&SERVADDR);
}

People who are familiar with MINA2 are basically familiar with the code above.

Performance how
function above is easy to use, how about performance? In Echo server, for example, the performance test results are as follows:

People who are familiar with MINA2 are basically familiar with the code above.

Performance how
function above is easy to use, how about performance? In Echo server, for example, the performance test results are as follows:
1. Experiment 1 (400 client)

400 concurrent clients, TPS Max 410,000, average 180,000, CPU footprint 164%

2. Experiment 2 (1000 client)

1000 concurrent clients, TPS Max 550,000, average 300,000, CPU footprint 169%

Original from: Gleasy Team Blog