Write an efficient C + + Redis client application using the ACL library

I. Overview

(can skip this paragraph directly) Redis has become more and more widely used as a NoSQL data service, and its greatest advantage over memcached is that it provides a richer data structure, so the application scenario is broader. The advent of Redis is a boon for the vast majority of web application developers, while a large number of open source personnel contribute client-side code, such as the Java language of the jedis,php language Phpredis/predis, and so on, these languages of the Redis library is rich and useful, and the C + + programmer Seems to be not so fortunate, officially provided the C version of Hiredis as the client library, many enthusiasts are based on the Hiredis two times package and development form a C + + client library, but these libraries (including the official Hiredis) most of the use of trouble, to users caused a lot of error opportunities. Always wanted to develop a more user-friendly C + + version of the Redis client library (note: The officially provided library is basically protocol-based, which means that users need to spend a lot of time populating each protocol field and having to analyze different result types that the server might return), but every time you see a redis 150 When multiple client commands are in retreat, it is a huge development effort to provide an easy-to-use C + + function interface for each command.

In the subsequent development of several projects by the official Hiredis library repeatedly destroyed, finally can not endure, decided to re-develop a new set of Redis client API, the library not only to implement these 150 multiple client commands, but also need to provide convenient and flexible connection pool and connection pool cluster management functions (fortunately ACL Library already has a common network connection pool and connection pool Cluster Management module, in addition, according to the previous practice, it is possible to provide more than 150 function interfaces, because the same command may be due to different parameter type scenarios to provide multiple function interfaces (the final result is to provide 3, 400 functions api,7000+ Line source, 2000+ costume file); After careful study of the Redis communication protocol, we started to design and develop (Redis protocol design is very simple and practical, that is, it can support binary, but also facilitate manual debugging). In the development process, a lot of reference to the Http://redisdoc.com website of the Chinese online translation version (very grateful to Huang classmate's hard work).

Ii. ACL Redis Library classification

According to the REDIS data structure type, divided into 12 large classes, each large class provides a different function interface, these 12 C + + classes are shown as follows:

1, Redis_key:redis all data types of the unified key operation class; Because REDIS data structure types are basic key-value types, where VALUE is divided into different data structure types;

2. Redis_connectioin: Classes related to Redis-server connection;

3. Redis_server: Classes related to redis-server service management;

4, the data type used in Redis_string:redis to denote the string;

5. The data type used to represent the hash table in Redis_hash:redis, each data object consists of "key-domain value pair set", that is, one KEY corresponds to multiple "domain value pairs", and each "domain value pair" consists of a field name and a field value;

6. The data type used to represent the list in Redis_list:redis;

7, the data type used in Redis_set:redis to denote the collection;

8. The data type used in Redis_zset:redis to denote an ordered set;

9. The data type used in Redis_pubsub:redis to denote "publish-subscribe";

10. The data types used in Redis_hyperloglog:redis to denote the hyperloglog cardinality estimation algorithm;

11. Data types used in Redis_script:redis to interact with LUA scripts;

12. The data type of multiple Redis commands that are executed in Redis_transaction:redis in a transactional manner (note: The transaction is very different from the transaction in the database, and the transaction process in Redis does not have a transaction rollback mechanism in the database. can only guarantee that many of these commands are executed or not executed);

In addition to the above 12 categories that correspond to the official Redis commands, several other classes are available in the ACL library:

13, Redis_command: The base class of the above 12 classes;

14, Redis_client:redis client network connection class;

15, Redis_result:redis command result class;

16, Redis_pool: For all the above commands to support the connection pool mode;

17, Redis_manager: For all of the above command allows to establish a connection pool cluster with multiple Redis-server services (that is, to establish a connection pool with each redis-server).

Iii. examples of ACL Redis use

1), here is a simple example of using the Redis client library in the ACL framework:

/ **
 * @param conn {acl :: redis_client &} redis connection object
 * @return {bool} whether the operation was successful
 * /
bool test_redis_string (acl :: redis_client & conn, const char * key)
{
// Create a command operation class object of type redis string, and connect the connection class object with the operation class
// object is bound
acl :: redis_string string_operation (& conn);
const char * value = "test_value";

// Add K-V value to redis-server
if (string_operation.set (key, value) == false)
{
const acl :: redis_result * res = string_operation.get_result ();
printf ("set key:% s error:% s \ r \ n",
key, res? res-> get_error (): "unknown error");
return false;
}
printf ("set key:% s ok! \ r \ n", key);

// Need to reset the state of the connection object, or call conn.reset () directly
string_operation.reset ();

// Get the value of the corresponding key from redis-server
acl :: string buf;
if (string_operation.get (key, buf) == false)
{
const acl :: redis_result * res = string_operation.get_result ();
printf ("get key:% s error:% s \ r \ n",
key, res? res-> get_error (): "unknown error");
return false;
}
printf ("get key:% s ok, value:% s \ r \ n", key, buf.c_str ());

// detect if the given key exists in redis-server, need to create redis key
// class object, and bind the redis connection object to it
acl :: redis_key key_operation;
conn.reset (); // Reset connection status
key_operation.set_client (conn); // Bind the connection object to the operation object
if (key_operation.exists (key) == false)
{
if (conn.eof ())
{
printf ("disconnected from redis-server \ r \ n");
return false;
}

printf ("key:% s not exists \ r \ n", key);
}
else
printf ("key:% s exists \ r \ n", key);

// delete the string object of the specified key
conn.reset (); // Reset the connection object state first
if (key_operation.del (key, NULL) <0)
{
printf ("del key:% s error \ r \ n", key);
return false;
}
else
printf ("del key:% s ok \ r \ n", key);

return true;
}

/ **
 * @param redis_addr {const char *} redis-server server address,
 * The format is: ip: port, such as: 127.0.0.1:6379
 * @param conn_timeout (int) timeout (in seconds) to connect to redis-server
 * @param rw_timeout (int) IO timeout (in seconds) to communicate with redis-server
 * /
bool test_redis (const char * redis_addr, int conn_timeout, int rw_timeout)
{
// Create a redis client network connection class object
acl :: redis_client conn (redis_addr, conn_timeout, rw_timeout);
const char * key = "test_key";
return test_redis_string (conn, key);
}

The simple example above is to add the string type data to the Redis-server and get the specified string data from Redis-server, and determine whether the object specifying the specified key exists on the redis-server--- ; Removes the data object for the specified key from Redis-server (that is, the string object in the example). Using the simple example above, the user should be aware of the following points:

A), in the design of the Redis library in the ACL, the Redis connection class object is separated from the command operation class object, and the 12 redis command operation classes correspond to the corresponding 12 command operation classes in the ACL Redis library;

b), when using the Redis command to manipulate the class, the Redis connection class object and the command action class object must be bound (in order to facilitate the operation of the class inside the network connection, protocol package and Protocol resolution, etc.);

c), when reusing a Redis connection class object, you need to first reset the state of the connection class object (that is, call: Acl::redis_client::reset ()), which is primarily to free the intermediate memory resources of the last command operation process;

D), a Redis connection class object can be used by more than one command class operation class object (need to bind once before use);

e), there are two ways to bind a Redis connection object to a command operand: You can either pass in a non-empty Redis connection object in the constructor, or call the Set_client method of the action object to bind.

      2), a slight modification to the above example, so that it can support the connection pooling method, the sample code is as follows:

/ **
 * @param conn {acl :: redis_client &} redis connection object
 * @return {bool} whether the operation was successful
 * /
bool test_redis_string (acl :: redis_client & conn, const char * key)
{
...... // The code is the same as the above code, omitted

return true;
}

// child thread processing class
class test_thread: public acl :: thread
{
public:
test_thread (acl :: redis_pool & pool): pool_ (pool) ()

~ test_thread () ()

protected:
// base class (acl :: thread) pure virtual function
virtual void * run ()
{
acl :: string key;
// Give each thread its own key for testing purposes, where thread_id ()
// function is a method of base class acl :: thread, used to get the thread unique ID number
key.format ("test_key:% lu", thread_id ());

acl :: redis_client * conn;

for (int i = 0; i <1000; i ++)
{
// Get a redis connection object from the redis client connection pool
conn = (acl :: redis_client *) pool_.peek ();
if (conn == NULL)
{
printf ("peek redis connection error \ r \ n");
break;
}

// Process the redis client command operation
if (test_redis_string (* conn) == false)
{
printf ("redis operation error \ r \ n");
break;
}
}

return NULL;
}

private:
acl :: redis_pool & pool_;
};

void test_redis_pool (const char * redis_addr, int max_threads,
int conn_timeout, int rw_timeout)
{
// create redis connection pool object
acl :: redis_pool pool (redis_addr, max_threads);
// Set the timeout and IO timeout for connection redis, the unit is second
pool.set_timeout (conn_timeout, rw_timeout);

// create a set of child threads
std :: vector <test_thread *> threads;
for (int i = 0; i <max_threads; i ++)
{
test_thread * thread = new test_thread (pool);
threads.push_back (thread);
thread-> set_detachable (false);
thread-> start ();
}

// wait for all child threads to exit normally
std :: vector <test_thread *> :: iterator it = threads.begin ();
for (; it! = threads.end (); ++ it)
{
(* it)-> wait ();
delete (* it);
}
}

In addition to creating threads and Redis connection pools, the above example and Example 1 have the same code as the functionality.

3), below for example 2 above, make a slight modification, so that it can support the Redis cluster Connection pool way, the sample code is as follows:

/ **
 * @param conn {acl :: redis_client &} redis connection object
 * @return {bool} whether the operation was successful
 * /
bool test_redis_string (acl :: redis_client & conn, const char * key)
{
... // Same as the above example code, omitted
return true;
}

// child thread processing class
class test_thread: public acl :: thread
{
public:
test_thread (acl :: redis_manager & manager): manager_ (manager) ()

~ test_thread () ()

protected:
// base class (acl :: thread) pure virtual function
virtual void * run ()
{
acl :: string key;
// Give each thread its own key for testing purposes, where thread_id ()
// function is a method of base class acl :: thread, used to get the thread unique ID number
key.format ("test_key:% lu", thread_id ());

acl :: redis_pool * pool;
acl :: redis_client * conn;

for (int i = 0; i <1000; i ++)
{
// Get a redis-server connection pool object from the connection pool cluster manager
pool = (acl :: redis_pool *) manager_.peek ();
if (pool == NULL)
{
printf ("peek connection pool failed \ r \ n");
break;
}

// Get a redis connection object from the redis client connection pool
conn = (acl :: redis_client *) pool_.peek ();
if (conn == NULL)
{
printf ("peek redis connection error \ r \ n");
break;
}

// Process the redis client command operation
if (test_redis_string (* conn) == false)
{
printf ("redis operation error \ r \ n");
break;
}
}

return NULL;
}

private:
(acl :: redis_manager & manager_;
};

void test_redis_pool (const char * redis_addr, int max_threads,
int conn_timeout, int rw_timeout)
{
// Create a redis cluster connection pool object
acl :: redis_manager manager (conn_timeout, rw_timeout);

// Add multiple redis-server server instance addresses
manager.set ("127.0.0.1:6379", max_threads);
manager.set ("127.0.0.1:6380", max_threads);
manager.set ("127.0.0.1:6381", max_threads);

// Set the timeout and IO timeout for connection redis, the unit is second
pool.set_timeout (conn_timeout, rw_timeout);

// create a set of child threads
std :: vector <test_thread *> threads;
for (int i = 0; i <max_threads; i ++)
{
test_thread * thread = new test_thread (manager);
threads.push_back (thread);
thread-> set_detachable (false);
thread-> start ();
}

// wait for all child threads to exit normally
std :: vector <test_thread *> :: iterator it = threads.begin ();
for (; it! = threads.end (); ++ it)
{
(* it)-> wait ();
delete (* it);
}
} 

This example modifies only a few pieces of code to support the cluster Redis connection pooling process by creating a cluster Connection pool object (which can add multiple Redis-server service addresses) and getting a connection pool object from the Cluster Connection pool object---> Gets a connection from the connection pool object---> The connection object is bound to the Redis action class object.

Iv. Summary

The above describes the use of new Redis libraries in the ACL framework and the process of processing, the library of complex protocols and network processing is hidden in the implementation of the internal, so that users feel like in the call of this function. The use of ACL threads is mentioned in Example 2, 3), and articles about using threads in more detail in the ACL library are described in: "Using the Acl_cpp Library to write multithreaded program."

Download: http://sourceforge.net/projects/acl/

Svn:svn://svn.code.sf.net/p/acl/code/trunk

Github:https://github.com/zhengshuxin/acl

Domestic Mirror: Http://git.oschina.net/zsxxsz/acl/tree/master

bbs:http://www.acl-dev.com/

QQ Group: 242722074

Write an efficient C + + Redis client application using the ACL library