Ha, another super simple queue (MQ) implementation of the ~

Open source Message queue has been many, but most of it is heavy, the actual environment, many may just use a little bit of function, kill chicken use sledgehammer, really some wasted. Many times, we only want a green leaf, but they give us the whole spring, it is difficult to digest. In the Dir spirit, also pondering a super simple queue implementation. Said to be super simple, uh, absolutely super simple, the queue storage uses Redis for persistent storage, using Netty to provide HTTP-style queue out/in. The Jedis used by the Redis client. And then, then it's gone. One. The Redisredis built-in subscription publishing model (Publish/subscribe), the flaw is, does not store, once the subscriber is disconnected, will not be able to receive the message, once again connected, the publisher posted during the disconnection of the message is not available. Can only be implemented by the list array, using the Rpush/lpop Combination command to implement the FIFO queue model, of course, Redis also provides blocking version of the Blpush/brpush/blpop/brpop, see how we use the actual environment. Java clients use Jedis and provide a rich interface. However, it is important to note that connection pooling is required, otherwise it is possible that the Jedis connection is not sufficient in large numbers of cases.

private static Jedispool pool; static {  ResourceBundle bundle = Resourcebundle.getbundle ("Redis");  if (bundle = = null) {   throw new IllegalArgumentException (     "Cannot find the signverprop.properties");  }  Jedispoolconfig config = new Jedispoolconfig ();  Config.setmaxactive (integer.valueof (Bundle    . getString ("redis.pool.maxActive"));  Config.setmaxidle (integer.valueof (Bundle    . getString ("Redis.pool.maxIdle"));  Config.setmaxwait (integer.valueof (Bundle    . getString ("redis.pool.maxWait"));  Pool = new Jedispool (config, bundle.getstring ("Redis.server"),    integer.valueof (bundle.getstring ("Redis.port") )); }

Two. Netty is a very mature NIO framework that uses it to provide access to HTTP-style queues. Well, currently only provides HTTP mode in the queue, out of the queue and so on.
The HTTP form is: / HTTP Server name: Port/Queue operation primitive/queue name? msg= Message Content
Queue operation Primitives, only get and put;get are out of queue, put is in queue.
Returned as JSON: {s:0, M: ' Error message/message content '}
S is the abbreviation for success, with a value of 0, which means that the false,1 corresponds to success. M is the abbreviation for message, the error message/message content, and when the message is added, the number of inserted messages is returned by default with UTF-8. Into queue: http://localhost:8080/put/demo?msg= message Content
Out Queue: Http://localhost:8080/get/demo HTTP method, especially suitable for the push of message data between LAN. Three. The entrance interpretation uses the Queuedaemon encapsulation of the Netty code: The invocation is simple, the program starts the Portal: while the HTTP way queue request processor is Httprequesthandler: four. Queue processor This is very simple, directly using the Jedis client can, here has been occupied a connection, not released. Five. AB pressure test native configuration: Pentium (R) dual-core CPU E5200 2.50ghz,2.00 GB of memory, Windows XP system, REDIS-2.4.5-WIN32 version 32 (non-Linux version), all running on one machine. inserting a bytes text message queue: 1781.24 requests/sec

D:\apache2.2\bin>ab-c 10-n 100000 "http://localhost:8080/put/demo?msg= Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaa "This is Apachebench, Version 2.3 < $Revision: 655654 $>copyright 1996 Adam Twiss, Zeus T Echnology LTD, http://www.zeustech.net/Licensed to the Apache software Foundation, http://www.apache.org/Benchmarking localhost (be patient) completed 10000 requestscompleted 20000 requestscompleted 30000 requestscompleted 40000 requestscompleted 50000 requestscompleted 60000 requestscompleted 70000 requestscompleted 80000 requestscompleted 90000 requestscompleted 100000 requestsfinished 100000 requestsserver Software:Server Hostname: Localhostserver port:8080document Path:/put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa  Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaadocument Length:12 bytesconcurrency Level:10time taken for tests:56.141 Secondscomplete Requests:100000fail Ed requests:99991 (connect:0, receive:0, length:99991, exceptions:0) Write errors:0total Transferr ed:8188895 byteshtml Transferred:1588895 bytesrequests per second:1781.24 [#/sec] (mean) time per request:5.614 [MS] (mean) time per re quest:0.561 [MS] (mean, across all concurrent requests) Transfer rate:142.45 [kbytes/sec] Receivedconnecti  On times (ms) min MEAN[+/-SD] Median maxconnect:0 0 1.7 0 16processing:0 5 10.7 0 781waiting:0 5 10.4 0 766total:0 6 10.8 0 781Percentage of th     E requests served within a certain time (ms) 50% 0 66% 16 75% 16 80% 16 90% 16 95% 16 98% 99% 100% 781 (longest request)

Insert a bytes text message queue (add keep-alive support): 1875.18 requests/sec

D:\apache2.2\bin>ab-k-C 10-n 100000 "http://localhost:8080/put/demo?msg= Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaa "This is Apachebench, Version 2.3 < $Revision: 655654 $>copyright 1996 Adam Twiss, Zeus technol Ogy LTD, http://www.zeustech.net/Licensed to the Apache software Foundation, http://www.apache.org/Benchmarking localhost (be patient) completed 10000 requestscompleted 20000 requestscompleted 30000 requestscompleted 40000 requestscompleted 50000 requestscompleted 60000 requestscompleted 70000 requestscompleted 80000 requestscompleted 90000 requestscompleted 100000 requestsfinished 100000 requestsserver Software:Server Hostname: Localhostserver port:8080document Path:/put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaadocument Length:17 bytesconcurrency Level:10time taken for tests:53.328 Secondscomplete Requests:100000faile D requests:0write errors:0keep-alive requests:0total transferred:8300000 byteshtml Transferre d:1700000 bytesrequests per Second:   1875.18 [#/sec] (mean) time per request:5.333 [MS] (mean) time per request:0.533 [MS] (mean, across all con Current requests) Transfer rate:151.99 [Kbytes/sec] receivedconnection times (ms) min MEAN[+/-SD] m   Edian maxconnect:0 0 1.5 0 16processing:0 5 7.8 0 203waiting:0 5  7.8 0 203total:0 5 7.9 0 203Percentage of the requests served within a certain time (MS) 50% 0 66% 0 75%-80%-90%-95%-98% (99% 100%) 203 longest St

Get bytes message (with keep-alive): 1875.73 requests/sec

D:\apache2.2\bin>ab-k-C 10-n 100000 "Http://localhost:8080/get/demo" This is Apachebench, Version 2.3 < $Revision: 655654 $>copyright 1996 Adam Twiss, Zeus technology LTD, http://www.zeustech.net/Licensed to the Apache software Found ation, http://www.apache.org/Benchmarking localhost (be patient) completed 10000 requestscompleted 20000 requestscompleted 30000 requestscompleted 40000 requestscompleted 50000 requestscompleted 60000 requestsCompleted 70000 requestscompleted 80000 requestscompleted 90000 requestscompleted 100000 requestsfinished 100000 requestsServer        Software:server hostname:localhostserver port:8080document Path:/get/demodocument Length: 523 bytesconcurrency Level:10time taken for tests:53.313 secondscomplete requests:100000failed request       S:0write errors:0keep-alive requests:0total transferred:58900000 byteshtml transferred: 52300000 bytesrequests per second:1875.73 [#/sec](mean)  Time per request:5.331 [MS] (mean) time per request:0.533 [MS] (mean, across all concurrent requests) Transfer        rate:1078.91 [Kbytes/sec] receivedconnection times (ms) min MEAN[+/-SD] Median Maxconnect:           0 0 1.9 0 16processing:0 5 7.5 0 94waiting:0 4 6.9 0 94Total:  0 5 7.6 0 94Percentage of the requests served within a certain time (ms) 50% 0 66% 0 75% 80%-90%-95%-98%-99%-100% 94 (longest request)

Other questions

1. Temporarily no support for security/authorization, this is actually easy to achieve
2. The queue processor is simple and can be used directly with Jedis
3. Shard the queue data (sharding) and write a Queueservice implementation
4. The memory persistence of Redis is not assured, the use of Diskstore storage mode is good, in Redis configuration, does not involve the program
5. Distributed storage on the queue, write a queueservice implementation, other can not change
6. Not suitable for the strict subscription/release model, which is suitable for multiple publishers/individual subscriber environments
7. HTTP requests return content in JSON format, XML format, not required for the time being
8. LAN environment, the system to send messages between the push/notification,

Project: http://code.google.com/p/nettyqueue/Reference:

1. Httpsqs

Http://www.blogjava.net/yongboy/archive/2012/03/20/372308.html

Ha, another super simple queue (MQ) implementation of the ~