Common open Source messaging systems

Source: Internet
Author: User
Tags benchmark stomp rabbitmq

The role of messaging systems: asynchronous processing, cutting spikes, and reducing coupling between components.

Select the messaging system to consider the following aspects of your business needs:

    1. is persisted
    2. Throughput capacity
    3. Highly Available
    4. Distributed scaling capabilities
    5. Compatible with existing protocols
    6. Easy to maintain
    7. Other, such as message loss and duplicate processing
    8. Avoid single point of failure
    9. Load Balancing
Common Message system protocols:
    1. STOMP
    2. Amqp
    3. Protocol similar to MEMCACHE
    4. HTTP
    5. Custom Format
1, 2 is a good optional open source components: 1. Kafka/metaq: Widely used within Linkedin (similar to the Java version of the domestic Metaq)

Due to the priority of throughput, it is more suitable for message collection and processing of big data, such as log analysis, real-time report of user behavior information, gathering and analysis of cluster state information.

    1. Prioritize persistent design and rely on page cache to manage memory
    2. High throughput 112mb/s 11K msgs/s (more than BEANSTALKD >70x throughput capability)
    3. Supports asynchronous replication
    4. High-availability, Zookeeper-based cluster design, support for consumer-fail re-load balancing
    5. Kafka provides PHP class library
    6. Support for Ganglia JMX monitoring
    7. The need for policies to avoid duplicate messages, the way Consumers Update Zookeeper offset (Metaq has provided several ways to avoid message duplication)
    8. Metaq provides HTTP interface

http://www.mail-archive.com/[email protected]/msg02082.html
Https://github.com/neophenix/StateOfTheMQ/blob/master/state_of_the_mq.pdf?raw=true
Http://s.urge.omniti.net/i/content/slides/Surge2012-ErikOnen_Kafka_Messaging-Paradigms.pdf
Http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf
Http://dirlt.com/kafka.html
Http://dirlt.com/index.html
Http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes

2. Nsq–golang

No center design, automatic node registration and discovery. Can be considered as the basis for an internal communications framework.

Https://github.com/bitly/nsq
* Pursuit of simple deployment
* The pursuit of high availability, avoid single point of failure, no center design
* Ensure message delivery
* Producer consumers automatically discover, consumers connect to all producers, push to consumer mode
* HTTP Interface available

Https://speakerdeck.com/snakes/nsq-nyc-golang-meetup
https://github.com/davegardnerisme/nsqphp

http://www.davegardner.me.uk/blog/tag/nsq/

3. BEANSTALKD
    1. Supports persistent binlog design, no loss of restart message
    2. So so
    3. No high-availability design
    4. The same distributed extension approach as memcached
    5. Various class libraries
    6. Have Web management tools
    7. Supports synchronous calls and waits for return
    8. Only similar to Memcache TCP ASCII protocol, single file deployment
    9. Support Message priority
    10. 9K jobs/s into queue 5K jobs/s out queue
    11. Single point of failure
    12. No master-slave synchronous replication mechanism
    13. Best standalone Multi-instance deployment

Https://github.com/kr/beanstalkd/wiki/Tools
Https://github.com/pda/pheanstalk

4. Redis

Need to package pub/sub yourself

    • Redis-based replication high availability
Other common open source messaging systems: ZeroMQ: Lightweight Base Message Library

Suitable for scenarios that do not need to be persisted and that require their own encapsulation

    1. does not support persistence, only provides message distribution, the best performance
    2. No Broker design, no center fault
RabbitMQ
    • 2500 job/s into queue 1300 job/s out queue
    • Suitable for small messages
    • Distributed no single point design
    • The bottom layer is the Erlang implementation
      There are comments: RabbitMQ could not enqueue/dequeue fast enough.

Https://blogs.vmware.com/vfabric/2013/04/how-fast-is-a-rabbit-basic-rabbitmq-performance-benchmarks.html

Restmq

http://restmq.com/

Memcacheq

http://memcachedb.org/memcacheq/

Httpsqs

https://code.google.com/p/httpsqs/

Gearman

Http://gearman.org/presentations
https://code.google.com/p/shard-query/

Kestrel

http://robey.github.io/kestrel/
Http://robey.github.io/kestrel/docs/guide.html

HornetQ

Poor performance not considered [3]

Resque

3800 jobs/s into queue jobs/s out queue
Https://github.com/blog/542-introducing-resque
Redis-based Message Queuing

Starling

Https://github.com/starling/starling

Squirrelmq

https://code.google.com/p/squirrel-message-queue/

Sparrow–ruby

https://code.google.com/p/sparrow/

Apache ActiveMQ

ActiveMQ crashed constantly under load.

STOMP HTTP Protocol

Http://stomp.github.io/stomp-specification-1.2.html

Reference:

Http://hiramchirino.com/stomp-benchmark/ec2-c1.xlarge/index.html
https://blog.serverdensity.com/queueing-mongodb-using-mongodb/
[3] http://x-aeon.com/wp/2013/04/10/a-quick-message-queue-benchmark-activemq-rabbitmq-hornetq-qpid-apollo/
https://news.ycombinator.com/item?id=5531192
http://www.javaplex.com/blog/high-performance-message-queues-get-reviewed/
Http://adam.heroku.com/past/2010/4/24/beanstalkasimpleand fastqueueingbackend/

Related articles

Common open Source messaging systems

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.