Common open-source Message Systems-website announcements, notifications, and messages

The role of the messaging system: asynchronous processing, peak reduction, and reduced coupling between components. Choose the message system based on business needs to consider the following aspects: whether persistent throughput capacity high availability distributed scalability compatible with existing protocols easy to maintain others, for example, message loss and repeated processing to avoid single point of failure (SPOF) server load balancer common message system protocols: STOMP

The role of the messaging system: asynchronous processing, peak reduction, and reduced coupling between components.

Choose a message system based on business needs to consider the following aspects:

1. Persistent or not
2. Throughput capacity
3. High Availability
4. Distributed scalability
5. Compatible with existing protocols
6. Easy to maintain
7. Others, such as message loss and repeated processing
8. Avoid SPOF
9. Server load balancer

Common message system protocols:

1. STOMP
2. AMQP
3. Protocols similar to MEMCACHE
4. HTTP
5. Custom Format

The following 1 and 2 are good open source components: 1. Kafka/MetaQ: widely used inside Linkedin (similar to domestic MetaQ with Java version)

Throughput is a top priority for collecting and processing large data volumes of messages, such as log analysis, real-time user behavior reports, and cluster status information collection and analysis.

1. The persistence design is preferred and memory is managed by page cache.
2. High throughput: 112 MB/s 11 K msgs/s (higher than beanstalkd> 70x throughput)
3. Supports asynchronous replication
4. High availability, Zookeeper-based cluster design, support for re-load balancing after consumer failure
5. Kafka provides PHP class libraries
6. Support for ganglia JMX monitoring
7. The policy is required to avoid repeated messages. the consumer must update the offset of Zookeeper (MetaQ has provided several methods to avoid repeated messages)
8. MetaQ provides HTTP interfaces

Https://kafka.apache.org/

Http://metaq.taobao.org/

2. NSQ-Golang

Non-central design, automatic node registration, and discovery. It can be considered as the basis of the internal communication framework.

* Simple Deployment
* High Availability, single point of failure avoidance, and no center design
* Ensure message delivery
* Automatic discovery by producer and consumer, connection by consumer to all producers, push to consumer
* Provides HTTP interfaces

Https://github.com/bitly/nsq

3. Beanstalkd

1. Supports persistent binlog design, so messages are not lost after restart.
2. Average
3. No high availability design
4. The same distributed extension method as memcached
5. Various class libraries
6. Web management tools
7. Support synchronous call and wait for return
8. Only Memcache-like tcp ascii protocol for single-file deployment
9. Message priority supported
10. 9 K jobs/s incoming queue 5 K jobs/s outgoing queue
11. Single point of failure
12. Master-slave synchronous replication mechanism
13. It is best to deploy multiple instances on a single machine

Http://kr.github.io/beanstalkd/

4. Redis

You need to encapsulate Pub/Sub

• Redis-based replication High Availability

Other common open-source Message Systems: ZeroMQ: lightweight basic message Library

Only applicable to scenarios that do not require persistence and require encapsulation by yourself

1. Persistence is not supported. only message delivery is supported, with the best performance.
2. No Broker design, no center failure

RabbitMQ

• 2500 job/s inbound queue 1300 job/s outbound queue
• Suitable for small messages
• Distributed, non-SPOF design
• Erlang implementation at the underlying layer
  Commented: RabbitMQ cocould not enqueue/dequeue fast enough.

RESTMQ

Http://restmq.com/

MemcacheQ

Http://memcachedb.org/memcacheq/

HTTPSQS

Https://code.google.com/p/httpsqs/

Gearman

Http://gearman.org/presentations
Https://code.google.com/p/shard-query/

Kestrel

Http://robey.github.io/kestrel/
Http://robey.github.io/kestrel/docs/guide.html

HornetQ

Poor performance is not considered

Resque

3800 jobs/s inbound queue 300 jobs/s outbound queue
Https://github.com/blog/542-introducing-resque
Redis-based message queue

Starling

Https://github.com/starling/starling

SquirrelMQ

Https://code.google.com/p/squirrel-message-queue/

Sparrow-Ruby

Https://code.google.com/p/sparrow/

Apache ActiveMQ

ActiveMQ crashed constantly under load.

Stomp http protocol

Http://stomp.github.io/stomp-specification-1.2.html