Look at the IT architecture in the Big Data era (1) Industry Message Queuing comparison

One, MQ (Message Queue)

That is, Message Queuing, which is commonly used for application system decoupling, message asynchronous distribution, can improve the system throughput. MQ has a lot of products, there are open source, there are closed sources, such as ZEROMQ, RabbitMQ, ActiveMQ, Kafka/jafka, Kestrel, BEANSTALKD, HornetQ, Apache qpid, Sparrow, Starling, Amazon SQS, MSMQ, and even Redis can also be used to construct message queues. As for how to choose, depends on your needs. Due to job needs and interests, once wrote a series of blog posts about RABBITMQ, the RABBITMQ protocol, installation, configuration, management, monitoring, persistence, distribution, high availability, zoning, clustering, load balancing and so on in detail. This series of blog posts basically meet the general needs of the work.

Currently there are many MQ products in the industry, we make the following comparisons:

Second,RabbitMQ

is an open source message queue written using Erlang, which itself supports a lot of protocols: Amqp,xmpp, SMTP, STOMP, and so on, making it very heavyweight and more suitable for enterprise-class development. A broker framework is also implemented, which means that messages are queued in the central queue before they are sent to the client. Good support for routing (Routing), load balancing (payload balance), or data persistence.

For details, see:

• Janet: Looking at the IT architecture in the Big Data Era (2) Message Queuing rabbitmq-Basic Concept detailed introduction
• Janet: Looking at the IT architecture in the Big Data Era (3) rabbitmq-installation, configuration and monitoring of Message Queuing

third,Redis

is a key-value NoSQL database, development and maintenance is very active, although it is a Key-value database storage system, but it natively supports MQ function, so it can be used as a lightweight queue service. For RABBITMQ and Redis on-board and out-of-team operations, each execution 1 million times, every 100,000 times the execution time is recorded. The test data is divided into 128Bytes, 512Bytes, 1K and 10K four different sizes of data. The experiment shows that when the data is compared with the RABBITMQ, the performance of the Redis is higher than that of the 10k,redis when the data is larger than the size of the queue, and the Redis shows very good performance regardless of the data size.   And RABBITMQ's performance is much lower than Redis. Queue up 128B 512B 1K 10K 128B 512B 1K 10K Redis

Iv. ZeroMQ

Known as the fastest Message Queuing system, especially for large-throughput demand scenarios. ZMQ can implement advanced/complex queues that rabbitmq are not good at, but developers need to assemble multiple technical frameworks themselves, and technical complexity is a challenge to the success of this MQ application. ZEROMQ has a unique non-middleware model that you do not need to install and run a messaging server or middleware because your application will play the role of this service. You just need to simply reference the ZEROMQ library, you can use NuGet to install it, and then you can happily send messages between applications. However, ZEROMQ only provides a non-persistent queue, which means that if the down machine, the data will be lost. Among them, the storm of Twitter uses zeromq as the transmission of data streams.

Wu, ActiveMQ

is a sub-project under Apache. Similar to ZEROMQ, it is able to implement queues with agent and peer-to-peer technology. It is also similar to RABBITMQ, where a small amount of code makes it possible to implement high-level scenarios efficiently. RabbitMQ, ZeroMQ, and activemq all support commonly used multi-lingual client C + +, Java,. Net, Python, PHP, Ruby, etc.

Liu, Jafka/kafka

Kafka is a sub-project under Apache and is a high-performance cross-language distributed Publish/subscribe Message Queuing system, and Jafka is hatched on Kafka, an upgraded version of Kafka. Has the following characteristics: Fast persistence, can be in the O (1) of the system overhead for message persistence, high throughput, on a common server can achieve 10W/S throughput rate, complete distributed system, Broker, Producer, consumer are native automatically support distributed, Automatic implementation of complex equalization, support Hadoop data parallel loading, for the same as Hadoop, the same log data and offline analysis system, but also require real-time processing constraints, this is a feasible solution. Kafka the parallel loading mechanism of Hadoop to unify the online and offline message processing, which is the focus of the research system of this topic. Apache Kafka is a very lightweight messaging system in contrast to ACTIVEMQ, and is a well-functioning distributed system in addition to its very good performance.
Some other queue lists Hornetq, Apache qpid, Sparrow, Starling, Kestrel, BEANSTALKD, and Amazon Sqs are no longer analyzed.
Through the above specific comparison analysis, tonglink/q and IBM's functional difference is not small, the user commonly used functions have been fully supported. Although the functionality is slightly less than MQ, but "SOAP message support" and "database Transaction" the two features of little actual market demand and application, tonglink/q also did not spend more effort on the implementation of these two functions, and has been relentlessly digging the market demand, To provide users with more practical features, and constantly improve product performance and ease of use.
TONGLINK/Q provides the unique features of the user is generally recognized, such as file transfer function is the actual needs of many enterprises, and tonglink/q has only seized the domestic enterprise application characteristics, has received good market feedback.
TONGLINK/Q's product reliability and stability has also experienced the market more than 10 years of testing, showing impressive results. In finance, telecommunications, transportation, electricity and other fields, tonglink/q has a large number of application cases. For example, the People's Bank of China in the national Cheque image Exchange System in full use of Dongfang Tong technology tonglink/q Products, the realization of the cheque in the nationwide interoperability, the system is running stably, the national cheque use gradually increased. This fully proves the adaptability of tonglink/q for large-scale applications.
1:ACTIVEMQ as few as possible to establish a connection, multiple sends can share a connection. If a connection is established each time it is not comparable to MSMQ, that is, MSMQ performance is much higher than ACTIVEMQ. However, if the connection request is reduced then it has a higher performance than MSMQ.

2:ACTIVEMQ message capacity is much higher than MSMQ

3:ACTIVEMQ is the most efficient when sending and receiving synchronization

4:ACTIVEMQ performance in high capacity (millions) 5:activemq more flexible handling and expansion

Each MQ product is described in detail in the following sections

Look at the IT architecture in the Big Data era (1) Industry Message Queuing comparison