"System Architecture" reads "Large Web site Architecture series: Distributed Message Queuing" collation

Source: Internet
Author: User
Tags system log rabbitmq

article AddressExpand reading:
RabbitMQ + PHP (i) Getting started with installing RabbitMQ + PHP (ii) AMQP extension installation RabbitMQ + PHP (iii) Case Demo1. Some words and techniques 1) Zookeeper Register Center Storm Cluster2. Core Ideas 1) "Overview"Message Queue Middleware is an important component in distributed system, which mainly solves the problems of application coupling, asynchronous message and traffic cutting front. Achieve high performance, high availability, scalability and eventual consistency architecture. is an indispensable middleware for large-scale distributed systems. Currently in production environment, using more message queue has ACTIVEMQ,RABBITMQ,ZEROMQ,KAFKA,METAMQ,ROCKETMQ etc 2) "Application Scenario" asynchronous processing, application decoupling, traffic cutting front and message communication four scenes. 3) "Message Middleware Sample" E-commerce System log Collection system 4) "Log System" ELK Static (Sina Kafka log processing application case)--Kafka: Receive Message Queuing for user logs. --Logstash: Do log parsing, unified into JSON output to elasticsearch. --Elasticsearch: The core technology of real-time log Analysis service, a schemaless, real-time data storage service, organization of data through index, both powerful search and statistical functions. --Kibana: Data visualization component based on Elasticsearch, strong data visualization capability is an important reason for many companies to choose Elk Stack. 5) "Zookeeper"ZookeeperDistributed Service Framework is a sub-project of Apache Hadoop, which is mainly used to solve some data management problems commonly encountered in distributed applications, such as: Unified Naming Service, State Synchronization service, cluster management, management of distributed application configuration items, etc.6) "Storm"is aimplement a real-time computing system
7) "Two message modes"

-Point-to-point (peer mode):

A) there is only one consumer (Consumer) per message (that is, once consumed, the message is no longer in the message queue)
b) There is no dependency on time between the sender and the receiver, that is, when the sender sends a message, regardless of whether the receiver is running, it does not affect the message being sent to the queue
c) The recipient must answer the queue successfully after receiving the message successfully

--Publish Subscription (Pub/sub mode)

A) Each message can have multiple consumers
b) There is a time dependency between the Publisher and the Subscriber. Subscribers to a topic (TOPIC) must create a subscriber before they can consume the publisher's message.
c) In order to consume messages, subscribers must remain in a running state.
3. Common Message Queuing middleware 1) "ActiveMQ "      ActiveMQ is Apache's most popular and powerful open source messaging bus that supports Java .2) "RabbitMQ"-RABBITMQ is a popular open source Message Queuing system developed in Erlang language. RABBITMQ is the standard implementation of the AMQP (Advanced Message Queuing protocol). Support for multiple clients such as Python, Ruby,. NET, Java, JMS, C, PHP, ActionScript, XMPP, stomp, etc., support Ajax, persistent. Used to store forwarded messages in distributed systems, with ease of use, scalability, high availability, and so on-installation of RABBITMQ3) "ZeroMQ"Known as the fastest message queue, it is actually similar to the socket of a series of interfaces, the difference between him and the socket is: the ordinary socket is end-to-end (1:1 of the relationship), and ZMQ is can be n:m relationship, people on the BSD socket understanding more is the point-to-point connection, A point-to-point connection requires an explicit connection, a destroy connection, a selection protocol (TCP/UDP), and a processing error, and ZMQ masks these details to make your network programming easier. ZMQ is used for the communication between node and node, which can be either a host or a process. 4) "Kafka"Kafka is a high-throughput distributed publish-subscribe messaging system that handles all of the action flow data in a consumer-scale website. This kind of action (web browsing, search and other user actions) is a key factor in many social functions on modern networks. This data is usually resolved by processing logs and log aggregations due to throughput requirements. This is a viable solution for the same log data and offline analysis system as Hadoop, but requires real-time processing constraints.  The purpose of Kafka is to unify online and offline message processing through Hadoop's parallel loading mechanism, and also to provide real-time consumption through the cluster machine. Reference Document: RabbitMQ + PHP (i) Getting started with installing RabbitMQ + PHP (ii) AMQP extension installation RabbitMQ + PHP (iii) Case Demo

"System Architecture" reads "Large Web site Architecture series: Distributed Message Queuing" collation

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.