Talk about Pipeline-filter mode again

This paper discusses the responsibility chain model at the design level and the Pipeline-filter model at the architectural level based on the detours that I have recently implemented on the RABBITMQ message bus. Another purpose of writing this article is to correct my previous blog post on the Pipeline-filter model:"The tail loop of the Pipeline-filter model variant", if you want to see why I did that before, you can read the article first, But whether you look at it or not, it doesn't affect the wording of this article.

Message Bus needs extensibility

At present, this message bus realizes the Produce/consume, Request/response, Publish/subscribe, broadcast and other kinds of message communication scenarios. These scenarios involve the processing of messages.

I want to implement a plugin-based message processing. They need to be granular and can be reused between various message communication patterns, easily extensible, and can automatically concatenate these different plugin into a single pipeline based on the configuration file. This pattern is called a chain of responsibility or Pipeline-filter mode (if you are careful, you might classify pipeline-filter as an architectural pattern, see Posa Volume 4), if you've done Java Web development, It's always easy to relate it to filter. Yes, this model of filter is in fact the chain of responsibility mode.

The mainstream understanding of the two models

In fact, we usually talk about the chain of responsibility and Pipeline-filter, most of the attention is focused on "into" the Consciousness: the data (usually encapsulated in a context object) on the call chain is processed sequentially by each filter, advancing forward. But the implementation of filter in Java Web technology focuses on "in and Out":

This is the result of a combination of factors:

1. HTTP has a request and responds: Data processing is not unidirectional, it's a closed loop.
   
2. Its context is actually two objects: HttpServletRequest, HttpServletResponse, in the time to focus on the HttpServletRequest object, back when the focus on HttpServletResponse objects, Clear division of labor, non-interference
   
3. The practice of implementing this filter-chain is usually recursive, whereas recursive invocation involves the process of loading and unwinding the stack in the execution of the method. The critical point is the method's internal invocation of the recursive method (see chain.dofilter above. The code before the call can be considered as a stack, which is called first, and the code after the call can be seen as a stack, which is called when all the stacks are completed and then sequentially out of the stack. The service of the last servlet in the diagram is considered to be the recursive break point, and after execution, it will begin to retire (process HttpServletResponse) process
   

Here's a simple filter example: it first determines if the HTTP request header contains information that supports GZIP compression: if it is not included, go directly to the next step, and if it is included, extract the response object and gzip compress the output stream. There is no doubt that extracting the response object should be done after the servlet is executed, but the filter begins to execute before the servlet, and it relies on the execution mechanism of the recursive call to the stack. It also has processing for httpservletrequest and HttpServletResponse objects:

public void DoFilter (ServletRequest servletrequest, Servletresponse servletresponse, Filterchain Filterchain) throws IOException, servletexception {        logger.info ("[DoFilter] enter into Compressionfilter");        HttpServletRequest req = (httpservletrequest) servletrequest;        HttpServletResponse resp = (httpservletresponse) servletresponse;        String encoding = Strings.nulltoempty (Req.getheader ("accept-encoding"));        if (Encoding.contains ("gzip")) {            Compressionresponsewrapper warppedresp = new Compressionresponsewrapper (resp);            Warppedresp.setheader ("content-encoding", "gzip");            Filterchain.dofilter (req, warppedresp);            Gzipoutputstream Gzos = Warppedresp.getgzipoutputstream ();            Gzos.finish ();        } else {            Filterchain.dofilter (req, resp);        }    }

By the way, just focus on moving forward, just put the recursive call in the last sentence (that is, there is no code logic behind the recursive call).
Problems encountered in message bus the first problem: As mentioned in the previous article, when receiving messages, due to the RABBITMQ official Java client, provides a blocking waiting push consumption API, this mode is not friendly to client users, Usually we need to build a separate event loop based on it (on another thread), with OnMessage's event callback as an asynchronous API implementation more friendly, so it involves the processor in chain cross-threading problem, and in the event Loops are looped on several processors at the tail of the chain (message waiting and message processing). The second problem: Web containers separate request objects and response objects for the HTTP zone, making them clearly responsible, non-intrusive, and out-of-order. In the message bus, the semantics of two context objects do not exist in the process of a single processing of messages (only called produce or consume alone), which misleads me into cutting chain into and out. AOP Pre-aspect and Post-aspect (This implementation is an improved version of the article I mentioned above). The third question: this is a big problem, I did ignore the Java implementation of the filter "redirecting Investments" design, and always think of posa that book overcame architecture level design, which resulted in one of the processors is to borrow objects from the object pool, The return logic is not after the recursive call point of the processor (but in post-aspect). The improvement scenario addresses the first question: I changed my mind, if we were to include the entire chain of the asynchronous receive message to the event loop approached (which was just a implementation of blocking wait and triggering onmessage), we solved the need for a processor in chain to cross-threading issues , and the processing logic that receives the message is no longer split and reused as a processor, but is reused in the form of inheritance and subclasses, which solves the so-called "tail loop" problem in chain.
For the second and third problem: discard the previously so-called chain cut into pre-aspect and post-aspect processing, using the Java filter in the design of the recursive call to the critical point of the distinction between "into", "retreat" logic.

Written in the last

Pipeline The term is not too accurate in the expression, some people use it to express the chain of responsibility model, some people will refer to Pipeline&filter. In the exchange of nature does not have to be too serious, but I think from this article can give a good distinction: the responsibility chain mode is the design pattern, oriented to the implementation of the program; Pipeline&filter is an architectural pattern. and different points, I think you see.

When I realized this part of the code, I was just looking at Posa volume four, which talked about the architecture pattern pipeline-filter. And I think that's the difference between design patterns and schema patterns: Design patterns are oriented toward code implementations, and in code implementations there is no focus on the deployment topology of the server running the code, and the schema pattern is a higher level of abstraction, and volume four itself is for distributed systems. So the so-called filter is actually a distributed data processing component, which is naturally a purely overcame pipeline.

In fact, this design has now gone through the third edition. The first version is the way it was described in that article, completely posa expression; second edition, I intentionally looked at the Java filter implementation, but was confused by its two context objects, think this is a specific scenario implementation (request, there is a response), And I really did not look squarely at the implementation of recursive calls to play a major role. The third edition, based solely on the design pattern of the responsibility chain.

Code See: Banyan

Talk about Pipeline-filter mode again