Reactor Introduction and Practice

This is a creation in Article, where the information may have evolved or changed.

Suitable for reading: This article is suitable for the spring, Netty and other frameworks, as well as the Java 8 Lambda, Stream and other features have a basic understanding of the spring 5 to understand the reactive programming characteristics of technical staff reading.

First, preface

In recent years, with the advent of new technologies and languages such as node. js, Golang, Java's server-side development language has been a big challenge. While Java's market share remains large and will not change in a short time, the Java community is not indifferent to the challenges. Instead, the Java community is actively addressing these challenges and constantly improving its ability to respond to high-concurrency server-side development scenarios.

In response to the high concurrency of server-side development, in 2009, Microsoft presented a more elegant way to implement asynchronous programming--reactive programming, Chinese-language called reactive programming. Other technologies quickly followed, as ES6 introduced a similar asynchronous programming approach through Promise. The Java community has not lagged much behind, and Netflix and Typesafe have provided RxJava and Akka Stream technology, allowing the Java platform to have a framework for reactive programming.

In fact, earlier, the NIO framework like Mina and Netty can actually handle high-concurrency server-side development tasks, but such techniques are relatively small tools in the hands of a handful of senior developers. For more ordinary developers, the difficulty seems to be a bit larger, so it is not easy to popularize.

Many years have passed, and by the year 2017, although there have been many companies in the practice of reactive programming. But overall, the scope of application is still small. The reason for this is the lack of easy-to-use technology to popularize reactive programming and integrate with such things as MVC frameworks, HTTP clients, and database technologies.

Finally, on September 28, 2017, the tool to solve the problem surfaced--spring 5 officially released. Spring 5 Its greatest significance is to be able to move the popularity of reactive programming technology to advance a big step. As a framework for supporting Spring 5 reactive programming behind the scenes, Reactor also released the 3.1.0 version accordingly.

This article will introduce you to reactive programming (reactive programming), the introduction of Reactor, and practical skills. The practical content of this article comes from the author's experience of using Spring 5 and Reactor to transform practical projects.

Ii. introduction of Reactor

Let's introduce Reactor technology. The Reactor framework was developed by Pivotal Corporation (a company that develops technologies such as Spring), realizing the reactive programming idea, in accordance with the reactive STREAMS specification (reactive Streams is by Netflix , Typesafe, Pivotal and other companies). The name has a reactor meaning, reflecting the powerful performance behind it.

REACTIVE programming

Reactive programming, Chinese called reactive programming, is a high-performance application programming way. It was first proposed by Microsoft and introduced into the. NET platform, followed by the introduction of similar technologies by ES6. On the Java platform, the early adoption of reactive programming technology is the RxJava framework of Netflix's open source. Now everyone is more familiar with the Hystrix is based on the development of RxJava.

Reactive programming is not mysterious, and the basic idea is understood by comparing it with the familiar iterator pattern:

Event	iterable (Pull)	Observable (push)
Retrieve data	T next()	onNext(T)
Discover error	ThrowsException	onError(Exception)
Complete	!hasNext()	onCompleted()

The Observable column in the table above represents the way the API is used for reactive programming. Visible, it is an extension of the common observer pattern. If the iterator is considered to be a pull mode, then the Observer pattern is the push mode. The Subscriber (publisher) actively pushes the data to the Subscriber (Subscriber), triggering the onNext method. The other two methods are triggered when exceptions and finishes are completed. What if Publisher publishes the message too quickly and exceeds the subscriber processing speed. This is the origin of backpressure, reactive programming framework needs to provide mechanisms that enable subscriber to control the speed of consumer messages.

On the Java platform, Netflix (developed RxJava), Typesafe (developed Scala, Akka), Pivatol (developed Spring, Reactor) jointly developed a project called Reactive STREAMS (specification), Used to develop specifications and interfaces related to reactive programming. Its main interface has these three:

• Publisher
• Subscriber
• Subcription

Among them, the Subcriber onNext onError three methods mentioned in the above table are included onCompleted .

For reactive Streams, we only need to understand their thoughts, including the basic ideas and backpressure ideas.

Imperative vs Reactive

For the two styles of iterable and Observale mentioned in the table above, there is another salutation, namely the imperative (instruction programming) and reactive (reactive programming) styles. In fact, pull the model and push the model of another expression, we understand the ideas can be. For imperative, written by foreigners sometimes used, literal translation is the instruction of programming, in fact, we all usually use Java, Python and other languages to write code common style, code execution sequence and writing order basically consistent (here does not consider the JVM command rearrangement)

Main modules of the Reactor

The Reactor framework has two main modules: Reactor-core and REACTOR-IPC. The former is mainly responsible for the implementation of reactive programming related core API, which is responsible for the realization of high performance network communication, which is based on Netty.

Main classes of Reactor

In Reactor, there are not many classes that are often used, mainly the following two:

• MonoImplements an org.reactivestreams.Publisher interface that represents a publisher of 0 to 1 elements.
• FluxThe org.reactivestreams.Publisher interface is also implemented, representing the publisher of 0 to n elements.

Classes that may be used

• SchedulerA scheduler that represents a back-driven reactive flow, typically implemented by a variety of thread pools.

Web Flux

Spring 5 introduces a high-performance Web framework based on Netty rather than servlets, but is not used in much the same way as traditional servlet-based Spring MVC.

Example of MVC interface in Web Flux

@RequestMapping("/demo")@RestControllerpublic class DemoController {    @RequestMapping(value = "/foobar")    public Mono<Foobar> foobar() {        return Mono.just(new Foobar());    }}

The biggest change is that the return value changes from the Foobar represented object to the Mono<Foobar> (or Flux<T> )

Of course, the actual program does not have a single line of code like the example. about how to develop practical applications, these are the parts of the Reactor that are described in detail later.

Reactive Streams, Reactor, and Web Flux

Some of the concepts of reactive programming, as well as Reactor and Web Flux, are described above. The reader may see some confusion here. Here is a description of the relationship between the three. It's actually very simple:

Reactive STREAMS is the norm, Reactor realizes reactive Streams. Web Flux is based on Reactor and implements the reactive programming framework in the web domain.

In fact, for most business developers, when writing reactive code, we usually only have access to Publisher this interface, which corresponds to Reactor Mono Flux . For Subscriber and Subcription These two interfaces, the Reactor must also have the corresponding realization. However, these are used by frameworks such as Web Flux and Spring Data reactive. If middleware is not developed, it is usually not accessible to developers.

For example, in Web Flux, your method simply returns Mono or Flux can. Your code basically only Mono deals with or Flux . Web Flux is implemented when the Subscriber onNext business developer writes Mono or transforms the Flux HTTP Response back to the client.

Iii. introduction of Reactor

Next, we introduce the use of the main methods in the two classes of Mono and Flux in Reactor.

Like the Stream introduced in Java 8, the way Reactor is used is basically three steps: the creation of the start phase, the processing of the intermediate stage, and the final phase of consumption. Just creating and consuming may be done through a framework like Spring 5 (such as invoking the HTTP interface via the WebClient in Web Flux, the return value is a Mono). But we still need a basic understanding of how these phases are developed.

1. Create Mono and Flux (start phase)

The beginning of programming with Reactor must first create Mono or Flux. There are times when we do not need to create ourselves, but to implement such as WebClient in Webflux or Spring Data reactive get a Mono or Flux.

Invoking the HTTP interface using Webflux WebClient

WebClient webClient = WebClient.create("http://localhost:8080");public Mono<User> findById(Long userId) {    return webClient            .get()            .uri("/users/" + userId)            .accept(MediaType.APPLICATION_JSON)            .exchange()            .flatMap(cr -> cr.bodyToMono(User.class));}

Querying the user with Reactivemongorepository

public interface UserRepository extends ReactiveMongoRepository<User, Long> {    Mono<User> findByUsername(String username);}

But sometimes, we also need to proactively create a Mono or Flux.

How "normal" is created

The simple way to create it is primarily just to use methods like this to create

Mono<String> helloWorld = Mono.just("Hello World");Flux<String> fewWords = Flux.just("Hello", "World");Flux<String> manyWords = Flux.fromIterable(words);

When does this create the way to use it? It is generally used when you get an object after a series of non-IO-type operations. The next step is to use Reactor for high-performance IO operations based on this object, which can be used to convert the previously obtained objects into Mono or Flux.

The way to create "art"

The above is the result of a synchronous call we create Mono or Flux , but sometimes we need to create Mono or Flux from the result of a non-reactive asynchronous call. How does that work?

If this async method returns one CompletableFuture , then we can CompletableFuture create a Mono based on this:

Mono.fromFuture(aCompletableFuture);

If this asynchronous call does not return CompletableFuture and has its own callback method, how can it be created Mono ? We can use the static <T> Mono<T> create(Consumer<MonoSink<T>> callback) method:

Mono.create(sink -> {    ListenableFuture<ResponseEntity<String>> entity = asyncRestTemplate.getForEntity(url, String.class);    entity.addCallback(new ListenableFutureCallback<ResponseEntity<String>>() {        @Override        public void onFailure(Throwable ex) {            sink.error(ex);        }        @Override        public void onSuccess(ResponseEntity<String> result) {            sink.success(result.getBody());        }    });});

After using Webflux, Asyncresttemplate is deprecated and is just a demo.

2. Processing Mono and Flux (intermediate stage)

The methods of Mono and Flux in intermediate stage mainly include,,,,, and filter map flatMap then zip reduce so on. These methods are used in the same way as the methods in Stream. The introduction of these methods will be placed in the next section, "Reactor", the main introduction of these methods are not easy to understand and use problems prone points.

3. Consumption of Mono and Flux (end stage)

The way to directly consume Mono or Flux is to invoke the subscribe method. If developed in the Web Flux interface, return Mono or flux directly. The Web Flux Framework will do the final Response output for us.

Iv. Reactor Advanced

Next I'll look at some of the slightly more complex issues that I've encountered when using Reactor to develop actual projects, as well as workarounds.

Question one: map , flatMap and then when are they used?

This paragraph covers the following classes and methods:

• Method:Mono.map
• Method:Mono.flatMap
• Method:Mono.then
• Class:Function

In the process of dealing with Mono Flux intermediate links, there are three similar methods: map , flatMap and then . These three methods can be said to be a very high frequency method in Reactor.

Traditional command-type programming

Object result1 = doStep1(params);Object result2 = doStep2(result1);Object result3 = doStep3(result2);

The corresponding reactive programming

Mono.just(params)    .flatMap(v -> doStep1(v))    .flatMap(v -> doStep2(v))    .flatMap(v -> doStep3(v));

It is easy to see how the method plays a role in the comparison between the two sections of the code above flatMap , map and the then method has a similar effect. But what is the difference between these methods? Let's take a look at the signatures of these three methods ( Mono for example):

• flatMap(Function<? super T, ? extends Mono<? extends R>> transformer)
• map(Function<? super T, ? extends R> mapper)
• then(Mono<V> other)

Visible, the most complex is the flatMap method, second map , then the simplest. From the name of the method, flatMap and map all are used for mapping. And then then the next step, which is best for chained calls, but why the above example uses flatMap instead then ?

thenThe surface looks like the next step, but it only represents the next step in the order of execution, and does not indicate that next depends on the previous one. This semantics then is different from the method in ES6 Promise. The then parameters from the method are just one Mono and cannot accept the result of the previous step. flatMapand map The parameters are all one Function . The entry parameter is the result of the previous step execution.

flatMapand map The difference is that the return value of the entry in the flatMap Function request is a Mono (do not understand the definition of the Function interface), and map the incoming parameter Function only requires the return of a normal object. Because we often need calls or methods in business WebClient processing ReactiveXxxRepository , the return values of these methods are Mono (or Flux ). So to concatenate these calls into a whole chain call, you have to use flatMap , not map .

Therefore, we should understand correctly flatMap , map and the then use of these three methods and the meaning behind, so as to correctly practice reactive programming.

Question two: How to implement concurrent execution

This paragraph covers the following classes and methods:

• Method:Mono.zip
• Class:Tuple2
• Class:BiFunction

Concurrent execution is a common requirement. Reactive programming is an asynchronous programming method, but asynchronous does not mean concurrency parallel.

In the traditional way of imperative development, concurrent execution is realized by the way of line Cheng future.

Future<Result1> result1Future = doStep1(params);Future<Result2> result2Future = doStep2(params);Result1 result1 = result1Future.get();Result2 result2 = result2Future.get();// Do merge;return mergeResult;

Because the above code has some asynchronous effects inside, the Future.get() method is blocked. So, when we use Reactor to develop reactive code with concurrent execution scenarios, it's definitely not going to work that way. At this point, you need to use the Mono Flux methods in and zip . Here we take Mono the example to demonstrate. The code is as follows:

Mono<CustomType1> item1Mono = ...;Mono<CustomType2> item2Mono = ...;Mono.zip(items -> {    CustomType1 item1 = CustomType1.class.cast(items[0]);    CustomType2 item2 = CustomType2.class.cast(items[1]);    // Do merge    return mergeResult;}, item1Mono, item2Mono);

In the code above, the resulting item1Mono and item2Mono the process is parallel. For example, a database query operation is performed while invoking an HTTP interface. This can speed up the execution of the program.

However, there is a problem with the above code, that is, the zip method requires a forced type conversion. Coercion of type conversions is not secure. So we need a more elegant way.

Fortunately, zip there are many overloaded forms of methods. In addition to the most basic forms, there are several types of safe forms:

static <T1, T2> Mono<Tuple2<T1, T2>> zip(Mono<? extends T1> p1, Mono<? extends T2> p2);static <T1, T2, O> Mono<O> zip(Mono<? extends T1> p1, Mono<? extends T2> p2, BiFunction<? super T1, ? super T2, ? extends O> combinator); static <T1, T2, T3> Mono<Tuple3<T1, T2, T3>> zip(Mono<? extends T1> p1, Mono<? extends T2> p2, Mono<? extends T3> p3);

For merge operations with no more than 7 elements, there is a type-safe zip method to choose from.

Take the two-element merger as an example, and describe how to use it:

Mono.zip(item1Mono, item2Mono).map(tuple -> {    CustomType1 item1 = tuple.getT1();    CustomType2 item2 = tuple.getT2();    // Do merge    return mergeResult;});

In the above code, the map parameters of the method are one Tuple2 , representing a two-tuple array, corresponding to Tuple3 , and so on Tuple4 .

In addition, for concurrent execution of two elements, you can also zip(Mono<? extends T1> p1, Mono<? extends T2> p2, BiFunction<? super T1, ? super T2, ? extends O> combinator) merge the results directly from the method. Method is a pass-through implementation of the BiFunction merge algorithm.

Issue three: Aggregation after a collection cycle

This paragraph covers the following classes and methods:

• Method:Flux.fromIterable
• Method:Flux.reduce
• Class:BiFunction

Another slightly more complex scenario is to manipulate a type of object into a collection class (List, set), and then manipulate the original object. Using imperative style code is easy to write:

List<SubData> subDataList = data.getSubDataList();for (SubData item : subDataList) {    // Do something on data and item}// Do something on data

is not simply to the point of the extreme. But when we want to use reactive-style code to implement the above logic, it's not that simple.

To implement the above logic in reactive-style code, we mainly use Flux the reduce method. Let's first look reduce at the signature of the method:

<A> Mono<A> reduce(A initial, BiFunction<A, ? super T, A> accumulator);

From the method signature we can see that reduce the function of the method is to speak a Flux poly synthesis one Mono . The first parameter in the parameter is the Mono initial value of the element in the return value.

The second parameter is the BiFunction logic used to implement the aggregation operation. In a generic parameter <A, ? super T, A> , the first A type that represents the result of each aggregation operation (because it needs to operate on each element in the collection) is the BiFunction.apply first entry of the method, ? super T representing each element in the collection, which acts as BiFunction.apply the second entry for a method; c11/> represents the result of the aggregation operation, which acts as BiFunction.apply the return value of the method.

Next look at the example:

Data initData = ...;List<SubData> aList = ...;Flux.fromIterable(aList)    .reduce(initData, (data, itemInList) -> {        // Do something on data and itemInList        return data;    });

In the example code above, initData and data the same type as us, but the name cannot be duplicated. After executing the above code, the reduce method returns Mono<Data> .

V. Summary

This paper introduces some concepts of reactive programming and the basic usage of Spring's Reactor framework, and describes how to solve some slightly more complicated problems with Reactor. The examples in this article come from the practical process of using Reactor to transform real projects, because energy and time are limited, and the examples above have many limitations. But hopefully this article will play a role, so that we can see more about reactive programming and Reactor use of practical sharing.

For simplicity and clarity, the above example calls directly to the related methods in Reactor. But here is a suggestion, is to practice reactive programming, more need to continue to use methods such as extraction method for refactoring to improve code readability, or project code will only be less readable than the traditional style of code.

In addition to the problem of code readability, there are many more issues to consider in the practice of reactive programming. For example, the introduction of the NIO framework such as Netty brings the technical complexity, the difficulty of unit testing, and the integration of other framework technologies, and so on. So, for new technologies like reactive programming, we have to explore on the one hand and evaluate the technical risks it brings and the value it brings to match your project.

Of course, as a whole, the prospect of reactive programming is very bright because it leads to higher throughput. It can be said that the future high-performance Java WEB applications are basically the world of reactive programming technology, but also the Java platform against Golang, node. JS and other new technologies.

This article is over!

========== Recruitment Information ==========

At present, Iqiyi Art member technical team needs an architect, a number of senior and senior back-end development engineers, welcome to know the technology, the ideal of small partners to the resume smashed. E-mail: yanglifan@qiyi.com