Coroutine in Java-quasar fiber implementation--reprint

Transfer from 1190000006079389?from=groupmessage&isappinstalled=0 Introduction

Speaking of coroutine, Many people will think of Go,lua,erlang and other languages, in fact, there are quite a number of JVM implementations, such as picothread,kilim,quasar, This article mainly introduces one of the Coroutine Implementation--Quasar Fiber,quasar Fiber is relatively more popular, if you have not previously contacted the Association (user-level lightweight thread), You can see what is fibers, coroutine

So why use a co-process?
The process can be programmed to achieve or close to pure asynchronous performance, without the asynchronous callback hell, although there are many mechanisms or patterns to solve or decouple callback Hell problems, but synchronous programming is easier to maintain and understand (style of contention is another topic, Interested to see the comparison of Akka and Fiber)
Compared to OS thread,fiber, whether in memory resources or scheduling is more lightweight than the former, relative to the thread blocking, fiber blocking can reach a few orders of magnitude more concurrency, more efficient use of CPU resources ( The worker thread running fiber does not have a block)
specifically, we can look at why and when use Fiber

It's like a magical thing, I realized it.
Compared to the callback interface callback of the asynchronous framework, coroutine this pause and restore without the support of the jvm, it is more difficult to understand, how to do it? Do you have any magic? In fact, there are many implementations of coroutine in the JVM (implementing-coroutines-in-java), Quasar fiber is a bytecode modification technique that weaves the necessary context Save/restore code when compiling or loading, pausing by throwing exceptions , restore the Jvm's method call stack and local variables according to the saved context (continuation), Quasar fiber provide the appropriate Java class library to implement, the application has a certain degree of intrusion (very Small)

Quasar Fiber mainly has instrument + continuation + Scheduler several parts composed

• Instrument Do some coding, such as save/restore of context before and after park

• Continuation Saves the information of the method invocation, such as local variables, references, and so on, the User-state stack, which is the biggest difference between the asynchronous framework and akka-based fixed callback interface.

• Scheduler scheduler, responsible for assigning fiber to specific OS thread execution

The following details the implementation of the next quasar fiber, it is best to read the quasar official documents, not very long

Instrumentweaving

The operation of the quasar fiber requires the weaving of instructions to save and restore the call stack, Quasar provides three ways of weaving (AOT, javaagent, ClassLoader)
Quasar will perform a static call-site analysis of our code, weaving the code to save and restore the call stack where necessary.
Which methods require call site analysis? This requires an explicit mark (jdk9 not required), as follows

• Method with suspendable annotations

• Method with Suspendexecution

• The Classpath method is the class or interface specified under/meta-inf/suspendables,/meta-inf/suspendable-supers, or subclass
  Method,quasar that meet the above conditions will do call site analysis, perhaps for efficiency, Quasar did not do call site analysis of all methods

What directives in the method need to be instrument (before and after they are woven into the relevant instructions)?

• Calling methods with suspendable annotations

• Call method with Suspendexecution

• The calling method is Classpath under/meta-inf/suspendables,/meta-inf/suspendable-supers the specified class or interface, or child class
  Primarily to address issues where Third-party libraries cannot add suspendable annotations

• Methods that are called through reflection

• Dynamic method Call Methodhandle.invoke

• Java Dynamic Agent Invocationhandler.invoke

• Java 8 Lambdas Call

Note that instrument is in class loading time, not runtime, so here call site analysis such as virtual Invoke command is the compile time decision, here is more prone to make mistakes, I summed up the following two points
1. Code based on interface or base class compilation, if the implementation class is likely to be suspend, you need to add suspendable annotation or suspend exception in the interface or base class
2. If the implementation class is suspend, you need to add suspendable annotation or suspend exceptions, and of course you can declare all the implementation classes as Suspendable (if The suspend call is not found in the method, The method will not be instrument, so there is no overhead, although this overhead is very tiny)

Let's take a quick look at what code quasar instrument has woven into it.

As can be seen, quasar instrument mainly in park () before saving the relevant local variables and pc, and then fiber recovery execution time through the PC (switch case jumps program counter, Non-register Pc) Jump to park () After the code and restore the local variables, in addition to the method call before and after the Push/pop related contiuation

Instrument will also replace the Thread.park in Juc (java.util.concurrent) with Fiber.park so that park to thread becomes park to fiber, so use the Juc code, You can run on the fiber without Modification.
quasar, while weaving code, adds instrumented annotations to the classes and methods being processed to check whether instrumented,instrumented annotations contain a suspendablecallsites array at run Time. Line number to store the method body suspendable call
Contiuations/stack for more details please see Contiuations Chapter

Quasarinstrumentor

Either way of weaving, the byte stream of class is processed by creating quasarinstrumentor
Quasarinstrumentor uses ASM internally to process the byte stream of class, and Suspendableclassifier class to determine if instrument is Required.
Suspendableclassifier has two subclasses, defaultsuspendableclassifier and simplesuspendableclassifier, respectively.
DefaultSuspendableClassifierScanning the implementation of Classpath under suspendableclassifier, and calling its interface to determine if instrument is required, will also invoke Simplesuspendableclassifier
SimpleSuspendableClassifierJudging By/meta-inf/suspendables and/meta-inf/suspendable-supers

The suspendable-supers in the Quasar-core.jar package contains interfaces such as Java NIO and Juc lock/future, because these interfaces cannot change signatures, and Quasar weaving is compiled or loaded. It is not possible to know whether the implementation class is suspendable, so it is necessary to explicitly specify

Method Instrument Implementation Details

Here is the entire quasar fiber is the implementation of the principle of the most critical place, but also the most questionable place, we are interested to see the source code, about 1000 lines of ASM operation, can consolidate the knowledge of the JVM and in-depth understanding of the principle of fiber, here I do not intend to introduce too much ASM knowledge , mainly from the implementation of the logical introduction

Instrumentclass inherits the Classvisitor of ASM and weaves the Suspendable method before and after
The Instrumentmethod is created in the Instrumentclass visitend, and the specific weaving instructions are processed in Instrumentmethod.
Combining the above instrument sample Code map, Consider a few questions first

• How to find suspend call

• How to save, Restore local variables, stack frames, etc.

• Switch case jump how to weave in

• Suspend call in the try catch block, how to handle

• Under what circumstances can not be woven before and after suspend call normal operation

1.怎么找到suspend call
Instrumentmethod.callssuspendables This method will traverse the instructions of the method,
If instruction is method invoke, then the suspend call is judged (see the above section for Logic)
If instruction is suspend call, the instrunction and source line numbers are recorded in the two arrays, Suspcallsbcis and suspcallssourcelines, respectively. For later logic use

2.switch case跳转织入是如何实现的
Now that we know how to find suspend call in method, How to split these suspend calls into the instrument example diagram (switch case,pc ...)
This splitting process is instrumentmethod.collectcodeblocks
Assign the label array according to the suspend call array computed above, and then jump to the label based on the PC counter (see further Chapters)
The label is used in the JVM for jump commands such as (goto,ifeq,tableswitch, etc.)
The quasar will store the context of the weave in the recovery instructions and the original instructions generated by the code to the corresponding label

3.怎么保存、恢复局部变量,栈帧

- 在方法开始执行1.调用Stack.nextMethodEntry，开启新的method frame- 在方法结束执行1.Stack.popMethod, 进行出栈- 在调用Suspendable方法之前,增加以下逻辑1.调用Stack.pushMethod 保存栈帧信息2.依次调用Stack.put保存操作数栈数据3.依次调用Stack.put保存局部变量- 在Suspendable方法调用后1.依次调用Stack.get恢复局部变量2.依次调用Stack.get恢复操作数栈  恢复局部变量和操作数栈的区别是前者在get后调用istore

Because Stack.put has 3 parameters, each put here is actually multiple JVM directives

aload_x //如果是保存操作数栈，这条指令不需要，因为值已经在操作数栈了aload_x //load Stack引用iconst_x //load Stack idxinvokestatic  co/paralleluniverse/fibers/Stack:push (Ljava/lang/Object;Lco/paralleluniverse/fibers/Stack;I)V

/**   

Java compile time to know the local variable table and the number of operands, the above put or get dependent on the information, stack specific logic see the following chapters

4.什么情况下在suspend call前后可以不织入也能正常运行
This is actually an optimization, that is, if there is only one suspend call inside the method, and there is no following instruction

• Side effects, including method invocations, property settings

• Jump forward

• Monitor Enter/exit

then, Quasar does not instrument it, and it does not require collectcodeblocks analysis, because it does not need to save, restore local variables

5.suspend call在try catch块中，如何处理
If suspend call is in a large try catch, and we need to split it in the middle with switch case, it seems like a tricky question,
So before you weave the code, you need to slice the try catch that contains suspend call, and suspend calls is contained in a try catch alone, adding a new try by ASM MethodNode.tryCatchBlocks.add Catch block,
Quasar first gets Methodnode trycatchblocks to traverse, if suspend Call's instruction sequence is within the try catch block, then it needs to be sliced to weave the code

Fiber

The following describes the classes and interfaces provided to the user in the following quasar fiber
Strand is an abstraction of thread and fiber unity in quasar, fiber is a user-level thread implementation of strand, thread is the implementation of strand Kernel-level thread

Fiber mainly has several functions

New

@SuppressWarnings("LeakingThisInConstructor")public Fiber(String name, FiberScheduler scheduler, int stackSize, SuspendableCallable<V> target)

Properties	type	Description
Name	String	Fiber name
Scheduler	Fiberscheduler	scheduler, default = Fiberforkjoinscheduler
StackSize	Int	Stack size, Default 32
Target	Suspendablecallable<v>	Specific business code, in Suspendablecallable.run ()

The constructor mainly accomplishes the following several things

• Set state to State.new

• Initialize stack (for saving fiber call stack information, continuations Implementation)

• Check if Target is instrumented

• Encapsulates the current fiber into a task that can be dispatched by scheduler, which defaults to Fiberforkjointask

• Save the Thread's inheritablethreadlocals and Contextclassloader to fiber

Start

Fiber.start () logic is relatively simple, as follows

• Switch fiber state to state.started

• Invoke Task's submit, commit to Scheduler run
  Here the default is Fiberforkjoinscheduler,fiberforkjoinscheduler will be submitted to the internal forkjoinpool, and hash to one of the work queue

Exec

The worker thread of the fiber scheduler gets the task from the work Quere and calls the Fiber.exec ()
Fiber.exec () The main steps are as follows

• Cancel Timeout Task

• The threadlocals, inheritablethreadlocals, Contextclassloader of thread are exchanged with fiber respectively, which realizes local to fiber instead of local to Thread. Special attention is needed here.

• So code based on thread local and context ClassLoader can basically run on fiber

• State = state.running;

• Run business logic (method Fiber.run ())

• State = state.terminated;

What to do when fiber paused
Fiber task switching is done in two ways, one is fiber task ends normally, and one is fiber task throw suspendexecution

Fiber.exec () will catch suspendexecution, and surrender the execution authority, the following steps

• Stack sp = 0; Fiber resume execution requires recovery from the first frame

• Set Fiber Status Timed_waiting/waiting

• threadlocals, inheritablethreadlocals, Contextclassloader of thread for recovery threads

The call stack information has been saved to the Stack (see section Instrument) before Park (), so there is no need to handle

Park

Suspend the execution of the current fiber and surrender the right of execution
Fiber task Status: parked, PARKING, RUNNABLE

Fiber Status: RUNNING-waiting

The Fiber.park method is as follows and can only be called at the current Fiber

static boolean park(Object blocker, ParkAction postParkActions, long timeout, TimeUnit unit) throws SuspendExecution

Park's main logic is as follows

• Setting the fiber state

• If timeout is set, a new scheduledfuturetask is added to Fibertimedscheduler for time-out checking

• Set Fiber.postpark = postparkactions, used to execute after an exception is caught by the Exec method above

• Throw exception, transfer Execution permissions, follow the Exec chapter handover execution permissions

Unpark

Resuming the execution of fiber
Fiber Task Status: Parked-RUNNABLE

Fiber Status: Waiting-RUNNING

Unpark mainly do two things, one is to set the state, the other is to fiber task Re-submit to Scheduler

In addition to manually calling Fiber's Park,unpark to pause and resume fiber, You can use the Fiberasync class to encapsulate callback-based asynchronous calls into fiber blocking, The Third-party library based on fiber Comsat is achieved by replacing the bio with NIO and then encapsulating it into fiberasync, Fiberasync can refer to Http://blog.paralleluniverse ....

State switching

Fiber running state consists of two parts, one is the state of the fiber itself, and the other is the state of the scheduler Task.

Fiber status

Status	Description
NEW	Strand created but not started
STARTED	Strand started but not yet running
RUNNING	Strand is running
Waiting	Strand is blocked
Timed_waiting	Strand is blocked with a timeout
TERMINATED	Strand has terminated

Task state, with the default Fiberforkjointask as an example

Status	Description
RUNNABLE	Can run
Leased	Unpark state is runnable, set to leased
Parked	Stop it
PARKING	Stop in

Run State switch diagram

Continuation

Fiber/coroutine = continuation + Scheduler can be seen, continuation in Fiber is essential, he saved the Fiber when the implementation of the necessary data, such as Pc,sp

The implementation of continuation in Quasar is the Stack class

Stack

The Stack class is an implementation class of quasar to Fiber continuation, which is called by Quasar Instrument to save and restore method call stack information

Properties	type	Description
Sp	Int	Frame ordinal representing the current action
Datalong	long[]	Holds primitives on stack as well as each method ' s entry
DataObject	object[]	Holds refs on stack, preventing JVM GC reclamation methods local objects

Each long in the Datalong represents a method frame, which is defined as follows

• entry (PC) : bits, program counter, for Swich case jump

• num Slots : bits, How many slots the current method frame occupies

• Prev Method Slots : Three bits, The last method frame occupies the number of slots, mainly used for pop jump

I've simply drawn a stack example where Pc,slots,prev slots is separated by commas and xxxxxx represents extra data for method frame
The sequence number and contents of the following IDX and data code datalong, respectively

idx	Data	Description
5	0L	Where the next frame is stored, the SP points to that node
4	xxxxxxx	Method 2 local variable C
3	xxxxxxx	Method 2 local variable B
2	7, 3, 2	Method 2,pc counter is 7, occupies 3 slots, the previous method occupies 2 slots
1	xxxxxxx	Method 1 Local Variable A
0	0,	Method 1,pc counter is 1, occupies 2 slots, the previous method occupies 0 slots

Quasar will weave stack/continuation logic into the instrument phase, as follows

• Call the Suspendable method before calling Stack.pushmethod

• At the beginning of the suspendable method, call Stack.nextmethodentry

• At the end of the suspendable method, call Stack.popmethod

Let's look at the logic of these methods in turn

Stack.pushMethod

• Save current PC

• Save current number of slots

• Set the next frame to 0L

Stack.nextMethodEntry

• Move the SP to the current frame position

• Sets the number of slots for the previous freme to the Prev slots field of the current frame

Stack.popMethod

• Perform a stack operation according to the Prev slots of the current frame

Scheduler

scheduler, as the name implies, is the execution of fiber code place, quasar with Forkjoinpool as the default scheduler thread pool,

Forkjoinpool's advantages are not emphasized here, we focus on how to use Forkjoinpool to dispatch fiber task in Quasar

Fiberforkjoinscheduler

The default fiber Task Scheduler in quasar is the Forkjoinpool class for Juc wrapper, forkjoinpool specific details Reference Forkjoinpool

//主要属性private final ForkJoinPool fjPool;//具体执行task的线程池private final FiberTimedScheduler timer;//监控fiber timeout的schedulerprivate final Set<FiberWorkerThread> activeThreads;//保存fiber worker线程

Fiberforkjointask

Wrapper the Forkjointask of Fiber.

//主要属性private final ForkJoinPool fjPool;private final Fiber<V> fiber;

Fibertimedscheduler

Quasar self-implemented Timeout Scheduler for Fiber timeout processing
Fibertimedscheduler Default Work queue is singleconsumernonblockingproducerdelayqueue, which is a multi-production single consumption of the Lock-free queue, The internal is a lock-free based on the Skip list priority chain list, interested can see the specific implementation, also worth a look
Scheduler implementation logic is relatively simple, from the Singleconsumernonblockingproducerdelayqueue internal priority queue to fetch data, If the timeout is called Fiber.unpark ()

Monitor

Fiber can be monitored through jmx, the accumulation of work queue, the number of fiber, scheduling delay, etc.

Comsat

Comsat provides libraries based on Quasar fiber, making integration with fiber easier, such as with servlets, springboot, and Drapwizard

https://github.com/puniverse/...

Comsat (or Comsat) is a set of open source libraries this integrate Quasar with various web or enterprise technologies (li KE HTTP services and database access). With comsat, you can write Web applications that is scalable and performing and, at the same time, is simple to code and Maintain.

Comsat is not a web Framework. In fact, the It does not add new APIs at all (with one exception, Web Actors, mentioned later). It provides implementation to popular (and often, standard) APIs like Servlet, jax-rs, and JDBC, so can be used Efficien tly within Quasar Fibers.

Problems encountered and solutions

I encountered a lot of problems when integrating fiber in the application, some problems also reflect the quasar fiber is not very perfect, listed here for everyone to refer to the next

Netty Poolbytebufallocator in Fiber call causes memory Leak

Due to the processing of quasar bytecode, threadlocal is called on Fiber, actually "local to Fiber" instead of "local to Thread", if you want to bypass Fiber underlying threadlocal, Need to use truethreadlocal

Netty Poolbytebufallocator$poolthreadlocalcache used threadlocal, if run on fiber, each time poolthreadlocalcache.get () will return a new Poolthreadcache object (because each request is a new fiber processing, non-webactor Mode)

In the Poolthreadcache constructor, Threaddeathwatcher.watch is called, and the object returned by the current thread and Poolthreadlocalcache.get () Add to the global Threaddeathwatcher list so that the memory pool can be freed when the related thread stops

But for fiber there will be a problem, poolthreadlocalcache.get () constantly return the new object, and then add to the threaddeathwatcher, and is really running fiber fiber-fork/ The Joinpool worker thread does not terminate, resulting in more watcher lists in threaddeathwatcher, resulting in memory leak,100% full GC time

Summary: fiber The object returned on threadlocal, escaping to the global object, and Netty only frees the memory when the real thread (os thread) terminates

Workaround: do not use the Netty object pool, or the mock Netty code to replace with truethreadlocal

[quasar] Warning:can ' t determine super class of XXX at startup

Quasar This alarm will only appear at boot time, can be ignored, Quasar temporarily no switch can swith off

Fabio:as for the first warning, this is the relevant code and it basically means Quasar ' s instrumentor couldn ' t load a cl The "superclass". This can happen because the class isn't present or, more likely, because the ClassLoader where that instrumentation code Is running doesn ' t allow to access it. Adding to that the strange warning about the agent isn't being running, I think the latter is most probably the Case.

If The application runs you can just ignore the warnings (they should is printed only at instrumentation time, so Bootstra P/warming Stage) or if you can share a minimal project setup I could help have a deeper look into figure out what ' s happen ing Exactly.

Https://groups.google.com/for ...

FJP worker Runtime If there is a suspected blocking, there will be warning hogging the CPU or blocking a thread

You can disable the warning by setting a system property with "-dco.paralleluniverse.fibers.detectrunawayfibers=false"

Standalone Tomcat + Quasar Agent fiberhttpservlet report NPE

[quasar] Error:unable to instrument class Co/paralleluniverse/fibers/servlet/fiberhttpservlet

From the full logs I see that my setup and your setup is different though:i ' m using an embedded Tomcat while you ' re Runn ing Tomcat as a standalone servlet container and using the agent for Instrumentation. Unfortunately the agent doesn ' t currently work with standalone Tomcat and your need to use the instrumenting loader.

Official Recommendation: standalone Tomcat + quasarwebappclassloader or inline container + Quasar Agent

Warning:uninstrumented methods on the "call stack" (marked with * *)

Quasar cannot modify the third party libraries to @suspend, can explicitly put the relevant method into the Meta-inf/suspendables

Standalone Tomcat + Quasarwebappclassloader unabletoinstrumentexception (harmless)

It's a comsat bug, but it's harmless and can be ignored.

Unabletoinstrumentexception:unable to instrument co/paralleluniverse/fibers/fiber#onresume () V because of catch for Suspendexecution
Google Group Comsat issues 25

Coroutine in Java-quasar fiber implementation--reprint