Erlang send analysis and parameter significance, erlangsend

Erlang send analysis and parameter significance, erlangsend
Erlang send is a basic message sending function used by a process to send a message to another process. This function can be used for local node process communication or remote node Process Communication.
Preface recently, some colleagues encountered the problem of message accumulation caused by erlang: send, which aroused my strong attention. I also read this code. Here is a simple example.
Function prototype:Erlang: send (Dest, Msg, Options)-> ResOptions can be either of the following:NosuspendIf the sender wocould have to be susponded to do the send, nosuspend is returned instead.
NoconnectIf the destination node wowould have to be auto-connected before doing the send, noconnect is returned instead.
Literally, it means that when nosuspend encounters a situation where a process will be suspended, it directly returns nosuspendnoconnect. When a remote node does not have a connection, it will not automatically send a message, and it will directly return noconnect.
However, erlang directly describes how to use nosuspendWarning with caution.

As with erlang: send_nosuspend/2, 3: Use with extreme care!

Why is this message sent or not when nosuspend is returned?
The source code analysis shows the erlang code. erlang: send is implemented by bif. Here we describe R16B02:

/*** Bif. c send_3 () function to implement erlang: send/3 */BIF_RETTYPE send_3 (BIF_ALIST_3) {Eterm ref; Process * p = BIF_P; Eterm to = BIF_ARG_1; Eterm msg = BIF_ARG_2; eterm opts = BIF_ARG_3; int connect =! 0; // set the initial value to 1, indicating that the non-0 value int suspend =! 0; // same as Eterm l = opts; Sint result; while (is_list (l) {// traverse the parameter list if (CAR (list_val (l) = am_noconnect) {connect = 0; // if the parameter has noconnect, connect value 0} else if (CAR (list_val (l) = am_nosuspend) {suspend = 0; // same as above} else {BIF_ERROR (p, BADARG);} l = CDR (list_val (l);} if (! Is_nil (l) {BIF_ERROR (p, BADARG);} // call do_send to send a message. If the result is greater than 0, it indicates the reds to be deducted for this message, otherwise, the error code result = do_send (p, to, msg, suspend, & ref); if (result> 0) {ERTS_VBUMP_REDS (p, result ); // deduct the redsif (ERTS_IS_PROC_OUT_OF_REDS (p) goto yield_return; BIF_RET (am_ OK);} switch (result) {case 0: /* May need to yield even though we do not bump reds here... */if (ERTS_IS_PROC_OUT_OF_REDS (p) goto yield_re Turn; BIF_RET (am_ OK); break; // case SEND_TRAP: if (connect) {// connect is not equal to 0 BIF_TRAP3 (dsend3_trap, p, to, msg, opts);} else {// connect is equal to 0. Return noconnect BIF_RET (am_noconnect) ;}break; // case SEND_YIELD error: SEND_YIELD: if (suspend) {// suspend is not equal to 0 ERTS_BIF_YIELD3 (bif_export [BIF_send_3], p, to, msg, opts);} else {// suspend is equal to 0, return nosuspend BIF_RET (am_nosuspend);} bre Ak; // when the SEND_YIELD_RETURN error occurs, the case SEND_YIELD_RETURN: // suspend is equal to 0, and the nosuspendif (! Suspend) BIF_RET (am_nosuspend); yield_return: values (p, am_ OK); case when: ASSERT (is_internal_ref (ref); then (values, p, ref, am_nosuspend, am_ OK ); case SEND_BADARG: BIF_ERROR (p, BADARG); break; case SEND_USER_ERROR: BIF_ERROR (p, EXC_ERROR); break; case SEND_INTERNAL_ERROR: BIF_ERROR (p, EXC_INTERNAL_ERROR); break; default: ASSERT (! "Illegal send result"); break;} ASSERT (! "Can not arrive here"); BIF_ERROR (p, BADARG );}

Let's take a look at the do_send () function:

/** Bif. c do_send () function to send messages to other processes, or ports, or remote processes * return the reds sent by the message, or the error code * // The following are possible error codes returned by do_send # define SEND_TRAP (-1) # define SEND_YIELD (-2) # define SEND_YIELD_RETURN (-3) # define SEND_BADARG (-4) # define SEND_USER_ERROR (-5) # define SEND_INTERNAL_ERROR (-6) # define SEND_AWAIT_RESULT (-7) Sintdo_send (Process * p, Eterm to, Eterm msg, int suspend, Eterm * refp) {Eterm portid; Port * pt; Process * rp; DistEntry * dep; Eterm * Tp; // if the target process is on the local node if (is_internal_pid (to) {if (IS_TRACED (p) trace_send (p, to, msg ); if (ERTS_PROC_GET_SAVED_CALLS_BUF (p) save_cils (p, & exp_send); rp = erts_proc_lookup_raw (to); if (! Rp) return 0; // find the process and execute the final send_message} else if (is_external_pid (to) {// if the target process is on a remote node, dep = external_pid_dist_entry (); if (dep = erts_this_dist_entry) {// The dist_entry corresponding to the remote node is abnormal. The collaboration process to erts_dsprintf_buf_t * dsbufp = Disable (); erts_dsprintf (dsbufp, "Discarding message % T from % T to % T in an old" "incarnation (% d) of this node (% d) \ n", msg, p-> common. id, to, external_pid_creatio N (to), erts_this_node-> creation); erts_send_error_to_logger (p-> group_leader, dsbufp); return 0 ;}// remote message call remote_send to send return remote_send (p, dep, to, to, msg, suspend);} else if (is_atom (to) {// if the passing parameter is atomic, try to find the process Eterm id = erts_whereis_name_to_id (p, to); rp = erts_proc_lookup (id); if (rp) goto send_message; // the process cannot be found. Check whether the target port is pt = erts_port_lookup (id, ERTS_PORT_SFLGS_INVALID_LOOKUP ); if (pt ){ Portid = id; goto port_common;} if (IS_TRACED (p) trace_send (p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF (p) save_cils (p, & exp_send ); return SEND_BADARG;} else if (is_external_port (to) & (external_port_dist_entry (to) = erts_this_dist_entry) {// The dist_entry corresponding to the remote node is abnormal, collaboration Process to failure erts_dsprintf_buf_t * dsbufp = erts_create_logger_dsbuf (); erts_dsprintf (dsbufp, "Discarding message % T from % T to % T in Old "" incarnation (% d) of this node (% d) \ n ", msg, p-> common. id, to, external_port_creation (to), erts_this_node-> creation); increment (p-> group_leader, dsbufp); return 0;} else if (is_internal_port ()) {// if it is a local port int ret_val; portid = to; pt = erts_port_lookup (portid, region); port_common: ret_val = 0; if (pt) {int ps_flags = suspend? 0: ERTS_PORT_SIG_FLG_NOSUSPEND; * refp = NIL; // execute the port operation switch (erts_port_command (p, ps_flags, pt, msg, refp) {case ERTS_PORT_OP_CALLER_EXIT: /* We are exiting... */return SEND_USER_ERROR; case when:/* Nothing has been sent */if (suspend) erts_suspend (p, ERTS_PROC_LOCK_MAIN, pt); // if nosuspend returns SEND_YIELD, the Message has not sent return SEND_YIELD; case ERTS_PORT_OP_BUSY_SCHEDULED:/* Message was sent */If (suspend) {erts_suspend (p, ERTS_PROC_LOCK_MAIN, pt); ret_val = SEND_YIELD_RETURN; break;} // There is no break here. if nosuspend performs the next step, message sent/* Fall through */case when: if (is_not_nil (* refp) {ASSERT (is_internal_ref (* refp); ret_val = SEND_AWAIT_RESULT;} break; case when: case ERTS_PORT_OP_BADARG: case ERTS_PORT_OP_DONE: break; default: ERTS_INTERNAL_ERROR ("Unexpected e Rts_port_command () result "); break;} if (IS_TRACED (p)/* trace once only !! */Trace_send (p, portid, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF (p) save_cils (p, & exp_send); if (SEQ_TRACE_TOKEN (p )! = NIL # ifdef USE_VM_PROBES & SEQ_TRACE_TOKEN (p )! = Am_have_dt_utag # endif) {seq_trace_update_send (p); seq_trace_output (SEQ_TRACE_TOKEN (p), msg, SEQ_TRACE_SEND, portid, p);} if (limit (p )) {KILL_CATCHES (p);/* Must exit */return SEND_USER_ERROR;} return ret_val;} else if (is_tuple ()) {/* Remote send * // if to is an atom, only the Remote message is sent. int ret; tp = tuple_val (to); if (* tp! = Make_arityval (2) return SEND_BADARG; if (is_not_atom (tp [1]) | is_not_atom (tp [2]) return SEND_BADARG; /* sysname_to_connected_dist_entry will return NULL if there is no dist_entry or the dist_entry has no port, but remote_send () will handle that. * // if dist_entry is found, the local process message or port form is used to send dep = forward (tp [2]); if (dep = erts_this_dist_entry) {Eterm id; erts_deref_dist_entry (d Ep); if (IS_TRACED (p) trace_send (p, to, msg); if (distinct (p) save_cils (p, & exp_send); id = erts_whereis_name_to_id (p, tp [1]); rp = erts_proc_lookup_raw (id); if (rp) goto send_message; pt = erts_port_lookup (id, region); if (pt) {portid = id; goto port_common;} return 0;} // If dist_entry cannot be found, send ret = remote_send (p, dep, tp [1], to, msg, suspend ); If (dep) erts_deref_dist_entry (dep); return ret;} else {if (IS_TRACED (p)/* XXX Is this really neccessary ??? */Trace_send (p, to, msg); if (ERTS_PROC_GET_SAVED_CALLS_BUF (p) save_cils (p, & exp_send); return SEND_BADARG ;} // The following process processes send_message of the current node process: {ErtsProcLocks rp_locks = 0; Sint res; # ifdef ERTS_SMPif (p = rp) rp_locks | = ERTS_PROC_LOCK_MAIN; # endif/* send to local process */res = erts_send_message (p, rp, & rp_locks, msg, 0); if (erts_use_sender_punish) res * = 4; else res = 0; erts_smp_proc_unlock (rp, p = Rp? (Rp_locks &~ ERTS_PROC_LOCK_MAIN): rp_locks); return res ;}}

Let's look at the erts_send_message () function of the local process message processing:

/** Erl_message.c erts_send_message () function to send local messages to the Process */worker (Process * sender, Process * Handler er, ErtsProcLocks * receiver_locks, Eterm message, unsigned flags) {Uint msize; optional * bp = NULL; Eterm token = NIL; Sint res = 0; # ifdef USE_VM_PROBES DTRACE_CHARBUF (sender_name, 64); DTRACE_CHARBUF (receiver_name, 64); Sint tok_label = 0; sint tok_lastcnt = 0; Sint tok_serial = 0; # e Ndif BM_STOP_TIMER (system); BM_MESSAGE (message, sender, receiver); BM_START_TIMER (send); # ifdef USE_VM_PROBES * sender_name = * receiver_name = '\ 0 '; if (DTRACE_ENABLED (message_send) {erts_snprintf (sender_name, sizeof (DTRACE_CHARBUF_NAME (sender_name), "% T", sender-> common. id); erts_snprintf (receiver_name, sizeof (DTRACE_CHARBUF_NAME (receiver_name), "% T", Cycler-> common. id) ;}# endif if (SEQ_TRACE _ TOKEN (sender )! = NIL &&! (Flags & ERTS_SND_FLG_NO_SEQ_TRACE) {// The sender processes are marked with a trace mark sequential_trace_token; the latter processes the process tracing process // see the http://www.erlang.org/doc/man/erlang.html#process_flag-3/** limited space, some irrelevant code is omitted here */} else if (sender = worker ER) {// The process sends a message to itself/* if the receiving process is shutting down, then the message is discarded */# ifdef ERTS_SMPErtsProcLocks need_locks = (~ (* Receiver_locks) & (Response | ERTS_PROC_LOCK_STATUS); if (need_locks) {* receiver_locks | = need_locks; if (Response er, need_locks) = EBUSY) {if (need_locks = locks) {locate (receiver, ERTS_PROC_LOCK_STATUS); need_locks = unlock | ERTS_PROC_LOCK_STATUS;} locate (receiver, need_locks) ;}} if (! Handler (handler) # endif {ErlMessage * mp = message_alloc (); DTRACE6 (message_send, sender_name, receiver_name, size_object (message), tok_label, tok_lastcnt, tok_serial ); mp-> data. attached = NULL; ERL_MESSAGE_TERM (mp) = message; ERL_MESSAGE_TOKEN (mp) = NIL; # ifdef USE_VM_PROBES ERL_MESSAGE_DT_UTAG (mp) = NIL; # endif mp-> next = NULL; // move the message to the end of the private heap in SMP (pure pointer operation) ERTS_SMP_MSGQ_MV_INQ2PRIVQ (batch ER); // append the message to the end of the Message Queue (pure pointer operation) LINK_MESSAGE_PRIVQ (batch er, mp); // res gets the length of the process Message Queue res = worker er-> msg. len; if (IS_TRACED_FL (receiver, F_TRACE_RECEIVE) {trace_receive (receiver er, message) ;}} BM_SWAP_TIMER (send, system );} else {// process sends a message to another process # ifdef ERTS_SMPErlOffHeap * ohp; Eterm * hp; erts_aint32_t state; BM_SWAP_TIMER (send, size); msize = size_object (message ); BM_SWAP_TIMER (size, send); // the recipient's process allocates a heap space of msize to store the message hp = erts_alloc_message_heap_state (msize, & bp, & ohp, Cycler, receiver_locks, & state); BM_SWAP_TIMER (send, copy); // copy the message to the recipient's process. The returned message may be a binary refc binarymessage = copy_struct (message, msize, & hp, ohp); BM_MESSAGE_COPIED (msz); BM_SWAP_TIMER (copy, send); DTRACE6 (message_send, sender_name, receiver_name, msize, tok_label, tok_lastcnt, tok_serial ); // res get the length of the message queue of the receiver process res = queue_message (sender, receiver, receiver_locks, & state, bp, message, token # ifdef USE_VM_PROBES, NIL # endif); BM_SWAP_TIMER (send, system); # elseErlMessage * mp = message_alloc (); Eterm * hp; BM_SWAP_TIMER (send, size); msize = size_object (message); BM_SWAP_TIMER (size, send ); // check that the recipient's process has insufficient memory. Execute GCif (Cycler-> stop-Cycler-> htop <= msize) {BM_SWAP_TIMER (send, system); erts_garbage_collect (Cycler, msize, extends er-> arg_reg, extends er-> arity); BM_SWAP_TIMER (system, send);} hp = extends er-> htop; extends er-> htop = hp + msize; BM_SWAP_TIMER (send, copy); // process referenced binary data (modify reference count) message = copy_struct (message, msize, & hp, & receiver-> off_heap); BM_MESSAGE_COPIED (msize ); upload (copy, send); DTRACE6 (message_send, sender_name, receiver_name, (uint32_t) msize, tok_label, tok_lastcnt, tok_serial); ERL_MESSAGE_TERM (mp) = message; consume (mp) = NIL; # ifdef USE_VM_PROBESERL_MESSAGE_DT_UTAG (mp) = NIL; # endifmp-> next = NULL; mp-> data. attached = NULL; LINK_MESSAGE (receiver er, mp); // res gets the length of the message queue of the receiver process res = receiver er-> msg. len; // notify the recipient that the process message is delivered to erts_proc_policy_new_message (receiver ER); if (IS_TRACED_FL (receiver er, F_TRACE_RECEIVE) {trace_receive (receiver, message);} BM_SWAP_TIMER ); # endif/* # ifndef ERTS_SMP */} return res ;}

Let's take a look at the remote_send () function:

/*** Bif. c remote_send () function for remote message sending * dist_entry is the erlang distributed interface */static Sint remote_send (Process * p, DistEntry * dep, Eterm to, Eterm full_to, Eterm msg, int suspend) {Sint res; int code; ErtsDSigData dsd; ASSERT (is_atom (to) | is_external_pid (to); // check the dist_entry status code = erts_dsig_prepare (& dsd, dep, p, ERTS_DSP_NO_LOCK ,! Suspend); switch (code) {case ERTS_DSIG_PREP_NOT_ALIVE: case ERTS_DSIG_PREP_NOT_CONNECTED: res = SEND_TRAP; break; // return SEND_YIELD case when: ASSERT (! Suspend); res = SEND_YIELD; break; // nosuspend even if dist_entry is not connected, it will go here and try to send the message case ERTS_DSIG_PREP_CONNECTED: {if (is_atom ()) code = erts_dsig_send_reg_msg (& dsd, to, msg); else code = erts_dsig_send_msg (& dsd, to, msg ); /** Note that functions have been bumped on calling * process by erts_dsig_send_reg_msg () or * erts_dsig_send_msg (). */if (code = ERTS_DSIG_SEND_YIELD) res = SEND_YIELD_RETURN; else re S = 0; break;} default: ASSERT (! "Invalid dsig prepare result"); res = SEND_INTERNAL_ERROR;} if (res> = 0) {if (IS_TRACED (p) trace_send (p, full_to, msg ); if (ERTS_PROC_GET_SAVED_CALLS_BUF (p) save_cils (p, & exp_send);} return res ;}

Erts_dsig_send_msg () calls dsig_send at the underlying layer to put the message in the sending queue, and the port_task worker thread is responsible for delivering the message to other nodes. This area is large, mainly including dist. c, erl_port_task.c, erl_process.c, erl_node_tables.c, and io. c. I will talk about it later.


Problem DiscussionHere, answer the previous questions
1. erlang: does send contain nosuspend/noconnect, which may cause message accumulation? When erlang is suspended (that is, the process is {status, sucommitted ded}), it will cause message accumulation.

30> Pid = spawn(fun() -> receive M -> M end end).<0.68.0>31> process_info(Pid,messages).{messages,[]}32> erlang:suspend_process(Pid).true33> Pid ! hello.hello34> process_info(Pid,status).{status,suspended}35> process_info(Pid,messages).{messages,[hello]}

2. Why should nosuspend be used with caution in the erlang document? Because, when a message cannot be sent, the sender process will lose the scheduling permission, but this method will notify the caller of the busy port in the way of return value, and will not suppress the sender process.

3. When erlang: send returns nosuspend, is the message sent out or not? As submitted earlier, the returned nosuspend may be a SEND_YIELD error or a SEND_YIELD_RETURN error. The SEND_YIELD error indicates that the message was not successfully sent, and the SEND_YIELD_RETURN error indicates that the message has been successfully sent.

Extension FunctionsIt can be understood as the basic scheduling measurement unit of Erlang. Erlang VM is scheduled Based on callback to ensure the quasi-real-time performance of scheduling. The higher the function value of a process, the more scheduling opportunities it receives.
Trap_sendThe above Code mentioned the SEND_TRAP error.

    case SEND_TRAP:       if (connect) {          BIF_TRAP3(dsend3_trap, p, to, msg, opts);       } else {          BIF_RET(am_noconnect);      }      break; 

Check the BIF_TRAP3 Code. It is actually a macro. The key to modifying the 'register 'information of the current process is to set freason to TRAP.

#define BIF_TRAP3(Trap_, p, A0, A1, A2) do {\      Eterm* reg = ERTS_PROC_GET_SCHDATA((p))->x_reg_array;\      (p)->arity = 3;\      reg[0] = (A0);\      reg[1] = (A1);\      reg[2] = (A2);\      (p)->i = (BeamInstr*) ((Trap_)->addressv[erts_active_code_ix()]); \      (p)->freason = TRAP;\      return THE_NON_VALUE;\ } while(0)

Now let's take a look at how this process works:

/** Beam_emu.c process_main () thread entry function to implement VM scheduling * the following section intercepts the bif processing Process */OpCase (call_bif_e): {Eterm (* bf) (Process *, Eterm *, beamInstr *) = GET_BIF_ADDRESS (Arg (0); // obtain the actual execution function Eterm result of bif Based on the parameter; BeamInstr * next; PRE_BIF_SWAPOUT (c_p ); c_p-> fcall= fcall- 1; if (fcall< = 0) {save_call( c_p, (Export *) Arg (0);} PreFetch (1, next ); ASSERT (! ERTS_PROC_IS_EXITING (c_p); reg [0] = r (0); result = (* bf) (c_p, reg, I); // execute the bif function ASSERT (! ERTS_PROC_IS_EXITING (c_p) | is_non_value (result); Evaluate (c_p); ERTS_HOLE_CHECK (c_p); Evaluate (c_p); PROCESS_MAIN_CHK_LOCKS (c_p ); if (c_p-> mbuf | MSO (c_p ). overhead> = BIN_VHEAP_SZ (c_p) {Uint arity = (Export *) Arg (0)-> code [2]; result = erts_gc_after_bif_call (c_p, result, reg, arity); E = c_p-> stop;} HTOP = HEAP_TOP (c_p); fcils = c_p-> fcils; if (is_value (result) {r (0) = result; CHECK_TERM (r (0); NextPF (1, next);} else if (c_p-> freason = TRAP) {// SET_CP (c_p, I + 2); SET_ I (c_p-> I); SWAPIN; r (0) = reg [0]; Dispatch (); // after this step, the function directed to dsend3_trap will be called, involving VM implementation. I will not talk about it here

Which function does dsend3_trap point:

/* Dist. c erlang distributed upper-layer implementation function */static Export * trap_function (Eterm func, int arity) {return erts_export_put (am_erlang, func, arity ); // obtain the function address from the exported function table} void init_dist (void) {init_nodes_monitors (); nodedown. reason = NIL; nodedown. bp = NULL; sums (& no_nodes, 0); sums (& no_caches, 0);/* Lookup/Install all references to trap functions */dsend2_trap = trap_function (am_dsend, 2); dsend3_trap = trap_function (am_dsend, 3); // The function directed by dsend3_trap is erlang: dsend/3/* dsend_nosuspend_trap = trap_function (am_dsend_nosuspend, 2 ); */dlink_trap = trap_function (am_dlink, 1); dunlink_trap = trap_function (am_dunlink, 1); functions = trap_function (functions, 3); functions = trap_function (am_dgroup_leader, 2 ); dexit_trap = trap_function (am_dexit, 2); dmonitor_p_trap = trap_function (am_dmonitor_p, 2 );}

Why is erlang subject to trap? Erlang: send () bif and then execute the non-bif function (erlang: dsend/3) erlang: send/3 only when the node connection fails. erlang: dsend/3. At this time, the current process will lose the CPU scheduling permission and put it to the next scheduling to execute this function. In addition, a lot of functions are deducted once this time.
It's not too early. It's a bit sleepy. Find time and study the actual impact of different functions, how to implement soft real-time
Reference: http://blog.csdn.net/mycwq/article/details/42845385