ZEROMQ Source Code Analysis 3------ZEROMQ Message _ Code Analysis

This paper mainly talks about the ZEROMQ message system zmq_msg_t and its related operation functions.

Because TCP is a type of byte-throttling protocol, wood has boundaries, so the development of the message boundary is left to the application layer. There are usually two ways to achieve this:

1. Add delimiters to the transmitted data.

2. Add the Size field to each message.

And ZEROMQ could say that the second way was chosen.

Let's look at the basic data structure of zmq_msg_t.

/*  A message. Note that ' content ' isn't ' a ' pointer to the raw data.          *  /* Rather it is pointer to zmq::msg_content_t                      structure  /* (for definition).                             * *
typedef struct
{   
    void *content;
    unsigned char flags;
    unsigned char vsm_size;//small message size
    unsigned char vsm_data [zmq_max_vsm_size];//Small message memory area
} zmq_msg_t

Where the content pointer points to the structure of the zmq::msg_content_t, flags indicate some flags of the message

This side vsm_size and Vsm_data is for very small message of small messages to do some optimization, directly on the stack allocated memory, you can see the following message function of the specific operation.

  Shared message buffer. Message data are either allocated in one
    //  continuous blocks along with this structure-thus avoiding one
    //< C4/>malloc/free pair or they are stored in used-supplied.  in the "latter case", FFN member stores pointer to the "function to
    "//  used to deallocate the data. If the buffer is actually shared (there
    //  are at least 2 references to it) RefCount member contains number OF
  //  references.
    
    struct msg_content_t
    {
        void *data;
        size_t size;
        ZMQ_FREE_FN *FFN;
        void *hint;
        zmq::atomic_counter_t refcnt;
    };

Where data points to the true message data, size represents the byte size of the message data, ZMQ_FREE_FN *FFN points to the release function, refcnt represents the reference count of the message, hint is currently unknown.

Let's take a look at the basic message manipulation functions:

int Zmq_msg_init_size (zmq_msg_t *msg_, size_t size_)
{
    if (size_ <= zmq_max_vsm_size) {
        msg_-> Content = (zmq::msg_content_t*) zmq_vsm;
        Msg_->flags = (unsigned char) ~zmq_msg_mask;
        Msg_->vsm_size = (uint8_t) size_;
    }
    else {
        msg_->content =
            (zmq::msg_content_t*) malloc (sizeof (zmq::msg_content_t) + size_);
        if (!msg_->content) {
            errno = Enomem;
            return-1;
        }
        Msg_->flags = (unsigned char) ~zmq_msg_mask;
    
        zmq::msg_content_t *content = (zmq::msg_content_t*) msg_->content;
        Content->data = (void*) (content + 1);
        Content->size = Size_;
        CONTENT->FFN = NULL;
        Content->hint = NULL;
        New (&CONTENT->REFCNT) zmq::atomic_counter_t (); Set reference count
    } return
    0;
}   

For small messages, we set the content to point to address ZMQ_VSM, which is 32, the equivalent of a magic number, used to represent small messages.

For relatively large messages, we malloc (sizeof (zmq::msg_content_t) + size_), where Size_ is the variable-length message size, and then set content->data = (void *) ( Content + 1), pointing to this variable length of memory area.

Here is an operation that initializes the MSG flags:

        Msg_->flags = (unsigned char) ~zmq_msg_mask;

The definition of flags:

/* Message  flags. Zmq_msg_shared is strictly speaking not a message flag
/*  (it has no equivalent in the wire format), Howeve R, making  it a flag     * * *  allows us to pack the Stucture Tigher       and thus improve. *
#define ZMQ_MSG_MORE 1//00000001
#define ZMQ_MSG_SHARED 128//10000000
#define ZMQ_MSG_MASK 129/* Me Rges all the flags *//10000001

We can simply analyze this so that the initial value of the MSG flags is 01111110, which can be | Zmq_msg_more or | Zmq_msg_shared.

Nima... Feel like talking nonsense ...

Let's take a look at another two message initialization functions:

int Zmq_msg_init (zmq_msg_t *msg_)
{
    msg_->content = (zmq::msg_content_t*) zmq_vsm;
    Msg_->flags = (unsigned char) ~zmq_msg_mask;
    msg_->vsm_size = 0;
    return 0;
}

Initializes a MSG message, sets it to a small message type, initializes the flags, and initializes the size of the small message to 0.

int Zmq_msg_init_data (zmq_msg_t *msg_, void *data_, size_t size_,
    zmq_free_fn *ffn_, void *hint_)
{
    msg_-& Gt;content = (zmq::msg_content_t*) malloc (sizeof (zmq::msg_content_t));
    Alloc_assert (msg_->content);
    Msg_->flags = (unsigned char) ~zmq_msg_mask;
    zmq::msg_content_t *content = (zmq::msg_content_t*) msg_->content;
    Content->data = Data_;
    Content->size = Size_;
    CONTENT->FFN = ffn_;
    Content->hint = Hint_;
    New (&CONTENT->REFCNT) zmq::atomic_counter_t ();
    return 0;
}

The function initializes the message based on the existing data and corresponding size, the destroyed function, and the hint.

This message points to the memory area that the data pointer refers to.

Let's take a look at the function that gets the address of the data so that you can manipulate the data memory area of the message.

void *zmq_msg_data (zmq_msg_t *msg_)
{
    Zmq_assert (msg_->flags | Zmq_msg_mask) = = 0xff);

    if (msg_->content = = (zmq::msg_content_t*) zmq_vsm) return
        msg_->vsm_data;
    if (msg_->content = = (zmq::msg_content_t*) zmq_delimiter) return
        NULL;

    Return ((zmq::msg_content_t*) msg_->content)->data;
}

If it's a small message, then it's the Vsm_data address on the stack, and the big news is the content->data on the heap.

If it is delimiter, it returns a null pointer. We'll talk about the usefulness of delimiter later.

The Zmq_msg_size (1) function is similar to the way it is handled, except that the message size is returned.

size_t zmq_msg_size (zmq_msg_t *msg_)
{
    Zmq_assert (msg_->flags | Zmq_msg_mask) = = 0xff);

    if (msg_->content = = (zmq::msg_content_t*) zmq_vsm) return
        msg_->vsm_size;
    if (msg_->content = = (zmq::msg_content_t*) zmq_delimiter) return
        0;

    Return ((zmq::msg_content_t*) msg_->content)->size;
}

Next we look at the Zmq_msg_close (1) function to see how to destroy the message.

int Zmq_msg_close (zmq_msg_t *msg_) {//Check the validity tag. if (Unlikely (Msg_->flags |
        Zmq_msg_mask)!= 0xff) {errno = Efault;
    return-1;
    }//For vsms and delimiters there are no.  if (msg_->content!= (zmq::msg_content_t*) zmq_delimiter && msg_->content!= (zmq::msg_content_t*)
        ZMQ_VSM) {//If the content is isn't shared, or if it is shared and the reference.
        Count has dropped to zero, deallocate it.
        zmq::msg_content_t *content = (zmq::msg_content_t*) msg_->content; if (!) ( Msg_->flags & zmq_msg_shared) | |
            !content->refcnt.sub (1)) {//We used "placement new" operator to initialize the reference.
            Counter so we call it destructor now.

            content->refcnt.~atomic_counter_t ();
            if (CONTENT->FFN) CONTENT->FFN (Content->data, content->hint);
 Free (content);       }//Remove the validity tag from the message.

    msg_->flags = 0;
return 0;
 }

The flag detection of the message is similar to the previous, and the end of the function also removes the initialized flags, placing them in 0, indicating that the message has been discarded.

For small messages, message data is allocated on the stack, so no manual destruction is required.

For large messages, the message data is allocated on heap, so we see if the message flags use zmq_msg_shared sharing mode and destroy them if they are not used. If shared mode is used, then we decrement the reference count of the message, and once the reference count is 0, we destroy the message content.

The destruction process is as follows:

1. Destroy the reference count, because we used placement new when we created the reference count so we need to call its destructor here.

2. Call this function if you have the appropriate custom destroy function registered. This is primarily used to use Zmq_msg_init_data () to manage the space that data points to.

3. Call Free (1) to release the heap space, note that if you use Zmq_msg_init_size () to initialize the message, you will release the space of data, because this space is the allocation of more than the size of the amount allocated (below the content) of the block.

As for Zmq_msg_move and zmq_msg_copy, the message is mainly move and copy.

int Zmq_msg_move (zmq_msg_t *dest_, zmq_msg_t *src_) {#if 0//Check the validity tags. if (Unlikely (Dest_->flags |
          Zmq_msg_mask)!= 0xFF | | (Src_->flags |
        Zmq_msg_mask)!= 0xff)) {errno = Efault;
    return-1;
    } #endif Zmq_msg_close (dest_);
    *dest_ = *src_;
    Zmq_msg_init (SRC_);
return 0;
    int zmq_msg_copy (zmq_msg_t *dest_, zmq_msg_t *src_) {//Check the validity tags. if (Unlikely (Dest_->flags |
          Zmq_msg_mask)!= 0xFF | | (Src_->flags |
        Zmq_msg_mask)!= 0xff)) {errno = Efault;
    return-1;

    } zmq_msg_close (Dest_);
    Vsms and delimiters require no special handling.  if (src_->content!= (zmq::msg_content_t*) zmq_delimiter && src_->content!= (zmq::msg_content_t*) ZMQ_VSM) {//One reference is added to shared messages.
        non-shared messages//are turned into shared messages and reference count are set to 2. Zmq::msg_content_t *content = (zmq::msg_content_t*) src_->content;
        if (Src_->flags & zmq_msg_shared) Content->refcnt.add (1);
            else {src_->flags |= zmq_msg_shared;
        Content->refcnt.set (2);
    } *dest_ = *src_;
return 0;
 }

Zmq_msg_move (1) Resets the source message to an initialized empty message.

Zmq_msg_copy (1) DST and SRC share the message content references, with a major focus on reference-counting changes. If you are already in shared mode, increase the reference count, otherwise set to shared mode and set the reference technology to 2.

With this knowledge, let's take a look at some examples of basic message operations:

Receive 0MQ string from socket and convert to C string static char * S_RECV (void *socket) {zmq_msg_t message; Create message Structure zmq_msg_init (&message); Initializes an empty message zmq_recv (socket, &message, 0); Receive Message int size = Zmq_msg_size (&message); Computes the size of the message char *string = malloc (size + 1); Assign string to heap space that points to size + 1, and that extra 1 bytes is a ' memcpy ' space (String, Zmq_msg_data (&message), size); The data address of the message is obtained by Zmq_msg_data (1), and the Zmq_msg_close (&message) is copied into the string. Release or destroy message string [size] = 0;
Set ' I ' return (string);
    /Convert C string to 0MQ string and send to socket static int s_send (void *socket, char *string) {int rc; zmq_msg_t message; Creates a message structure zmq_msg_init_size (&message, strlen (string)); Initialized to Message memcpy (Zmq_msg_data (&message), String, strlen (string) as the length of the string (excluding ' "); Copies the contents of the string (excluding ' I ') to the message     RC = zmq_send (socket, &message, 0);
    Send Message assert (!RC); Zmq_msg_close (&message); //release and Destroy message return (RC); }

The process of sending and receiving messages can be seen from the comments above. ZEROMQ has associated libraries (CZMQ) that encapsulate these operations, but you can also encapsulate them yourself.

Zmq_delimiter:

The delimiter type message was mentioned earlier. This type of message is similar to the Terminator's meaning, and is used primarily in pipelines that are sent and received. Because ZEROMQ sends the message to the pipe first, then Poller runs on another thread, reads the data from the pipe to the socket buffer, so you can send a delimiter type of message to terminate the pipe and destroy it. In the course of our analysis of the pipeline later, you can see the code to do the work.

Summarize:

This paper simply introduces the data structure of the message defined in ZEROMQ and related operations, basically zeromq the data structure of the message using variable Long data structure to store data, but also to the small message for memory allocation optimization above, directly using stack allocation, Instead of using heap dynamic allocation, message content has reference counts that can be shared.

When we see the message flags, we find that we have omitted a #define Zmq_msg_more 1 symbol, which is used by multipart messages. The next time we will analyze this piece of content, and will look at the message sent to the pipeline, poller it out to send to the socket buffer details and the process of receiving messages in turn, please wait.

I hope that interested friends can contact me and study together. Kaka11.chen@gmail.com