Research on NATs

Nats is the internal neural system of cloudfoundry. It is a lightweight messaging Middleware Based on eventmachine and using the "publish-subscribe" mechanism. Based on the features of EM, Nats has the ability to process highly concurrent requests in the ruby environment. Nats does not persist the message itself, so the matching and subscription process is concise and efficient. At present, Nats Server is a single point of dependency to be addressed in CF.

Nats main dependent gem packages include: eventmachine, json_pure, daemons, thin

The NATs code structure is relatively simple:

Source Code address: https://github.com/derekcollison/nats/

Lib
NATs/client. RB NATs Client
Entry class for NATs/server. RB to start NATs Server
Actual NATs/Server/server. RB NATs server code
NATs/Server/connection. RB network connection and Message Processing
NATs/Server/sublist. RB subscriber management
NATs/Server/options. RB NATs startup options
NATs/EXT/supplement to jruby and ruby1.8

In addition, under the cluster branch on GitHub, we can also see the NATs cluster code:

Lib
NATs/Server/cluster. RB NATs server cluster code
NATs/Server/route. RB NATs routing cluster code

Note: before reading the code, we 'd better familiarize ourselves with the use of NATs according to the README section on GitHub.

I./lib/NATs/server. Rb

This is the server packaging class, which starts two services through eventmachine.

1. NATs server, Em-based Core Message Server

NATSD::Server.start_http_serverEventMachine::start_server(NATSD::Server.host, NATSD::Server.port, NATSD::Connection)

Here, we can see the close relationship between NATs and Em. Please try to compare the following two pieces of code in NLP:

The left part is a simple example of starting an "echo server" we have seen before, while the right side is the method for starting the core NATs server in NATs. Yes, Nats is a typical example of EM-based network programming. In the next part of NATs # connection, we can see more such examples.

2. HTTP Monitor server, a thin-based monitoring server. Like other CF components, this monitor server is mainly used to respond to/varz and/healthz requests.

 # Check to see if we need to fire up the http monitor port and server  if NATSD::Server.options[:http_port]    begin      NATSD::Server.start_http_server    rescue => e      log "Could not start monitoring server on port #{NATSD::Server.options[:http_port]}"      log_error      exit(1)    end  end

View the definition of http_port in/lib/NATs/Server/options. RB. We can see how to start monitor:

Opts. On ("-M", "-- http_port port", "Use http port "). That's right. Add the parameter-M.

For example, if we access localhost: http_port/varz, we can see the NATs-server information.

{  "in_bytes": 0,  "out_bytes": 0,  "cpu": 3.2,  "start": "Tue Oct 09 17:15:54 +0800 2012",  "connections": 0,  "options": {    "max_pending": 2000000,    "users": [      {        "pass": "nats",        "user": "nats"      }    ], ... ...

If we access/healthz, we will receive an "OK", which is similar to the cloudfoundry component mechanism:

@healthz = "ok\n"... ...map '/healthz' do  run lambda { |env| [200, RACK_TEXT_HDR, NATSD::Server.healthz] }end

Ii. sublist, the core data organization

Sublist is used to describe the subscriber (subs) that subscribes to a topic. Topics registered by subs are separated into tokens by dots. NATs supports two wildcards:> and *. Their definitions are as follows:

# "*" matches any token, at any level of the subject. like *.foo foo.*bar
# ">" Matches any length of the tail of a subject and can only be the last token # E. g. 'foo.> 'Will Match' Foo. bar ', 'foo. bar. baz', 'foo. foo. bar. bax.22'
For example:
If the topic of the publisher is foo. Bar, the subscriber subscribed to foo. Bar and foo. * will respond to the subscriber.
If the topic of the publisher is foo. Bar. Hoge, the subscriber subscribed to foo.> will respond to the subscriber, And the subscriber scheduled to foo. * is invalid.
The topic matching mechanism of NATs is described through two data structures: Level and Node
SublistLevel = Struct.new(:nodes, :pwc, :fwc)
SublistNode  = Struct.new(:leaf_nodes, :next_level)
If you want to understand the meaning of the above data structure, we can start with the sublist # insert method to add the subscriber below:
...    for token in tokens      # This is slightly slower than direct if statements, but looks cleaner.      case token        when FWC then node = (level.fwc || (level.fwc = SublistNode.new([])))        when PWC then node = (level.pwc || (level.pwc = SublistNode.new([])))        else node = ((level.nodes[token]) || (level.nodes[token] = SublistNode.new([])))      end      level = (node.next_level || (node.next_level = SublistLevel.new({})))    end    node.leaf_nodes.push(subscriber)...
By reading the following insert method, we can analyze the relationship between node and level as follows:
In combination, we can see that:
Level storage content includes
 
 

  
  
PwC, composed of * matched nodes
  
  
Fwc, directed by>
  
  
Nodes, a hash, the key is the token, and the value is the corresponding node
 
 
The content stored by node includes:
 
 

  
  
Leaf_nodes: all subscribers corresponding to the topic when it matches the level to which the node belongs
  
  
Next_level, which can be further matched
 
 
Therefore, when NATs adds a subscriber, it first needs to traverse the sub topic:
1. Check whether the node corresponding to the token exists in each level from the root of sublist, such as node1 = nodes ['a'] (or associated with the wildcard, for example, node3), record the next_level of the node. Of course, if there is no such node in the current sublist, it will be created directly.
2. Continue traversal. If the topic tokens have been traversed, add all subscribers (subs) for this subscription to leaf_nodes, for example, node1, obtained from the previous traversal)
3. Otherwise, traverse continues Based on next_level until the traversal ends in step 2.
After the insert operation is complete, the sublist structure is roughly shown in.
When NATs needs to find the corresponding subs Based on the specified topic, it actually searches in the following order: (each token has an iteration)
Token-> node-> level-> token-> node-> level->... -> token-> node-> leaf_nodes. All right, the leaf_nodes array subscribes to the token. token... the subscriber set of the topic token.
Under such a mechanism, the query conditions for wildcards and non-Wildcards are completely similar, but if a token is PwC, it should match the nodes for subscribing to any token in step 2; fwc means that this token must be the last token of a topic, and we only need to get the leaf_nodes of the node associated with this fwc.
With the foundation of the above sublist, the implementation of the following matching method is quite understandable:
  ...      tokens = subject.split('.')    matchAll(@root, tokens)  ...    def matchAll(level, tokens)    node, pwc = nil, nil # Define for scope    i, ts = 0, tokens.size    while (i < ts) do      return if level == nil      # Handle a full wildcard here by adding all of the subscribers.      @results.concat(level.fwc.leaf_nodes) if level.fwc      # Handle an internal partial wildcard by branching recursively      lpwc = level.pwc      matchAll(lpwc.next_level, tokens[i+1, ts]) if lpwc      node, pwc = level.nodes[tokens[i]], lpwc      #level = node.next_level if node      level = node ? node.next_level : nil      i += 1    end    @results.concat(pwc.leaf_nodes) if pwc    @results.concat(node.leaf_nodes) if node  end
When we request a subscriber list for a topic, no matter what the topic is, the actual matching process can be divided into three situations:
1. the level to be traversed does not contain wildcards. In this case, the matching process is to traverse the tokens array and find the last node in the order mentioned above. The leaf_nodes recorded on this node is the set of subscribers we need.
2. The retrieved level contains * wildcards. At this point:
If it is already the last traversal (e.g. sublist records the subscription of. * Subs, then there is. b), the loop will jump out and the result is assigned to the leaf_nodes of the node that points to PwC for the last time.
If the traversal has not ended (e.g. sublist records the subscription of. *. b's subs, then there is. c. B's matching request), we will recursively perform the same matching operation from the tokens and level following *, and this process will continue until the final subscriber gets the backtracking.
Go back to the original code segment. B and the subs in the later sections have been found and put into results. Next, we will continue to query sublist from C. Therefore, subscriber like A. c. B will also be added to results. If the node associated with the c topic cannot be queried from this level, the traversal will also end.
3. the level to be traversed contains> wildcards. This is the best solution. Because it is the last Token, you only need to get the leaf_nodes of the node associated with the fwc.
The structure of sublist is actually relatively simple, but we are still very arrogant. You 'd better give a few examples and proceed with the above process. I believe sublist will be much clearer.
Iii./lib/NATs/Server/server. Rb
This is the specific implementation class of the server. The main functions of the server are described in this section.
1. subscribe and unsubscribe
def subscribe(sub)  @sublist.insert(sub.subject, sub)enddef unsubscribe(sub)  @sublist.remove(sub.subject, sub)end
2. Send a message to a subscriber.
Def deliver_to_subscriber (sub, subject, reply, MSG)
The methods in ../Server/connection. RB need to be used here:
conn.queue_data("MSG #{subject} #{sub.sid} #{reply}#{msg.bytesize}#{CR_LF}#{msg}#{CR_LF}")
Connection. RB will be introduced later. Here we will only look at the relevant methods.
    def flush_data      return if @writev.nil? || closing?      send_data(@writev.join)      @writev, @writev_size = nil, 0    end    def queue_data(data)      EM.next_tick { flush_data } if @writev.nil?      (@writev ||= []) << data      @writev_size += data.bytesize      flush_data if @writev_size > MAX_WRITEV_SIZE    end
Recall the knowledge of EM # next_tick: The next_tick method schedules a block to the next iteration of the reactor for execution. The thread that executes the task is the main reactor thread. In fact, here we want to execute the send_data method to send data.
3. route a message to all subscribers
The corresponding method is route_to_subscribers (subject, reply, MSG). The main parts of this method are as follows:
   ... ...          @sublist.match(subject).each do |sub|          # Skip anyone in the closing state          next if sub.conn.closing          unless sub[:qgroup]            deliver_to_subscriber(sub, subject, reply, msg)          else            if NATSD::Server.trace_flag?              trace("Matched queue subscriber", sub[:subject], sub[:qgroup], sub[:sid], sub.conn.client_info)            end            # Queue this for post processing            qsubs ||= Hash.new            qsubs[sub[:qgroup]] ||= []            qsubs[sub[:qgroup]] << sub          end        end          return unless qsubs        qsubs.each_value do |subs|          # Randomly pick a subscriber from the group          sub = subs[rand*subs.size]          if NATSD::Server.trace_flag?            trace("Selected queue subscriber", sub[:subject], sub[:qgroup], sub[:sid], sub.conn.client_info)          end          deliver_to_subscriber(sub, subject, reply, msg)        end   ... ...
We can see that the core of this method is to traverse sublist by subject, and then call deliver_to_subscriber to implement the routing process.
Note: Here is a queue subscriber judgment. If you need to use qgroup, Nats first generates qsubs and then randomly retrieves a subscriber from the Group for routing.
The usage of queue subscriber in readme indicates that a group of subs subscribed to the same topic is divided into one group, and then a sub is selected for message forwarding according to the above mechanism.
4. Start the preceding HTTP Monitoring Server
http_server = Thin::Server.new(@options[:http_net], port, :signals => false)
Iv./lib/NATs/Server/connection. Rb 
Connection is mainly responsible for receiving and analyzing data. it overwrites the methods in the EM # connection class, so NATs connection is passed to Em as the connection parameter of the previous NATs server. For more information about the role of connectioin in Em, see relevant articles on eventmachine.
As mentioned above, queue_data focuses on its receive_data method.
There are two connection modes: Waiting for control information (awaiting_control_line) waiting for data messages (awaiting_msg_payload)
The connection is initially in awaiting_control_line and waits for control information.
Control Information type:
Pub_op, sub_op, unsub_op, ping, Pong, connect, info, unkonwn
Here we will only introduce the processing process of the two most important control information:
Pub_op:
Control Information Format: pub msg_sub msg_reply msg_size
Msg_sub is the topic, msg_reply is the message response, and msg_size is the message length.
After receiving the control information:
1. The connection will be transferred to the awaiting_msg_payload mode.
@parse_state = AWAITING_MSG_PAYLOAD
2. In awaiting_msg_payload mode, connection receives a message from the client, and then intercepts the published message content.
msg = @buf.slice(0, @msg_size)
3. Route it to the subscribed subscriber for processing.
Server.route_to_subscribers(@msg_sub, @msg_reply, msg)
4. After processing, it is still restored to the awaiting_control_line mode, and the new control information will continue to arrive.
Sub_op
Control Information Format: Sub msg_sub qgroup Sid
Msg_sub indicates the topic of the message, qgroup indicates the queue group information, and Sid indicates the ID of the subscription connection.
After receiving this control information:
1. generate a new subscriber Based on msg_sub, qgroup, and Sid. The Subscriber object includes the topic to be registered.
subscriber = Subscriber.new(self, sub, sid, qgroup, 0) 
2. Add the subscriber to the subscriber set of the server specified by the SID.
@subscriptions[sid] = sub
3. Call server # subscribe to add the subscriber to sublist for future topic-based query.
Server.subscribe(sub)
The subscribe method actually calls the sublist # insert method.
 
V. NATs Client
The client is responsible for sending operation commands to the NATs server. We only explain the following methods:
1. In the start method, the client needs to establish a connection to the server:
EM.run { @client = connect(*args, &blk) }
And use the connection handle as the @ client variable.
2. The connect method uses the EM connect method to establish a connection to the NATs server:
client = EM.connect(@uri.host, @uri.port, self, opts)client.on_connect(&blk) if blkreturn client
After the connection is established successfully, the associated block is executed.
3. The subscribe method saves the subscriber information on the client and sends sub commands to the NATs server.
 
  def subscribe(subject, opts={}, &callback)    return unless subject    sid = (@ssid += 1)    sub = @subs[sid] = { :subject => subject, :callback => callback, :received => 0 }    sub[:queue] = opts[:queue] if opts[:queue]    sub[:max] = opts[:max] if opts[:max]    send_command("SUB #{subject} #{opts[:queue]} #{sid}#{CR_LF}")    # Setup server support for auto-unsubscribe    unsubscribe(sid, opts[:max]) if opts[:max]    sid  end
Refer to the send_commant method. Other control information is sent to the server through a similar send_command method.
Vi. Others
The source code analysis of NATs is here, but if you really want to understand NATs, you still need to do more experiments to see how eventmachine works. In fact, the reason why cloudfoundry chose a lightweight message-oriented middleware such as Nats is that this ruby-based and em-based code performs well in high concurrency, and Ruby's concise and efficient network programming technology. These two factors are very important in the design of distributed systems.
However, for Private clouds of enterprises, stability and reliability often account for a high proportion, so the NATs of a single node is easy to worry about. Since NATs official cluster is still in beta and there is no clear deadline, some foreign countries have suggested the following alternative solutions:
-Amqp (rabbitmq)
-Storm
-Restms
-XMPP
The selection of middleware is complicated, but we should consider the support of Ruby clients, sub/pub message mechanisms, and wildcards, server or broker-based work methods.