Hive,skynet and Go languages

This is a creation in Article, where the information may have evolved or changed.

Hive,skynet and Go languages

2013-09-25

Hive,skynet and Go languages

Both Hive and Skynet are open source projects for the cloud-inspired gods. Skynet is an open-source concurrency framework based on the actor model. Hive is a redesigned framework that Skynet simplifies and removes some of the "historical baggage". Go is a programming language for Google Open source.

Why put these things together? Because I looked at their code and found that there are amazing similarities in a lot of places, these are the things that Daniel has accumulated over a long period of time, which is very valuable, so this article will take them together to analyze.

Evolution of Skynet to Hive

I have written some articles about [[Http://zenlife.tk/actor.html][actor model]] and [[][skynet]], and I didn't follow up on the latest Skynet code. According to the Cloud Wind, Skynet has been used in the project, not very good big changes, so re-wrote The Hive project.

1. The more streamlined core

Hive is more streamlined than Skynet, for example, Skynet did RPC stuff, and hive was removed. The reason is that something like this is done on the core layer, which complicates the core and does not necessarily satisfy the various scenarios of the application layer. It's better to move to the top to achieve, and keep the core layer thin.

1. Hive directly binds Lua

Since the core code like Skynet implements message passing, it is bound to be a dynamic language to do the upper layer, and instead of making a LUA service module, it is better to bind Lua directly. I think this has brought a lot of benefits. You can look at the differences in the code, which is the previous definition of actor in Skynet:

struct skynet_context {    void * instance;    struct skynet_module * mod;    uint32_t handle;    int ref;    char result[32];    void * cb_ud;    skynet_cb cb;    int session_id;    uint32_t forward;    struct message_queue *queue;    bool init;    bool endless;    CHECKCALLING_DECL};

The core is SkynetCB and messagequeue, one is the callback function, and the other is Message Queuing. And in hive, you can say Skynet is a bit C, and Hive is Lua. Its actor is so defined:

struct cell {    int lock;    int ref;    lua_State *L;    struct message_queue mq;    bool quit;    bool close;};

Its actor is a Lua state, a messagequeue.

A skynet_context is a service in which each service in Skynet writes a callback function that handles a particular message.

struct skynet_message {    uint32_t source;    int session;    void * data;    size_t sz;};

This is the definition of the message and then the callback function for the message processing:

typedef int (*skynet_cb)(struct skynet_context * context, void *ud, int type, int session, uint32_t source , const void * msg, size_t sz);

With Lua, you can pass LUA functions directly in the message! This is a qualitative leap, the flexibility is greatly improved. In another actor, there is a LUA runtime environment, you have to send it a message, you can directly send a LUA function in the past to let it execute well. This promotion is equivalent to the overflight that the two parties agreed to allow the sending and processing of what messages to RPC masking the underlying details.

1. Processing of network messages

In Skynet, the comparison distinguishes between external and internal messages, and external messages such as those from the network. It uses a gate service exclusively, and all network messages come in through the gate and distributed to each actor. The gate side manages all connections and can do some memory management optimizations for Ringbuffer. But the cloud wind behind the discovery and external interaction is inevitable, so there are various services may not go directly to the gate processing network services, the system also has a number of Epoll port, which has an impact on efficiency.

As a result, the Epoll non-blocking operation is encapsulated directly in hive and provided to the upper-level synchronous interface.

There are a lot of other details I didn't really look at, either.

Comparison of Hive and go

I've seen both hive and go code, and I've found some ideas that are surprisingly similar. To fully benefit the multi-core concurrency advantage, they all chose the goroutine,hive, which is the coroutine of the LUA framework in the go, very light weight. There is no way to handle data dependencies between processes, not through shared memory, but through communication. In go, each actor in channel,hive comes with a message queue. If the encounter with Concord execution is not down, it will be temporarily yield, the direct conditions to continue to meet. Go is the low overhead of saving a goroutine with a segmented stack implementation, while hive is more provincial and directly leverages the LUA virtual machine.

At the bottom of the implementation, they are open a few physical threads, and constantly take a co-process execution, if you want to yield the concord in the queue to wait for the opportunity to re-take out the execution. In terms of scheduling go to do a bit better, after all, the hive code volume is small.

However, Hive is more bullish on the save context. According to Yunfeng said to save a coroutine as long as 200 to 300B, each lua_state less than 10K, and go of each goroutine will need at least 4k, even with the use of segmented stack technology, so there is no LUA light weight. As long as the stack to implement the save context can not be more lightweight, no way. and the sub-stack brings a large negative effect is the compatibility with C, in fact, CGO is not so useful. Lua's use of virtual machines is perfect for compatibility with C. However, it is not at all cost, and the transfer of C data to the LUA stack data is an additional overhead.

In the network processing, from the Skynet revised hive and go approach is the same, the underlying is the epoll/kqueue mechanism of asynchronous IO, the upper layer to provide users with the blocking IO interface. I think this is the humane approach, the asynchronous plus callback that is absolutely anti-human.

The underlying implementation is also the same, the call to the upper layer of the network API resulting in blocking, will be the current process to yield out. There is a background thread that keeps doing poll, and if it receives data, it wakes up the corresponding port's association.

The different place is the channel aspect. Go provides the first class channel, which is more versatile. While hive is affected by more Erlang, each actor binds a message queue.

Although I am a go language fan, after all, not a low-end brain residue. After looking at the hive code, sometimes I even feel that hive is doing a little better. The perfect compatibility with C is a big advantage, for example, memory management can choose to let the Lua garbage collection or manual management. Even after the use is complete, releasing a lua_state directly does not have to be garbage collected. Although go can also apply for a large amount of memory manually after management, but it is not as good as the direct use of C. Random comparison, the code is short and easy to control aspects, hive wins. Coroutine's overhead, hive wins. In DSL design, Hive wins. In terms of the flexibility of memory management, hive wins.

About online Games

Recently want to find something interesting to do, I think of online games. Finding a suitable game for research is not easy. Nothing is direct open-source online games, open source server framework, or the source code leaked.

First a diamonon, this is an open-source online game. Look at the 2d interface, the Ugly storm. 3d is not compiled. Casually look at its source code, feel more frustrated, still stay in very early times. IO multiplexing is also used by Select, not poll. Server convenient also did not make any distinction, the whole compiled out a server file, including the Loginserver,gameserver,database. The exact point is that database is not, the data is directly persisted to the local file system.

Then is Planeshift, this is also an open-source online game, 3d. Compiled to half gave up. Think of the well-known T-end m-end and so on, C + + What the least love, and too much code is not going to study the desire.

Then consider the source code leakage of some games, legends of what the code should not be difficult to get, but the game has not played, research is not interesting. In fact, find a self-played more familiar or pretty good. Eventually returned to the Darkeden. This game I most familiar with, played for a long time. 10 also read a little bit of service-side code, although the version is relatively old.

If you can find a service-side code and the matching client, you can still play. The initial plan, Loginserver can be used in my familiar with the go language to write, gameserver with hive. The database can avoid the first try to avoid it. The first to ask for a can run up things, mainly to the packet fix. It's a bit tricky to write so much. At present, only the plan, the specific implementation of the next article will be left ...