Go language uses CGO to optimize

This is a creation in Article, where the information may have evolved or changed.

Reprinted from Dada's blog

I used CGO to optimize the data storage method of the game memory database, and reduce the number of objects. However, the program placed in the online environment after a paragraph error, directly leading to the process exit, had to temporarily removed the optimization of the part, removed after the program continued to run stably for two weeks.

After the optimization code was removed, I re-tidied the code. Sort it out, I think the optimization logic for a table with string fields is too complex, and it's difficult to control the boundary condition.

Here's an example:

typeMyTablestruct{Namestring}funcInsertmytable (myTable myTable) {namelen: = c.size_t (Len(Mytable.name)) Name: = C.calloc(1, Namelen) c.memcpy (name, unsafe. Pointer (*reflect. Stringheader) (unsafe. Pointer (&mytable.name)). Data), Namelen) (*reflect. Stringheader) (unsafe. Pointer (&mytable.name)). Data =UIntPtr(name)}funcUpdatemytable (myTable myTable) {namelen: = c.size_t (Len(Mytable.name)) Name: = C.calloc(1, Namelen) c.memcpy (name, unsafe. Pointer (*reflect. Stringheader) (unsafe. Pointer (&mytable.name)). Data), Namelen) (*reflect. Stringheader) (unsafe. Pointer (&mytable.name)). Data =UIntPtr(name) C.free (unsafe. Pointer (*reflect. Stringheader) (unsafe. Pointer (&oldmytable.name)). Data)}funcDeletemytable (myTable myTable) {c.free (unsafe. Pointer (*reflect. Stringheader) (unsafe. Pointer (&mytable.name)). Data)}

The above code is a simulation of the problems encountered in the project, not the real code, the real code is actually more complicated than this, because the object will be used for transaction submission, but also need to control the object after the transaction commits to release the String Type field, in the update also need to determine whether the string is changed and so on.

Why do we need to handle strings? Because if the string is not processed, when the go string is assigned to the memory block created by CGO, go does not know that the string is referenced, which results in a useful string being recycled by GC.

The same applies to nested structures, such as:

typestruct {    ChildTable *MyChildTable}

If a go created Mychildtable object is assigned to a childtable field of a CGO maintained MyTable object, the Go GC is not traced to this referential relationship, and when the MyTable object is still in effect, The Go object referenced by the internal childtable field has been reclaimed and a segment error occurs if the program accesses the ChildTable object.

However, the situation of the child table is better handled, as long as the original new (mychildtable) to replace with their own implementation of newmychildtable (), with CGO to apply for memory, their own manual release, there will be no problem, the boundary condition is not as many as the string. The code looks like this:

sizeofMyChildTable := unsafe.SizeOf(MyChildTable{})func newMyChildTable() *MyChildTable {    return (*MyChildTable)(C.calloc(1, C.size_t(sizeofMyChildTable)))}

Troubleshooting a segment error is difficult, so I want to do the elimination first, the most complex string optimization logic is removed, and the memory table containing the String type field is not optimized. Fortunately, the game is not much use of strings, only a few tables useful to the string, slightly reduce the optimization effect to improve the stability of the program, or cost-effective.

The new version of the program, which removed the string optimization, has been stabilized for a week, officially verifying the effectiveness of GC optimization for CGO.