Etherdream: Using C programs in JavaScript

JavaScript is a flexible scripting language that makes it easy to handle business logic. When communication is required, most of us choose JSON or XML format.

However, when the data length is very harsh, the efficiency of the text protocol is very low, and the binary format has to be used.

Last year, when I was tossing a WAF with a back-and-forth end, I was confronted with this problem.

Because the front-end script needs to collect a lot of data, and eventually is implicitly written in a cookie, so the available length is very limited, only dozens of bytes.

If you use JSON without thinking, a marker field {"enableXX": true} takes up half the length of the light. In binary, however, the token true or false is only 1 bits, which can save a hundredfold of space.

At the same time, the data also through the verification, encryption and other links, only the use of binary format, to facilitate the invocation of these algorithms.

Elegant implementation

However, JavaScript does not support binary.

The "no support" here is not "impossible", but not "graceful implementation". The invention of language is used to solve problems gracefully. Even without language, humans can use machine instructions to write programs.

If you do not want to manipulate binary with JavaScript, it will end up like this:

var flags = +enableXX1 << 16 | +enableXX2 << 15 | ...

Although it can be achieved, but very ugly. Various hard-coded, various bit operations.

However, for languages that are inherently binary-enabled, they look elegant:

union {    struct {        int enableXX1: 1;        int enableXX2: 1; ... }; int16_t value;} flags;flags.enableXX1 = enableXX1;flags.enableXX2 = enableXX2;

Developers just need to define a description. When used, the field offsets how much, how to read and write, these details do not care at all.

In order to achieve a similar effect, initially encapsulated a JS version of the structure:

// 最初方案：封装一个 JS 结构体var s = new Struct([    {name: ‘month‘, bit: 4, signed: false}, ...]);s.set(‘month‘, 12);s.get(‘month‘);

The details are hidden and look more elegant.

Elegant but not perfect

However, this total feeling is not the most perfect. The structure of this stuff, which is supposed to be provided by the language, is now implemented with extra code, and is still running.

In addition, the back-end decoding is implemented in C, so we have to maintain two sets of code. Once the data structure or algorithm has changed, it is troublesome to update JS and C at the same time.

So wondering, can you share a set of C code for both front-end and back-end?

In other words, you need to be able to compile C into JS to run.

Meet Emscripten

Can compile C into JS tool has a lot of, the most professional to number Emscripten.

Emscripten is very simple to use, similar to the traditional C compiler, except that it generates JS code.

EMCC hello.c-o hello.html//hello.c# Include <stdio.h>#include <time.h>  int  Main () {time_t now; time (&now); printf ( "Hello World:%s", CTime (&now)) ; return 0;}         

After compiling, you can run:

It's fun ~ Everyone can try, here is not more introduction.

Practical defects

However, we are not interested in interesting, but practical.

In fact, even a Hello world compiled by JS is over-the-line, up to hundreds of KB. Even if you compress and then GZIP, you still have dozens of KB.

At the same time Emscripten uses the Asm.js specification, memory access is implemented through TypedArray.

This means that users below IE10 will not be able to run. This is unacceptable, too.

Therefore, we have to make the following improvements:

• Reduced volume

• Increased compatibility

First pinned Emscripten itself, to see if we can set the parameters to achieve our goal.

After a few attempts, however, it did not succeed. That can only be done by yourself.

Reduced volume

Why is the final script so big and what's in it? Analysis of the following content, roughly there are several parts:

• Accessibility features

• Interface Simulation

• Initialize operation

• Run-time functions

• Program logic

Accessibility features

Examples include string and binary conversions, callback wrappers, and so on. None of this is essential, so we can write a special callback function for ourselves.

Interface Simulation

Provide the interface of file, terminal, network, rendering and so on. Before I saw a client game that was ported with Emscripten, it seems to simulate a lot of interfaces.

Initialize operation

Initialization of global memory, runtime, various modules.

Run-time functions

Pure C can only do simple calculations, and many functions rely on run-time functions.

However, there are some commonly used functions, and the implementation behind them is complex. For example, malloc and free, the corresponding JS has nearly 2000 lines!

Program logic

This is the C program really corresponding JS code. Because the compilation is LLVM optimized, the logic may become unrecognizable.

This part of the code is small, we really want to.

In fact, if the program does not use some special functions, the logic function alone, can still be run!

Considering our C program is very simple, so the simple rough extraction, it is no problem.

C program corresponding to the JS logic is located // EMSCRIPTEN_START_FUNCS and // EMSCRIPTEN_END_FUNCS between. Filter out the run-time functions, and the rest is 100% of the logic code.

Increased compatibility

Then resolve the memory access compatibility issues.

In the old version of Emscripten, it is possible to choose whether to use TypedArray. If not, then through the JS Array to achieve. But now this parameter has been removed and only TypedArray can be used.

First of all understand, why use TypedArray.

Emscripten applied for a chunk of ArrayBuffer to simulate memory, and then associated some HEAP of the beginning variables.

These different types of HEAP share the same block of memory, which enables efficient pointer manipulation.

Browsers that do not support TypedArray, however, are clearly unable to run. So we have to provide a polyfill compatible.

However, it is almost impossible to achieve this by analysis-because TypedArray and arrays are accessed through the index:

var buf = new Uint8Array(100);buf[0] = 123; // setalert(buf[0]); // get

However [] , the operator cannot be rewritten in JS, so it is difficult to turn it into a setter and getter. Moreover, the TypedArray is not supported by the low version of IE, not to consider the characteristics of ES6.

Then pondering the private interface of IE. For example, using the Onpropertychange event to simulate setter. However, this is extremely inefficient and the getter is still not easy to achieve.

After some consideration, decided not to use the hook way, but directly from the source to solve-modify the grammar!

We use regular to find out the assignment operation in the source code:

= val;

Replace with:

HEAP_SET(index, val);

Similarly, the read operation is:

HEAP[index]

Replace with:

HEAP_GET(index)

In this way, the original index operation becomes a function call. We can take over the memory read and write, and there are no compatibility issues!

Then implement 8, 16, 32-bit unsigned versions. It is very simple to simulate by JS Array. The trouble is to simulate the float type, but the C program does not use floating point, so it is not implemented.

If TypedArray is supported, the native interface is used; otherwise, the version is simulated with an Array.

In this way, the performance of the high-version browser is ensured, and the functionality of the old browser is taken into account.

Done

Solve these shortcomings, we can happily use the C logic in JS.

Scripts that only care about business logic. For example, what data is collected so that the code is very elegant:

Data is stored, encrypted, encoded, and these binary operations are implemented through C.

Compile-time use -Os parameters to optimize the volume, the final JS thin compression, less than 2 KB, very small refining.

So, this "front-end waf" development is much easier. " We only need to maintain a code, we can compile both front and back two versions!

All data structures and algorithms are implemented by C. The front end is compiled into JS code, and the backend is compiled into a LUA module for Nginx-lua use.

The front and back of the script, all you need to focus on business functions, completely without the data level of detail.

Beta version

In fact, there is a third version-the local version.

Because all the C code is together, it is easy to write a test program.

This eliminates the need to start WebServer, open the browser to test. Just simulate some data and run the program directly to test it, very lightweight.

With the help of the IDE, it's easier to debug.

Summary

Each language has its own merits and demerits. Combining the advantages of different languages can make the program more elegant and perfect.

Https://www.cnblogs.com/index-html/p/using_c_in_javascript.html

Etherdream: Using C programs in JavaScript