How does programming learn to optimize server-side rendering in node. js? Figure

How does programming learn to optimize server-side rendering in node. js? Figure
At Airbnb, we've spent years migrating all of the front-end code to the React architecture, with Ruby on Rails taking up a daily share of the WEB application. In fact, we will soon be turning to another new service that provides full server-side rendering pages via node. js. This service will render most of the HTML for all Airbnb products. This rendering engine differs from other backend services because it is not developed in Ruby or Java, but it is also different from common I/O intensive node. JS Services.
Speaking of node. js, you might start to imagine a highly asynchronous application that can handle thousands of connections at the same time. Your service pulls data from everywhere, handles them with lightning power, and then returns it to the client. You're probably dealing with a whole bunch of WebSocket connections, and you're confident about your lightweight concurrency model and think it's a great fit to do these tasks.
But server-side rendering (SSR) breaks your assumptions about this beautiful vision because it is computationally intensive. User code in node. JS runs on a single thread, so the compute operations can be performed concurrently (as opposed to I/O operations), but they cannot be executed in parallel. node. JS can handle large amounts of asynchronous I/O in parallel, but is constrained in terms of computation. As the proportion of compute parts increases, CPU contention begins, and concurrent requests will have a greater impact on latency.
In Promise.all ([fn1,fn2]), for example, if FN1 or fn2 is an I/O intensive Promise, such parallel execution can be achieved:
If Fn1 and fn2 are computationally intensive, they will execute like this:

An operation must wait for another operation to complete before it can run because there is only one thread of execution.
This occurs when the server process needs to process multiple concurrent requests while server-side rendering is in progress. Requests being processed will cause other requests to be delayed:
In practice, requests are usually made up of many different asynchronous phases, although the calculations are still dominated. This can lead to a worse crossover. If our request consists of a chain like Renderpromise () then (out = Formatresponsepromise (out)) and then (BODY = res.send (body)). So the cross of the request might be this:
In this case, two requests can take up to twice times to process the completion. As the concurrency increases, the problem becomes more serious.
One goal of the SSR is to be able to use the same or similar code on both the client and the server. There is a huge difference between the two environments, where the client context is essentially single-tenant and the server context is multi-tenancy. Things that the client can run normally, such as a singleton or global state, can lead to bugs, data leaks, and various kinds of confusion on the server side.
Both of these issues are related to concurrency. Everything is fine when the load level is low, or in the development environment.
This is completely different from the case of the Node application. The reason we use JavaScript is because it provides library support and support for browsers, not because of its concurrency model. The above example shows that the cost of the asynchronous concurrency model is beyond the benefits it can bring.
Lessons learned from Hypernova:
Our new rendering service, Hyperloop, will be the primary interactive service for AIRBNB users. As a result, its reliability and performance are critical to the user experience. With the gradual use of new services in the production environment, we will refer to the lessons learned from the early SSR service Hypernova.
Hypernova work in a different way from the new service. It is a purely renderer, and the Rails monomer app monorail calls it, which returns the HTML fragment of the rendered component. In most cases, "fragment" is part of the entire page, and Rails only provides an external layout. The sections on the page can be stitched together using the ERB. However, in either case, Hypernova does not acquire data and the data is provided by Rails.
That is, Hyperloop and Hypernova have similar operational characteristics in terms of computation, and Hypernova provides a good test base to help us understand how the page content in a production environment is replaced.
The user requests access to our Rails main application monorail, which assembles props for the React component that needs to be rendered, and sends a request with these props and component names to Hypernova. Hypernova uses the received props to render the component, generates the HTML and returns it to Monorail,monorail to embed the HTML fragment into the page template and send all the content to the client.
If Hypernova rendering fails (due to errors or timeouts), the components and props are embedded in the page, and perhaps they can be successfully rendered on the client. Therefore, we think that Hypernova is an optional dependency and we can tolerate some timeouts and failures. I set the timeout based on the SLA P95, as expected, our time-out baseline was slightly below 5%.
When deploying during peak traffic load, we can see a maximum of 40% request timeouts from monorail to Hypernova. We can see the badrequesterror:aborted error rate spikes from the Hypernova.
Example Deployment Timeout peaks:
We attribute these timeouts and errors to slow startup times, such as GC startup initialization, lack of JIT, fill cache, and so on. The newly released React or Node is expected to provide sufficient performance improvements to mitigate slow startup issues.
I suspect this could be due to poor load balancing or capacity issues during deployment. When we run multiple compute requests concurrently on the same process, we see an increase in latency. I added a middleware to record the number of requests processed concurrently by the process.
We blamed the latency on the concurrent request waiting for the CPU to start the delay. Judging from our performance metrics, we cannot distinguish between the time spent waiting for execution and the time it takes to actually process the request. This also means that the latency associated with concurrency is the same as that of new code or new features-which in effect increases the processing cost of a single request.
Obviously, we can't blame the badrequesterror:request aborted error on the startup delay. This error comes from the message resolver, especially before the client aborts the request before the server fully reads the request message body. The client closes the connection and we are unable to get the valuable data needed to process the request. This is more likely to happen, such as: we begin to process the request, and then the event loop is blocked by another request rendering, and when we go back to where we were previously interrupted, we find that the client has disappeared. Hypernova's request message body is also large, with an average of several hundred bytes, which only makes things worse.
We decided to use two existing components to solve this problem: the reverse proxy (Nginx) and the Load balancer (HAProxy).
Reverse Proxy and load balancing:
To take advantage of multicore CPUs on Hypernova instances, we run multiple Hypernova processes on a single instance. Because these are independent processes, concurrent requests can be processed in parallel.
The problem is that each Node process will be consumed throughout the request time, including reading the request message body from the client. While we can read multiple requests in parallel in a single process, this causes the compute operations to cross when rendering. Therefore, the use of the Node process depends on the speed of the client and the network.
The workaround is to use a buffered reverse proxy to handle communication with the client. To this end, we used Nginx. Nginx reads the client's request into the buffer and passes the full request to the Node server after a full read.High Old man reading notesExcerpt good words and sentiment appreciation, this transfer process is carried out on the local machine, using loopback or UNIX domain sockets, which is faster and more reliable than the communication between the machines.
By using Nginx to handle read requests, we are able to achieve higher Node process utilization.
We also use Nginx to process a subset of requests without sending them to the node. JS process. Our service discovery and routing tiers check the connectivity between hosts by/ping low-cost requests. Handling these in Nginx can reduce the throughput of the node. JS process.
Next is load balancing. We need to be wise to decide which node. JS processes should receive the requests. The cluster module allocates requests through the Round-robin algorithm, which is good when the request delay changes very small, such as:
But when there are different types of requests that take different processing times, it is less useful. Subsequent requests must wait for all previous requests to complete, even if another process can handle them.
A better distribution model should look like this:
Because this minimizes the wait time and can return the response more quickly.
This can be done by putting the request into the queue and only assigning the request to the idle process. To this end, we used the HAProxy.
When we implement these in Hypernova, we completely eliminate the time-out peaks and badrequesterror errors at deployment. Concurrent requests are also a major contributor to latency, and the latency is reduced as new scenarios are implemented. With the same timeout configuration, the time-out rate baseline changes from 5% to 2%. 40% failures during deployment were also reduced to 2%, a major victory. Now, the odds of a user seeing a blank page are already low. In the future, deployment stability is critical for our new renderer because the new renderer does not have a hypernova rollback mechanism. Not clear

How does programming learn to optimize server-side rendering in node. js? [Figure]