Here we will discuss a more important function, as mentioned in the previous article, to combine the sources. Why should we discuss it separately? In fact, it is mainly from the perspective of personal work. Previously, the cache in the company needed such a function. The existing squid function is not perfect and is not suitable for our business. Then we added this feature to the cache and nginx respectively, but the current nginx version is already supported by native. Haha, you may not know that our nginx version is 0.7.x. We were still thinking about it during development. Maybe the official support will be available soon, but who knows how long it will be. Go to the topic and see how to design and implement nginx. Here we combine the fetch source into fetch merged. By default, nginx does not enable it. To use this function, run proxy_cache_lock. Here we will post the official wiki instructions again:
Syntax: Proxy_cache_lock on | off;
Default: Proxy_cache_lock off;
Context: HTTP, server, location this directive appeared in version 1.1.12. when enabled, only one request at a time will be allowed to populate a new cache element identified according to theproxy_cache_key directive by passing a request to a proxied server. other requests of the same cache element will either wait
A response to appear in the cache, or the cache lock for this element to be released, up to the time set by the proxy_cache_lock_timeout directive. if it is enabled, there is a member in the ngx_http_cache_t structure called lock, which will be set to 1. This structure has been mentioned earlier. In a cache, there is always a structure to communicate with the cache object corresponding to the request. Similarly, this structure is used in nginx. Let's look at the process. When an object is retrieved from the source, its corresponding control structure already exists and is also visible in the system. That is to say, when a request is retrieving the source, if the subsequent same request comes again, this object will inevitably be found. Obviously, objects at this time are different from normal ones. In cache, we usually use pending and OK States to distinguish the two. Nginx does not process these statuses. when an object is not fully cached, a member named updating sets 1 and the exists member sets 0. These two members play an important role in the entire cache process, not just fetch merged. If you want to read the code, you must pay attention to the use of the two members. When fetch merged is enabled, nginx will wait for subsequent requests for a period of time (the default value is 5 seconds ). During this period, nginx checks whether the cache ends normally every MS (whether the updating is set to 0). If the cache ends, then the cached file will be found in the subsequent processing process, and the hit will be ready. If it takes a long time and the maximum wait time is reached, the fetch merged Mechanism for the cached file will be cancel, so that subsequent requests will be the same as normal, continue to the backend fetch file. Of course, the cancel processing will not affect the fetch merged of other files. As mentioned above, in fetch merged, the default wait time is 5 seconds, which can be controlled by configuration. That is, proxy_cache_lock_timeout. For details about how to use it, go to the official wiki. As you may see. Nginx is a little tricky because it causes latency to the client to reduce the backend pressure. Especially for large files that take a long time to get the source, you need to wait for the client, and wait for the time to get the source again. I think many companies may not recognize nginx, especially CDN companies. Generally, it is reasonable to distribute data while receiving data. However, it may be a bit difficult to handle. Let's discuss this solution with you in the next time. First, we need the first request to fetch data from the backend. During this period, when the same request comes, we need a mechanism to find the request that is taking the source, and then merge it. Nginx uses the rbtree, and All cached objects are registered in the tree in the form of a node (in this regard, I know a lot about the use of hash tables, implementation of conflicting linked lists ). The first step is to find the source object. The next step is to use this request for data distribution. The simplest way is to add a queue to the request structure, the requests to be merged are mounted to the above, and the retrieved data is then distributed to each request in the queue. But the problem is not so simple: when the origin fetch request ends, the general processing is to release the request, then we need to ensure that the data is distributed before the request is released, or other mechanisms can be used to release the request to complete subsequent distribution. For the former, we have to add reference counting in the request struct, or other counting methods. A request can be released only when the count is 0. This kind of processing is acceptable, but if you are engaged in secondary development on the original system, then you need to add the counting check code in a lot of violent places, which is really a nightmare. We use the latter. Specifically, we mount a global hash table to the merged request. The merged request will add an entry to the table, which we may call a node, this node manages a queue and manages all merge requests for the same file. What we need to do is let the node be destroyed after the data is completely distributed. This process is not like adding a count in a request. The code coupling is much smaller, but the memory consumption of only one hash table is exceeded. Distribution is actually very simple. The first preparation is to add some Members in the request to mark the number of sent members and the number of requests to be sent next time. In terms of event processing, events closed by the client can be discovered by registering the read event processing function. For writing, when the source is being retrieved, we do not need to set any processing functions. At this time, sending relies on distribution of the source fetch request. The distribution of retrieval requests depends on the trigger of read/write. When the retrieval ends, no event is triggered to distribute the requests. This can also be done. When the last distribution is performed after the source fetch is completed, a new read/write event processing function is set for each merged request. In this way, subsequent distribution can be driven by read/write events, and well-cached files are available. Of course, there are many specific implementation methods, but the idea is generally like this. In this model, how can we properly and correctly handle exceptions? Our processing follows the simple and crude principle: if the client is shut down, you can simply kill it from the queue. If there is a problem with the source fetch connection, close each request connection in the queue in turn. In fact, there is no need to consider more friendly processing methods here, because the source is down and generally cannot be restored instantly, so it is often useless to find a "stepmother" for these merged requests. But if there is a load balancing mechanism like nginx, you can try to find a "stepmother ". Here are two implementations of multi-process source merging: Intra-process merging and inter-process merging. For in-process, the hash table used by the merge request node is in-process, so the same file will have the maximum number of concurrent connections to obtain the source. This concurrency is completely acceptable. If we put this hash in the shared memory, there is hope that the process can be perfectly merged. But is that all right? Merging processes is tricky. Who is responsible for distribution? In-process merge, the retrieval of source requests can shoulder the responsibility of distribution, because the requests in the queue belong to this process. If we place the hash table and its queue in the shared memory, we need to distinguish which processes belong to them during distribution. Which processes do not belong to them? Cache based on disk files is easier to process. Because requests in various processes can go to the disk to check whether new content exists each time there is a writable event, so as to send the data. If there is no new data, add the event again, so that each epoll_wait will report many events. This is only one implementation. Let's look at the full-memory cache. All its files are stored in the memory. The path to sending data by viewing disk files is completely useless, so how can we distribute the content? In other words, how can we be notified of merging requests in other processes except for the process where the source request is located, when Will new content be sent? We have discussed a scheme before, that is, to set some timers for these requests and check whether the content is updated for a while. However, this is similar to the nginx implementation because of the existence of the timer, the client adds latency. The method for adding events is discussed earlier. Therefore, regardless of the disk cache or memory cache, it is also the most difficult and critical issue to notify you when to distribute data. Our goal is to make other processes receive notifications asynchronously, rather than waiting (the timer and event are here), so the signal looks like: the source process sends a signal to other processes, tell them that the content has been updated. The process of receiving the signal completes data distribution in the signal processing function. However, this will cause the signal to fly in the system, and the particularity of the signal processing function. The feasibility of this solution remains to be discussed. After investigation, we chose to merge our programs in the process. At present, it seems that our programs fully meet our needs. Although nginx solves the problem of inter-process merge, it costs the client latency, which is unacceptable in our CDN business. Therefore, it does not affect performance much. There are still many problems to be solved if real-time asynchronous merge is implemented. We will discuss it with you here.
