Erlang: rabbitmq source code analysis 4. file_handle_cache Implementation Analysis

The file operation of rabbitmq uses file_handle_cache to package Erlang's prim_file module to a layer. Added the writebuffer and control logic for the number of opened files.

File_handle_cache is also a gen_server2, but generally only open operations send messages to gen_server2, read and write operations, including writebuffer, are executed and maintained in client process.

File_handle_cache stores several tables:

1. Elders: {pid, eldestunusedsince}, stores the PID of each client and its oldest handle opening time, without the key

2. Clients: # cstate, which stores information about each client. The key is PID.

Next we will look at the implementation of file_handle_cache from the four most basic file operation functions:

Open:

1. file_handle_cache is shared read but not shared write for the same file. Therefore, the <key = {path, fhc_file}, value = # file {reader_count, has_writer }> if has_writer = true and the open model is write, {error, writer_exists} is returned };

2. Create a closed handle. The handle-related information is stored in the process Dictionary of the client. <key = {ref, fhc_handle}, value = # handle>

3. file_handle_cache stores a gb_tree in the client's process dictionary to save the start time of all handle opened by the client. This time is used to adjust the number of handle of client open.

1-3 is done in the client process.

4. gen_server: Call open, enters the Process Loop of file_handle_cache. If the number of files opened by file_handle_cache does not exceed the preset number of files, update clients table and state and Return OK.

5. Return to the client process and call prim_file: open to open the file. Therefore, the operation to open the file is actually done in client process.

6. If the number of files opened by file_handle_cache exceeds the preset number of files,

6.1 if the PID has opened many files, close is returned. After the client receives the close command, soft_close all the previously opened handle. The so-called soft close means that the buffer of the corresponding handle file is written, sync before closing. Close all previously opened handle and try again to open the handle.

6.2 If This PID does not open any file, you can start with another PID, calculate the oldest time of each PID (the oldest time of opening the file), and accumulate the average value.

Generally, the client registers a cleanup function by calling register_callback before opening the file.

6.2.1 If the average value is greater than two seconds, it indicates that many old files have not been closed, and the function is cleared for each client call. The parameter is the average value of the oldest time. If the cleanup function is not registered, ignore it.

6.2.1 If the average value is less than two seconds, there are not many old files that are not closed. Find the PID of all the registration cleaning functions. Assume that N open blocks are used, and N of them are used to clean up n opened handle.

Close:

1. Delete {ref, fhc_handle} In processdictionary}

2. The writebuffer of handle must be prim_file: Write

3. If there is a write without sync, call prim_file: sync

4. Call prim_file: Close

5. Delete the handle time from the gb_tree.

6. Update {path, fhc_file}, read_count, write_count, and so on in process dictionary.

Read: there is no special read and no buffer is used.

1. Although it has already been opened, it may be closed because too many files are opened, so let's take a look. If it is closed, reopen it.

2. The writebuffer of handle must be prim_file: Write

3. Call prim_file: Read to obtain the result.

Append:

1. Like read, it is also possible to reopen.

2. If the client does not set writebuffer during open, it will directly call prim_file: Write (HDL, data ).

3. the so-called writebuff is for a single handle, that is, to create an empty list for buffer during open, and to call prim_file if buffersize> limit set during client open at append: write writes all the buffer into the file. If buffersize <limit, data is only put into the buffer and the result is returned.



Erlang: rabbitmq source code analysis 4. file_handle_cache Implementation Analysis