Analysis of read/write processes in the MD module-3

 

In this section, we will analyze the magical and NB handle_stripe. For version 2.6.21, raid5 and raid6 use the functions 'handle_stripe6 'and 'handle_stripe5, handle_stripe6, and handle_stripe5, respectively, as long as you understand the principle of raid6 and understand handle_stripe5, then handle_stripe6 naturally understands.

In the previous section, you have inserted the requested bio into the appropriate strip. Then, you need to process this strip .. This task is completed by the handle_stripe function. This section describes how it handles normal read/write operations. Other parts will be involved later. Since reading is relatively simple, let's talk about reading first.

For reading, the bio to be requested has been added to the toread linked list of a device in the Strip. Call handle_stripe to process the strip. The steps are as follows: (assume that all the device buffers in the strip are empty)

A. At the beginning, we will count the number of r5dev read requests to be processed, find one, to_read ++, and count the status of the buffer zone and the number of failed disks.

B. At this time, the to_read value is not 0.

View plaincopy to clipboardprint?

If (to_read | non_overwrite | (syncing & (uptodate <disks) | expanding ){

It can be seen that there are still a lot of conditions for entering this judgment: Read requests, non-full block write, synchronization, resizing, etc. Indeed, these operations need to read data first. For each device buffer, if there is a Read Request (dev-> to_read) and the buffer status is empty (! Test_bit (R5_LOCKED, & dev-> flags )&&! Test_bit (R5_UPTODATE, & dev-> flags), the buffer flag location is want (set_bit (R5_LOCKED, & dev-> flags); set_bit (R5_Wantread, & dev-> flags);) indicates to read data from the underlying layer.

 

C. At the end of handle_stripe5, count the devices that have read requests. If a read request exists, initialize the req in r5dev, including setting the callback function bi_end_io = raid5_end_read_request and the start sector and length of the req. Finally, use generic_make_request to issue the req.

D. After the request is processed, raid5_end_read_request is called. If the data is successfully read, The R5_UPTODATE of the device buffer is set to valid and the R5_LOCKED bit is cleared, indicating that the buffer status is clean. So far, we have read data from the disk to the device buffer of the Strip, but this is only read to the buffer, and did not fill the data in the original request bio. Therefore, you need to process the Strip once and set the Strip STRIPE_HANDLE to be valid. Call release_stripe to put the strip in handle_list for processing and wake up the daemon thread raid5d.

E. raid5d extracts the strip from handle_list and calls handle_stripe again to process the strip. This time, we found that the buffer status is R5_UPTODATE (test_bit (R5_UPTODATE, & dev-> flags) & dev-> toread conditions are met ), call the copy_data function to copy the data in the buffer to the corresponding bio segment. If bi_phys_segments is equal to 0, it indicates that this bio has been processed and can be returned to the upper layer. Then it is added to the linked list of return _ bi.

F. Final execution

While (bi = return_bi )){

Int bytes = bi-> bi_size;

Return_bi = bi-> bi_next;

Bi-> bi_next = NULL;

Bi-> bi_size = 0;

Bi-> bi_end_io (bi, bytes,

Test_bit (BIO_UPTODATE, & bi-> bi_flags)

? 0:-EIO );

}

 

 

The upper-layer processing is completed.

 

The write request is complicated and involves delayed write. The following is a step-by-step analysis:

A. Just like reading requests, the number of write requests to be processed by r5dev in the strip is also counted. If yes, to_write ++ also counts the number of non-full-block writes, if this r5dev is not full block write, non_overwrite ++;

B. If a device in the strip is marked as a non-full block write, that is, the R5_OVERWRITE bit is valid, you need to read the data (why should you read the data below) and the device buffer is want, indicates that data needs to be read from the disk.

C. Determine the write method used for the write operation. We know that raid5 still writes new verification data while writing data, which determines that data writing must first limit data. There are two write methods here. To put it bluntly, there are two calculation verification methods: rmw (read-modify-write) and rcw (reconstruct-write ). the first method is to read the device with write requests and the data on the verification disk, and then perform xor on the read data and the data to be written to obtain a new verification value. The 2nd method is to read the data on a device with non-full write requests and no write requests. here we can see why to mark a non-full write, because of the use of rcw, therefore, it is not enough to read data from devices without writing requests. Because a non-full block write exists, you also need to read data from a non-full block write device, and then copy some of the data, in this way, the data on a non-full block write device can be used to calculate the new verification. Rcw uses the newly constructed data, read data, and data to be written (excluding non-full block writes) to calculate the new verification.

D. Take rmw as an example. First, we need to read the data on the write request and verify the disk (if the buffer status is empty). Then we can see a judgment on the Strip status, test_bit (STRIPE_PREREAD_ACTIVE, & sh-> state ). If STRIPE_PREREAD_ACTIVE is valid, it indicates that the pre-read is activated and the read request can be sent. If it is invalid, set_bit (STRIPE_DELAYED, & sh-> state); indicates that the strip must be processed in a delayed manner. Here we will continue the analysis with delayed processing entries. In my understanding, latency processing of a stripe is essentially the process of reading as little data as possible.

E. The strip is set to STRIPE_DELAYED. This round of handle_stripe ends. In the release_stripe function, the strip is added to delayed_list, then activate the block Device Driver blk_plug_device (conf-> mddev-> queue). When the timer expires (3 ms by default ), the kernel calls the q-> unplug_fn method of the md column through some class functions. This method is implemented by raid5_unplug_device. This method calls the raid5_activate_delayed function and extracts the strip from delayed_list, remove the STRIPE_DELAYED mark, set STRIPE_PREREAD_ACTIVE to be valid, add the Strip to handle_list, wake up the raid5d thread, and process the strip.

F and handle_stripe functions are called again. If the STRIPE_PREREAD_ACTIVE bit is found to be valid, the device buffer is in the want state and the read request is sent to the lower layer. After the read request is processed, the buffer state is clean and the strip is added to handle_list for further processing.

The g and handle_stripe functions are called again. When all the data to be read is read, The compute_parity5 function is called to calculate the new check value. The buffer status is dirty, it indicates that new data needs to be written to the disk, and the to_writen linked list is left empty and connected to the written linked list. Set set_bit (R5_Wantwrite, & sh-> dev [I]. flags), clear STRIPE_PREREAD_ACTIVE, and wake up other strip waiting for pre-read.

I. The raid5_end_write_request callback function is called. The data is successfully written and the buffer status is clean. At this time, the data has been written to the disk, but the request has not been processed, so add the Strip to handle_list for processing.

J. handle_stripe was called once. It was found that the written linked list is not empty and the buffer zone is clean, and the request can be returned.

 

To sum up, the process is still complicated after a simple read/write command is processed. What we mentioned above is just a normal read/write operation. We know that raid5 can allow a disk to fail. If a disk fails, what is the read/write processing result? The next article continues the analysis.