TFS expansion block optimization solution

TFS currently uses extended blocks to solve file write and update problems. The number of extended blocks is determined by the disk space available, the size of the primary block, the size of the extended block, and the block_ratio of the dataserver (DS) configuration item. The number of TFS primary blocks and extended blocks is determined when the file system is formatted, and all block storage spaces are allocated in advance. DS periodically reports the storage space usage to the namserver. DS uses the larger ratio of the primary block and the extended block usage as the DS storage space usage.

Problems caused by extended block usage

1. Ideal; the usage of the expansion block and the main block is very close or the same;
2. Extended block usage is much lower than the usage of the main block: this trend continues to waste a lot of extended block space.
3. Extended block usage is much higher than that of the primary block: when calculating the available space, DS determines the usage of the extended block. This trend will result in a waste of space. In this case, even if DS has a lot of available space NS, it is considered that DS is full, so that data cannot be written to the DS. (This situation occurs when the expansion block is configured very small and the max_block_size of the NS end is configured too large ).

 

Why do I need expansion blocks?

1. When allocating a block, NS considers whether the block is full (beyond the threshold). The block usage collected by ns is not real-time (with periodic updates included in the heartbeat packet ); the block allocated by ns to the client may be full when the client connects to DS for write operations. DS must satisfy write requests on the block allocated by ns, you need to expand the block space (because of the TFS file name feature, the write must be completed within the block ).
2. The client's sequence of one open, multiple writes, and close operations also results in the failure to determine whether the remaining length of a block is sufficient when the open block is allocated.
3. The update operation must meet the requirements of the original block by keeping the file name unchanged. If the length of the updated file is larger than the original length of the file, when the original block does not have enough space, you need to use the extended block.

The problem of extended block makes TFsThe usage of storage space on the data server is reduced, and there are two ways to solve the expansion block problem:

1. Abandon extension blocks: any block is equivalent. When the block is full, the file can be stored on any block;
2. Use the primary block as an extension block: Split the primary block into multiple extension blocks.

Solution 1Implementation ideas

Similar to Linux soft-link files, when a file is written or updated, but the block a storage space allocated by ns is insufficient, the file is stored on any other block B, and the <blockid and fileid> information of the corresponding file on B is stored on Block A as the file content, and the file is marked as a linked file; when such a file is read, then, read the content to parse the <blockid, fileid>, read the actual data of the file, and return it to the client. Because block migration exists, the target of the linked file must be the logic block ID, so that even if the block migration takes the file data, it can still be read. If the physical block ID is saved, when the block is migrated, the file data cannot be read.

 

The linked file solution completely abandons the use of extended blocks, so there is no waste of storage space. When a linked file is read, you need to read the target block indirectly once (the overhead is similar to the current expansion block solution). However, if the target block has been migrated to another DS instance, the overhead of reading files will be increased (even the NS obtains the DS information of the block and reads the file data from the DS ). When the remaining space of a block is smaller than the average file size, many linked files may appear at the end of the block, because many write buckets are insufficient, resulting in the use of linked files.

Solution 2Implementation ideas

All buckets are formatted as the primary block. When you need to expand the block, you can allocate a primary block and split it into multiple expansion blocks (to simplify implementation, you can directly fix the size of the extended block, which has little impact). select one of them for use. When other blocks need to be extended, you can continue to allocate them from this block, after all the extended blocks are removed, the next primary block is allocated and the splitting continues.

 

 

This solution effectively solves the problem of storage space waste, because the extended block is allocated on demand rather than pre-allocated, regardless of the ratio of the extended block to the primary block, all can be used until the space is used up (you need to consider reserving some storage space for file updates ).

Solution 3 proposed by Feige

The general idea is similar to solution 2, but the use of extended blocks is not divided into fixed-size units, but the number of allocated units. files do not kill Block Storage. When the primary block space is insufficient, all file data is written to the extended block, and the index item adds the block number of the file.

Problems

1. If there are many updates, many files on the block will be stored in the extended block (the metadata of a block increases), and the number of blocks to be read during compression and replication (random Io ).
2. The remaining space on the primary block is wasted, and the wasted space is determined by the size of the last write operation.

 

Based on the TFS fixed block size and update support, the expansion block is a forced choice. The key problem is whether the expansion block is fixed in size (solution 2) or variable in size (solution 3 ), the management of fixed-size blocks is relatively simple, but if some blocks are not updated after use, the space behind the blocks will be wasted; variable-Size Blocks (each block corresponds to the data of a file) require more metadata for Management (larger index files, longer block chains ), there is also a waste of storage space. I personally think solution 2 is a good choice and a compromise between efficiency and management costs.