QoS mechanism in Linux-DM-ioband (4)

This article continues the previous article and analyzes the workflow of DM-ioband through several typical scenarios.

The first scenario Is Example 1 in the http://sourceforge.net/apps/trac/ioband/wiki/dm-ioband/man/examples, first call the command to create two ioband device,

# echo "0 $(blockdev --getsize /dev/sda1) ioband /dev/sda1 1 0 0 none weight 0 :80" | dmsetup create ioband1# echo "0 $(blockdev --getsize /dev/sda2) ioband /dev/sda2 1 0 0 none weight 0 :40" | dmsetup create ioband2

DMSetup calls ioband_ctr to create the ioband device. The Parameter order can be seen from the annotations of ioband_ctr.

/*
* Create a new band device:
* Parameters: <device> <device-group-ID> <io_throttle> <io_limit>
* <Type> <Policy> <policy-Param...> <group-ID: group-Param...>
*/

Where<Device-group-ID>YesIoband_device-> g_nameTo uniquely identify an ioband device.P.s. Here ioband1 and ioband2 actually correspond to the same ioband_device, and their ioband-group-ID is 1. However<Device>Real Block devices, such as/dev/sda1 and/dev/sda2. Dm_get_device will be called to add this block deviceDm_table-> Devices
List. The following callAlloc_ioband_device (<device-group-ID>, <io_throttle>, <io_limit>)To create an ioband_device.<Device-group-ID>Ioband_device with a value of 1, so it simply ends ioband_device-> g_ref ++. There is a global KernelIoband_device_listAll ioband_device belongs to this list_head.Ioband_device-> g_list
Organize.

The following calls policy_init. The input parameters include:<Policy> <policy-Param...> <group-ID: group_param...>First, compare the Policy Name, query the policy named weight in the global variable dm_ioband_policy_type, and then call the correspondingPolicy_weight_initFunction.G_groups of p.s. ioband_device links attach to all ioband_group above. The list_head variable corresponds to the ioband_group
C_list member. ioband_group can be obtained through the container_of macro.. The obtained struct ioband_policy_type is assigned to ioband_device-> g_policy.

When policy_weight_init is called, the input parameter is<Policy-Param...> <group-ID: group-Param...> these parameters. The first parameter of weight policy is<Token base>, Followed<Group-ID: group-param>Array, policy_weight_init is incorrect
<Group-ID: group-param>.

Next, we will call ioband_group_init to create the default ioband group. In this scenario, two default ioband_groups appear for the same ioband_device, and c_id is ioband_id_any. Ioband_device-> g_root_groups saves the same<Device-group-ID>All the root groups and groups add ioband_device-> g_root_groups to ioband_group-> c_sibling through list_add_tail. We can see that each ioband_group-> c_sibling stores all ioband_groups of the same level, all ioband_groups in ioband_device-> g_root_groups are sibling each other.In this scenario, the c_parent of two ioband_group,
C_children is empty, and c_sibling has two ioband_groups.

For other ioband_group members: c_children indicates all sub-groups under this group, c_parent indicates parent group, and c_sibling indicates sibling group. Because ioband_group is organized according to the structure of the red/black tree, c_group_root indicates the root of the Red-black tree, And c_group_node indicates the node of the current group in the red-black tree.

Finally, call ioband_group_type_select and select ioband_group_type. The type here is none.

############################

OK. Now let's look at the request execution flow. After the request enters the device Mapper, it will eventually fall into the ioband_map function. The ioband_map execution steps are as follows:

1. Call ioband_group_get to obtain the ioband_group corresponding to bio. If the t_getid function pointer is null, it indicates the default group. Otherwise, the corresponding function dm_ioband_group_type is called Based on bio to obtain the group type.
2. Because<Device-group-ID>Ioband_device with a value of 1 has only two default groups. When the Group initialization function policy_weight_ctr, a token is assigned to it. Policy_weight_ctr will call yy_weight_param and finally call set_weight to set weight.
3. Set_weight first fills in ioband_group-> c_weight, calculates the sum of c_weight of c_sibling, and obtains the base_token and base_io_limit values (that is, the token and io_limit values of parent, if the parent is null, It is the corresponding value of ioband_device), and then call _ set_weight. In _ set_weight, token is allocated based on the proportion of c_weight to base_token and io_limit is allocated based on base_io_limit. If necessary, modify the corresponding value in the parent.
   At this point, the ioband_group-> c_token, ioband_group-> c_token_initial, ioband_group-> c_token_bucket values are set to the allocated token values.
4. When processing the request, determine whether is_token_left has the remaining token available. If yes, call consume_token so that bio is executed. Otherwise, the delayed_work mechanism will be called to delay processing.
5. If nr_blocked (DP), room_for_bio_sync (DP, blk_rw_sync/async), issue_list, and pushback_list are empty, the three conditions are true, it indicates that the remaining bio blocks are all exhausted due to token. Call DP-> g_restart_bios to release a new token, which is completed by make_global_epoch.
6. Make_global_epoch is a very interesting function. First, it finds the group with the most tokens in ioband_device. If the group has too many tokens and the Group has a high Io load(Iopriority (GP) * process_threshold> ioband_ioprio_base & nr_issued (DP)> = DP-> g_io_throttle)In this case, the group will be given priority and the token will not be given out. Otherwise, an epoch will be added for ioband_device. for all groups
   Ioband_group-> c_token_initial count token

In this scenario, a default group has always had IO requests, but the other has never. This mechanism ensures that no group will token-Starving.

------------------------------------------------- Gorgeous split line -----------------------------------------------------

The second scenario is Example 3 of the http://sourceforge.net/apps/trac/ioband/wiki/dm-ioband/man/examples. In this scenario, the ioband2 has a uid = 1000, Weight = 20 group.

First, SET _ ioband_message to uid.

Second, when ioband_group_attach (struct ioband_group * head, 0, 1000, null) is called in _ ioband_message, head is the pointer of the original default group, but now head-> c_type is changed to uid group type in dm_ioband_group_type. Parent_id is 0, and group ID is 1000. Then call ioband_group_init (DP, Head, null, GP, 1000, null). Here, GP is the newly allocated memory used to save ioband_group. Here GP-> c_id
= ID. In this case, the ID of ioband_group is not ioband_id_any in the default group, but uid 1000. The newly allocated ioband_group is attached to g_groups of ioband_device. Therefore, this ioband_device currently has three ioband_groups. At the same time, under the red and black tree of the ioband2 group, there are two nodes, one belongs to the default group and the other belongs to the UID group. The c_dev and c_target values of the two groups are the same.

Finally, call ioband_set_param in _ ioband_message to set weight. Finally, call policy_weight_param (GP, "weight", "20 ").

Now there are three ioband_groups under ioband_device. How do I know which group the request belongs? Do you still remember ioband_group_get in ioband_map? For a request sent by a process with the objective of ioband2 user = 1000, the first parameter in ioband_group_get is the pointer of the default group. Because the c_type of the default group has changed to uid group type, ioband_group_find will be called to search for ioband_group with c_id = 1000 on the red-black tree. This search algorithm will either return
The rb_node with c_id 1000 or the rb_node of the default group with c_id ioband_id_any is returned. Therefore, bio sent by processes with uid = 1000 is not returned to the default group. The bio from processes with uid = 1000 will be sent to ioband_group with c_id = 1000.

Summary:<Device-group-ID>ContainsG_refIt is an ioband_device of 2, which has two block devices:/dev/mapper/ioband1,/dev/mapper/ioband2, and three ioband_groups. the token is shared by weight.