Start with the swift-ring-builder command and understand the ring construction process.

Some people who know siwft know that ring is a very core component of swift and determines how data is distributed in the cluster. Swift determines the number of partitions in the Cluster Based on the configured partition_power (2 to the power of partition_power), and based on consistent hashAlgorithmAllocate partitions to different nodes and distribute data to corresponding partitions.

Therefore, building a ring is a process that swift initialization must go through. Simply put:

• New Ring creation process:

1. The ring-builder calculates the number of partitions that should be allocated to each device based on the device weight. (The partition_power of 2 is used to obtain the total number of partitions, and then allocated based on the number of weight and devices)
2. Ring-builder allocates copies of each partition to the corresponding device.

 

• Create a new ring based on an old ring:

1. Recalculate the number of partitions on each device;
2. Collect the partitions to be reassigned:
   • 1) add all partitions on the removed device to the gathered list;
   • 2) Add the partition that needs to be allocated due to adding a new device to the gathered list;
   • 3) Add the multiple partitions on all devices to the gathered list.
3. Use the "new ring creation process" method to allocate the partitions in the gathered list to devices.

How is the swift-ring-builder Command executed? The purpose of this article is to introduce the swift-ring-builder command. Through the source code, we can find that the functions of the swift-ring-builder command are basically implemented through the relevant methods of the ringbuilder instance, therefore, more principles and details will be summarized after reading the source code of ringbuilder. So, do not spray me to sell dog meat. ^ _~

 

1. What does swift-ring-builder do?

Rings is manually created using the swift-ring-Builder tool. Swift-ring-builder associates the partition with the device and writes the data to an optimized Python data structure, compress, serialize, and write data to the disk so that data created by rings can be imported to the server. The rings update mechanism is very simple. The server checks the last update date of the file that creates the rings to determine which update it and the version in its memory need to re-load the rings to create data. The "Python Data Structure" mentioned in this section is a dictionary output structure as follows:

 Def  To_dict (Self ):  "Returns a dict that can be used later with copy_from to restore a ringbuilder. swift-ring-builder uses this to pickle. dump the dict to a file and later load that dict into copy_from. """          Return { '  Part_power  '  : Self. part_power, '  Replicas  '  : Self. replicas,  '  Min_part_hours  '  : Self. min_part_hours,  '  Parts  '  : Self. parts,  '  Devs  '  : Self. devs, '  Devs_changed  '  : Self. devs_changed,  '  Version  '  : Self. version,  '  _ Replica2part2dev  '  : Self. _ replica2part2dev,  '  _ Last_part_moves_epoch  ' : Self. _ last_part_moves_epoch,  '  _ Last_part_moves  '  : Self. _ last_part_moves,  '  _ Last_part_gather_start  '  : Self. _ last_part_gather_start,  '  _ Remove_devs  ' : Self. _ remove_devs}

The basic structure of the swift-ring-builder command is as follows:

Swift-ring-builder <builder_file> <action> [Params]
 

Swift-ring-builder generates the builder file stored in the file specified by <builder_file> according to <action> according to the corresponding action, and generates the file xxx.ring.gz created by the guidance. Before that, it will back up the original <builder_file> and xxx.ring.gz to the Backups folder.

Figure 1 builder fileand ring.gz created by swift-ring-Builder

Figure 2 builder fileand ring.gz backed up by swift-ring-Builder

It is very important to save <builder_file>. Therefore, you need to store multiple copies of the ring creation file. Once the ring creation file is completely lost, it means that we need to re-create a ring completely, so that almost all partitions will be allocated to different new devices, therefore, data copies will be moved to new locations, resulting in a large amount of data migration, resulting in the system being unavailable for a period of time. 2. Swift-ring-builder command Swift-ring-builder contains multiple commands:

 
Add create list_parts rebalance Remove search set_infoset_min_part_hoursset_weightset_replicasvalidatewrite_ring
 

 
Next we will list these commands and make relevant explanations. You can directly run the swift-ring-builder command to obtain the English document content.
 

 Swift-ring-builder <builder_file>  Shows information about the ring and the devices.
Displays information about devices in the ring and ring. A region attribute is added to the device in swift-1.8.0.Swift -Ring-builder <builder_file>AddZ <zone>-<IP >:< port>/<device_name >_< meta> <weight> [z <zone>-<ip>: <port>/<device_name >_< meta> <weight>  ]...Adds devices to the ring with the given information. No partitions will be assigned to the new device  Until After running '  Rebalance  '  . This is so you can  Make  Multiple device changes and rebalance them all just once.
    Add a new device to the ring using the provided information. The add operation does not allocate partitions to new devices. partitions are allocated only after the 'rebalance' command is run.  
 Therefore, this mechanism allows you to add multiple devices at a time and perform only one rebalance to allocate partitions for these devices.

 

 Swift  -Ring-builder <builder_file>Create<Part_power> <replicas> <min_part_hours>  Creates <Builder_file> 2 ^ <Part_power> partitions and <replicas> . <Min_part_hours> is number of hours to restrict moving a partition More  Than once.
Use the <part_power> power partition of 2 and the <replicas> Number of replicas to create <builder_file>. <min_part_hour> is the minimum interval between two consecutive moves of a partition.Swift -Ring-builder <builder_file>List_parts<Search-value> [<search-value>  ]...Returns  2  Column list of all the partitions that are assigned to any of the devices matching the search values given. The first column is the assigned Partition Number and the second column is the number of device matches  For  That partition. The list is ordered from most number of matches to least. If there are a lot of devices to match against, this command cocould take  While  To run.
  Returns a list of two columns, including all partitions of all devices that match the search value. 
The first column is the associated Partition Number.
The second column is the device number that matches the partition. 
 The list is sorted by matching numbers in ascending order. If many devices match the search result, this command needs to run for a while. 

 
Swift-Ring-builder <builder_file>RebalanceAttempts to rebalance the ring by reassigning partitions that Haven'T beenRecently reassigned.
RThe ebalance command attempts to rebalance the ring by allocating partitions that have not been reassigned recently.
 

 Swift  -Ring-builder <builder_file>Remove<Search-value> [search- Value...]Removes the device (s) from the ring. This shoshould normally just be used  For  A device that has failed. For a device you wish to decommission, it  '  S best To set its weight 0 , Wait   For It to drain all its data, Then  Use this remove command. This will not take effect  Until After running '  Rebalance  '  . This is so you can  Make  Multiple device changes and rebalance them all just once.
 The remove command removes the device from the ring. In general, this command should only be used on devices that fail.
  If you want to retire a device, the best way is to set its weight to 0, after it removes all the data on it, run this command to remove the device. 
 The remove operation does not re-allocate partitions. partitions are allocated only after the 'rebalance' command is run. Therefore, this mechanism allows you to add and delete devices at a time and perform a rebalance only once to allocate partitions for these devices. 

 Swift  -Ring-builder <builder_file>Search<Search-value>  Shows information about matching devices.
Display the information of the matched deviceSwift -Ring-builder <builder_file>Set_info<Search-value> <IP >:< port>/<device_name >_< meta> [<search-value> <ip>: <port>/<device_name >_< meta>  ]...For each search -Value, resets the Matched Device '  S information. This information isn '  T used to assign partitions, so you can use      '  Write_ring  '  Afterward to rewrite the current ring with the newer device information. Any of the parts are optional  In  The final <Ip>: <port>/<device_name >_< meta>Parameter; just give what you want to change. For instance set_info d74 _  "  Snet: 5.6.7.8  "  Wocould just update the meta data  For Device ID   74  .
  The set_info command resets the information of each device that matches <search-value>. This information is not used to re-allocate partitions. Therefore, you can use 'write _ ring' to directly overwrite the current ring.
  <Ip>: any part of the <port>/<device_name >_< meta> parameter is optional. You only need to provide the part you want to change. 
 For example, set_info d74 _ "snet: 5.6.7.8" only updates the metadata of the device whose ID is 74 to "snet: 5.6.7.8" 

   swift   -ring-builder 
    
     set_min_part_hours  
     
    
     changes the  
  
    to the given 
    
    . this shoshould be set to however  
     long  A full replication/update cycle takes. we 
     ' 
     re working on a way  to determine this 
     more  
    
      easily than scanning logs. 
     
  the set_min_part_hours command sets 
     
       to the 
      
       . 
       
 
        at least one complete replication/update cycle must be set. We are trying to find a way to determine this time more easily than reading logs  
      
     
    
   
  

  swift-ring-builder 
    
     set_weight  
     
     
       [
       
      
     
    
      weight]...  resets the devices   '  weights. no partitions will be reassigned to or from  the device  until  after running  '  rebalance   '. this is so you can  make   multiple device changes and rebalance them all just once. 
   Reset the weight of the device. After the set_weight operation, the partition on the device is not re-allocated. The partition is allocated only after the 'rebalance 'command is run.  
  therefore, this mechanism allows you to add multiple devices at a time, perform the rebalance operation only once to allocate partitions for these devices.  

Swift-ring-builder <builder_file>Set_replicas<Replicas>
Changes the replica count to the given <replicas>. <replicas> May
Be a floating-point value, in which case some partitions will have
Floor (<replicas>) Replicas and some will have ceiling (<replicas>)
In the correct proportions.A rebalance is needed to make the change take effect.

The set_replicas command is used to set the number of replicas in the parameter <replicas>.
<Replicas> can be a floating point number. Therefore, in some scenarios, the number of replicas in some partitions may be floor (<replicas>) or (<replicas> ), this depends on the correct proportion.

You need to execute a rebalance command to make the copy settings take effect. This command is added to swift-1.8.0.

 Swift -Ring-builder <builder_file>  ValidateJust runs the validation routines on the ring.Only run the builder's validate method to make the ring take effect.

Swift  -Ring-builder <builder_file>  Write_ringJust rewrites the distributable Ring  File . This is Done  Automatically after a successful rebalance, so really this is only useful after one or  More      '  Set_info  '  Callwhen no rebalance is needed but you want to send out the new device information.
  The write_ring command is only used to override the ring file in a partial environment. This command is automatically executed after a rebalance operation is successfully executed. 
 Therefore, it is only used when you run the 'set _ info' command once or multiple times, but you do not want to rebalance but want to retain new information. 

 

3. Parameter format

The <search_value> format is as follows:

 
D <device_id> Z <zone>-<ip>: <port>/<device_name >_< meta>
 

Any part of this format is optional, for example:

Z1 matches Devices In Zone 1  Z1 - 1.2 . 3.4 Matches Devices In Zone 1 With the IP 1.2 . 3.4  1.2 . 3.4 Matches Devices In Any zone with the IP 1.2 . 3.4  Z1:  5678 Matches Devices In Zone 1 Using Port 5678 :  5678 Matches devices that use port 5678 / Sdb1 matches devices with the device name sdb1_shiny matches devices with shiny  In  The meta data _  "  Snet: 5.6.7.8  " Matches devices with snet: 5.6 . 7.8   In  The meta data [:: 1 ] Matches Devices In Any zone with the IP :: 1  Z1 -[:: 1 ]: 5678 Matches Devices In Zone 1 With IP :: 1 And Port 5678 

The following is an example of the most accurate definition:

D74z1-1.2.3.4:5678/Sdb1 _"Snet: 5.6.7.8"
 

 

4. Return code meaning

 
0=Operation successful1=Operation completed with warnings2= Error