Sync_by_blk for TFs data migration

This document records the data migration process between two TFs 2.2.16 systems.

Source Environment Introduction:

TFS master nameserver: 192.168.1.225/24 (VIP 229)

TFs from nameserver: 192.168.1.226/24

TFS Data Server 1: 192.168.1.226/24 (start three mount points and allocate 20 GB space for each mount point)

TFS Data Server 1: 192.168.1.227/24 (start three mount points, each of which allocates 20 GB space)

TFS Data Server 1: 192.168.1.228/23 (start three mount points, each of which allocates 20 GB space)

 

Target Environment Introduction:

TFS namserver: 192.168.1.12/24 (VIP not configured)

TFS dataserver: 192.168.1.12/24 (start three mount points, each of which allocates 20 GB space)

I. view the status of the current source server

# /usr/local/tfs/bin/ssm -s 192.168.1.229:8108 show > machine -a

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4B/F1/wKioL1Q1533AwEVUAAQ0w1yqjnU811.jpg "Title =" image 1.png "alt =" wkiol1q1533awevuaaq0w1yqjnu811.jpg "/>

Ii. view the current target server status

# /usr/local/tfs/bin/ssm -s 192.168.1.12:8108 show > machine -a

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4B/EF/wKiom1Q153CS8-mvAAL4ofughHA396.jpg "Title =" image 2.png "alt =" wKiom1Q153CS8-mvAAL4ofughHA396.jpg "/>

3. Export the block_id of the block with data from the source.

#/Usr/local/TFS/bin/SSM-s 192.168.1.229: 8108 show> block>/tmp/1.txt show> exit # WC-L/tmp/1.txt 8406/tmp/1.txt filter out version, filecount, size, del_file, the del_size fields are all 0 block_id, and the remaining blocks contain actual data. # Grep-V '0 0 0 0 0'/tmp/1.txt>/tmp/2.txt

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4B/EF/wKiom1Q156STFzBuAAG4JaZRLQM741.jpg "Title =" image 3.png "alt =" wkiom1q156stfzbuaag4jazrlqm741.jpg "/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4B/F1/wKioL1Q15_TBl96aAAIdlpom8uc475.jpg "Title =" image 4.png "alt =" wkiol1q15_tbl96aaaidlpom8uc475.jpg "/>

Obtain the block_id # Cut-C 1-10/tmp/2.txt>/tmp/3.txt

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4B/F1/wKioL1Q16CSicFthAADtC7DHH1o947.jpg "Title =" image 5.png "alt =" wkiol1q16csicfthaadtc7dhh1o947.jpg "/>

Indent space # For I in $ (CAT/tmp/3.txt); do echo $ I>/tmp/4.txt; done # WC-L/tmp/4.txt 119/tmp/4.txt

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4B/F1/wKioL1Q16ErC6ZFQAADcd25GGbU765.jpg "Title =" image 6.png "alt =" wkiol1q16erc6zfqaadcd25ggbu765.jpg "/>

4. Use the sync_by_blk provided by TFs for data migration

# /usr/local/tfs/bin/sync_by_blk -s 192.168.1.229:8108 -d 192.168.1.12:8108 -f /tmp/4.txt

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4B/F1/wKioL1Q16HbT3gc5AAICK7X-k3c528.jpg "Title =" image 7.png "alt =" wKioL1Q16HbT3gc5AAICK7X-k3c528.jpg "/>

The log shows that 15515 files are successfully synchronized, the number of failures is 0, and the number of unsynchronized files is 532.

# /usr/local/tfs/bin/ssm -s 192.168.1.12:8108show > block

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4B/EF/wKiom1Q16GbQ2QoJAARj32KzRmY735.jpg "Title =" image 8.png "alt =" wkiom1q16gbq2qojaarj32kzrmy735.jpg "/>

V. data comparison

# /usr/local/tfs/bin/ssm -s 192.168.1.229:8108show > block

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4B/F1/wKioL1Q16MCS0skoAALE892MctE254.jpg "Title =" image 9.png "alt =" wkiol1q16mcs0skoaale892mcte254.jpg "/>

Total number of filecount on source is 16349

The total number of filecount on target is 15515

16349-15515 = 834

It seems that 834 files are missing, and 834 minus the unsynchronized 532 equals 302, And the deleted 3 files are still missing. Of course, this may only be statistical information, which cannot be explained.

Vi. Log Analysis

# cd logs/# wc -l sync_unsync_file 532 sync_unsync_file

Through log analysis, it is found that there are only 16 files in the 532 files that are not synchronized after the file name is de-duplicated.

# uniq sync_unsync_file |wc -l16

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4B/F1/wKioL1Q16PyyvYxgAAMlfB5AXRI249.jpg "Title =" image 10.png "alt =" wkiol1q16pyyvyxgaamlfb5axri249.jpg "/>

In fact, it is okay to directly access these files using modules provided by nginx.

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/4B/EF/wKiom1Q16Ofwk3wAAAFdJEyILmA935.jpg "Title =" image 11.png "alt =" wkiom1q16ofwk3waaafdjeyilma935.jpg "/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4B/F1/wKioL1Q16SuwqBFBAASi-0mSKXU326.jpg "Title =" image 12.png "alt =" wKioL1Q16SuwqBFBAASi-0mSKXU326.jpg "/>

This article is from the "Bo Yue" blog and will not be reproduced!

Sync_by_blk for TFs data migration