Glusterfs Self-heal and Re-balance Operations

In my previous article "Introduction to GlusterFS" (File System) and Installation–part 1 ' was just a brief overview o f the file system and its advantages describing some basic commands. It's worth mentioning about the important features,  self-heal  and  re-balance, in this article without which explanation on  GlusterFS  Would be is of no use. Let us get familiar with the terms  self-heal  and  re-balance.What does we mean by self-heal on replicated volumes?

This feature was available for replicated volumes. Suppose, we have a replicated volume [minimum replica Count 2]. Assume that due to some failures one or more brick among the replica bricks go off for a while and user happen to delete A file from the mount point which would get affected only on the online brick.

When the offline brick comes online at a later time, it's necessary to has that file removed from this brick also i.e. a Synchronization between the replica bricks called as healing must is done. Same is the case with creation/modification of files on offline bricks. GlusterFS has a inbuilt self-heal daemon to take care of these situations whenever the bricks become online.

Replicated Volume

What does we mean by re-balance?

Consider a distributed volume with only one brick. For instance we create ten files on the volume through mount point. Now all the files were residing on the same brick since there was only brick in the volume. On adding one and more brick to the volume, we could have to re-balance the total number of files among the same bricks. If a volume is expanded or shrunk in GlusterFS, the data needs to being re-balanced among the various bricks included in the Volume.

Distributed Volume

Performing self-heal in GlusterFS

1. Create A replicated volume using the following command.

$ gluster Volume Create Vol replica 2 192.168.1.16:/home/a 192.168.1.16:/home/b

Note: Creation of a replicated volume with bricks on the same server could raise a warning for which you had to PR Oceed ignoring the same.

2. Start and Mount the volume.

$ gluster Volume start vol$ mount-t glusterfs 192.168.1.16:/vol/mnt/

3. Create a file from mount point.

$ touch/mnt/foo

4. Verify the same on the replica bricks.

$ ls/home/a/foo$ Ls/home/b/foo

5. Now send one of the bricks offline by killing the corresponding Glusterfs daemon using the C3>pid got from volume status information.

$ Gluster Volume Status Vol

Sample Output

Status of Volume:volgluster Processportonlinepid----------------------------------------------------------------- -------------Brick 192.168.1.16:/home/a49152 Y3799 Brick 192.168.1.16:/home/b49153 Y3810 NFS Server on localhost2049 Y 3824 self-heal Daemon on localhostn/a Y3829

Note: See the presence of Self-heal daemon on the server.

$ kill 3810

$ Gluster Volume Status Vol

Sample Output

Status of Volume:vol Gluster processportonlinepid----------------------------------------------------------------- -------------Brick 192.168.1.16:/home/a49152 Y3799 Brick 192.168.1.16:/home/bn/a nn/a NFS Server on localhost2049 Y382 4 self-heal Daemon on localhostn/a Y3829

Now the second brick is offline.

6. Delete the file foo from mount point and check the contents of the brick.

$ rm-f/mnt/foo$ ls/home/a$ Ls/home/bfoo

you see foo are still there in second brick.

7. now bring back the brick online.

$ gluster Volume start force$ gluster Volume Status Vol

Sample Output

Status of Volume:vol Gluster processportonlinepid----------------------------------------------------------------- -------------Brick 192.168.1.16:/home/a49152 Y3799 Brick 192.168.1.16:/home/b49153 Y4110 NFS Server on localhost2049 Y 4122 self-heal Daemon on localhostn/a Y4129

Now the brick is online.

8. Check the contents of bricks.

$ ls/home/a/$ ls/home/b/

File has been removed from the second brick by the self-heal daemon.

Note: In case of larger files it is a while for the self-heal operation to be successfully done. You can check the Heal status using the following command.

$ gluster Volume Heal Vol Info

Performing re-balance in GlusterFS

1. Create a distributed volume.

$ gluster Create volume distribute 192.168.1.16:/home/c

2. Start and Mount the volume.

$ gluster Volume start distribute$ mount-t glusterfs 192.168.1.16:/distribute/mnt/

3. Create ten files.

$ touch/mnt/file{1..10}$ ls/mnt/file1 file10 file2 file3 file4 file5 file6 file7 file8 file9$ ls/home/cfile1 File10 file2 file3 file4 file5 file6 file7 file8 file9

4. Add Another Brick to volume distribute.

$ gluster Volume Add-brick distribute 192.168.1.16:/home/d$ ls/home/d

5. do re-balance.

$ gluster Volume rebalance distribute Startvolume rebalance:distribute:success:Starting rebalance on volume distribute has been successful.

6. Check the contents.

$ ls/home/cfile1 file2 file5 file6 file8 $ ls/home/dfile10 file3 file4 File7 file9

Files has been re-balanced.

Note: You can check the re-balance status by issuing the following command.

$ gluster Volume rebalance distribute status

Sample Output

node           rebalanced-files      size          scanned    failures     skipped   statusrun time in secs ---------       -----------          ---------      --------   ---------   -------   -- ------     ----------------- localhost           5                 0Bytes           15           0         0        completed         1.00 volume rebalance:  Distribute: success:

With the this I plan to conclude the this series on GlusterFS. Feel free to comment here with your doubts regarding the Self-heal and re-balance features.

Glusterfs Self-heal and Re-balance Operations