Basic Idea of GlusterFS Performance Tuning

The cluster NAS system built based on GlusterFS is a complex system engineering. Its overall performance is closely related to specific application features, hardware and software configurations. During system deployment, if the performance is significantly different from the estimation or experience, you need to optimize the performance according to the actual situation. According to the composition of the Gluster system, performance tuning mainly starts from three parts: hardware, OS system, and GlusterFS file system. Through testing and experiment analysis, the key points affecting performance are obtained, then the optimization method is implemented in a targeted manner. Typical scenarios that cause performance problems include:

(1) Incorrect disk configuration;

(2) Incorrect network configuration;

(3) hardware faults;

(4) the number of servers is insufficient;

(5) incorrect file access protocol;

1. hardware optimization

To a certain extent, the performance of GlusterFS depends on hardware infrastructure, mainly involving servers, CPUs, memory, disks, networks, and other components. The hardware performance directly determines the maximum theoretical performance of the system. When performance cannot meet application requirements, we should first analyze whether the hardware configuration is sufficient. If the hardware configuration has obvious problems, you can directly upgrade the hardware configuration to directly improve the performance, such as more CPU with higher performance, more memory, and more or faster disks, more network interfaces or faster networks.

2. OS system optimization

Generally, the default parameter settings are used to adapt to more application loads, but the performance is usually not optimal, such as I/O scheduling algorithms, Cache parameters, process scheduling affinity, disk file system parameters, mount parameters, and network communication parameters. You can customize these parameters based on theoretical analysis and experimental tests based on specific application features to achieve higher performance.

3. GlusterFS File System Optimization

The underlying core of Gluster is the GlusterFS distributed file system. To meet different application load requirements, it provides many adjustable system parameters. The main parameters related to performance tuning include:

(1) Global Cache-Size; default value: 32 MB

(2) Write-Cache-Size for each file. The default value is 1 MB.

(3) number of I/O concurrency. The default value is 16.

(4) Read-ahead switch, default value On

(5) strip size, default value: KB

The above system parameter settings do not have a universally applicable experience. Therefore, theoretical analysis and experimental testing are required based on actual conditions to determine the optimal system parameters. If the hardware configuration is sufficient, the system parameter settings should be appropriately upgraded to achieve higher overall performance.

For details about GlusterFS, click here
GlusterFS: click here

Use GlusterFS as KVM backend storage

Distributed Storage System GlusterFS initial experience

GlusterFS globally unified namespace

Design new Xlator extension GlusterFS

GlusterFS Rebalance Analysis