PG Setup and Planning

Calculating the PG Count formula for a single pool

Poolpgcount= ((TARGETPGSPEROSD)? ( Osdnumber)? (datapercent)) /poolsizeParameter introduction

name	Description	Notes
Pool PG Count	Number of PG in a single pool
Target PGs Per OSD	Number of PGS per OSD	A) If the number of OSD in the future cluster is no longer basic, Target PGs per OSD =100 b) If the number of OSD in the future cluster may increase to twice times the current size, Target PGs per OSD =200 c) If the number of OSD in the future cluster grows larger than the current twice times and less than 3 times times, Target PGs per OSD =300
OSD number	The total number of OSD in the cluster, by default it is all OSD	If the rules for SSD and SATA devices are split (such as SSD and SATA into two zones) through the crush rules, the number of OSD corresponding to rule must be completed separately
Data Percent	Pool occupies a percentage of the total capacity of the OSD (pre-valuation)
Pool Size	Replicas size per pool, default is 3	If using Erasure Coded pools abbreviation EC pool,pool Size = k+m

About calculation Results Rounding instructions

The final result of the calculation should be a power of 2. The power of 2 is used to improve the efficiency of the crush algorithm.
If the result is calculated, find the two 2 power values adjacent to the result, if the result is more than 25% of the smaller two power values, select the larger two power as the final result, and vice versa select the smaller 2 power value.

Other Notes

The purpose of the calculation formula is to ensure that the whole cluster has enough PG to distribute the data evenly on each OSD, and can effectively avoid the problems caused by the high pg/osd ratio when recovery and backfill.
If there are empty pool or other pool in inactive state in the cluster, these pool will not affect the data distribution of existing cluster, but these pool still consume the memory and CPU resources of the cluster.

ExampleEnvironment Introduction:

A cluster consists of 100 OSD, 40 of which are SSD (via crush rules as SSD Zone) and 60 OSD SATA (SATA zone via crush rules), and the cluster size is expected to grow to twice times the current scale.

Demand:

Create a new 4 pool for the entire cluster, as follows:

name	Description	Notes
Ssd_poola	Pool size=3, estimated capacity occupancy ratio 60%	SSD Zone
Ssd_poolb	Pool size=3, estimated capacity occupancy ratio 40%	SSD Zone
Sata_poola	EC pools,pool size = k+m= 3+2, estimated capacity occupancy ratio 80%	SATA Zone
Sata_poolb	Pool size=3, estimated capacity occupancy ratio 20%	SATA Zone

name	Description	Notes
Ssd_poola	poolpgcount= (200)? ( 40)? (0.6)) /3=1600≈2048	1600 with adjacent 2-power numerical distribution of 1024<1600<2048
Ssd_poolb	poolpgcount= (200)? ( 40)? (0.4)) /3=1066≈1024	SSD Zone
Sata_poola	poolpgcount= (200)? ( 60)? (0.8)) /5=1920≈2048	SSD Zone
Sata_poolb	poolpgcount= (200)? ( 60)? (0.2)) /5=800≈1024	SSD Zone

Reference http://ceph.com/pgcalc/

PG Setup and Planning