Considerations for Hadoop Multi-drive configuration

<!--Hdfs-site.xml -< Property><name>Dfs.datanode.fsdataset.volume.choosing.policy</name><value>Org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy</value></ Property>< Property><name>Dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold</name><value>64424509440</value></ Property><!--Yarn-site.xml -< Property>  <name>Yarn.nodemanager.local-dirs</name>  <value>/data1,/data2,/data3/,/data4</value></ Property>

The impact of IO on Hadoop is large, and we need to be aware of the configuration of Hdfs-site.xml and yarn-site.xml on multiple drives.

For example, to reserve the appropriate space for a hard drive, in addition to being used as a copy store, you can leave a portion of the intermediate result storage for the NM to do temporary calculations.

The first parameter is the selection policy of the copy store, which is stored according to the amount of free space, and the second parameter is the standard chosen when balancing, and the difference is greater than the number of G.

Considerations for Hadoop Multi-drive configuration