Name |
Value |
Description |
Dfs.default.chunk.view.size |
32768 |
The content display size for each file in the HTTP access page of Namenode, usually without setting. |
Dfs.datanode.du.reserved |
1073741824 |
The amount of space reserved for each disk needs to be set up, mainly for non-HDFS files, default is not reserved, 0 bytes |
Dfs.name.dir |
/opt/data1/hdfs/name, /opt/data2/hdfs/name, /nfs/data/hdfs/name |
The metadata used by NN is saved, and it is generally recommended to keep one copy on NFS, used as a 1.0 ha scenario, or on multiple drives on a single server |
Dfs.web.ugi |
Nobody,nobody |
Users and groups used by the Web Tracker page server used by NN,JT, etc. |
Dfs.permissions |
true | False |
DFS permissions are turned on, I generally set false, through the development of tools to train others interface operation to avoid misoperation, set to true sometimes you will encounter data because the permissions can not access. |
Dfs.permissions.supergroup |
SuperGroup |
Set the HDFS Super privilege group, which is supergroup by default, and the user who started Hadoop is typically superuser. |
Dfs.data.dir |
/opt/data1/hdfs/data, /opt/data2/hdfs/data, /opt/data3/hdfs/data, ... |
True Datanode data save path, can write multiple hard disks, comma separated |
Dfs.datanode.data.dir.perm |
755 |
Path permission for local folder used by Datanode, default 755 |
Dfs.replication |
3 |
The number of copies of the HDFS data block, by default 3, in theory the more the number of copies run faster, but more storage space is required. The rich can tune 5 or 6. |
Dfs.replication.max |
512 |
Sometimes the DNS temporary failure recovery causes the data to exceed the default backup count. The largest number of copies, usually of no use, is not written in the config file. |
Dfs.replication.min |
1 |
Minimum number of copies, function ibid. |
Dfs.block.size |
134217728 |
The size of each file block, we use 128M, the default is 64M. This calculation needs 128*1024^2, I have encountered someone directly write 128000000, very romantic. |
Dfs.df.interval |
60000 |
Disk usage Statistics automatic refresh time in milliseconds. |
Dfs.client.block.write.retries |
3 |
The maximum number of times the data block is written, which does not catch a failure before this number of times. |
Dfs.heartbeat.interval |
3 |
The heartbeat detection interval for the DN. Seconds |
Dfs.namenode.handler.count |
10 |
The number of threads that are expanded after nn startup. |
Dfs.balance.bandwidthPerSec |
1048576 |
Maximum bandwidth per second used when doing balance, using bytes as units instead of bit |
Dfs.hosts |
/opt/hadoop/conf/hosts.allow |
A host Name list file, where the host is allowed to connect the NN, must write absolute path, the contents of the file is empty is considered to be all. |
Dfs.hosts.exclude |
/opt/hadoop/conf/hosts.deny |
The rationale is the same, but here is a list of host names that are forbidden to access the NN. This is useful for removing the DN from the cluster. |
Dfs.max.objects |
0 |
The number of Dfs maximum concurrent objects, the files in HDFs, and the directory blocks are considered to be an object. 0 means no Limit |
Dfs.replication.interval |
3 |
NN computes the internal interval of the copied block, usually without writing to the configuration file. The default is good |
Dfs.support.append |
true | False |
The new Hadoop supports the append operation of the file, which is to control whether the file is allowed to append, but the default is false, because there are additional bugs. |
dfs.datanode.failed.volumes.tolerated |
0 |
The maximum number of bad drives that can cause the DN to be hung, the default 0 is that the DN will be shutdown as long as 1 hard drives are broken. |
Dfs.secondary.http.address |
0.0.0.0:50090 |
SNN Tracker page listener address and port |
Dfs.datanode.address |
0.0.0.0:50010 |
DN of the service listening port, Port 0 will be random listening to the port, through the heartbeat to notify the NN |
Dfs.datanode.http.address |
0.0.0.0:50075 |
The tracker page of the DN listens to the address and port |
Dfs.datanode.ipc.address |
0.0.0.0:50020 |
The IPC listening port of DN, write 0 to listen at random port via heartbeat transmission to NN |
Dfs.datanode.handler.count |
3 |
The number of service threads started by DN |
Dfs.http.address |
0.0.0.0:50070 |
Tracker page listening address and port for nn |
Dfs.https.enable |
true | False |
If the tracker of NN is listening on the HTTPS protocol, the default is False |
Dfs.datanode.https.address |
0.0.0.0:50475 |
HTTPS for DN Tracker page listening address and port |
Dfs.https.address |
0.0.0.0:50470 |
HTTP Tracker page listening address and port for nn |
Dfs.datanode.max.xcievers |
2048 |
Equivalent to the maximum number of open files under Linux, the document does not have this parameter, when the Dataxceiver error occurs, you need to increase the size. Default 256 |