Horse soldier hadoop2.7.3_ using Java to access HDFs

• Accessing HDFs through a Java program:
• The HDFS system will store the data used in the Core-site.xml specified by the Hadoop.tmp.dir, which defaults to/tmp/hadoop-${user.name}, because the/tmp directory will be deleted when the system restarts. Therefore, the directory location should be modified. Modify Core-site.xml (modified on all sites)

  12345

  <property>    <name>hadoop.tmp.dir</name>    <value>/var/hadoop</value></property>

• HDFs Namenode-format
• The permissions system on Windows and the permissions system on Linux, during testing, for simplicity, you can turn off permission checking on Namenode hdfs-site.xml, adding the configuration:

  12345

  <property>    <name>dfs.permissions.enabled</name>    <value>false</value></property>

  Restart namenode:hadoop-daemon.sh Stop Namenode, hadoop-daemon.sh start Namenode
• Reading files from HDFs

  Url.seturlstreamhandlerfactory (New fsurlstreamhandlerfactory ()); InputStream in = new URL ("hdfs:// 192.168.56.100:9000/test.data "). OpenStream (); Ioutils.copybytes (in, System.out, 4096, true);              

• Learn to use the FileSystem class

  Configuration conf = newConfiguration (); Conf.set ("Fs.defaultfs", "hdfs://192.168.56.100:9000"); FileSystem FileSystem =filesystem.get (conf);                Boolean B = filesystem.exists (New Path ("/hello"));                                System.out.println (b);                Boolean success = Filesystem.mkdirs (New Path ("/mashibing")); SYSTEM.OUT.PRINTLN (Success); Success = Filesystem.delete (New Path ("/mashibing"), true); SYSTEM.OUT.PRINTLN (Success); Fsdataoutputstream out = filesystem.create (new Path ("/test.data"), true); FileInputStream fis = new FileInputStream ("C:/test/core-site.xml"); Ioutils.copybytes (FIS, out, 4096, true); filestatus[] statuses = filesystem.liststatus (new Path ("/"));//system.out.println (statuses.length); for( Filestatus status:statuses) {System.out.println (Status.getpath ()); System.out.println (Status.getpermission ()); System.out.println (Status.getreplication ()); }

• Thinking of how Baidu network disk implementation?
• What does Core-site do with it? What does Hdfs-site do with it?

Horse soldier hadoop2.7.3_ using Java to access HDFs