Hadoop HDFs Upload file permissions issue

Problem Description:

Hadoop in the virtual machine under the Linux system

Local files are uploaded to the specified directory on the Hadoop platform by writing code locally through eclipse

The code is as follows:

@Test Public voidUpload ()throwsIOException {Configuration conf=NewConfiguration (); Conf.set ("Fs.defaultfs", "Hdfs://lujie01:9000/"); FileSystem FS=filesystem.get (conf); Path Path=NewPath ("Hdfs://lujie01:9000/aa/lulu.txt"); Fsdataoutputstream OutputStream=fs.create (path); FileInputStream InputStream=NewFileInputStream ("D:/lulu.txt");            Ioutils.copy (InputStream, OutputStream); }

　

There are more simple ways to upload files:

Copyfromlocalfile method for direct use of filesystem

Fs.copyfromlocalfile (New Path ("D:/lulu.txt"), New Path ("Hdfs://lujie01:9000/aa/lulu.txt"));

The user identity of this machine cannot be authenticated by Hadoop, and cannot be uploaded until the user is set or the permissions of the/aa folder under Hadoop are changed.

The error is as follows:

Workaround One:

Set the permissions of the/aa folder to any user readable, writable, actionable

Execute the test program again, run normally, and the client can view the file Lulu.txt in AA. Indicates the upload was successful, note that the owner here is Lujie, the local user name of the computer

Workaround Two:

Set the arguments in the run configuration to change the user name to the user name of the Linux system Hadoop

Workaround Three:

Specify the user as Hadoop directly in the code

FileSystem fs = Filesystem.get (New URI ("Hdfs://lujie01:9000/"), conf, "Hadoop");

@Test Public voidUpload ()throwsIOException, Interruptedexception, urisyntaxexception {Configuration conf=NewConfiguration (); Conf.set ("Fs.defaultfs", "Hdfs://lujie01:9000/"); FileSystem FS = Filesystem.get (New URI ("Hdfs://lujie01:9000/"), conf, "Hadoop"); Path Path=NewPath ("Hdfs://lujie01:9000/aa/lulu1.txt"); Fsdataoutputstream OutputStream=fs.create (path); FileInputStream InputStream=NewFileInputStream ("D:/lulu.txt");            Ioutils.copy (InputStream, OutputStream); }

　　　　

After the upload is successful, the client view

　　　

Hadoop HDFs Upload file permissions issue