PHP calls the shell to upload local files into Hadoop's HDFs

PHP calls the shell to upload local files into Hadoop's HDFs

Originally used to upload thrift, but its low upload efficiency, another person heinous, had to choose other methods.

?

Environment:

PHP operating Environment for Nginx + PHP-FPM

?

Because Hadoop has permission control enabled, there is no permission to use PHP directly to invoke Shel for uploading. The PHP execution command appears to be nobody for both the account and the level of PHP running. Therefore, there are two workarounds: first, create a directory in the shell of Hadoop and assign the owner of the directory to the nobody user of the nobody group. Two, the original directory permission to release, using 777. The following is a detailed procedure:

?

Method one:./hadoop fs-chown-r Nobody:nobody/resources

Note:/resources is a user directory and needs to be changed according to circumstances

?

Method two:./hadoop fs-chomod-r 777/resources

Note:/resources is a user directory and needs to be changed according to circumstances

?

How PHP calls the shell:

Public Function Uploadbyshell ($local, $hdfs) {$shell =  "Sh/usr/local/hadoop/hadoop-0.20.2/bin/hadoop FS- Copyfromlocal ". $local. " " . $hdfs; $shell. = "; Sh/usr/local/hadoop/hadoop-0.20.2/bin/hadoop fs-chmod 777 ". $hdfs $res = Null;system ($shell, $res), $flag = false;if ($res = = "0") {$flag = true;} return $flag;}

?

?

?

?

?

