PHP calls SHELL to upload local files to Hadoop hdfs

PHP used Thrift to upload local files to Hadoop's hdfs by calling SHELL, but the upload efficiency was low. another user pointed out that he had to use other methods .? Environment: The php runtime environment is nginx + php-fpm? Because hadoop enables permission control, PHP calls SHELL to upload local files to Hadoop hdfs.

Thrift was originally used for uploading, but its Upload efficiency was low. another user had to use other methods.

?

Environment:

The php runtime environment is nginx + php-fpm.

?

Because hadoop has enabled permission control, it does not have permission to directly use php to call shel for upload. From the command execution in php, the account and level of php running are both nobody. Therefore, there are two solutions: 1. create a directory in the hadoop shell and assign the directory owner to the nobody user in the nobody Group. 2. open the original directory permission and use 777. The procedure is described as follows:

?

Method 1:./hadoop fs-chown-R nobody: nobody/resources

Note:/resources is the user directory and needs to be changed as needed

?

Method 2:./hadoop fs-chomod-R 777/resources

Note:/resources is the user directory and needs to be changed as needed

?

Php shell call method:

Public function uploadByShell ($ local, $ hdfs) {$ shell = "sh/usr/local/hadoop/hadoop-0.20.2/bin/hadoop fs-copyFromLocal ". $ local. "". $ hdfs; $ shell. = "; sh/usr/local/hadoop/hadoop-0.20.2/bin/hadoop fs-chmod 777 ". $ hdfs; $ res = null; system ($ shell, $ res); $ flag = false; if ($ res = "0") {$ flag = true ;} return $ flag ;}

?

?

?

?

?