Hadoop thrift:php access to Hadoop resources via thrift

PHP can connect hbase via thrift, and PHP can also read Hadoop resources (HDFS resources) through thrift.
Get ready:
PHP needs a thrift libary
packages:hadoop-0.20.2\src\contrib\thriftfs\gen-php
Source:
<?php
$globals [' thrift_root '] = RootPath. '/lib/thrift ';
Require_once ($globals [' Thrift_root ']. /thrift.php ');
Require_once ($globals [' Thrift_root ']. /transport/tsocket.php ');
Require_once ($globals [' Thrift_root ']. /transport/tbufferedtransport.php ');
Require_once ($globals [' Thrift_root ']. /protocol/tbinaryprotocol.php ');
Require_once ($globals ["Thrift_root"]. "/packages/hadoopfs/thrifthadoopfilesystem.php");
$hadoop _socket = new Tsocket ("localhost", 59256);
$hadoop _socket-> setsendtimeout (10000); Ten seconds
$hadoop _socket-> setrecvtimeout (20000); Twenty seconds
$hadoop _transport = new Tbufferedtransport ($hadoop _socket);
$hadoop _protocol = new Tbinaryprotocol ($hadoop _transport);
$hadoopclient = new Thrifthadoopfilesystemclient ($hadoop _protocol);
$hadoop _transport-> Open ();
try {
Create directory
$dirpathname = new Hadoopfs_pathname (Array ("pathname" => "/user/root/hadoop"));
if ($hadoopclient-> exists ($dirpathname) = = True) {
echo $dirpathname-> pathname. "Exists.\n";
} else {
$result = $hadoopclient-> mkdirs ($dirpathname);
}
Put file
$filepathname = new Hadoopfs_pathname (Array ("pathname" => $dirpathname-> pathname. "/hello.txt"));
$localfile = fopen ("Hello.txt", "RB");
$hdfsfile = $hadoopclient-> Create ($filepathname);
while (true) {
$data = Fread ($localfile, 1024);
if (strlen ($data) = = 0)
Break
$hadoopclient-> Write ($hdfsfile, $data);
}
$hadoopclient-> Close ($hdfsfile);
Fclose ($localfile);
Get file
echo "Read file:\n";
Print_r ($filepathname);
$data = "";
$hdfsfile = $hadoopclient-> open ($filepathname);
Print_r ($hdfsfile);
while (true) {
$data = $hadoopclient-> Read ($hdfsfile, 0, 1024);
if (strlen ($data) = = 0)
Break
Print $data;
}
$hadoopclient-> Close ($hdfsfile);
echo "liststatus:\n";
$result = $hadoopclient-> liststatus ($dirpathname);
Print_r ($result);
foreach ($result as $key => $value) {
if ($value-> isdir = = "1")
print "dir\t";
Else
print "File\t";
Print $value-> block_replication. "\ T". $value-> length. "\ T". $value-> Modification_time. "\ T". $value-> permission. "\ T". $value-> owner. "\ T". $value-> Group. "\ T". $value-> path. "\ n";
}
$hadoop _transport-> Close ();
catch (Exception $e) {
Print_r ($e);
}
?>
Start the thrift of Hadoop
Hadoop-0.20.2\src\contrib\thriftfs\scripts\start_thrift_server.sh 59256
Problem One:
Creating files in the system directory instead of creating files in the Hadoop directory
Reason:
Load the default profile at thrift startup
Workaround:
modifying start_thrift_server.sh files
top=/usr/local/hadoop-0.20.2
Classpath= $classpath: $top/conf
Problem Two:
java.lang.NullPointerException
At Org.apache.hadoop.thriftfs.hadoopthriftserver$hadoopthrifthandler.write (hadoopthriftserver.java:282)
At org.apache.hadoop.thriftfs.api.thrifthadoopfilesystem$processor$write.process (unknown source)
At org.apache.hadoop.thriftfs.api.thrifthadoopfilesystem$processor.process (unknown source)
At Com.facebook.thrift.server.tthreadpoolserver$workerprocess.run (unknown source)
At Java.util.concurrent.threadpoolexecutor$worker.runtask (threadpoolexecutor.java:886)
At Java.util.concurrent.threadpoolexecutor$worker.run (threadpoolexecutor.java:908)
At Java.lang.thread.run (thread.java:662)
Reason:
The map hash ID returned by Java is a long and PHP (32-bit) cannot store a long type of data, resulting in loss of precision after conversion to float data.
Private long NextID = new Random (). Nextlong ();
Java return Data: 4207488029786584864 This article links http://www.cxybl.com/html/wlbc/Php/20120607/28513.html