【Nutch】Linux下應用nutch-1.0Web前端實現HDFS資料檢索

啟動hadoop叢集

配置/usr/program/apache-tomcat-6.0.18/webapps/nutch-1.0/WEB-INF/classes下的設定檔

hadoop-site.xml:

<property>
        <name>fs.default.name</name>
        <value>hdfs://gc03vm12:9000</value>
        <description> </description>
</property>

nutch-site.xml:

<property>
  <name>http.agent.name</name>
  <value>nutch-1.0</value>
  <description>HTTP 'User-Agent' request header.</description>
</property>

<property>
  <name>http.robots.agents</name>
  <value>nutch-1.0,*</value>
  <description>The agent strings we'll look for in robots.txt files,
  comma-separated, in decreasing order of precedence. You should
  put the value of http.agent.name as the first agent name, and keep the
  default * at the end of the list. E.g.: BlurflDev,Blurfl,*
  </description>
</property>

<property>
  <name>searcher.dir</name>
  <value>data</value>            爬蟲資料的存放目錄，hdfs的相對路徑
  <description> index dir </description>
</property>

data目錄所處位置如下：
[nutch@gc03vm12 nutch-1.0]$ bin/hadoop dfs -ls
Found 5 items
drwxr-xr-x   - nutch supergroup          0 2010-05-19 01:54 /user/nutch/data

重新啟動tomcat

[nutch@gc03vm12 apache-tomcat-6.0.18]$ bin/shutdown.sh
[nutch@gc03vm12 apache-tomcat-6.0.18]$ bin/startup.sh

搜尋