Install and configure Mahout-distribution-0.7 in the Hadoop Cluster

Install and configure Mahout-distribution-0.7 in the Hadoop Cluster

System Configuration:

Ubuntu 12.04

Hadoop-1.1.2

Jdk1.6.0 _ 45

Mahout is an advanced application of Hadoop. To run Mahout, you must install Hadoop in advance. Mahout can only be installed on one NameNode node of the Hadoop cluster, but not on other data nodes.

1. Download and decompress the Binary Package.

To begin.

Hadoop @ ubuntu :~ $ Tar-zxvf mahout-distribution-0.7.tar.gz

2. Configure the environment variable:/etc/profile

Make environment variables take effect: source/etc/profile

3. Start hadoop

4. mahout -- help

Check whether Mahout is properly installed and check whether some algorithms are listed.

Of course, this method is not accurate and can be verified through the following steps.

5. Prepare for mahout

A. download a file synthetic_control.data,: http://archive.ics.uci.edu/ml/databases/synthetic_control/synthetic_control.data

B. Create the test directory testdata and import the data to the tastdata directory (the directory name can only be testdata)

Hadoop @ ubuntu :~ /$ Hadoop fs-mkdir testdata

Hadoop @ ubuntu :~ /$ Hadoop fs-put/usr/local/mahout-distribution-0.7/synthetic_control.data testdatac. Use the kmeans Algorithm

Hadoop @ ubuntu :~ /$ Hadoop jar/usr/local/mahout-distribution-0.7/mahout-examples-0.7-job.jar org. apache. mahout. clustering. syntheticcontrol. kmeans. Job

6. view results

Hadoop @ ubuntu :~ /$ Hadoop fs-lsr output if you see the following results, the algorithm runs successfully and your installation succeeds.

ClusteredPoints clusters-0 clusters-1 clusters-10 clusters-2 clusters-3 clusters-4 clusters-5 clusters-6 clusters-7 clusters-8 clusters-9 data

This is the directory tree under my-eclipse:

Note:

ClusteredPoints: stores the final cluster result. Both cluster-id and documents-id are displayed. The key-value Type of the result of reading clusteredPoints using mahout seqdumper is (IntWritable, WeightedVectorWritable)

Clusters-N: The result of the N-th clustering. n indicates the number of samples of a certain type, c indicates the center of each attribute, and r indicates the radius of each attribute. Clusters-N result type is (Text, Cluster)

Data: stores raw data. files in this folder can be read using mahout vectordump. The original data is in the vector format. Others can only be read using mahout seqdumper, you can also use mahout seqdumper to read vector files. However, vectordump is used to read the numeric results without the corresponding key. If you read them using seqdumper, you can see the key, that is, the corresponding url, the value reads a class description instead of an array vector.

7. analysis results

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)