Cloud Sail Big Data College _hdfs and yarn start-up mode

Yarn's shell operation and management 7.1 starting yarn

Yarn has 2 daemon threads: ResourceManager, NodeManager.

[[email protected] hadoop-2.2.0] $sbin/yarn-daemon.sh start ResourceManager

[[email protected] hadoop-2.2.0] $sbin/yarn-daemon.sh start NodeManager

7.2YARN Web Management interface

Yarn Management Address:

ResourceManager: Host name:8088 . In this environment: http://hadoop-yarn.dragon.org:8088

Namenodemanager: Host name:8042 . In this environment: http://hadoop-yarn.dragon.org:8042

650) this.width=650; "Src=" http://img.blog.csdn.net/20150102142902250?watermark/2/text/ ahr0cdovl2jsb2cuy3nkbi5uzxqvy2xvdwr5agfkb29w/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/dissolve/70/gravity/ Center "style=" border:none; "/>

7.3 Running the Maprecuce program

(1) There are many example programs in the $HADOOP _home/share/hadoop/mapreduce directory :

Anexample program must is given as the first argument.

Validprogram names is:

Aggregatewordcount:an Aggregate Basedmap/reduce Program This counts the words in the input files.

Aggregatewordhist:an Aggregate Basedmap/reduce Program This computes the histogram of the words in the input files.

BBP:A Map/reduce Program This usesbailey-borwein-plouffe to compute exact digits of Pi.

Dbcount:an example Job, the Count Thepageview counts from a database.

DISTBBP:A Map/reduce Program This uses ABBP-TYPE formula to compute exact bits of Pi.

GREP:A Map/reduce Program This counts thematches of A regex in the input.

Join:a job that effects A join over sorted,equally partitioned datasets

Multifilewc:a job that counts words fromseveral files.

Pentomino:a map/reduce Tile Laying programto find solutions to pentomino problems.

Pi:a Map/reduce Program, that estimates piusing A quasi-monte Carlo method.

RANDOMTEXTWRITER:A map/reduce Program Thatwrites 10GB of random textual data per node.

RANDOMWRITER:A map/reduce Program Thatwrites 10GB of random data per node.

Secondarysort:an example defining asecondary sort to the reduce.

SORT:A map/reduce Program This sorts thedata written by the random writer.

Sudoku:a Sudoku Solver.

Teragen:generate data for the Terasort

Terasort:run the Terasort

teravalidate:checking Results of Terasort

WORDCOUNT:A Map/reduce Program This countsthe words in the input files.

WORDMEAN:A Map/reduce Program This countsthe average length of the words in the input files.

WORDMEDIAN:A map/reduce Program This countsthe median length of the words in the input files.

Wordstandarddeviation:a Map/reduce programthat counts the standard deviation of the length of the words in the Inputfiles .

(2) How to run these programs

Running these examples is performed through the $hadoop_home/bin/yarn jar command, such as: The following example is the Execute PI function:

[Email protected] hadoop-2.2.0]# bin/yarn jar share/hadoop/mapreduce/ Hadoop-mapreduce-examples-2.2.0.jar

Description: Red section: Command Blue section: The path where the jar file resides

Cloud Sail Big Data College _hdfs and yarn start-up mode