cloudera hadoop installation

Inkfish original, do not reprint the commercial nature, reproduced please indicate the source (http://blog.csdn.net/inkfish). (Source: http://blog.csdn.net/inkfish)Pig is a project that Yahoo! has donated to Apache, and is currently in the Apache Incubator (incubator) stage, with the version number v0.5.0. Pig is a Hadoop-based, large-scale data analysis platform that provides the sql-like language called Pig Latin, which translates the data analysis

;NBSP;JDK's full name of the TAR package to extract 2. Configure environment variables to modify the configuration file using the Vi/etc/profile vi/etc/profile command, Add the following: Export Java_home=/java/jdk1.8.0_73export jre_home= $JAVA _home/jreexport class_home= $JAVA _home/libexport PATH= $PATH: $JAVA _home/bin use Source/etc/profile to update the profile using Java–version to see if the success is successful, such as: hadoop User Trust 1.

Create a hadoop folder under the/usr directory and grant the hadoop user permission (master) [[emailprotected]usr]$sudomkdirhadoop[[emailprotected]usr]$ls-altotal156drwxr-xr-x.2rootroot4096Jul3100:17hadoop[[emailprotected]usr]$sudochown-Rhadoop:hadoophadoop[[emailprotected]usr]$ls-altotal156drwxr-xr-x.2hadoophadoop4096Jul3100:17hadoop Install hadoop in the/usr

Original address: http://www.linuxidc.com/Linux/2014-03/99055.htmWe use MapReduce for data analysis. When the business is more complex, the use of MapReduce will be a very complex thing, such as you need to do a lot of preprocessing or transformation of the data to be able to adapt to the MapReduce processing mode, on the other hand, write a mapreduce program, Publishing and running jobs will be a time-consuming task.The appearance of pig makes up for this shortcoming well. Pig allows you to foc

Hadoop 2.7.1 high-availability installation configuration based on QMJ Hadoop 2.7.1 high-availability installation configuration based on QMJ 1. Modify the Host Name and hosts file 10.205.22.185 nn1 (main) function namenode, resourcemanager, datanode, zk, hive, sqoop10.205.22.186 nn2 (standby) function namenode, resour

space) Now, after jkd is installed, configure the environment variables below 4.3 open/etc/profile (vim/etc/profile) Add the following content at the end: JAVA_HOME =/usr/java/jdk1.7.0 _ 40 (the version number 1.7.40 must be modified based on the download details) CLASSPATH =.: $ JAVA_HOME/lib. tools. jar PATH = $ JAVA_HOME/bin: $ PATH Export JAVA_HOMECLASSPATH PATH 4.4. source/etc/profile 4.5 verify whether the installation is successful: j

=" border-top:0px; border-right:0px; Background-image:none; border-bottom:0px; padding-top:0px; padding-left:0px; margin:0px; border-left:0px; padding-right:0px "border=" 0 "alt=" clipboard "src=" http://s3.51cto.com/wyfs02/M00/6B/F3/ Wkiol1u7nkgbqs9gaageg2yscne517.jpg "" 425 "height=" 508 "/> Specific installation links can be in reference to the steps, but there are a few points to note. Host and Slave Unified create a dedicated user to run

can be backed up), its main job is to help nn merge edits log, Reduce NN startup time SNN execution merging time fs.checkpoint.period default 3,600 seconds based on profile settings edits log size fs.checkpoint.size rules Datanode Storage data (block) to start the DN thread will report to nn blocks information by sending the heartbeat to NN to maintain its contact (3 seconds), if nn 10 minutes did not receive the heartbeat of the DN, then think it has been lost, and copy it on the block to othe

Premise:Make sure the iptables is off and SELinux is disabled1. Prepare the hardware1 sets of Namenode and 3 DatanodeNamenode 192.168.137.100Datanode1 192.168.137.101Datanode2 192.168.137.102Datanode3 192.168.137.1032. Build Hadoop users on 4 machines (can also be other user names)Useradd Hadoop3. Install JDK 1.6 on 4 machinesPost-installation java_home on/JDKConfiguring Environment variablesVim/etc/bashrcE

"Pseudo" fractional installation of Hadoop 2.6 compared to "full" fractional installation, 99% of the operation is the same, the only difference is not to configure the slaves file, here are a few key configurations:(Install JDK, create user, set SSH password, set environment variable these preparations, you can refer to the