Hive on Tez detailed configuration and run tests

hive on Tez detailed configuration and run tests

tags (space delimited): Tez Hadoop Hive HDFs yarn

Environment: hadoop-2.5.2 hive-0.14 tez-0.5.3
There are two ways to configure hive on Tez: Configure in Hive in Hadoop

Comparison:
The second way: when there is a stable hadoop cluster, but do not want to move the cluster, you can consider the second way to configure, the second way configuration only hive program can dynamically switch the execution engine: set hive.execution.engine=mr;//tez/ While other MapReduce programs can only run on yarn;
The first way: the intrusion line is strong, the original Hadoop cluster has an impact, need to be configured in the Mapred-site.xml of Hadoop: Mapreduce.framework.name for Yarn-tez, If this configuration means that all Mr Tasks performed through this Hadoop cluster can only be submitted by the Tez method, the hive will run on the Tez instead of the other configuration.
So in the beginning, want to find the second kind of configuration way to go a lot of detours

Before you start, you need to compile the Tez source code here.

root@localhost:/opt/work# wget http://www.eu.apache.org/dist/tez/0.5.3/apache-tez-0.5.3-src.tar.gz root@localhost :/opt/work# tar zxvf apache-tez-0.5.3-src.tar.gz root@localhost:/opt/work# CD apache-tez-0.5.3-src root@localhost:/ opt/work/apache-tez-0.5.3-src# mvn Clean package-dskiptests=true-dmaven.javadoc.skip=true//compilation process long Ah, wait ... Midway error can be terminated after the execution of the MVN command multiple compilation, after the successful compilation of the directory structure as follows root@localhost:/opt/work/apache-tez-0.5.3-src# ll total dosage 204 drwxrwxr-x 15 500 500 40 96 May 16:29./drwxr-xr-x-root root 4096 May 26 16:34. /-rw-rw-r--1 500 500 5753 December 5 06:25 building.txt-rw-rw-r--1 500 500 60199 December 5 06:25 CHANGES.txt drwxrwxr- X 4 500 500 4096 May 16:30 docs/-rw-rw-r--1 500 500 66 December 5 06:25. Gitignore lrwxrwxrwx 1 500 500 3 3 December 5 06:25 install.md-docs/src/site/markdown/install.md-rw-rw-r--1 500 500 14470 December 5 06:25 keys-rw-rw-r --1 500 500 11358 December 5 06:25 license.txt-rw-rw-r--1 500 500 164 December 5 06:25 notice.txt-rw-rw-r--  1 500 500 34203 December 5 06:25 pom.xml-rw-rw-r--1 500 500 1433 December 5 06:25 readme.md drwxr-xr-x 3 root root 409 6 May 16:29 target/drwxrwxr-x 4 500 500 4096 May-16:29 tez-api/drwxrwxr-x 4 500 500 4096 May-16:29 tez- Common/drwxrwxr-x 4 500 500 4096 May 16:29 tez-dag/drwxrwxr-x 4 500 500 4096 May 16:30 tez-dist/drwxrwxr- X 4 500 500 4096 May 16:29 tez-examples/drwxrwxr-x 4 500 500 4096 May 16:29 tez-mapreduce/drwxrwxr-x 5 5 00 500 4096 May 16:29 tez-plugins/drwxrwxr-x 4 500 500 4096 May 16:29 tez-runtime-internals/drwxrwxr-x 4 5  00 500 4096 May 16:29 tez-runtime-library/drwxrwxr-x 4 500 500 4096 May 16:29 tez-tests/drwxrwxr-x 3 500 500 4096 December 5 06:25 tez-tools/root@localhost:/opt/work/apache-tez-0.5.3-src# ll tez-dist/target/total dosage 40444 drwxr-xr-x 5 root root 4096 May 16:30./drwxrwxr-x 4 500 500 4096 May 26 16:30.. /drwxr-xr-x 2 root root 4096 May 16:30 archive-tmp/drWxr-xr-x 2 root root 4096 May 16:30 maven-archiver/drwxr-xr-x 3 root root 4096 May 16:30 tez-0.5.3/-rw-r-- r--1 root root 10625995 May 16:30 tez-0.5.3-minimal.tar.gz-rw-r--r--1 root root 30757128 May 16:30 Tez-0.5.3.tar.  gz-rw-r--r--