Debug startup in hive

Recently, a bug in debug hive used printing logs to track the source code. This method is inefficient (re-compile and replace the online jar package every time you change the source code ), java applications support remote debug, and hive is no exception. It is mainly implemented through hive -- debug.

The following problems are encountered when running hive -- Debug:

ERROR: Cannot load this JVM TI agent twice, check your java command line for duplicate jdwp options.Error occurred during initialization of VMagent library failed to init: jdwp

According to the error message, we can see that it is caused by repeated jdwp parameters. Hive is actually a shell script that tracks its running status:

cd ${HIVE_HOME}/bin;sh -x ./hive --debug

View the call status of hive -- Debug:

 if [ "$DEBUG" ]; then   if [ "$HELP" ]; then     debug_help     exit 0   else     get_debug_params "$DEBUG"     export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS $HIVE_MAIN_CLIENT_DEBUG_OPTS"   fifi

Here is:

HIVE_MAIN_CLIENT_DEBUG_OPTS=‘ -XX:+UseParallelGC -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y‘

The hive_main_client_debug_opts variable is in $ {hive_home}/bin/EXT/debug. Sh. This script controls the debug parameters, such as the port

get_debug_params(){  set_debug_defaults  parse_debug $1  # For Debug -XX:+UseParallelGC is needed, as it is a (unfortunately not perfect)  # workaround for JVM 6862295 bug, that affects some JVMs still in use  if does_jvm_support_ti; then    export HIVE_MAIN_CLIENT_DEBUG_OPTS=" -XX:+UseParallelGC     -agentlib:jdwp=transport=dt_socket,server=y,$port,$main_suspend"    export HIVE_CHILD_CLIENT_DEBUG_OPTS=" -XX:+UseParallelGC     -agentlib:jdwp=transport=dt_socket,server=y,$child_suspend"  else    export HIVE_MAIN_CLIENT_DEBUG_OPTS=" -XX:+UseParallelGC     -Xdebug -Xrunjdwp:transport=dt_socket,server=y,$port,$main_suspend"    export HIVE_CHILD_CLIENT_DEBUG_OPTS=" -XX:+UseParallelGC     -Xdebug -Xrunjdwp:transport=dt_socket,server=y,$child_suspend"  fi

The Command run by hive is:

exec ${HADOOP_HOME}/bin/hadoop jar hive-cli-0.13.1.jar org.apache.hadoop.hive.cli.CliDriver --hiveconf hive.aux.jars.path=xxxxx

In $ {hadoop_home}/bin/hadoop, when hadoop_opts is set, the variable hadoop_client_opts is referenced, which leads to repeated jdwp parameters.

You only need to comment out the following line:

HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"

This is actually a bug:

Related Bug ID

Https://issues.apache.org/jira/browse/HADOOP-9455

Https://issues.apache.org/jira/browse/HIVE-3936

Bug description:

HADOOP_CLIENT_OPTS appended twice causes JVM failures

In the official Bug ID, we can see that hive0.13 fixes this bug. The specific patch is as follows:

diff --git bin/hive bin/hiveindex 40e2c75..434ea6c 100755--- bin/hive+++ bin/hive@@ -251,7 +251,6 @@ if [ "$DEBUG" ]; then   else     get_debug_params "$DEBUG"     export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS $HIVE_MAIN_CLIENT_DEBUG_OPTS"-    export HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"   fi fi

After testing, this path does not take effect.

This article from the "Food light blog" blog, please be sure to keep this source http://caiguangguang.blog.51cto.com/1652935/1564148

Debug startup in hive