Hive How to close task conjecture execution

A few days ago, when solving a colleague's hive bug, it was necessary to close task conjecture execution (speculative execution),

After checking the information on the Internet, someone mentioned the need

Set Hive.mapred.map.tasks.speculative.execution=false

Set Hive.mapred.reduce.tasks.speculative.execution=false

But after the execution found job or start speculative task, a little strange, clearly set AH.

Find hive corresponding code, found only hive.mapred.reduce.tasks.speculative.execution parameters, no hive framework corresponding to the set map task conjecture execution switch option.

And this job is exactly map-only job, no reducer, so the above two parameters are set, no effect.

In fact, speculative execution is a property of the MapReduce framework.

In jobconf, there are mapred.map.tasks.speculative.execution and mapred.reduce.tasks.speculative.execution.

Hive is also getting hive.mapred.reduce.tasks.speculative.execution values and set to Mapred.reduce.tasks.speculative.execution when assembling jobconf.

After you figure out why, you can fix the problem by setting Mapred.map.tasks.speculative.execution,mapred.reduce.tasks.speculative.execution to False

Set Mapred.map.tasks.speculative.execution=false;

Set Mapred.reduce.tasks.speculative.execution=false;

More Wonderful content: http://www.bianceng.cnhttp://www.bianceng.cn/database/extra/