How JAVA code sets the log printing level for spark __java

problem Scenario : When using spark SQL to increase where condition filtering, there will be a lot of filtered records (hundreds of thousands of) that are printed, resulting in a particularly slow run of spark SQL.

var df2 = sqc.sql ("select * from t_commisionratedef WHERE t_insurancetype = ' 630 ' ORDER by T_insurancetype DESC")

When executed, print screenshots:

Workaround :

Set Spark log level, do not print filter info information

1 Code Use proxy settings: The key reference code is as follows

Import Org.apache.log4j.Level

Import Org.apache.log4j.Level

Logger.getlogger ("Org.apache.spark"). Setlevel (Level.error)
Logger.getlogger ("Org.apache.hadoop"). Setlevel (Level.error)
Logger.getlogger ("Org.apache.zookeeper"). Setlevel (Level.error)

Code screenshot: