Spark Streaming debugging Skills _streaming

Spark streaming if running in local mode, log log is very clear. If log logs are running in yarn mode, driver logs can be seen through the Reource manager log. But executor's log does not see, we often make mistakes in the executor, such as the typical error: If we connect hbase to access the data, we will initialize the connection in driver, missing the excutors, resulting in a program error. If you have a try Cache,executor error in your code that does not affect the driver run, the result is wrong. We need to view executor log: Because executor is running in yarn mode, we view the log from the local containner. CDH version yarn container log again under the following path:

[Root@hadoop-3 ~]# ll/var/log/hadoop-yarn/container/application_1429701572510_0022/container_1429701572510_0022_ 01_000002/
Total dosage 932
-rw-r–r–1 yarn yarn 339015 April 11:53 stderr
-rw-r–r–1 yarn yarn 613851 April 11:53 stdout (our output log) HBase Initialize the connection, the official website also has the explanation, in executor to initialize:

 Userlog.foreachrdd (new function2<javapairrdd<string, Iterable<string>>, Time, Void> () {@Over Ride public Void Call (javapairrdd<string, iterable<string>> Stringiterablejavapairrdd, time) throws E xception {if (!stringiterablejavapairrdd.partitions (). IsEmpty ()) {STRINGITERABLEJAVAPAIRRDD.FOREACHPA 
                Rtition (New voidfunction<iterator<tuple2<string, iterable<string>>>> () {@Override public void Call (Iterator<tuple2<string, iterable<string>>> tuple2iterator) throws Exce
                    ption {//Initialize HBase connection hbaseconnectionfactory.init ();
            while (Tuple2iterator.hasnext ()) {//Specific logical Code}});
        return null;
}
}); 

The above code has a bug that initializes a lot of hbase connection, and finally throws something like the following exception:
caused by:java.net.SocketException: Too many open files
Add a judgment,

 Userlog.foreachrdd (new function2<javapairrdd<string, Iterable<string>>, Time, Void> () {@ Override Public Void Call (javapairrdd<string, iterable<string>> Stringiterablejavapairrdd, time) thro WS Exception {if (!stringiterablejavapairrdd.partitions (). IsEmpty ()) {Stringiterablejavapairrdd.forea Chpartition (New voidfunction<iterator<tuple2<string, iterable<string>>>> () {@Over Ride public void Call (Iterator<tuple2<string, iterable<string>>> tuple2iterator) throws Exception {//Initialize HBase connection if (hbaseconnectionfactory.getconnection () = NULL | |
                    HBaseConnectionFactory.getConnection.isClosed ()) hbaseconnectionfactory.init ();
            while (Tuple2iterator.hasnext ()) {//Specific logical Code}});
        return null; }
});