Problems caused by using string concatenation in MapReduce

Today, we encountered a string concatenation problem when optimizing the MapReduce job for the business party.

string concatenation code in reduce:

while (It.hasnext ()) {Line      = It.next (). toString ();      Artid+=line+ ",";      ct++;}

Using string concatenation, the result is that only one of the job jobs has been run, and the time is up to 6 hours, and the other completed reduce only took more than 10 minutes.

Consulted the business side, artid some will be very long, this time will always be string stitching.

I looked at it again. The amount of data for the incomplete and completed reduce processing is not very different, the unfinished processing of 60M data, the completed processing of 50M of data.

1, view the Java process of the reduce heap memory usage, there is no heap memory is not enough to cause the situation has been FGC.

2. View the CPU usage of the reduce's Java process:

[Email protected] ~]$ top-p 39257top-14:48:37 up 393 days, 20:34,  2 users,  load average:8.75, 15.84, 15.53Tas KS: Total  ,   1 running,  sleeping,   0 stopped,   0 zombiecpu (s): 28.8%us,  7.2%sy,  0.0%ni , 61.7%id,  0.9%wa,  0.0%hi,  1.3%si,  0.0%stmem:  49374704k Total, 41905644k used,  7469060k free,  2167520k buffersswap:  8388600k Total,   341168k used,  8047432k free, 27925176k Cached  PID USER      PR  NI  VIRT  RES  SHR S%cpu%MEM    time+  COMMAND39258 mapred   0 2432m 1.6g  15m R 94.1  3.4 303:47.63 java39263 mapred   0 2432m 1.6g  15m S  5.7  3.4  11:31.72 java39265 mapred   0 2432m 1.6g  15m S  5.3  3.4  11:32.03 java39268 mapred   0 2432m 1.6g  15m S  5.3  3.4  11:34.02    java39271 mapred   0 2432m 1.6g  15m S  5.3  3.4  11:32.73 java

Found that 39258 of this thread's CPU usage is high.

The

prints out the thread stack information for the Java process of the reduce and finds 39258 thread-related information, 39258 hexadecimal is 0x995a, and the nid=0x995a thread can be found:

"Main" prio=10 tid=0x00007f36e000d000 nid=0x995a runnable [0x00007f36e6ab6000] Java.lang.Thread.State:RUNNABLE A T Java.util.Arrays.copyOfRange (arrays.java:3209) at java.lang.string.<init> (string.java:215) at JAVA.L Ang. Stringbuilder.tostring (stringbuilder.java:430) at WeMedia.create.ArticleInfoReduce.reduce (Articleinforeduce.java : At WeMedia.create.ArticleInfoReduce.reduce (articleinforeduce.java:13) at Org.apache.hadoop.mapreduce.Re Ducer.run (reducer.java:171) at Org.apache.hadoop.mapred.ReduceTask.runNewReducer (reducetask.java:627) at org . Apache.hadoop.mapred.ReduceTask.run (reducetask.java:389) at Org.apache.hadoop.mapred.yarnchild$2.run ( yarnchild.java:167) at java.security.AccessController.doPrivileged (Native Method) at Javax.security.auth.Sub Ject.doas (subject.java:396) at Org.apache.hadoop.security.UserGroupInformation.doAs (Usergroupinformation.java : 1550) at org.apache.hadoop.mapred.Yarnchild.main (yarnchild.java:162) 

The thread stack information shows that the thread CPU usage is high because of the string concatenation, although the code directly uses a string, but the JVM is converted into a StringBuilder operation, supposedly converted into a StringBuilder effect should not be low, But the actual results are not ideal.

Finally let the business party display modified the string stitching part of the code, using StringBuffer, re-executed the job, the job all run completed only more than 10 minutes, and 6 hours before the modification, the efficiency of the operation increased many times.

Summary:

Although the direct use of string splicing, the bottom of the actual use of StringBuilder to operate, but the efficiency is not satisfactory, it is best to display the use of StringBuilder or StringBuffer bar, do not send hope for the bottom conversion.

Problems caused by using string concatenation in MapReduce