Problem Description :
Today, a colleague reflects the program has a problem, let help to view the background log, found that the background log error information as follows:
Java.net.SocketException:Too many open files at Java.net.Socket.createImpl (socket.java:460) at Java.net.Soc Ket.connect (socket.java:587) at Org.apache.commons.net.SocketClient.connect (socketclient.java:163) at Org.ap Ache.commons.net.SocketClient.connect (socketclient.java:184) at Com.asiainfo.goods.wo.store.scheduler.util.FtpUtil.downloadFileByFileName (ftputil.java:270) at Com.asiainfo.goods.wo.store.scheduler.job.TargetUserJob.dealWithFtpByRequestId (targetuserjob.java:186) at Com.asiainfo.goods.wo.store.scheduler.job.TargetUserJob.execute (targetuserjob.java:80) at Com.asiainfo.goods.presale.scheduler.job.QuartzJobFactory.execute (quartzjobfactory.java:68) at Org.quartz.core.JobRunShell.run (jobrunshell.java:202) at Org.quartz.simpl.simplethreadpool$workerthread.run ( simplethreadpool.java:573) 2018-04-08 18:45:00 [Com.asiainfo.goods.wo.store.scheduler.job.targetuserjob]-[error] : 216-file doesnot exist===targetcustu00011201804081800575321.zip
Problem Analysis :
You can tell by the above error that the program is causing too many files to open.
resolution process :
1. View the upper limit of open files set under current system user
[[email protected] log]$ ulimit-acore file size (blocks,-c) 0data seg size (Kbytes,-D) unlimitedscheduling Prio Rity (-e) 0file size (blocks,-f) unlimitedpending signals (-I ) 256705max locked memory (Kbytes, L) 64max Memory Size (Kbytes,-m) unlimitedopen files (-N) 65536pipe Size (bytes,-p) 8POSIX Messag e Queues (bytes,-Q) 819200real-time priority (-R) 0stack size (Kbytes,-s) 10240cpu time (seconds, -T) Unlimitedmax user Processes (-u) 20000virtual memory (Kbytes,-V) unlimitedfile locks (-X) Unlimited
Note: Under current users, you can open up to 65,536 file descriptors per process.
2. View the number of processes currently open by the application process
[[email protected] log]$ Lsof-p 2526 | wc-l65785
Note: The current application opens a file of 65785 that is clearly exceeding the 65536 limit, causing the process to subsequently fail to open the new file.
3. View a large number of deleted files for a separate process via the lsof command
Note: Many files no longer exist, but the file's descriptor is still open.
[[email protected] log]$ Lsof-p 2526 | grep deleted | wc-l65274
Note: There are 65,274 files in deleted. Visible, most of the file descriptor occupies are deleted files.
4. Close the application process and release the open file
[[email protected] log]$ kill-9 2526[[email protected] log]$ lsof-p 2526 | Wc-l0
5. Restart the application and view the open files
[Email protected] log]$ Ps-ef | grep Scheduler_hdfs | Grep-v grep | awk ' {print $} ' 29639[[email protected] log]$ lsof-p 29639 | Wc-l485[[email protected] log]$ lsof-p 29639 | grep deleted | Wc-l0
Note: After the application restarts, the open files are released. The daemon can be processed correctly.
Document creation Time: April 8, 2018 21:25:33
Java Program background error Java.net.SocketException:Too many open files