Shell process exits after SSH connection disconnects

problem Description: when SSH connects remotely to the server and then runs a service./catalina.sh start, and then the terminal opened and closed (disconnected SSH connection), found that the service is interrupted, resulting in the Web page is inaccessible.

Workaround:Use the Nohup command to allow the program to continue running in the background when the window is closed (switching SSH connections). Unix/linux generally like to let a program run in the background, many use & at the end of the program to let the program run automatically. For example we want to run MySQL in the background:/usr/local/mysql/bin/mysqld_safe--user=mysql & But joining many of our programs is not a daemon like mysqld, maybe our program is just a normal program, Generally this program uses & end, but if the terminal is closed, then the program will be closed. But in order to be able to run in the background, then we can use the Nohup command, such as we have a test.php need to run in the background, and want to be able to run regularly in the background, then use nohup:nohup/root/test.php & Tip: [~]$ Appending output to nohup.out well, it proves that the operation was successful, and that the program running outputs were placed in the Nohup.out file of the current directory. nohup Command Description:  Use: Run the command without hanging up.   Syntax: nohup Command [Arg ...] [&]  Description: The nohup command runs commands specified by the command parameter and any associated ARG parameter, ignoring all hang-up (SIGHUP) signals. Use the Nohup command to run a program in the background after logging off. To run the Nohup command in the background, add & (the symbol representing "and") to the end of the command.   Regardless of whether the output of the Nohup command is redirected to the terminal, the output is appended to the Nohup.out file in the current directory. If the nohup.out file for the current directory is not writable, the output is redirected to the $HOME/nohup.out file. If no file can be created or opened for appending, then the command specified by the commands parameter is not callable. If the standard error is a terminal, then all output of the specified command to the standard error is redirected to the same file descriptor as the standard output.   Exit Status: The command returns the following exit Values:  126 can find but cannot invoke the command specified by the command parameter. The   127 nohup command has an error or cannot find the command specified by the commands parameter.   Otherwise, the exit status of the Nohup command is the command parameter that specifies the exit status of the commands.   NOHUP command and its output file   nohup command: If you are running a process and you feel that the process will not end when you exit the account, you can use the Nohup command. This command can continue to run the process after you exit the account/close the terminal. Nohup is the meaning of not hanging (n ohang up).   The general form of the command is: Nohup command &  submit job using NOHUP commands   If the job is submitted with the Nohup command, Then, by default, all output of the job is redirected to a file named Nohup.out unless an output file is specified: (that is, the file name of the custom output)   nohup command > Myout.file 2>&1 &  in the example above, the output is redirected to the Myout.file file.   Use jobs to view tasks.   closed with FG%n.   Two other commonly used FTPTools Ncftpget and Ncftpput, can be implemented in the background FTP upload and download, so I can use these commands in the background to upload and download files.   thinking: Question 1 Why is the program no longer running when SSH is closed?

Culprit: SIGHUP Signal
Let's see why shutting down the window/disconnecting will cause the running program to die.

In Linux/unix, there are several concepts:
Process group: A collection of one or more processes, each with a unique process group ID, which is the ID of the process leader process.
Session Duration: A collection of one or more process groups, with a unique session-period-first process (session leader). The ID of the session ID that is the lead process.
The session period can have a separate control terminal (controlling terminal). The session first process connected to the control terminal is called the control process (controlling processes). The process that is currently interacting with the terminal is called the foreground process group. The remaining process groups are called background process groups.
According to POSIX.1 definition:
Hang-up signal (SIGHUP) The default action is to terminate the program.
When the terminal interface detects a disconnected network connection, the hang-up signal is sent to the control process (session-first process).
If the session first process terminates, the signal is sent to the foreground process group for that session period.
A process exit causes an orphan process group to occur if any of the orphaned process group processes are in the stop state, sending Sighup and sigcont signals to all processes in the process group . (For the orphan process, refer to:http://blog.csdn.net/hmsiwtv/article/details/7901711 )
Conclusion: Therefore, when the network is disconnected or the terminal window is closed, that is, after SSH disconnects, the control process receives a SIGHUP signal exit, which causes the other process to exit during the session.

In short: After SSH is opened, Bash is his subroutine, once SSH is off, the system will kill all related processes!! Cause once SSH is closed, the task in execution is canceled.

Example:
Let's look at an example. Open the two SSH terminal window and run the top command in one of them.
[email protected] root]# Top

In another terminal window, find the process ID of top 5180 and its parent process ID is 5128, which is the login shell.
[Email protected] root]# Ps-ef|grep Top
Root 5180 5128 0 01:03 pts/0 00:00:02 top
Root 5857 3672 0 01:12 pts/2 00:00:00 grep top

Using the Pstree command, you can see this relationship more clearly:
[Email protected] root]# pstree-h 5180|grep Top
|-SSHD-+-SSHD---bash---top


Using the PS-XJ command, you can see that the login shell (PID 5128) and top are in the same session period, the shell is the session first process, the process group Pgid is the 5128,top process group Pgid is 5180, is the foreground process group.
[Email protected] root]# Ps-xj|grep 5128
5126 5128 5128 5128 pts/0 5180 S 0 0:00-bash
5128 5180 5180 5128 pts/0 5180 S 0 0:50 Top
3672 18095 18094 3672 pts/2 18094 S 0 0:00 grep 5128

Close the first SSH window, and in the other window you can see that top is also killed.
[Email protected] root]# Ps-ef|grep 5128
Root 18699 3672 0 04:35 pts/2 00:00:00 grep 5128

2 Why does the daemon even turn off ssh if SSH is turned on, without affecting its operation?
Because their program is special, such as Httpd–k start running this, he does not belong to the SSHD process group, but a separate process group, so even if the ssh closed, and he does not have any relationship!
[[email protected] ~]# Pstree |grep http
|-httpd
[Email protected] ~]# pstree |grep Top
|-SSHD-+-SSHD---bash---top


Conclusion: The daemon's start command itself is special, unlike the General command, such as Mysqld_safe, once used, is the daemon run. So it's impossible to transform the general program into a daemon.

Issue 3 using the background to run the command & can get rid of the program SSH process Group control it is SSH shutdown, daemon continue to run?
We do an experiment: Find/-name ' *http* ' &
Do you see this command to run after you log out of ctrl+d and then enter the system?
The answer is: the command was aborted!!

Because he still belongs to this SSH process group even add & can't get rid of!!
[Email protected] ~]# Pstree |grep Find
|-SSHD-+-SSHD---Bash---find

The conclusion is: as long as SSH open the execution of the General Command, not the daemon, regardless of &amp, once SSH is closed, the system will be terminated with Sighup

Question 4 nohup can solve the problem
But in order to be able to log out again and still be able to run the background, then we can use nohup this command, we now start looking for Find/-name ' *http* ' &
, and want to run in the background,
Then use Nohup:nohup Find/-name "*httpd*"
At this point, the default program runs the output information into the current folder in the Nohup.out file
Adding & does not affect this command just to get the program foreground or background to run. extension: Linux command nohup+screen commandIf you want to continue running the program that you just started after you close the SSH connection, you can use Nohup. But if you ask for the next day, when you open ssh, you can see the state of the program that was running yesterday, and then continue working, then Nohup is out of the process and needs to use screen for this purpose.

Although Nohup is easy to use, it is still relatively "crude", for simple commands can be dealt with, for the complex needs of human-computer interaction task is troublesome.
In fact, we can use a more powerful utility screen. A popular Linux distribution, such as Red Hat Enterprise Linux 4, usually comes with the screen utility, and if not, it can be downloaded from the official website of the GNU screen.

1) Use
Execute screen, press any key to enter sub-interface;
I use the ping command to start execution, if off, but want to close the SSH after the ping to continue to run, then press CTRL + A and then press D to pause the sub-interface, will show [detached] The word, this time I returned to the parent interface;
Use Screen–ls to view the status of the current sub-interface Screen-ls
There is a screen on:22292.pts-3.free (Detached)
1 Socket in/tmp/screens/s-root, here 22292 is actually the PID number of the sub-interface;

If you go back to the sub-interface with Screen–r 22292, a sudden play to ping sub-interface;

2) More help
You can see all the key bindings by C-a (Ctrl + a), the key bindings commonly used are:

C-a?
Show all key binding information
C-a W
Show list of all windows
C-a c-a
Switch to the previously displayed window
C-a C
Create a new window to run the shell and switch to that window
C-a N
Switch to the next window
C-a P
Switch to the previous window (relative to C-a N)
C-a 0..9
Switch to Window 0: 9
C-a A
Send c-a to current window
C-a D
Temporarily disconnecting a screen session
C-a K
Kill the current window
C-a [
Enter copy/rollback mode

Other common options:

-C file
Using the profile file instead of using the default $HOME/.SCREENRC
-d|-d [Pid.tty.host]
Instead of opening a new screen session, you are disconnecting other running screen sessions
-H num
Specifies that the history rollback buffer size is num rows
-list|-ls
List existing screen sessions in pid.tty.host format
-d-m
Start a session that starts in disconnected mode
-R sessionowner/[Pid.tty.host]
Reconnect a disconnected session. Connect to other users in multi-user mode screen session needs to specify Sessionowner, requires Setuid-root permissions
-S SessionName
Assigning a name to a session when creating a screen session
-V
Show screen version information
-wipe [Match]
Same-list, but delete disconnected sessions Other information:

1. Linux tips: Several ways to keep processes running reliably in the background, https://www.ibm.com/developerworks/cn/linux/l-cn-nohup/

Shell process exits after SSH connection disconnects