causes:
Last week to several clusters of machine upgrade package, each cluster about 2000 servers, but in the process of software delivery and batch execution of commands in the course of two have encountered problems, in the batch execution of the command will always be stuck in the back for a long time can not quit, finally had to manually kill the process.
If the SSHPT batch execution of the command, to the end of a long time, and later reported a TypeError exception, because the return value is not the result of the command output list but an exception object, this exception is not the key, is aunt of the result type judgment is not complete. The real reason should be that a machine causes the thread not to quit.
SSHPT supports timeout parameters,-T or--timeout
Sshpt-help-t <seconds>,--timeout=<seconds> timeout (in seconds) before giving up over an SSH Connection (DEFAULT:30)
Looking at Sshpt's code about timeout, we found that this timeout was only used in Ssh.connect, and the Exec_command call did not pass in timeout.
Def paramikoconnect (host, username, password, timeout, port=22): "" "connects to ' host ' and returns a paramiko transport object to use in further communications "" " # Uncomment this line to turn on Paramiko debugging (Good for troubleshooting why some servers report connection failures) #paramiko. Util.log_to_file (' Paramiko.log ') ssh = Paramiko. Sshclient () try: ssh.set_missing_host_ Key_policy (Paramiko. Autoaddpolicy ()) ssh.connect (host, port=port, username=username, password=password, timeout=timeout) #ssh. Incoming timeout in Connect except exception, detail: # connecting failed (For whatever reason) ssh = str (detail) return sshdef sudoexecute (Transport, command, password, run_as= ' root '): "" "Executes the given command via sudo as the specified user (run_as) using the given Paramiko transport object. returns stdout, stderr (after command Execution) "" " stdin, stdout, stderr = transport.exec_command (" sudo -s -u %s %s " % (Run_as, command)) #exec_ There is no timeout control in command if stdout.channel.closed is false: # if stdout is still open then sudo is asking us for a password stdin.write ('%s\n ' % password) Stdin.flush () return stdout, stderr
The Paramiko code is a support for the timeout parameter of the Exec_command passed in
class sshclient (Closingcontextmanager): def exec_command ( self, command, bufsize=-1, timeout=none, #支持timeout get_pty=False, environment=None, ): chan = self._transport.open_session (timeout=timeout) if get_pty: chan.get_pty () chan.settimeout (timeout) if environment: Chan.update_environmenT (Environment) chan.exec_command (command) stdin = chan.makefile (' WB ', bufsize) stdout = chan.makefile (' R ', bufsize) stderr = chan.makefile_stderr (' R ', bufsize) return stdin, stdout, stderr
The network card rate change causes the Paramiko module timeout to fail, multithreading timeout control solution.