EzDevInfo.com

Creating multiple SSH connections at a time using Paramiko

The code below runs grep in one machine through SSH and prints the results:

import sys, os, string
import paramiko

cmd = "grep -h 'king' /opt/data/horror_20100810*"

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('10.10.3.10', username='xy', password='xy')
stdin, stdout, stderr = ssh.exec_command(cmd)
stdin.write('xy\n')
stdin.flush()

print stdout.readlines()

How can I grep five machines all at once (so that I don't have major delay), than put all that in five variables and print them all out.

How can you get the SSH return code using Paramiko?

client = paramiko.SSHClient()
stdin, stdout, stderr = client.exec_command(command)

Is there any way to get the command return code?

It's hard to parse all stdout/stderr and know whether the command finished successfully or not.

Running interactive commands in Paramiko

I'm trying to run an interactive command through paramiko. The cmd execution tries to prompt for a password but I do not know how to supply the password through paramiko's exec_command and the execution hangs. Is there a way to send values to the terminal if a cmd execution expects input interactively?

ssh = paramiko.SSHClient()
ssh.connect(server, username=username, password=password)
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("psql -U factory -d factory -f /tmp/data.sql")

Does anyone know how this can addressed? Thank you.

Nested SSH session with Paramiko

I'm rewriting a Bash script I wrote into Python. The crux of that script was

ssh -t first.com "ssh second.com very_remote_command"

I'm having a problem with the nested authentication with paramiko. I wasn't able to find any examples dealing with my precise situation, but I was able to find examples with sudo on a remote host.

The first method writes to stdin

ssh.connect('127.0.0.1', username='jesse', password='lol')
stdin, stdout, stderr = ssh.exec_command("sudo dmesg")
stdin.write('lol\n')
stdin.flush()

The second creates a channel and uses the socket-like send and recv.

I was able to get stdin.write to work with sudo, but it doesn't work with ssh on the remote host.

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('first.com', username='luser', password='secret')
stdin, stdout, stderr = ssh.exec_command('ssh luser@second.com')
stdin.write('secret')
stdin.flush()
print '---- out ----'
print stdout.readlines()
print '---- error ----'
print stderr.readlines()

ssh.close()

...prints...

---- out ----
[]
---- error ----
['Pseudo-terminal will not be allocated because stdin is not a terminal.\r\n', 'Permission denied, please try again.\r\n', 'Permission denied, please try again.\r\n', 'Permission denied (publickey,password,keyboard-interactive).\r\n']

The pseudo-terminal error reminded me of the -t flag in my original command, so I switched to the second method, using a Channel. Instead of ssh.exec_command and later, I have:

t = ssh.get_transport()
chan = t.open_session()
chan.get_pty()
print '---- send ssh cmd ----'
print chan.send('ssh luser@second.com')
print '---- recv ----'
print chan.recv(9999)
chan = t.open_session()
print '---- send password ----'
print chan.send('secret')
print '---- recv ----'
print chan.recv(9999)

...but it prints '---- send ssh cmd ----' and just hangs until I kill the process.

I'm new to Python and none too knowledgeable about networks. In the first case, why does sending the password work with sudo but not with ssh? Are the prompts different? Is paramiko even the right library for this?

How do you execute multiple commands in a single session in Paramiko? (Python)

def exec_command(self, command, bufsize=-1):
    #print "Executing Command: "+command
    chan = self._transport.open_session()
    chan.exec_command(command)
    stdin = chan.makefile('wb', bufsize)
    stdout = chan.makefile('rb', bufsize)
    stderr = chan.makefile_stderr('rb', bufsize)
    return stdin, stdout, stderr

When executing a command in paramiko, it always resets the session when you run exec_command. I want to able to execute sudo or su and still have those privileges when I run another exec_command. Another example would be trying to exec_command("cd /") and then run exec_command again and have it be in the root directory. I know you can do something like exec_command("cd /; ls -l"), but I need to do it in separate function calls.

Long-running ssh commands in python paramiko module (and how to end them)

I want to run a tail -f logfile command on a remote machine using python's paramiko module. I've been attempting it so far in the following fashion:

interface = paramiko.SSHClient()
#snip the connection setup portion
stdin, stdout, stderr = interface.exec_command("tail -f logfile")
#snip into threaded loop
print stdout.readline()

I'd like the command to run as long as necessary, but I have 2 problems:

1. How do I stop this cleanly? I thought of making a Channel and then using the shutdown() command on the channel when I'm through with it- but that seems messy. Is it possible to do something like sent Ctrl-C to the channel's stdin?
2. readline() blocks, and I could avoid threads if I had a non-blocking method of getting output- any thoughts?

How to ssh connect through python Paramiko with public key

i'm using Paramiko to connect through ssh to a server.

Basic authentication works well, but i can't understand how to connect with public key.

When i connect with putty, the server tell me this:

Using username "root".
Authenticating with public key "rsa-key@ddddd.com"
Passphrase for key "rsa-key@ddddd.com": [i've inserted the passphrase here]
Last login: Mon Dec  5 09:25:18 2011 from ...

I connect to it with this ppk file:

PuTTY-User-Key-File-2: ssh-rsa
Encryption: aes256-cbc
Comment: rsa-key@dddd.com
Public-Lines: 4
[4 lines key]
Private-Lines: 8
[8 lines key]
Private-MAC: [hash]

With basic auth the error i get (from the log) is:

DEB [20111205-09:48:44.328] thr=1   paramiko.transport: userauth is OK
DEB [20111205-09:48:44.927] thr=1   paramiko.transport: Authentication type (password) not permitted.
DEB [20111205-09:48:44.927] thr=1   paramiko.transport: Allowed methods: ['publickey', 'gssapi-with-mic']

I've tried to include that ppk file and set to auth_public_key, but didn't work.

Can you help me?

How do I change directories using Paramiko?

Paramiko stderr object won't reset

I posted the above question regarding a persistent error message that I receive using Paramiko. I do not think it is related to my next question, but it might be.

I can successfully connect to my server via SSH using Paramiko. I can execute commands like ls or pwd. What I can't seem to do is change directories. I can send the command "cd .." for example, but when I follow up with "pwd" it shows that I haven't changed directories. It just lists the initial directory I am in when I log in.

>>> stdin, stdout, stderr = myssh.exec_command("pwd")
>>> stdout.readlines()
['/big/dom/home/myid\n']
>>> stdin, stdout, stderr = myssh.exec_command("cd ../")
>>> stdout.readlines()
[]
>>> stdin, stdout, stderr = myssh.exec_command("pwd")
>>> stdout.readlines()
['/big/dom/home/myid\n']
>>>

Am I misunderstanding what is going on here? Should I not be able to change directories? Or if I can, should I be doing it in some other way than using exec_command?

--

I can't answer my own question for another 7 hours, so here's the answer:

This guy had it figured out: http://www.vertigrated.com/blog/2010/02/python-remote-ssh-with-paramiko/

You just have to send multiple commands with one exec_command, such as:

myssh.exec_command('cd ..; pwd')

Then stdout.readlines() will return the directory that you changed to.

'Put' in SFTP using PAramiko

I've installed and written the following Paramiko which is unable to put the file. It is easily able to 'get' a file and execute 'ls' commands on it.

#set username & password
username='runaway'
password='runaway'
port=22
source= '/Unzip.sh' 
destination ='/var/mpx/www/http'


#SFTP
client.load_system_host_keys()
print " hostname =%s \n username=%s \n password=%s \n" (hostname,username,password)
t = paramiko.Transport((hostname, port)) 
t.connect(username=username,password=password)
sftp = paramiko.SFTPClient.from_transport(t)
sftp.put(source,destination)
#sftp.close()
#t.close()

Using a 'put' command gives the following error & stack trace -

File "upload_file.py", line 84, in ?
    sftp.put(source,destination)
  File "/usr/lib/python2.4/site-packages/paramiko/sftp_client.py", line 522, in put
    fr = self.file(remotepath, 'wb')
  File "/usr/lib/python2.4/site-packages/paramiko/sftp_client.py", line 221, in open
    t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
  File "/usr/lib/python2.4/site-packages/paramiko/sftp_client.py", line 572, in _request
    return self._read_response(num)
  File "/usr/lib/python2.4/site-packages/paramiko/sftp_client.py", line 619, in _read_response
    self._convert_status(msg)
  File "/usr/lib/python2.4/site-packages/paramiko/sftp_client.py", line 649, in _convert_status
    raise IOError(text)
IOError: Failure

How do I overcome this?

Dynamic Hosts and Parallel Tasks with Fabric library

Is there a way to dynamically modify hosts in between multiple parallel tasks? This is what I have so far.

def init_hosts():
    env.hosts = [host1,host2,host3,host4]

@parallel
def task_1():
    if condition is False:
        env.hosts.remove(env.host)

@parallel
def task_2():
    run('uname -s')

Obviously I'm missing some env paramenters, but I only want task_2 to run on hosts that satisfy the condition in task_1. It appears task_2's host list is initialized on startup, because it's running on all hosts in the initial env.hosts list defined in init_hosts(). I also tried dynamically modifying and building roledefs, but had the same result.

Edit: Also, is there a way to setup a parallel execution queue such that multiple parallel tasks are executed in parallel rather than sequentially?

Edit: I managed to get my desired end result by having each task return output, and parsing the output to build a new host list to pass to execute():

def init_hosts():
    env.hosts = [host1,host2,host3,host4]

@parallel
def task_1():
    if condition is False:
        return False
    else:
        return True

@parallel
def task_2():
   run('uname -s')

def run_tests():
   results = execute(task_1)
   successful_hosts = [k for k in results.iterkeys() if results[k]]
   execute(test_2, hosts=successful_hosts)

This works, but it's gross for many reasons. Is there a better way?

How to suppress a third-party warning using warnings.filterwarnings

I am using Paramiko in my python code (for sftp). Everything works fine except that everytime I import or call a paramiko function. This warning would show up:

C:\Python26\lib\site-packages\Crypto\Util\randpool.py:40: RandomPool_Deprecation
Warning: This application uses RandomPool, which is BROKEN in older releases.  S
ee http://www.pycrypto.org/randpool-broken
  RandomPool_DeprecationWarning)

I know that this has to do with the fact that Paramiko is using some Deprecated functionalities of PyCrypto.

My question is, is there a way to suppress this warning programmatically ? I have tried this:

warnings.filterwarnings(action='ignore', \
category=DeprecationWarning, module='paramiko')

and even this:

warnings.filterwarnings(action='ignore', \
category=DeprecationWarning, module='randpool')

before 'import paramiko' statement and before paramiko-specific function calls, but nothing works. This warning keeps showing up no matter what. If it helps, here's the code in the third party library that prints the warning:

in randpool.py:

from Crypto.pct_warnings import RandomPool_DeprecationWarning
import Crypto.Random
import warnings

class RandomPool:
    """Deprecated.  Use Random.new() instead.

    See http://www.pycrypto.org/randpool-broken
    """
    def __init__(self, numbytes = 160, cipher=None, hash=None, file=None):
        warnings.warn("This application uses RandomPool, which is BROKEN in older releases.  See http://www.pycrypto.org/randpool-broken",
            RandomPool_DeprecationWarning)

If you know a way around this, please help me shut this warning off.

x11 forwarding with paramiko

I'm trying to run a command with paramiko that should be able to open an X window. The script I'm using would something as follows:

import paramiko                                    

ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect('192.168.122.55', username='user', password='password')
transport = ssh_client.get_transport()
session = transport.open_session()

session.request_x11()
stdin = session.makefile('wb')
stdout = session.makefile('rb')
stderr = session.makefile_stderr('rb')
session.exec_command('env; xterm')
transport.accept()

print 'Exit status:', session.recv_exit_status()
print 'stdout:\n{}'.format(stdout.read())
print 'stderr:\n{}'.format(stderr.read())
session.close()

Unfortunately, when I run the script above, I get this output:

Exit status: 1
stdout:
SHELL=/bin/bash
XDG_SESSION_COOKIE=8025e1ba5e6c47be0d2f3ad6504a25ee-1347286654.617967-1932974971
SSH_CLIENT=192.168.122.1 58654 22
USER=user
MAIL=/var/mail/user
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PWD=/home/user
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/user
LOGNAME=user
SSH_CONNECTION=192.168.122.1 58654 192.168.122.55 22
DISPLAY=localhost:10.0
_=/usr/bin/env

stderr:  
xterm: Xt error: Can't open display: localhost:10.0

If I run the following command in a terminal:

ssh -X user@192.168.122.55 'env; xterm'

then I get the same environment variables (some ports changed, though), so I'd say that my environment is correct. However, I'm still missing something to make paramiko work with x11 forwarding.

A couple of things I tried are:

• Use the handler parameter in request_x11: aside from printing values, I didn't get any further than with the default handler.
• Use the auth_cookie parameter in request_x11: tried to hardcode a cookie value that was being used according to the xauth list output. The idea of doing this was to avoid problems that might happen according to the documentation string in paramiko itself:

If you omit the auth_cookie, a new secure random 128-bit value will be generated, used, and returned. You will need to use this value to verify incoming x11 requests and replace them with the actual local x11 cookie (which requires some knoweldge of the x11 protocol).

Is there some other thing I could do to make it work or troubleshoot the problem?

Note: This has been previously asked in:

• superuser: the only response points to the request_x11 documentation I've already tried to use to no avail.
• stackoverflow: the accepted response suggests to use the handler parameter, but it's wrong.
• github: no answer provided for more than a year.

Paramiko "Unknown Server"

I'm trying to get started with the Paramiko library, but the library is throwing an exception as soon as I try to connect with the following simple program:

import paramiko
ssh = paramiko.SSHClient()
ssh.connect('127.0.0.1', username='boatzart', password='mypassword')

The error I get is:

Traceback (most recent call last):
File "test.py", line 6, in <module>
ssh.connect('127.0.0.1')
File "build/bdist.macosx-10.7-intel/egg/paramiko/client.py", line 316, in connect
File "build/bdist.macosx-10.7-intel/egg/paramiko/client.py", line 85, in missing_host_key
paramiko.SSHException: Unknown server 127.0.0.1

This occurs no matter which server I try.

Turning off logging in Paramiko

I am using the ssh client provided by Paramiko to create a function call 'remoteSSH' (the file name is remoteConnect.py):

import paramiko
import logging
logger = paramiko.util.logging.getLogger()
logger.setLevel(logging.WARN)

def remoteSSH(username,userpasswd):
    ....

Now I am calling the remoteSSH function in another Python module called getData() (getdata.py):

from remoteConnect import *
import logging
logger2=logging.getLogger()
logger2.setLevel(logging.INFO)

However, a call to logger2.info('ccc') also turns on all INFO level logging in the file that is importing the Paramiko module (i.e. remoteConnect.py)

How do I turn off logging in remoteConnect.py so that Paramiko does not spit out all the INFO level messages?

Could gunicorn cause an issue with exscript/paramiko?

I have a Django project running behind Nginx, and Gunicorn. One of the applications interacts with network devices using Exscript, which is in turn using Paramiko. Some devices do not work properly when they are behind Gunicorn.

The same exact code will work fine from within the django-admin shell. It will also work if I launch the built in django server, but I still get the error if I bypass nginx, and connect directly to Gunicorn.

I tried moving the functionality to a celery task, it had the same exact problem, but only behind Gunicorn.

I wrote a script using django-extensions that works from the command line, but will fail if called via subprocess. But only behind Gunicorn.

The devices that are failing all seem to be F5 LTMs, and it looks like the buffer on the exscript object is being modified somehow. If I had to guess I would say that Gunicorn, and Exscript/Paramiko are somehow stepping on each others memory, or perhaps Gunicorn is setting some environment variable that Exscript is picking up on.

In any case I am thoroughly stumped, and would appreciate any guidance on how to troubleshooot this.