EzDevInfo.com

How to set explicitly the terminal size when using pexpect

I have a ncurses app that checks terminal size at startup and exits immediately if it doesn't fit.

In Linux, the default size is 80x24, this app requires at least 25. The fix is easy, I'm just resizing the terminal emulation window (in X) before running the ncurses app.

I would like to automate the ncurses app with pexpect, but I'm getting stuck because it considers the terminal size smaller than required when launched through pexpect, so it doesn't run. Any way to specify the terminal size with pexpect explicitly at startup?

fabric vs pexpect

I've stumbled upon pexpect and my impression is that it looks roughly similar to fabric. I've tried to find some comparison, without success, so I'm asking here--in case someone has experience with both tools.

Is my impression (that they are roughly equivalent) correct, or it's just how it looks on the surface ?

Is there an implementation of 'expect' or an expect-like library that works in python3?

I would like to use an expect-like module in python3. As far as I know, neither pexpect nor fabric work with python3. Is there any similar package I can use? (If no, does anyone know if py3 support is on any project's roadmap?)

A perfectly overlapping feature set isn't necessary. I don't think my use case is necessary here, but I'm basically reimplementing a Linux expect script that does a telnet with some config-supplied commands, but extending functionality.

Is there autoexpect for pexpect?

I would like to generate Python Expect (pexpect) code automatically, does something like autoexpect exist for pexpect?

how to get console output from a remote computer (ssh + python)

I have googled "python ssh". There is a wonderful module pexpect, which can access a remote computer using ssh (with password).

After the remote computer is connected, I can execute other commands. However I cannot get the result in python again.

p = pexpect.spawn("ssh user@remote_computer")
print "connecting..."
p.waitnoecho()
p.sendline(my_password)
print "connected"
p.sendline("ps -ef")
p.expect(pexpect.EOF) # this will take very long time
print p.before

How to get the result of ps -ef in my case?

pexpect can't pass input over 1024 chars?

I'm currently passing some input to a process with pexpect with the following code:

p = pexpect.spawn('cat', timeout=5.0 )
p.maxread = 5000
p.setecho(False) # prevent the process from echoing stdin back to us
INPUT_LEN = 1024
p.sendline('a'*INPUT_LEN)
print p.readline() # pexpect.TIMEOUT: Timeout exceeded in read_nonblocking().

When INPUT_LEN < 1024, everything works fine, but for >= 1024 characters, the process does not receive the full input, causing raising a "pexpect.TIMEOUT" error on p.readline().

I've tried splitting my input into pieces smaller than 1024 characters, but this has the same issue:

p = pexpect.spawn('cat', timeout=5.0 )
p.maxread = 5000
p.setecho(False)
INPUT_LEN = 1024
p.send('a'*1000)
p.sendline('a'*(INPUT_LEN-1000))
print p.readline() # pexpect.TIMEOUT: Timeout exceeded in read_nonblocking().

Does anyone know how to make pexpect work with inputs over 1024 characters? I tried looking at the source, but it just seems to be calling os.write(...).

(As a side note, I've noticed the same truncation error occurs when I run "cat" from a shell and attempt to paste in >=1024 characters with "Cmd+V". However, everything works fine if I run "pbpaste | cat" .)

Thanks!

Update: The call to "os.write()" returns 1025, indicating a successful write, but os.read() returns "\x07" (the single character BEL), and then hangs on the next call, resulting in the timeout.

Dividing the os.write() call into two sub-1024 byte write()s, separated by a call to os.fsync(), does not change anything.

Pexpect and PyCharm - Inappropriate ioctl for device

I'm trying to run a basic Pexpect script:

import pexpect
ftp_process = pexpect.spawn('ftp')
ftp_process.interact()

When the code is run directly from a terminal, the code works as expected. If I run the code using PyCharm's run/debug I get the following error:

Traceback (most recent call last):
  File "/path/to/code/test.py", line 3, in <module>
    ftp_process.interact()
  File "/usr/local/lib/python3.4/site-packages/pexpect/__init__.py", line 1645, in interact
    mode = tty.tcgetattr(self.STDIN_FILENO)
termios.error: (25, 'Inappropriate ioctl for device')

It seems that how Pexpect interacts with PyCharm's run/debug window doesn't work by default. Is there some way to remedy this with a specific PyCharm setting? If not, is there some other way to work around this?

EDIT

The code above is simply a shortened example which results in the problem. The other abilities of pexpect (such as expect(), sendline(), etc) are still desired.

Why is Pexpect intermittently hanging (not detecting EOF) after executing certain commands?

Context:

I have some code written using pexpect, whose job is to give "live" output of a command. I.e. print something out when a command generates some output, or soon after, rather than waiting until the command completes and then interacting with its output.

All I'm doing is starting and stopping a service. I do this by spawning the process, and then outputting each line as it is printed, like so:

def watch(process):
    output = ""
    while True:
        try:
            line = process.read_nonblocking(timeout = -1)
            print(line, end ="")
            output += line
        except pexpect.EOF:
            break
    del process
    return output

while True:
    print("IN 1")
    process = pexpect.spawn("service",["zend-server", "stop"], timeout = None)
    watch(process)
    print("OUT 1")

    print("IN 2")
    process = pexpect.spawn("service",["zend-server", "start"], timeout = None)
    watch(process)
    print("OUT 2")

This code should just loop the service: start it and stop it over and over, printing the output of the start/stop as it goes. It prints the output fine. However, it eventually hangs right before "OUT 2". I can view the output, and see the service call stop its execution. The watch function never raises an EOF and exits, however.

This doesn't happen with every service. Some services loop indefinitely. zend-server, however, along with a few other unrelated commands, fail intermittently in the same way.

By "eventually hangs", I mean that it starts/stops the service a few (variable on each run) times, and hangs. It usually gums up after 4-6, though never on the first call--always at least on the second (hence the del statement; I figured I'd play it safe).

Python 2.6.6, CentOS (64) 6.3, Pexpect 2.3-6, FWIW

Question:

Why is pexpect hanging on certain commands? How should I resolve this issue? Using timeouts isn't a feasible solution, as some of these commands really can run for an arbitrarily long time. service zend-server stop is just the one I picked for an example because it doesn't take that long, and I can observe it finishing.

What I've Tried:

I've tried replacing the watch method with the following, which uses expect('\n'), but the results are the same: a variable number of restarts, and then an eventual hang.

I can also add pexpect.EOF into the array that is expected along with \n, and handle the return value to break out of the loop, it still hangs in the same place.

def watch2(process):
    output = ""
    done = False
    while True:
        try:
            if not process.isalive():
                line = process.readline()
                done = True
            else:
                process.expect(['\n'])
                line = process.before
             print(line)
            output += line
            if done:
                raise pexpect.EOF(0)
        except pexpect.EOF:
            break
    del process
    return output

What is a 'NoneType' object? [closed]

I'm getting this error when I run my python script:

TypeError: cannot concatenate 'str' and 'NoneType' objects

I'm pretty sure the 'str' means string, but I dont know what a 'NoneType' object is. My script craps out on the second line, I know the first one works because the commands from that line are in my asa as I would expect. At first I thought it may be because I'm using variables and user input inside send_command.

Everything in 'CAPS' are variables, everything in 'lower case' is input from 'parser.add_option' options.

I'm using pexpect, and optparse

send_command(child, SNMPGROUPCMD + group + V3PRIVCMD)
send_command(child, SNMPSRVUSRCMD + snmpuser + group + V3AUTHCMD + snmphmac + snmpauth + PRIVCMD + snmpencrypt + snmppriv)

pexpect setecho not working

I am trying to telnet to Cisco Router and give commands using pexpect. Its working, but the sendline() repeats in the output. even after using setecho to False. Code is:

'''
Created on Nov 19, 2012

@author: Amit Barik
'''

import pexpect

hostname = 'hostname'
login_cmd = 'telnet ' + hostname + '.net'
username = 'username'
password = 'pwd'
prompt = hostname + '#'

p = pexpect.spawn(login_cmd)
p.setecho(False)
p.logfile = open('Log.log', 'w+')

p.expect('Username:')
print '1',repr(p.before)

p.sendline(username)
p.expect('Password:')
print '2',repr(p.before)

p.sendline(password)
p.expect(prompt)
print '3',repr(p.before)

cmd = 'show clock'
p.sendline(cmd)
p.expect(prompt)
print 'Output for {0}'.format(cmd), repr(p.before)

Output is:

# On Python Console
Output for show clock 'show clock\r\n00:16:40.692 UTC Tue Nov 20 2012\r\n'

# On Log File
Username: username
username
Password: pwd

My Cisco Banner

hostname#show clock
show clock
00:16:40.692 UTC Tue Nov 20 2012
hostname#

Verify a file exists over ssh

I am trying to test if a file exists over SSH using pexpect. I have got most of the code working but I need to catch the value so I can assert whether the file exists. The code I have done is below:

def VersionID():

        ssh_newkey = 'Are you sure you want to continue connecting'
        # my ssh command line
        p=pexpect.spawn('ssh service@10.10.0.0')

        i=p.expect([ssh_newkey,'password:',pexpect.EOF])
        if i==0:
            p.sendline('yes')
            i=p.expect([ssh_newkey,'password:',pexpect.EOF])
        if i==1:
            p.sendline("word")
            i=p.expect('service@main-:')
            p.sendline("cd /opt/ad/bin")
            i=p.expect('service@main-:')
            p.sendline('[ -f email_tidyup.sh ] && echo "File exists" || echo "File does not exists"')
            i=p.expect('File Exists')
            i=p.expect('service@main-:')
            assert True
        elif i==2:
            print "I either got key or connection timeout"
            assert False

        results = p.before # print out the result

VersionID()

Thanks for any help.

python, set terminal type in pexpect

I have a script which uses pexpect to start a CLI program. It works a bit like a shell where you get a prompt where you can enter some commands.

The problem I have, I think, is that this program uses a coloured prompt.

This is what I do

import pprint
import pexpect

1 a = pexpect.spawn('program')
2 a.expect("prompt>")
3 print "---------start------------"
4 print(a.before)
5 a.sendline("command")
6 a.expect("prompt>")
7 print "---------before------------"
8 pprint.pprint(a.before)
9 print "---------after------------"
10 pprint.pprint(a.after)

This is the output:

> python borken.py
---------start------------
A lot of text here from the enjoying programs start-up, lorem ipsum ...  
---------before------------
' \x1b[0m\x1b[8D\x1b[K\x1b[1m\x1b[34m'
---------after------------
'prompt>'

For some reason the first prompt colour coding borkens up things and a.before at line 8 is garbled, normal print does not work, even if I see that the command at line 5 actually produced a lot of output.

Does someone know what the problem could be, or is it possible to set the terminal type in pexpect to avoid the colours?

I am using tcsh shell

Pexpect - silence ssh connection output

I'm using a simple pexpect script to ssh to a remote machine and grab a value returned by a command. Is there any way, pexpect or sshwise I can use to ignore the unix greeting? That is, from

    child = pexpect.spawn('/usr/bin/ssh %s@%s' % (rem_user, host))
    child.expect('[pP]assword: ', timeout=5)
    child.sendline(spass)
    child.expect([pexpect.TIMEOUT, prompt])
    child.before = '0'
    child.sendline ('%s' % cmd2exec)
    child.expect([pexpect.EOF, prompt])

    # Collected data processing
    result = child.before
    # logon to the machine returns a lot of garbage, the returned executed command is at the 57th position
    print result.split('\r\n') [57]
    result = result.split('\r\n') [57]

How can I simply get the returned value, ignoring, the "Last successful login" and "(c)Copyright" stuff and without having to concern with the value correct position?

Thanks !

Pexpect, running ssh-copy-id is hanging when trying to spawn a second process

I'm doing a Python script where I need to spawn several ssh-copy-id processes, and they need for me to type in a password, so i'm using PExpect.

I have basically this:

child = pexpect.spawn('command')
child.expect('password:')
child.sendline('the password')

and then I want to spawn another process, I don't care about this one anymore, whether it ended or not.

child = pexpect.spawn('command2')
child.expect('password:')
child.sendline('the password')

And the code is hanging at the second "spawn"

However, if I comment out the first call, the second one works, so i'm guessing that the fact that the first one is still running or something is keeping it from working.

Now, the other thing I haven't been able to do is wait until the first one stops. I've tried:
child.close() - it hangs (both with True and False as parameters) child.read(-1) - it hangs
child.expect(pexpect.EOF) - it hangs.
child.terminate() - it hangs (both with True and False as parameters)

Any ideas on what could be happening?
NOTE: I'm not a Python expert, and i have never used pexpect before, so ANY idea is more than welcome.

Thanks!

UPDATE: This is definitely related to ssh-copy-id, because with other processes, spawn works well even if they don't return. Also, apparently ssh-copy-id never returns an EOF.

pexpect - run script.sh over ssh

I'm having trouble programmatically running a local script over ssh.
I'm unsure if this is a problem with the shell variable substitution on the local host.

When manually running,

ssh monit@server1 'bash -s' < /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh

I get the expected output,

CPU pctUser pctNice pctSystem pctIowait pctIdle
all 11.21 0.00 1.50 0.31 86.98
0 0.00 0.00 0.00 0.00 100.00
1 3.00 0.00 1.00 0.00 96.00 ....

but I get

bash: /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh: No such file or directory

when running the following code,

splunk_bin_dir = '/u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin'
hostname = 'server1'
username = 'monit'
password = 'monit#_'


command = "/usr/bin/ssh %(username)s@%(hostname)s 'bash -s' < %(splunk_bin_dir)s/cpu.sh" % locals()
print command

ssh_new_conn = 'Are you sure you want to continue connecting'

p = pexpect.spawn(command, timeout=360)
# Handles the 3 possible connection outcomes:
# a) Ssh to the remote host for the first time, triggering 'Are you sure you want to continue connecting'
# b) ask you for password
# c) No password is needed at all, because you already have the key.
i = p.expect([ssh_new_conn,'[pP]assword:',pexpect.EOF])
print ' Initial pexpect command output: ', i
if i == 0:
    # send 'yes'
    p.sendline('yes')
    i = p.expect(['[pP]assword:',pexpect.EOF])
    print 'sent yes. pexpect command output', i
    if i == 0:
        # send the password
        p.sendline(password)
        p.expect(pexpect.EOF)
elif i == 1:
    # send the password
    p.sendline(password)
    p.expect(pexpect.EOF)
elif i == 2:
    print "pexpect faced key or connection timeout"
    pass

print p.before

These are the printed outputs,

/usr/bin/ssh monit@server1 'bash -s' < /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh
Initial pexpect command output: 1
bash: /u02/splunk/splunk/etc/apps/Splunk_TA_nix/bin/cpu.sh: No such file or directory

pexpect is bumping into the [pP]assword line so I guess the password is being correctly passed,