forked from OSchip/llvm-project
869 lines
44 KiB
HTML
869 lines
44 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<html>
|
|
<head>
|
|
<title>Pexpect - a Pure Python Expect-like module</title>
|
|
<link rel="stylesheet" href="clean.css" type="text/css">
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
|
<meta name="Author" content="Noah Spurrier">
|
|
<meta name="Keywords"
|
|
content="pexpect, Noah Spurrier, pypect, Python, Libes, TCL, Expect, pipe, popen, pyExpect, expectpy, expect-like, expect-alike, expect like">
|
|
<meta name="Description"
|
|
content="Pexpect is a pure Python Expect-like module. Pexpect makes Python a better tool for controlling other applications.">
|
|
</head>
|
|
<body bgcolor="#ffffff" text="#000000">
|
|
<div id="Header">
|
|
<h1>Pexpect version VERSION<br>
|
|
a Pure Python Expect-like module
|
|
</h1>
|
|
</div>
|
|
<div id="Content">
|
|
<p>Pexpect makes Python a better tool for controlling other
|
|
applications.</p>
|
|
<p>Pexpect is a pure Python module for spawning child applications;
|
|
controlling them; and responding to expected patterns in their output.
|
|
Pexpect works like Don Libes' Expect. Pexpect allows your script to
|
|
spawn a child application and control it as if a human were typing
|
|
commands.</p>
|
|
<p>Pexpect can be used for automating interactive applications such as
|
|
ssh, ftp, passwd, telnet, etc. It can be used to a automate setup
|
|
scripts for duplicating software package installations on different
|
|
servers. It can be used for automated software testing. Pexpect is in
|
|
the spirit of Don Libes' Expect, but Pexpect is pure Python. Unlike
|
|
other Expect-like modules for Python, Pexpect does not require TCL or
|
|
Expect nor does it require C extensions to be compiled. It should work
|
|
on any platform that supports the standard Python pty module. The
|
|
Pexpect interface was designed to be easy to use.</p>
|
|
<table border="0">
|
|
<tbody>
|
|
<tr>
|
|
<td align="right" valign="top">Send questions to:</td>
|
|
<td align="left"><a href="http://www.noah.org/email/"><img
|
|
src="email.png" alt="Click to send email." border="0" height="16"
|
|
width="100"></a></td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="license"></a>License: MIT style</h1>
|
|
<p>
|
|
Free, open source, and all that good stuff.<br>
|
|
<br>
|
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
of this software and associated documentation files (the "Software"), to deal
|
|
in the Software without restriction, including without limitation the rights
|
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
copies of the Software, and to permit persons to whom the Software is
|
|
furnished to do so, subject to the following conditions:<br>
|
|
<br>
|
|
The above copyright notice and this permission notice shall be included in all
|
|
copies or substantial portions of the Software.<br>
|
|
<br>
|
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
|
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
|
|
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
|
|
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
|
|
USE OR OTHER DEALINGS IN THE SOFTWARE.<br>
|
|
<br>
|
|
Pexpect Copyright (c) 2008 Noah Spurrier<br>
|
|
http://pexpect.sourceforge.net/
|
|
</p>
|
|
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="download"></a><a
|
|
href="http://sourceforge.net/project/showfiles.php?group_id=59762">Download</a></h1>
|
|
<p>Download the <a
|
|
href="http://sourceforge.net/project/showfiles.php?group_id=59762">
|
|
current version here</a> from the SourceForge site. Grab the current Pexpect tarball.
|
|
</p>
|
|
<h2>Installing Pexpect</h2>
|
|
<p>The Pexpect tarball is a standard Python Distutil distribution.</p>
|
|
<ol>
|
|
<li>download <span class="code">pexpect-VERSION.tar.gz</span></li>
|
|
<li><span class="code">tar zxf pexpect-VERSION.tar.gz</span></li>
|
|
<li><span class="code">cd pexpect-VERSION</span></li>
|
|
<li><span class="code">python setup.py install</span> <i>do this as root</i></li>
|
|
</ol>
|
|
<h2>Examples</h2>
|
|
<p>
|
|
Under the <span class="code">pexpect-VERSION</span> directory you should find
|
|
the <span class="code">examples</span> directory.
|
|
This is the best way to learn to use Pexpect.
|
|
See the descriptions of <a href="examples.html">Pexpect Examples</a>.
|
|
</p>
|
|
<h2><a name="doc"></a>API Documentation</h2>
|
|
<p>
|
|
<blockquote>
|
|
<a href="pexpect.html">pexpect</a> This is the main module that you want.<br>
|
|
<a href="pxssh.html">pxssh</a> Pexpect SSH is an extension of 'pexpect.spawn' that specializes in SSH.<br>
|
|
</blockquote>
|
|
the following are experimental extensions to Pexpect<br>
|
|
<blockquote>
|
|
<a href="fdpexpect.html">fdpexpect</a> fdpexpect extension of 'pexpect.spawn' that uses an open file descriptor.<br>
|
|
<a href="screen.html">SCREEN</a> This represents a virtual 'screen'.<br>
|
|
<a href="ANSI.html">ANSI</a> This parses ANSI/VT-100 terminal escape codes.<br>
|
|
<a href="FSM.html">FSM</a> This is a finite state machine used by ANSI.<br>
|
|
</blockquote>
|
|
</p>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="status"></a>Project Status</h1>
|
|
<p>Automated pyunit tests reach over 80%
|
|
code coverage on pexpect.py. I regularly test on Linux and BSD
|
|
platforms. I try to test on Solaris and Irix.
|
|
</p>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="requirements"></a>Requirements for use of Pexpect</h1>
|
|
<h2>Python</h2>
|
|
<blockquote>
|
|
<p>Pexpect was written and tested with Python 2.4. It should work on
|
|
earlier versions that have the <span class="code">pty</span> module. I
|
|
sometimes even manually test it with Python 1.5.2, but I can't easily
|
|
run the PyUnit test framework against Python 1.5.2, so I have less
|
|
confidence in Pexpect on Python 1.5.2.</p>
|
|
</blockquote>
|
|
<h2>pty module</h2>
|
|
<blockquote>
|
|
<p>Any POSIX system (UNIX) with a working <span class="code">pty</span>
|
|
module should be able to run Pexpect. The <span class="code">pty</span>
|
|
module is part of the Standard Python Library, so if you are running on
|
|
a POSIX system you should have it. The <span class="code">pty</span>
|
|
module does not run the same on all platforms. It should be solid on Linux
|
|
and BSD systems. I have taken effort to try to smooth the wrinkles out of the different platforms. To learn more
|
|
about the wrinkles see <a href="#bugs">Bugs</a> and <a href="#testing">Testing</a>.</p>
|
|
</blockquote>
|
|
<p>Pexpect does not currently work on the standard Windows Python (see
|
|
the pty requirement); however, it seems to work fine using <a
|
|
href="http://www.cygwin.com/">Cygwin</a>. It is possible to build
|
|
something like a pty for Windows, but it would have to use a different
|
|
technique that I am still investigating. I know it's possible because
|
|
Libes' Expect was ported to Windows. <i>If you have any ideas or
|
|
skills to contribute in this area then I would really appreciate some
|
|
tips on how to approach this problem.</i> </p>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="overview"></a>Overview</h1>
|
|
<p>Pexpect can be used for automating interactive applications such as
|
|
ssh, ftp, mencoder, passwd, etc. The Pexpect interface was designed to be
|
|
easy to use. Here is an example of Pexpect in action:</p>
|
|
<blockquote>
|
|
<pre class="code"># This connects to the openbsd ftp site and<br># downloads the recursive directory listing.<br>import pexpect<br>child = pexpect.spawn ('ftp ftp.openbsd.org')<br>child.expect ('Name .*: ')<br>child.sendline ('anonymous')<br>child.expect ('Password:')<br>child.sendline ('noah@example.com')<br>child.expect ('ftp> ')<br>child.sendline ('cd pub')<br>child.expect('ftp> ')<br>child.sendline ('get ls-lR.gz')<br>child.expect('ftp> ')<br>child.sendline ('bye')<br></pre>
|
|
</blockquote>
|
|
<p> Obviously you could write an ftp client using Python's own <span
|
|
class="code">ftplib</span> module, but this is just a demonstration.
|
|
You can use this technique with any application. This is especially
|
|
handy if you are writing automated test tools.</p>
|
|
|
|
<p>There are two important methods in Pexpect -- <span class="code"><b>expect()</b></span>
|
|
and <span class="code"><b>send()</b></span> (or <span class="code">sendline()</span>
|
|
which is like <span class="code">send()</span> with a linefeed).
|
|
The <span class="code">expect()</span> method waits for the child application
|
|
to return a given string. The string you specify is a regular expression, so
|
|
you can match complicated patterns. The <span class="code"><b>send()</b></span> method
|
|
writes a string to the child application. From the child's point of
|
|
view it looks just like someone typed the text from a terminal. After
|
|
each call to <span class="code"><b>expect()</b></span> the <span
|
|
class="code"><b>before</b></span> and <span class="code"><b>after</b></span>
|
|
properties will be set to the text printed by child application. The <span
|
|
class="code"><b>before</b></span> property will contain all text up to
|
|
the expected string pattern. The <span class="code"><b>after</b></span> string
|
|
will contain the text that was matched by the expected pattern.
|
|
The <span class="code">match</span> property is set to the <span class="code">re MatchObject</span>.
|
|
</p>
|
|
|
|
<p>An example of Pexpect in action may make things more clear. This example uses
|
|
<span class="code">ftp</span> to login to the OpenBSD site; list files
|
|
in a directory; and then pass interactive control of the ftp session to
|
|
the human user.</p>
|
|
<blockquote>
|
|
<pre class="code">import pexpect<br>child = pexpect.spawn ('ftp ftp.openbsd.org')<br>child.expect ('Name .*: ')<br>child.sendline ('anonymous')<br>child.expect ('Password:')<br>child.sendline ('noah@example.com')<br>child.expect ('ftp> ')<br>child.sendline ('ls /pub/OpenBSD/')<br>child.expect ('ftp> ')<br>print child.before # Print the result of the ls command.<br>child.interact() # Give control of the child to the user.<br></pre>
|
|
</blockquote>
|
|
<h2>Special EOF and TIMEOUT patterns</h2>
|
|
<p>
|
|
There are two special patterns to match the End Of File or a Timeout condition.
|
|
You you can pass these patterns to <span class="code">expect()</span>.
|
|
These patterns are not regular expressions. Use them like predefined constants.
|
|
</p>
|
|
<p>If the child has died and you have read all the child's output then ordinarily
|
|
<span class="code">expect()</span> will raise an <span class="code">EOF</span>
|
|
exception. You can read everything up to the EOF without generating an
|
|
exception by using the EOF pattern <span class="code">expect(pexpect.EOF)</span>.
|
|
In this case everything the child has output will be available in the <span
|
|
class="code">before</span> property.</p>
|
|
<p>The pattern given to <span class="code">expect()</span> may be a
|
|
regular expression or it may also be a <b>list</b> of regular expressions.
|
|
This allows you to match multiple optional responses. The <span class="code">expect()</span>
|
|
method returns the index of the pattern that was matched. For example,
|
|
say you wanted to login to a server. After entering a password you
|
|
could get various responses from the server -- your password could be
|
|
rejected; or you could be allowed in and asked for your terminal type;
|
|
or you could be let right in and given a command prompt. The following
|
|
code fragment gives an example of this:</p>
|
|
<blockquote>
|
|
<pre class="code">child.expect('password:')<br>child.sendline (my_secret_password)<br># We expect any of these three patterns...<br>i = child.expect (['Permission denied', 'Terminal type', '[#\$] '])<br>if i==0:<br> print 'Permission denied on host. Can't login'<br> child.kill(0)<br>elif i==2:<br> print 'Login OK... need to send terminal type.'<br> child.sendline('vt100')<br> child.expect ('[#\$] ')<br>elif i==3:<br> print 'Login OK.'<br> print 'Shell command prompt', child.after</pre>
|
|
</blockquote>
|
|
<p>If nothing matches an expected pattern then expect will eventually
|
|
raise a TIMEOUT exception. The default time is 30 seconds, but you can
|
|
change this by passing a timeout argument to expect():</p>
|
|
<blockquote>
|
|
<pre class="code"># Wait no more than 2 minutes (120 seconds) for password prompt.<br>child.expect('password:', timeout=120)</pre>
|
|
</blockquote>
|
|
<h2>Find the end of line -- CR/LF conventions<br>
|
|
Matching at the end of a line can be tricky<br>
|
|
$ regex pattern is useless.<br>
|
|
</h2>
|
|
<p>Pexpect matches regular expressions a little differently than what
|
|
you might be used to.
|
|
</p>
|
|
<p><i><b>The $ pattern for end of line match is useless</b></i>.
|
|
The $ matches the end of string, but Pexpect reads from the child
|
|
one character at a time, so each character looks like the end of a line.
|
|
Pexpect can't do a look-ahead into the child's output stream.
|
|
In general you would have this situation when using regular expressions
|
|
with any stream.<br>
|
|
<i>Note, pexpect does have an internal buffer, so reads are faster
|
|
than one character at a time, but from the user's perspective the regex
|
|
patterns test happens one character at a time.</i></p>
|
|
<p>The best way to match the end of a line is to look for the
|
|
newline: "\r\n" (CR/LF). Yes, that does appear to be DOS-style.
|
|
It may surprise some UNIX people to learn that terminal TTY device drivers
|
|
(dumb, vt100, ANSI, xterm, etc.) all use the CR/LF combination to signify
|
|
the end of line. Pexpect uses a Pseudo-TTY device to talk to the child application, so
|
|
when the child app prints "\n" you actually see "\r\n".
|
|
</p>
|
|
<p><b>UNIX uses just linefeeds to end lines of text, but not when it
|
|
comes to TTY devices!</b> TTY devices are more like the Windows world.
|
|
Each line of text end with a CR/LF combination. When you intercept data
|
|
from a UNIX command from a TTY device you will find that the TTY device
|
|
outputs a CR/LF combination. A UNIX command may only write a linefeed
|
|
(\n), but the TTY device driver converts it to CR/LF. This means that
|
|
your terminal will see lines end with CR/LF (hex <span class="code">0D 0A</span>).
|
|
Since Pexpect emulates a terminal, to match ends of lines you have to
|
|
expect the CR/LF combination.</p>
|
|
<blockquote>
|
|
<p class="code">child.expect ('\r\n')</p>
|
|
</blockquote>
|
|
<p>If you just need to skip past a new line then <span class="code">expect
|
|
('\n')</span> by itself will work, but if you are expecting a specific
|
|
pattern before the end of line then you need to explicitly look for the
|
|
\r. For example the following expects a word at the end of a line:</p>
|
|
<blockquote>
|
|
<p class="code">child.expect ('\w+\r\n')</p>
|
|
</blockquote>
|
|
<p>But the following would both fail:</p>
|
|
<blockquote>
|
|
<p class="code">child.expect ('\w+\n')</p>
|
|
</blockquote>
|
|
<p>And as explained before, trying to use '$' to match the end of line
|
|
would not work either:</p>
|
|
<blockquote>
|
|
<p class="code">child.expect ('\w+$')</p>
|
|
</blockquote>
|
|
<p>So if you need to explicitly look for the END OF LINE, you want to
|
|
look for the CR/LF combination -- not just the LF and not the $ pattern.</p>
|
|
<p>This problem is not limited to Pexpect. This problem happens any
|
|
time you try to perform a regular expression match on a stream. Regular
|
|
expressions need to look ahead. With a stream it is hard to look ahead
|
|
because the process generating the stream may not be finished. There is no
|
|
way to know if the process has paused momentarily or is finished and
|
|
waiting for you. <font color="#cc0000">Pexpect must implicitly always
|
|
do a NON greedy match (minimal) at the end of a input {### already said
|
|
this}.</font> </p>
|
|
<p>Pexpect compiles all regular expressions with the DOTALL flag. With
|
|
the DOTALL flag a "." will match a newline. See the Python <a
|
|
href="http://www.python.org/doc/current/lib/node115.html#l2h-733">documentation</a></p>
|
|
<h2>Beware of + and * at the end of input.</h2>
|
|
<p>Remember that any time you try to match a pattern that needs
|
|
look-ahead that you will always get a minimal match (non greedy). For
|
|
example, the following will always return just one character:</p>
|
|
<blockquote>
|
|
<p class="code">child.expect ('.+')</p>
|
|
</blockquote>
|
|
<p>This example will match successfully, but will always return no
|
|
characters:</p>
|
|
<blockquote>
|
|
<p class="code">child.expect ('.*')</p>
|
|
</blockquote>
|
|
<p>Generally any star * expression will match as little as possible</p>
|
|
<p>One thing you can do is to try to force a non-ambiguous character at
|
|
the end of your <span class="code">\d+</span> pattern. Expect that
|
|
character to delimit the string. For example, you might try making the
|
|
end of your pattrn be <span class="code">\D+</span> instead of <span
|
|
class="code">\D*</span>. That means number digits alone would not
|
|
satisfy the (<span class="code">\d+</span>) pattern. You would need
|
|
some number(s) and at least one <span class="code">\D</span> at the
|
|
end. </p>
|
|
<h2>Matching groups</h2>
|
|
<p>You can group regular expression using parenthesis. After a match,
|
|
the <span class="code">match</span> parameter of the spawn object will
|
|
contain the Python Match object. </p>
|
|
<h2>Examples</h2>
|
|
<p>Using "match" and groups...</p>
|
|
<h2>Debugging</h2>
|
|
<p>If you get the string value of a pexpect.spawn object you will get
|
|
lots of useful debugging information. For debugging it's very useful to
|
|
use the following pattern:</p>
|
|
<p>try:<br>
|
|
i = child.expect ([pattern1, pattern2, pattern3,
|
|
etc])<br>
|
|
except:<br>
|
|
print "Exception was thrown"<br>
|
|
print "debug information:"<br>
|
|
print str(child)<br>
|
|
</p>
|
|
<p>It is also useful to log the child's input and out to a file or the
|
|
screen. The following will turn on logging and send output to stdout
|
|
(the screen).<br>
|
|
</p>
|
|
<p> child = pexpect.spawn (foo)<br>
|
|
child.logfile = sys.stdout<br>
|
|
<br>
|
|
</p>
|
|
<hr noshade="noshade" size="1">
|
|
<h1>Exceptions</h1>
|
|
<p><b>EOF</b></p>
|
|
<p>Note that two flavors of EOF Exception may be thrown. They are
|
|
virtually identical except for the message string. For practical
|
|
purposes you should have no need to distinguish between them, but they
|
|
do give a little extra information about what type of platform you are
|
|
running. The two messages are:</p>
|
|
<blockquote>
|
|
<p class="code">End Of File (EOF) in read(). Exception style platform.</p>
|
|
<p class="code">End Of File (EOF) in read(). Empty string style
|
|
platform.</p>
|
|
</blockquote>
|
|
<p>Some UNIX platforms will throw an exception when you try to read
|
|
from a file descriptor in the EOF state. Other UNIX platforms instead
|
|
quietly return an empty string to indicate that the EOF state has been
|
|
reached.</p>
|
|
<p><b>Expecting EOF</b></p>
|
|
<p>If you wish to read up to the end of the child's output without
|
|
generating an <span class="code">EOF</span> exception then use the <span
|
|
class="code">expect(pexpect.EOF)</span> method.</p>
|
|
<p><b>TIMEOUT</b></p>
|
|
<p>The <span class="code">expect()</span> and <span class="code">read()</span>
|
|
methods will also timeout if the child does not generate any output for
|
|
a given amount of time. If this happens they will raise a <span
|
|
class="code">TIMEOUT</span> exception. You can have these method
|
|
ignore a timeout and block indefinitely by passing None for the timeout
|
|
parameter.</p>
|
|
<blockquote>
|
|
<p class="code">child.expect(pexpect.EOF, timeout=None)</p>
|
|
</blockquote>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="faq"></a>FAQ</h1>
|
|
<p><b>Q: Why don't shell pipe and redirect (| and >) work when I
|
|
spawn a command?</b></p>
|
|
<p>
|
|
|
|
A: Remember that Pexpect does NOT interpret shell meta characters such as
|
|
redirect, pipe, or wild cards (>, |, or *). That's done by a shell not the
|
|
command you are spawning. This is a common mistake. If you want to run a
|
|
command and pipe it through another command then you must also start a shell.
|
|
For example:
|
|
|
|
<pre>
|
|
child = pexpect.spawn('/bin/sh -c "ls -l | grep LOG > log_list.txt"')
|
|
child.expect(pexpect.EOF)
|
|
</pre>
|
|
|
|
The second form of spawn (where you pass a list of arguments) is useful in
|
|
situations where you wish to spawn a command and pass it its own argument list.
|
|
This can make syntax more clear. For example, the following is equivalent to
|
|
the previous example:
|
|
|
|
<pre>
|
|
shell_cmd = 'ls -l | grep LOG > log_list.txt'
|
|
child = pexpect.spawn ('/bin/sh', ['-c', shell_cmd])
|
|
child.expect (pexpect.EOF)
|
|
</pre>
|
|
|
|
</p>
|
|
<p><b>Q: Isn't there already a Python Expect?</b></p>
|
|
<p>A: Yes, there are several of them. They usually require you to
|
|
compile C. I wanted something that was pure Python and preferably a
|
|
single module that was simple to install. I also wanted something that
|
|
was easy to use. This pure Python expect only recently became possible
|
|
with the introduction of the pty module in the standard Python library.
|
|
Previously C extensions were required.</p>
|
|
|
|
<p><strong>Q: The before and after properties sound weird.</strong></p>
|
|
<p>Originally I was going to model Pexpect more after Expect, but then
|
|
I found that I could never remember how to get the context of the stuff
|
|
I was trying to parse. I hate having to read my own documentation. I
|
|
decided that it was easier for me to remember what before and after
|
|
was. It just so happens that this is how the -B and -A options in grep
|
|
works, so that made it even easier for me to remember. Whatever makes
|
|
my life easier is what's best.</p>
|
|
|
|
<p><b>Q: Why not just use Expect?</b></p>
|
|
<p>A: I love it. It's great. I has bailed me out of some real jams, but
|
|
I wanted something that would do 90% of what I need from Expect; be 10%
|
|
of the size; and allow me to write my code in Python instead of TCL.
|
|
Pexpect is not nearly as big as Expect, but Pexpect does everything I
|
|
have ever used Expect for.
|
|
<!-- :-P If I liked TCL then you wouldn't be reading this. My appologies to Don Libes -- Expect is cool, TK is cool, but TCL is only slightly better than Perl in my book. Hopefully after Expyct is done I will not need to use Expect anymore -- except for that lovely autoexpect tool. Damn, I wish I had that! --> </p>
|
|
|
|
<p><b>Q: Why not just use a pipe (popen())?</b></p>
|
|
<p>A: A pipe works fine for getting the output to non-interactive
|
|
programs. If you just want to get the output from <span class="code">ls</span>,
|
|
<span class="code">uname</span>, or <span class="code">ping</span>
|
|
then this works. Pipes do not work very well for interactive programs
|
|
and pipes will almost certainly fail for most applications that ask for
|
|
passwords such as telnet, ftp, or ssh.</p>
|
|
<p>There are two reasons for this. </p>
|
|
<p>First an application may bypass stdout and print directly to its
|
|
controlling TTY. Something like SSH will do this when it asks you for a
|
|
password. This is why you cannot redirect the password prompt because
|
|
it does not go through stdout or stderr.</p>
|
|
<p>The second reason is because most applications are built using the C
|
|
Standard IO Library (anything that uses <span class="code">#include
|
|
<stdio.h></span>). One of the features of the stdio library is
|
|
that it buffers all input and output. Normally output is <b><i>line
|
|
buffered</i></b> when a program is printing to a TTY (your terminal
|
|
screen). Every time the program prints a line-feed the currently
|
|
buffered data will get printed to your screen. The problem comes when
|
|
you connect a pipe. The stdio library is smart and can tell that it is
|
|
printing to a pipe instead of a TTY. In that case it switches from line
|
|
buffer mode to <i><b>block buffered</b></i>. In this mode the
|
|
currently buffered data is flushed when the buffer is full. This causes
|
|
most interactive programs to deadlock. Block buffering is more
|
|
efficient when writing to disks and pipes. Take the situation where a
|
|
program prints a message "Enter your user name:\n" and then waits for
|
|
you type type something. In block buffered mode, the stdio library will
|
|
not put the message into the pipe even though a linefeed is printed.
|
|
The result is that you never receive the message, yet the child
|
|
application will sit and wait for you to type a response. Don't confuse
|
|
the stdio lib's buffer with the pipe's buffer. The pipe buffer is
|
|
another area that can cause problems. You could flush the input side of
|
|
a pipe, whereas you have no control over the stdio library buffer. </p>
|
|
<p>More information: the Standard IO library has three states for a
|
|
FILE *. These are: _IOFBF for block buffered; _IOLBF for line buffered;
|
|
and _IONBF for unbuffered. The STDIO lib will use block buffering when
|
|
talking to a block file descriptor such as a pipe. This is usually not
|
|
helpful for interactive programs. Short of recompiling your program to
|
|
include fflush() everywhere or recompiling a custom stdio library there
|
|
is not much a controlling application can do about this if talking over
|
|
a pipe.</p>
|
|
<p> The program may have put data in its output that remains unflushed
|
|
because the output buffer is not full; then the program will go and
|
|
deadlock while waiting for input -- because you never send it any
|
|
because you are still waiting for its output (still stuck in the
|
|
STDIO's output buffer).</p>
|
|
<p>The answer is to use a pseudo-tty. A TTY device will force <i><b>line</b></i>
|
|
buffering (as opposed to block buffering). Line buffering means that
|
|
you will get each line when the child program sends a line feed. This
|
|
corresponds to the way most interactive programs operate -- send a line
|
|
of output then wait for a line of input.</p>
|
|
<p>I put "answer" in quotes because it's ugly solution and because
|
|
there is no POSIX standard for pseudo-TTY devices (even though they
|
|
have a TTY standard...). What would make more sense to me would be to
|
|
have some way to set a mode on a file descriptor so that it will tell
|
|
the STDIO to be line-buffered. I have investigated, and I don't think
|
|
there is a way to set the buffered state of a child process. The STDIO
|
|
Library does not maintain any external state in the kernel or whatnot,
|
|
so I don't think there is any way for you to alter it. I'm not quite
|
|
sure how this line-buffered/block-buffered state change happens
|
|
internally in the STDIO library. I think the STDIO lib looks at the
|
|
file descriptor and decides to change behavior based on whether it's a
|
|
TTY or a block file (see isatty()).</p>
|
|
<p>I hope that this qualifies as helpful.</p>
|
|
|
|
<h1>Don't use a pipe to control another application...</h1>
|
|
<p>Pexpect may seem similar to <span class="code">os.popen()</span> or
|
|
<span class="code">commands</span> module. The main difference is that
|
|
Pexpect (like Expect) uses a pseudo-TTY to talk to the child
|
|
application. Most applications do no work well through the system()
|
|
call or through pipes. And probably all applications that ask a user to
|
|
type in a password will fail. These applications bypass the stdin and
|
|
read directly from the TTY device. Many applications do not explicitly
|
|
flush their output buffers. This causes deadlocks if you try to control
|
|
an interactive application using a pipe. What happens is that most UNIX
|
|
applications use the stdio (#include <stdio.h>) for input and
|
|
output. The stdio library behaves differently depending on where the
|
|
output is going. There is no way to control this behavior from the
|
|
client end.<br>
|
|
</p>
|
|
|
|
<p><b>Q: Can I do screen scraping with this thing?</b></p>
|
|
<p>A: That depends. If your application just does line-oriented output
|
|
then this is easy. If it does screen-oriented output then it may work,
|
|
but it could be hard. For example, trying to scrape data from the 'top'
|
|
command would be hard. The top command repaints the text window. </p>
|
|
<p>I am working on an ANSI / VT100 terminal emulator that will have
|
|
methods to get characters from an arbitrary X,Y coordinate of the
|
|
virtual screen. It works and you can play with it, but I have no
|
|
working examples at this time.</p>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="bugs"></a>Bugs</h1>
|
|
<h2>Threads</h2>
|
|
<p>On Linux (RH 8) you cannot spawn a child from a different thread and
|
|
pass the handle back to a worker thread. The child is successfully
|
|
spawned but you can't interact with it. The only way to make it work is
|
|
to spawn and interact with the child all in the same thread. [Adam
|
|
Kerrison] </p>
|
|
<h2><a name="echo_bug"></a>Timing issue with send() and sendline()</h2>
|
|
<p>This problem has been addressed and should not effect most users.</p>
|
|
<p>It is sometimes possible to read an echo of the string sent with <span
|
|
class="code">send()</span> and <span class="code">sendline()</span>.
|
|
If you call <span class="code">sendline()</span> and then immediately
|
|
call <span class="code">readline()</span> you may get part of your
|
|
output echoed back. You may read back what you just wrote even if the
|
|
child application does not explicitly echo it. Timing is critical. This
|
|
could be a security issue when talking to an application that asks for
|
|
a password; otherwise, this does not seem like a big deal. <i>But why
|
|
do TTYs do this</i>?</p>
|
|
<p>People usually report this when they are trying to control SSH or
|
|
some other login. For example, if your code looks something like this: </p>
|
|
<pre class="code">child.expect ('[pP]assword:')<br>child.sendline (my_password)</pre>
|
|
<p><br>
|
|
<blockquote>
|
|
1. SSH prints "password:" prompt to the user.<br>
|
|
2. SSH turns off echo on the TTY device.<br>
|
|
3. SSH waits for user to enter a password.<br>
|
|
</blockquote>
|
|
When scripting with Pexpect what can happen is that Pexpect will response to the "password:" prompt
|
|
before SSH has had time to turn off TTY echo. In other words, Pexpect sends the password between
|
|
steps 1. and 2., so the password gets echoed back to the TTY. I would call this an SSH bug.
|
|
</p>
|
|
<p>
|
|
Pexpect now automatically adds a short delay before sending data to a child process.
|
|
This more closely mimics what happens in the usual human-to-app interaction.
|
|
The delay can be tuned with the 'delaybeforesend' attribute of the spawn class.
|
|
In general, this fixes the problem for everyone and so this should not be an issue
|
|
for most users. For some applications you might with to turn it off.
|
|
child = pexpect.spawn ("ssh user@example.com")
|
|
child.delaybeforesend = 0
|
|
</p>
|
|
<p><br>
|
|
</p>
|
|
<p>Try changing it to look like the following. I know that this fix
|
|
does not look correct, but it works. I have not figured out exactly
|
|
what is happening. You would think that the sleep should be after the
|
|
sendline(). The fact that the sleep helps when it's between the
|
|
expect() and the sendline() must be a clue.</p>
|
|
<pre class="code">child.expect ('[pP]assword:')<br>child.sendline (my_password)</pre>
|
|
<h2>Timing issue with isalive()</h2>
|
|
<p>Reading the state of isalive() immediately after a child exits may
|
|
sometimes return 1. This is a race condition. The child has closed its
|
|
file descriptor, but has not yet fully exited before Pexpect's
|
|
isalive() executes. Addings a slight delay before the isalive() will
|
|
help. In the following example <span class="code">isalive()</span>
|
|
sometimes returns 1:</p>
|
|
<blockquote>
|
|
<pre class="code">child = pexpect.spawn('ls')<br>child.expect(pexpect.EOF)<br>print child.isalive()</pre>
|
|
</blockquote>
|
|
<p>But if there is any delay before the call to <span class="code">isalive()</span>
|
|
then it will always return 0 as expected.</p>
|
|
<blockquote>
|
|
<pre class="code">child = pexpect.spawn('ls')<br>child.expect(pexpect.EOF)<br>time.sleep(0.1)<br>print child.isalive()</pre>
|
|
</blockquote>
|
|
|
|
<h2>Truncated output just before child exits</h2>
|
|
<p><i>So far I have seen this only on older versions of <b>Apple's MacOS X</b>.</i>
|
|
If the child application quits it may not flush its output buffer. This
|
|
means that your Pexpect application will receive an EOF even though it
|
|
should have received a little more data before the child died. This is
|
|
not generally a problem when talking to interactive child applications.
|
|
One example where it is a problem is when trying to read output from a
|
|
program like '<span class="code">ls</span>'. You may receive most of
|
|
the directory listing, but the last few lines will get lost before you
|
|
receive an EOF. The reason for this is that '<span class="code">ls</span>'
|
|
runs; completes its task; and then exits. The buffer is not flushed
|
|
before exit so the last few lines are lost. The following example
|
|
demonstrates the problem:</p>
|
|
<p> </p>
|
|
<blockquote>
|
|
<pre class="code">child = pexpect.spawn ('ls -l')<br>child.expect (pexpect.EOF)<br>print child.before <br> </pre>
|
|
</blockquote>
|
|
<p></p>
|
|
|
|
<h2>Controlling SSH on Solaris</h2>
|
|
<p>Pexpect does not yet work perfectly on Solaris.
|
|
One common problem is that SSH sometimes will not allow TTY password
|
|
authentication. For example, you may expect SSH to ask you for a
|
|
password using code like this:
|
|
</p>
|
|
<pre class="code">child = pexpect.spawn ('ssh user@example.com')<br>child.expect ('assword')<br>child.sendline ('mypassword')<br></pre>
|
|
You may see the following error come back from a spawned
|
|
child SSH:
|
|
<p></p>
|
|
<blockquote>Permission denied (publickey,keyboard-interactive). </blockquote>
|
|
<p>
|
|
This means that SSH thinks it can't access the TTY to ask you for your
|
|
password.
|
|
The only solution I have found is to use public key authentication with
|
|
SSH.
|
|
This bypasses the need for a password. I'm not happy with this
|
|
solution.
|
|
The problem is due to poor support for Solaris Pseudo TTYs in the
|
|
Python
|
|
Standard Library. </p>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="changes"></a>CHANGES</h1>
|
|
<h2>Current Release</h2>
|
|
<p>Fixed OSError exception when a pexpect object is cleaned up.
|
|
Previously you might have seen this exception:</p>
|
|
<blockquote>
|
|
<pre class="code">Exception exceptions.OSError: (10, 'No child processes') <br>in <bound method spawn.__del__ of<br><pexpect.spawn instance at 0xd248c>> ignored</pre>
|
|
</blockquote>
|
|
<p>You should not see that anymore. Thanks to Michael Surette.</p>
|
|
<p>Added support for buffering reads. This greatly improves speed when
|
|
trying to match long output from a child process. When you create an
|
|
instance of the spawn object you can then set a buffer size. For now
|
|
you MUST do the following to turn on buffering -- it may be on by
|
|
default in future version.</p>
|
|
<blockquote>
|
|
<pre class="code">child = pexpect.spawn ('my_command')<br>child.maxread=1000 # Sets buffer to 1000 characters.</pre>
|
|
</blockquote>
|
|
<div>
|
|
<p>I made a subtle change to the way TIMEOUT and EOF exceptions behave.
|
|
Previously you could either expect these states in which case pexpect
|
|
will not raise an exception, or you could just let pexpect raise an
|
|
exception when these states were encountered. If you expected the
|
|
states then the 'before' property was set to everything before the
|
|
state was encountered, but if you let pexpect raise the exception then
|
|
'before' was not set. Now the 'before' property will get set either way
|
|
you choose to handle these states.</p>
|
|
<h2><i>Older changes...</i></h2>
|
|
<p>The spawn object now provides iterators for a <i>file-like interface</i>.
|
|
This makes Pexpect a more complete file-like object. You can now write
|
|
code like this:</p>
|
|
<blockquote>
|
|
<pre class="code">child = pexpect.spawn ('ls -l')<br>for line in child:<br> print line<br></pre>
|
|
</blockquote>
|
|
<p>I added the attribute <span class="code">exitstatus</span>. This
|
|
will give the exit code returned by the child process. This will be set
|
|
to <span class="code">None</span> while the child is still alive. When
|
|
<span class="code">isalive()</span> returns 0 then <span class="code">exitstatus</span>
|
|
will be set.</p>
|
|
<p>I made a few more tweaks to <span class="code">isalive()</span> so
|
|
that it will operate more consistently on different platforms. Solaris
|
|
is the most difficult to support.</p>
|
|
<p> </p>
|
|
<p>You can now put <span class="code">TIMEOUT</span> in a list of
|
|
expected patterns. This is just like putting <span class="code">EOF</span>
|
|
in the pattern list. Expecting for a <span class="code">TIMEOUT</span>
|
|
may not be used as often as <span class="code">EOF</span>, but this
|
|
makes Pexpect more consitent.</p>
|
|
<p>Thanks to a suggestion and sample code from Chad J. Schroeder I
|
|
added the ability for Pexpect to operate on a file descriptor that is
|
|
already open. This means that Pexpect can be used to control streams
|
|
such as those from serial port devices. Now you just pass the integer
|
|
file descriptor as the "command" when contsructing a spawn open. For
|
|
example on a Linux box with a modem on ttyS1:</p>
|
|
<blockquote>
|
|
<pre class="code">fd = os.open("/dev/ttyS1", os.O_RDWR|os.O_NONBLOCK|os.O_NOCTTY)<br>m = pexpect.spawn(fd) # Note integer fd is used instead of usual string.<br>m.send("+++") # Escape sequence<br>m.send("ATZ0\r") # Reset modem to profile 0<br>rval = m.expect(["OK", "ERROR"])</pre>
|
|
</blockquote>
|
|
<h3>Pexpect now tests itself on Compile Farm!</h3>
|
|
<p>I wrote a nice script that uses ssh to connect to each machine on
|
|
Source Forge's Compile Farm and then run the testall.py script for each
|
|
platform. The result of the test is then recorded for each platform.
|
|
Now it's easy to run regression tests across multiple platforms.</p>
|
|
<h3>Pexpect is a file-like object</h3>
|
|
<p>The spawn object now provides a <i>file-like interface</i>. It
|
|
supports most of the methods and attributes defined for Python File
|
|
Objects. </p>
|
|
<p>I changed write and writelines() so that they no longer return a
|
|
value. Use send() if you need that functionality. I did this to make
|
|
the Spawn object more closely match a file-like object.</p>
|
|
<p>read() was renamed to read_nonblocking(). I added a new read()
|
|
method that matches file-like object interface. In general, you should
|
|
not notice the difference except that read() no longer allows you to
|
|
directly set the timeout value. I hope this will not effect any
|
|
existing code. Switching to read_nonblocking() should fix existing code.</p>
|
|
<p>I changed the name of <span class="code">set_echo()</span> to <span
|
|
class="code">setecho()</span>.</p>
|
|
<p>I changed the name of <span class="code">send_eof()</span> to <span
|
|
class="code">sendeof()</span>.</p>
|
|
<p>I modified <span class="code">kill()</span> so that it checks to
|
|
make sure the pid isalive().</p>
|
|
<p>I modified <span class="code">spawn()</span> (really called from <span
|
|
class="code">__spawn()</span>)so that it does not raise an expection
|
|
if <span class="code">setwinsize()</span> fails. Some platforms such
|
|
as Cygwin do not like setwinsize. This was a constant problem and since
|
|
it is not a critical feature I decided to just silence the error.
|
|
Normally I don't like to do that, but in this case I'm making an
|
|
exception.</p>
|
|
<p>Added a method <span class="code">close()</span> that does what you
|
|
think. It closes the file descriptor of the child application. It makes
|
|
no attempt to actually kill the child or wait for its status. </p>
|
|
<p>Add variables <span class="code">__version__</span> and <span
|
|
class="code">__revision__</span> (from cvs) to the pexpect modules.
|
|
This is mainly helpful to me so that I can make sure that I'm testing
|
|
with the right version instead of one already installed.</p>
|
|
<h3>Logging changes</h3>
|
|
<blockquote>
|
|
<p><span class="code">log_open()</span> and <span class="code">log_close()</span>
|
|
have been removed. Now use <span class="code">setlog()</span>. The <span
|
|
class="code">setlog()</span> method takes a file object. This is far
|
|
more flexible than the previous log method. Each time data is written
|
|
to the file object it will be flushed. To turn logging off simply call <span
|
|
class="code">setlog()</span> with None.</p>
|
|
</blockquote>
|
|
<h2>isalive changes</h2>
|
|
<blockquote>
|
|
<p>I renamed the <span class="code">isAlive()</span> method to <span
|
|
class="code">isalive()</span> to match the more typical naming style
|
|
in Python. Also the technique used to detect child process status has
|
|
been drastically modified. Previously I did some funky stuff with
|
|
signals which caused indigestion in other Python modules on some
|
|
platforms. It's was a big headache. It still is, but I think it works
|
|
better now.</p>
|
|
</blockquote>
|
|
<h3>attribute name changes</h3>
|
|
<blockquote>
|
|
<p>The names of some attributes have been changed. This effects the
|
|
names of the attributes that are set after called the <span
|
|
class="code">expect()</span> method.</p>
|
|
<table class="pymenu" border="0" cellpadding="5">
|
|
<tbody>
|
|
<tr>
|
|
<th class="pymenu">NEW NAME</th>
|
|
<th class="pymenu">OLD NAME</th>
|
|
</tr>
|
|
<tr>
|
|
<td><span class="code">before</span><br>
|
|
<i>Everything before the match.</i></td>
|
|
<td><span class="code">before</span></td>
|
|
</tr>
|
|
<tr>
|
|
<td><span class="code">after</span><br>
|
|
<i>Everything after and including the first character of the
|
|
match</i></td>
|
|
<td><span class="code">matched</span></td>
|
|
</tr>
|
|
<tr>
|
|
<td><span class="code">match</span><br>
|
|
<i>This is the re MatchObject from the match.<br>
|
|
You can get groups() from this.<br>
|
|
See '<span class="code">uptime.py</span>' in the examples tar ball.</i></td>
|
|
<td><i>New -- Did not exist</i></td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</blockquote>
|
|
<h3>EOF changes</h3>
|
|
<blockquote>
|
|
<p>The <span class="code">expect_eof()</span> method is gone. You
|
|
can now simply use the <span class="code">expect()</span> method to
|
|
look for EOF.</p>
|
|
<p>Was:</p>
|
|
<blockquote>
|
|
<p><span class="code">p.expect_eof ()</span></p>
|
|
</blockquote>
|
|
<p>Now:</p>
|
|
<blockquote>
|
|
<p><span class="code">p.expect (pexpect.EOF)</span></p>
|
|
</blockquote>
|
|
</blockquote>
|
|
<hr noshade="noshade" size="1">
|
|
<h1><a name="testing"></a>TESTING</h1>
|
|
<p>The following platforms have been tested:</p>
|
|
<!--
|
|
<table class="pymenu" border="0" cellpadding="5">
|
|
<tbody>
|
|
<tr>
|
|
<th class="pymenu">PLATFORM</th>
|
|
<th class="pymenu">RESULTS</th>
|
|
</tr>
|
|
<tr>
|
|
<td>Linux 2.4.9-ac10-rmk2-np1-cerf2<br>
|
|
armv4l</td>
|
|
<td><b><i>all tests passed</i></b></td>
|
|
</tr>
|
|
<tr>
|
|
<td>Linux 2.4.18 #2<br>
|
|
sparc64</td>
|
|
<td><b><i>all tests passed</i></b></td>
|
|
</tr>
|
|
<tr>
|
|
<td>MacOS X Darwin Kernel Version 5.5<br>
|
|
powerpc</td>
|
|
<td>
|
|
<p>failed more than one test.</p>
|
|
<p>Generally Pexpect works on OS X, but the nature of the quirks
|
|
cause a many of the tests to fail. See <a href="#bugs">bugs</a>
|
|
(Incomplete Child Output). The problem is more than minor, but Pexpect
|
|
is still more than useful for most tasks. The problem is an edge case.</p>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Linux 2.2.20<br>
|
|
alpha<br>
|
|
</td>
|
|
<td><b><i>all tests passed</i></b></td>
|
|
</tr>
|
|
<tr>
|
|
<td>Linux 2.4.18-5smp<br>
|
|
i686</td>
|
|
<td><b><i>all tests passed</i></b></td>
|
|
</tr>
|
|
<tr>
|
|
<td>OpenBSD 2.9 GENERIC#653<br>
|
|
i386</td>
|
|
<td><b><i>all tests passed</i></b></td>
|
|
</tr>
|
|
<tr>
|
|
<td>Solaris</td>
|
|
<td>
|
|
<p>failed <span class="code">test_destructor</span></p>
|
|
<p>Otherwise, this is working pretty well. The destructor problem
|
|
is minor. For some reason, the <i>second</i> time a pty file
|
|
descriptor is created and deleted it never gets returned for use. It
|
|
does not effect the first time or the third time or any time after
|
|
that. It's only the second time. This is weird... This could be a file
|
|
descriptor leak, or it could be some peculiarity of how Solaris
|
|
recycles them. I thought it was a UNIX requirement for the OS to give
|
|
you the lowest available filedescriptor number. In any case, this
|
|
should not be a problem unless you create hundreds of pexpect
|
|
instances... It may also be a pty module bug. </p>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Windows XP Cygwin</td>
|
|
<td>failed <span class="code">test_destructor</span>. That it
|
|
works at all is amazing to me. Cygwin rules!</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
-->
|
|
<h1> </h1>
|
|
<h1><a name="todo">TO DO</a></h1>
|
|
<p>Add an option to add a delay after each expect() or before each
|
|
read()/readline() call to automatically avoid the <a href="#echo_bug">echo
|
|
bug</a>.</p>
|
|
<p> </p>
|
|
</div>
|
|
<hr noshade="noshade" size="1">
|
|
<table border="0">
|
|
<tbody>
|
|
<tr>
|
|
<td> <a href="http://www.noah.org/email/"><img src="email.png"
|
|
alt="Click to send email." border="0" height="16" width="100"></a> </td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div id="Menu"><b>INDEX</b><br>
|
|
<hr noshade="noshade" size="1"> <a href="#license"
|
|
title="Python Software Foundation License">License</a><br>
|
|
<a href="#download" title="Download and setup instructions">Download</a><br>
|
|
<a href="#doc" title="Documentation and overview">Documentation</a><br>
|
|
<a href="#status" title="Project Status">Project Status</a><br>
|
|
<a href="#requirements" title="System requirements to use Pexpect">Requirements</a><br>
|
|
<a href="#overview" title="Overview of what Pexpect does">Overview</a><br>
|
|
<a href="#faq" title="FAQ">FAQ</a><br>
|
|
<a href="#bugs" title="Bugs and work-arounds">Known Bugs</a><br>
|
|
<a href="#changes" title="What's new with Pexpect">Recent Changes</a><br>
|
|
<a href="#testing" title="Test results on various platforms">Testing</a><br>
|
|
<a href="#todo" title="What to do next">To do</a><br>
|
|
<a href="http://pexpect.svn.sourceforge.net/viewvc/pexpect/trunk/pexpect/" title="browse SVN">Browse SVN</a><br>
|
|
<br>
|
|
<a href="http://sourceforge.net/projects/pexpect/"
|
|
title="The Pexpect project page on SourceForge.net"> <img
|
|
src="http://sourceforge.net/sflogo.php?group_id=59762&type=5"
|
|
alt="The Pexpect project page on SourceForge.net" border="0"
|
|
height="31" width="105"> </a> </div>
|
|
</body>
|
|
</html>
|