11. pipes — Interface to shell pipelines

Contents

The pipes module defines a class to abstract the concept of a pipeline — a sequence of converters from one file to another.

Because the module uses /bin/sh command lines, a POSIX or compatible shell for os.system() and os.popen() is required.

class pipes.Template[source]

An abstraction of a pipeline.

Example:

>>> import pipes
>>> t = pipes.Template()
>>> t.append('tr a-z A-Z', '--')
>>> f = t.open('pipefile', 'w')
>>> f.write('hello world')
>>> f.close()
>>> open('pipefile').read()
'HELLO WORLD'
pipes.quote(s)[source]

Deprecated since version 2.7: Prior to Python 2.7, this function was not publicly documented. It is finally exposed publicly in Python 3.3 as the quote function in the shlex module.

Return a shell-escaped version of the string s. The returned value is a string that can safely be used as one token in a shell command line, for cases where you cannot use a list.

This idiom would be unsafe:

>>> filename = 'somefile; rm -rf ~'
>>> command = 'ls -l {}'.format(filename)
>>> print command  # executed by a shell: boom!
ls -l somefile; rm -rf ~

quote() lets you plug the security hole:

>>> command = 'ls -l {}'.format(quote(filename))
>>> print command
ls -l 'somefile; rm -rf ~'
>>> remote_command = 'ssh home {}'.format(quote(command))
>>> print remote_command
ssh home 'ls -l '"'"'somefile; rm -rf ~'"'"''

The quoting is compatible with UNIX shells and with shlex.split():

>>> remote_command = shlex.split(remote_command)
>>> remote_command
['ssh', 'home', "ls -l 'somefile; rm -rf ~'"]
>>> command = shlex.split(remote_command[-1])
>>> command
['ls', '-l', 'somefile; rm -rf ~']

11.1. Template Objects

Template objects following methods:

Template.reset()[source]

Restore a pipeline template to its initial state.

Template.clone()[source]

Return a new, equivalent, pipeline template.

Template.debug(flag)[source]

If flag is true, turn debugging on. Otherwise, turn debugging off. When debugging is on, commands to be executed are printed, and the shell is given set -x command to be more verbose.

Template.append(cmd, kind)[source]

Append a new action at the end. The cmd variable must be a valid bourne shell command. The kind variable consists of two letters.

The first letter can be either of '-' (which means the command reads its standard input), 'f' (which means the commands reads a given file on the command line) or '.' (which means the commands reads no input, and hence must be first.)

Similarly, the second letter can be either of '-' (which means the command writes to standard output), 'f' (which means the command writes a file on the command line) or '.' (which means the command does not write anything, and hence must be last.)

Template.prepend(cmd, kind)[source]

Add a new action at the beginning. See append() for explanations of the arguments.

Template.open(file, mode)[source]

Return a file-like object, open to file, but read from or written to by the pipeline. Note that only one of 'r', 'w' may be given.

Template.copy(infile, outfile)[source]

Copy infile to outfile through the pipe.