Python and Bash:from os.system to Fabric
Radosław Ganczarek
● Linux fanboy (from Slackware to ArchLinux)● 9 months as devops (implementing scripts
for managing bup in Bash and build scripts in Python)
● Writing deployment scripts as a part of web developer job (Fabric, Bash)
● Running Bash workshops
About me
Agenda● Needs and applications● Subprocess● Beyond subprocess● Disk operations● From Bash to Python● Recapitulation
Needs and applications
BuildbotBuildoutAnsibleSaltSconsWAF
Build systems
System toolsPortage (Gentoo)YumWicdUbuntu Software CenterOpenStackImage Packaging SystemSupervisor
Bash replacementipython --profile=pysh%<command>!<command>dirlist = !lsdirlist.grep(‘^a.*’)
Subprocess
Before 2.4:os.systemos.spawn*os.popen*popen2.*commands.*
Subprocess2.4 and after:subprocess
Replacementsos.systemos.spawnlpos.popencommands.getoutput
subprocess.callsubprocess.Popensubprocess.Popensubprocess.check_output
Beyond subprocess
Everybody likes to proxy subprocess!
shfrom sh import sort, du, glob, ifconfig, ls, cat, sudo
print sort(du(glob("*"), "-sb"), "-rn")
print ifconfig()
ls(_out='myfiles')print cat('myfiles')
ls = ls.bake('ls -Al')
cishimport cish, os
cish.rm("build")cish.default.pip("install", "nose")cish.default.python("setup.py", "build")cish.default.nosetests()
env = cish.from_interpreter("path/to/python")env.python("setup.py", "build")env.nosetests()
env = cish.from_config()[os.environ["PYTHON_VERSION"]]env.python("setup.py", "build")
venv = cish.default.virtualenv("optional/location")venv.pip("install", "package_to_install_inside_virtualenv")
popenfrom popen import Sh
for line in Sh('du', '-sh', '~/') | 'head -n 10' | 'sort -n':print('GOT', line)
Sh('ls', '-la', '~') > '~/listing'
Sh('ls', '-la', '~') | 'sort -c' | ['uniq', '-c'] | 'tail' | Sh('wc') >\ '~/listing'
cmd = Sh('ls') | 'grep polka'print cmd.returncode
print Sh.pipe(['this is a \nmulti line\nstring.']) | 'wc'
invokefrom invoke import task, run
@taskdef find_files(directory, pattern): print run( 'find {dir} -iname "{pat}"'.format( dir=directory, pat=pattern ) )
$ invoke find_files . 'C*'
FabricFabfileEnvironmentTasksRemote vs local operationsRun on all hostsParallel execution
File operations
Library vs Bash vs EzPython:fnmatchshutil.abspathshutil.copy*shutil.moveos.chdiros.remove*...
Bash:matching with maskpwdcpmvcdrm...
Ez:regexez.pwdez.cpez.mvez.cdez.rm...
From Bash to Python
>>> list(braceexpand('item{1..3}'))['item1', 'item2', 'item3']>>> list(braceexpand('{a..c}'))['a', 'b', 'c']>>> list(braceexpand('index.html{,.backup}'))['index.html', 'index.html.backup']>>> list(braceexpand('python{2.{5..7},3.{2,3}}'))['python2.5', 'python2.6', 'python2.7', 'python3.2', 'python3.3']>>> list(braceexpand('{07..10}'))['07', '08', '09', '10']>>> list(braceexpand('{a..g..2}'))['a', 'c', 'e', 'g']>>> list(braceexpand('{4..1}'))['4', '3', '2', '1']
braceexpand
genpipelinefrom genpipeline import pipefilter, iter_source
@pipefilterdef double(target): while True: item = (yield) target.send(item*2)
@pipefilterdef printer(): while True: item = (yield) print(item)
iter_source(range(10)) | (double() | printer())
Bash is cool
Python cooler
Use them both
Recapitulation
Questions?
Radosław GanczarekSenior Python Developer @ STX Next
Twitter: @dreamwalker_inEmail: [email protected]
Thank you!
Top Related