Productivity tips - Introduction to linux for bioinformatics
-
Upload
bits -
Category
Technology
-
view
542 -
download
2
description
Transcript of Productivity tips - Introduction to linux for bioinformatics
Productivity
Joachim Jacob8 and 15 November 2013
Multiple commands
In bash, commands put on one line when be separated by “;”
$ wget http://homepage.tudelft.nl/19j49/t-SNE_files/tSNE_linux.tar.gz ; tar xvfz tSNE_linux.tar.gz
Multiple commands
Commands on a oneliner can also be separated by && or ||
&& Only execute the command if the preceding one finished correctly.
$ curl corz.org/ip && echo '\n'
|| (not a pipe!) - Inverse of the above. Only execute the command if the preceding one did not succesfully ends.
Piping a list of files with xargs
A pipe reads the output of a command.
Some commands requires the file name to be passed, instead of the content of the file. E.g. this doesn't work:
$ ls | less
$ ls | fileUsage: file [-bchikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type] [-e testname] [-F separator] [-f namefile] [-m magicfiles] file ... file -C [-m magicfiles] file [--help]
Piping a list of files with xargs
Some commands requires the file name to be passed, instead of the content of the file.
xargs passes the output of a command as a list of arguments to another program.
$ ls | xargs filebin: directorybuddy.sh: Bourne-Again shell script, ASCII text executableCompression_exercise: directoryDesktop: directoryDocuments: directoryDownloads: directoryFastQValidator.0.1.1.tgz: gzip compressed data, from Unix, last modified: Fri Oct 19 16:44:23 2012
.bashrc
~/.bashrc is a hidden configuration file for bash in your home.
It configures the prompt in your terminal.It contains aliases to commands.
alias example
When you enter a first word on the command line that bash does not recognize as a command, it will search in the aliases for the word.
You can specify aliases in .bashrc. An example:
Alias example
Some interesting aliases
alias ll='ls -lh'alias dirsize="du -sh */"alias uncom='grep -v -E "^\#|^$"'alias hosts="cat /etc/hosts"alias dedup="awk '! x[$0]++' "
Aliases are perfectly suited for storing one-liners: find some athttps://wikis.utexas.edu/display/bioiteam/Scott%27s+list+of+linux+one-liners
Alias exercise
→ exercise link
Finding stuff: locate
Extremely quick and convenient:locate
However, it won't find the newest files you created. First you need to update the database by running:updatedb
It accepts wildcards. Example:$ locate *.sam
Bonus: How to filter on a certain location?
Finding stuff: find
More elaborate tool to find stuff:$ find -name alignment.sam
Find won't find without specifying options:-name : to search on the name of the file-type : to search for the type: (f)ile, (d)irectory, (l)ink-perm : to search for the permissions (111 or rwx)…
This is the power tool to find stuff.
Finding stuff: find
The most powerful option of find:-exec Execute a command on the found entities.
Finding stuff: find
The most powerful option of find:-exec Execute a command on the found entities.
$ find -name \*.gz ./DRR000542_2.fastq.subset.gz./DRR000542_1.fastq.subset.gz./DRR000545_2.fastq.subset.gz./DRR000545_1.fastq.subset.gz$ find -name \*.gz -exec gunzip {} \;$ lsDRR000542_1.fastq.subset DRR000545_1.fastq.subsetDRR000542_2.fastq.subset DRR000545_2.fastq.subset
Command substitution in bash
In bash, the output of commands can be directly stored in a variable. Put the command between back-ticks.
$ test=`ls -l`$ echo $testtotal 7929624 -rw-rw-r-- 1 joachim joachim 15326 May 10 2013 0538c2b.jpg -rw-rw-r-- 1 joachim joachim 4914797 Nov 8 16:15 18d7alY
Command substitution in bash
A variable can also contain a list. A list contains several entities (e.g. files).
Extracting first 100k lines from compressed text file:
for filename in `ls DRR00054*tar.gz`; \ do zcat $filename | head -n 1000000 \
>${file%.gz}.subset; done
The output of ls is being put in a list. 'for' assigns one after the other the name of the file to the variable file. This variable is used in the
oneliner zcat | head.
Keywords.bashrc
;
alias
prompt
locate
find
Command substitution
Write in your own words what the terms mean
Break