Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö...

63

Transcript of Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö...

Page 1: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se
Page 3: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Shell and Bash

the Shell is a Command Line Interface (CLI)

Bash is one particular shell tcsh, zsh are also shell programs

Page 4: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Multiple files

● Same program, many files

Page 5: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Multiple files

● Same program, many files

Page 6: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Multiple files

● Same program, many files○ 10 files? Ok○ 1000 files? Not ok (inte bra!)

Page 7: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Multiple files

● Same program, many files○ 10 files? Ok○ 1000 files? Not ok (inte bra!)

● Reproducibility○ Self and others

Page 8: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Multiple files

● Same program, many files○ 10 files? Ok○ 1000 files? Not ok (inte bra!)

● Reproducibility○ Self and others

A solution - write a script!

Page 9: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Multiple files

● Same program, many files

Page 10: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Page 11: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Page 12: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Page 13: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Page 14: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Page 15: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Page 16: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Page 17: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Basic script

Still not OK for 1000 or more files!

Page 18: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Variables

● Assigningmy_variable=5my_variable=”nice text”

Page 19: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Variables

● Assigningmy_variable=5my_variable=”nice text”

● Using$my_variable

Page 20: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Variables

● Assigningmy_variable=5my_variable=”nice text”

● Using$my_variable

$ my_variable=”Rackham”

Page 21: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

● Assigningmy_variable=5my_variable=”nice text”

● Using$my_variable

$ my_variable=”Rackham” $ echo “Hello, $my_variable! ”

Variables

Page 22: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

● Assigningmy_variable=5my_variable=”nice text”

● Using$my_variable

$ my_variable=”Rackham” $ echo “Hello, $my_variable! ” Hello, Rackham!

Variables

Page 23: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Variables

Page 24: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Variables

Page 25: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Variables

Page 26: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Variables

Page 27: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loops

for var in 1 2 3;do echo $vardone

Page 28: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loops

for var in text works too;do echo $vardone

Page 29: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loops

for var in mix them 5;do echo $vardone

Page 30: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loops

Page 31: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loops

Debugging!

Page 32: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loops

Page 33: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Wildcard *

Page 34: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Page 35: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Page 36: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Page 37: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Page 38: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Page 39: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Page 40: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Loop over files

Page 41: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if condition; then

actionfi

Page 42: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if true; then

echo ”This is true”fi

result:This is true

Page 43: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if false; then

echo ”This is true”fi

result:

Page 44: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if [[ 5 < 9 ]]; then

echo ”This is true”fi

result:This is true

Page 45: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if [[ 5 > 9 ]]; then

echo ”This is true”fi

result:

Page 46: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if [[ 5 == 9 ]]; then

echo ”This is true”fi

result:

Page 47: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if [[ ”Hello” == ”Hello” ]]; then

echo ”This is true”fi

result:This is true

Page 48: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if [[ ”Hello” == ”Hi” ]]; then

echo ”This is true”fi

result:

Page 49: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● Control statement if [[ ”Hello” == ”Hel”* ]]; then

echo ”This is true”fi

result:This is true

Page 50: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● For all samples except dog

Page 51: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● For all samples except dog

Page 52: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● For all samples except dog

Ex: $file is /path/to/dog_1.bam

Page 53: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● For all samples except dog

Ex: $file is /path/to/dog_1.bam

basename $file

Page 54: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● For all samples except dog

Ex: $file is /path/to/dog_1.bam

basename $file

dog_1.bam

Page 55: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● For all samples except dog

Ex: $file is /path/to/dog_1.bam

basename $file

dog_1.bam

Page 56: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

If

● For all samples except dog

Ex: $file is /path/to/dog_1.bam

basename $file

dog_1.bam

Page 57: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Different languages

● Programming is programming○ Perl, Python, Bash, and more

Page 58: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Different languages

● Programming is programming○ Perl, Python, Bash, and more

Page 59: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Different languages

● Programming is programming○ Perl, Python, Bash, and more

Page 60: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Different languages

● Programming is programming○ Perl, Python, Bash, and more

Page 61: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Different languages

● Programming is programming○ Perl, Python, Bash, and more

● Start with one, git gud, (learn another)

Page 62: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Different languages

● Programming is programming○ Perl, Python, Bash, and more

● Start with one, git gud, (learn another)

PYTHON

Page 63: Advanced Linux Usage - GitHub Pages · Advanced Linux Usage 2018-02-13 Martin Dahlö martin.dahlo@scilifelab.uu.se Valentin Georgiev valentin.georgiev@icm.uu.se Jacques Dainat jacques.dainat@nbis.se

Laboratory time! (yet again)