Isabella Friis Jørgensen, Bioinformatics for Human Biologists, 19/9-16

Introduction to UNIX

12/09/16 1

What is UNIX?

12/09/16 2

•  Computer operating system •  Microsoft Windows •  MacOS X •  Android O/S

•  Developed in 1960’s •  Several variants – both commercial and academic

•  Most famous; Linux

•  Simple set of tools to perform functions

From SIB training portal, UNIX Fundamentals, October 2015

Why do we use UNIX?

12/09/16 3

•  Multiuser •  several users can connect and execute programs at the same time

•  Multitasking •  one user can run several programs at the same time

•  Life science have BIG datasets

•  How many genes? •  Most common gene? •  Rarest disease?

•  Large tool set predefined •  Reproducability •  Automation •  Allows for usage of remote servers

12/09/16 4

Inspired by UNIX – Peter Wad Sackett, DTU Bioinformatics, 2016

Let’s get started •  Start your Biomachine – check the guide

•  The shell/terminal can be found in the bottom •  Command line interface

•  Everything has to be typed •  Case sensitive

•  MyFile.txt, myfile.txt

12/09/16 5

Listing files ls ls –l Changing directories cd <folder> Creating directories mkdir <folder> Deleting directories rmdir <folder> Where are you? pwd Why are you here? Sorry – no answer to that one!

/

etc bin home var

UNIX data scripts folder

ex test.dat genes notes

Lecture1.txt Lecture2.txt

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

File navigation system

12/09/16 6

Coping files cp <file> <destination> Moving/renaming files mv <file> <destination> Deleting files rm <file> Changing file permissions chmod <ownership> ‘+/-’ <access> <file> Ownership indicates which owner that is affected (User, Group or Others) +/- shows if permissions are given or removed Access indicates the new level of access (Read, Write or eXecute)

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

File handling

From SIB training portal, UNIX Fundamentals, October 2015

12/09/16 7

Seeing the top of the file head <file> Seeing the end of the file tail <file> Seeing all of the file cat <file> Inspecting the file less <file> Using less; You can browse up and down in the file with PgUp/b and PgDn/space, search with /, but most importantly exit the application with q. g goes to the top and G to the bottom.

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

Inspecting a file

12/09/16 8

Counting the lines/words/bytes in the file wc <file> Merging files paste <file1> <file2> Extracting columns from a file cut <options> <file> Usually used as ’cut –f2,4 myfile.txt’ on tab files Sorting files sort <file> Extracting lines from a file grep <pattern> <file> This is an incredibly useful command, very versatile

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

Working with files

12/09/16 9

Many versions of grep – different capabilities grep HUMAN <file> Lines containing HUMAN, like POSTHUMAN grep –v HUMAN Lines without HUMAN grep –c HUMAN Count lines with HUMAN grep ”^HUMAN” Lines starting with HUMAN grep ”HUMAN$” Lines ending with HUMAN grep –e/E Regular expressions . any single character + one or more of preceeding characters * zero or more of preceeding characters grep –e ”H.+MAN” Matches HEMAN, HUMAN, HITMAN, etc

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

Let’s learn more about grep

12/09/16 10

Every program has three standard streams STDIN, STDOUT and STDERR

Defaults; STDIN = keyboard, STDOUT, STDERR = screen

Save output of a command in a file; > grep HUMAN orphans.sp > humanproteins.txt Append to the file with >>

Feed a file to a command; < wc < humanproteins.txt

Pipe (stream) the output of one command to the next; | grep HUMAN orphans.sp | wc

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

IO redirections and pipes

12/09/16 11

Printing text to the screen echo <text> Getting time and date information date Seeing what programs are running ps Stopping them kill <pid> Editing text files jedit <filename> If the file does not exit, it will be created

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

More useful commands

12/09/16 12

The UNIX manual man <command> The manual is shown with less. Also try google: man <command> whatis <command> Single line description of the command Plenty of UNIX guides on the net. Here are some of the best. http://www.ee.surrey.ac.uk/Teaching/Unix/ http://unixhelp.ed.ac.uk/ Google is your friend

Inspired by UNIX - Peter Wad Sackett, DTU Bioinformatics, 2016

Information and help

Now it’s your turn

12/09/16 13

Test different UNIX commands using excerpts of real biological data

ex1.acc

Gene ID or SwissProt ID

ex1.dat

GeneBank Accession number Experimental data

Now it’s your turn

12/09/16 14

Test different UNIX commands using excerpts of real biological data

Exercise at course page

Orphans.sp

Gene ID = Gene_Organism

GeneBank Accession number

Statistical data