Introduction to Unix
Advanced Computing Center for Research and Education
http://www.accre.vanderbilt.edu
What is Unix?✦Unix is an operating system (un)like Windows✦Originally created in the late 1960’s at AT&T Bell Labs✦Designed to be a programmers OS✦Turned out to be a portable, multi-tasking, multi-user OS (we take those things for granted now, but they were revolutionary at the time!)
✦There are many different “flavors” of Unix...
Different “flavors” of Unix have different niches
✦ IBM’s AIX and HP’s HP-UX are heavily used in business✦Oracle’s (formerly Sun’s) Solaris once powered 70% of the web / e-mail servers on the Internet
✦Apple’s MacOS X is a user-friendly desktop✦Linux is very high performance and free, which makes it ideal for HPC clusters
Unix ArchitectureShell
Kernel (OS)
Physical
Devices
✦Only the Kernel (the operating system) can manipulate physical devices (disks, the network, printers, etc.)
✦Users interact with the kernel via a shell (command interpreter)
✦Two primary shells: 1) bash, 2) tcsh
Unix Philosophy✦Everything is a file. Executables, program source, and text documents are all files. Directories are files that list other files. Devices (disks, network, etc.) are files.
✦You really want to do what you said you want to do. Therefore Unix should do it as quickly and quietly as possible. I.e. No “Are you sure? (Y/N)” messages by default.
✦This has also been stated as, “Unix gives you enough rope to hang yourself ... and then a couple of feet more just to be sure.”
Unix Commands✦The format of Unix commands is command [ options ] [ arguments]
✦ id is a command✦ -f is an option to the ps command✦example1 is an argument to the ls command (and -l is an option)
The most important command of all
✦The man command displays manual pages
✦Example at left is the output of man ls
✦Displays synopsis✦Gives description of each option and, if applicable, argument to the command
Command History / Editing
✦The shell maintains a history of the commands you have previously executed (1000 commands by default)
✦Up and down arrow keys scroll thru your command history
✦Left and right arrow keys scroll thru the command✦Edits can be made by adding, deleting, or replacing parts of the command
✦Pressing the enter / return key executes the command
Hierarchical Filesystem/
bin usr home scratchetc tmp
chmod
cp
date
grep
mv
rm
vi
nikkiannie codybin lib
bin docs src
libc.so
libgpfs.so
libjpeg.so
libstdc++.so
diff
find
gcc
id
make
perl
ssh
prog1.c
prog2.f77
prog3.cpp
myprog.sh
dothis.pl
dothat.py
Working With Directories✦pwd prints your present working directory
✦ cd changes directories (with no arguments always changes to your home directory)
✦mkdir makes a directory✦ rmdir removes a (empty) directory
✦ ls lists directories (and files)
Working With Files✦file tells you what type of file a file is
✦more displays the contents of a file
✦ cp copies files✦mv renames (moves) files
✦ rm removes files✦The -i option makes cp, mv, and rm “interactive”
Editing Files✦There are three text editors available...✦emacs - popular with programmers✦ nano (pico) - basic, easy to use editor✦ vi - the original Unix text editor; has steeper learning curve, but is the fastest of the three
✦Which should you use?✦Whichever one you like!
File Permissions✦All files / directories have one user (or owner - nikki in this example) and one group (guest) associated with them
✦There are three sets (user, group, other) of three permissions (read, write, execute)
✦Only the owner may change the owner, group, or the permissions
Read, Write, Execute
Read Write Execute
FilesYou can look at the file contents
You can modify the file contents
You can run the program
DirectoriesYou can ls the
directory
You can create and delete files
You can cd to the directory
Changing Permissions✦ chmod (change mode)✦alphabetic method - add or take away (r)ead, (w)rite, or e(x)ecute for the (u)ser, (g)roup, and / or (o)ther
✦ numeric method - read = 4, write = 2, execute = 1; total up what you want for the user, group, and other
Pattern Matching
✦ * - matches zero or more instances of any character✦ ? - matches one instance of any character✦ [abc] - matches any one character within the brackets; “a”, “b”, or “c” in this example
✦ [0-9] - matches any one character within the range defined within the brackets
✦ [A-z] - matches all letters, plus most punctuation characters, because this is an ASCII range (use “[A-Za-z]” instead to match only letters
Aliases
✦bash syntax - alias rm=”rm -i”✦ tcsh syntax - alias rm ‘rm -i’✦Now when you type rm, the shell will automatically replace it with rm -i
✦alias ll=”ls -la”✦alias mroe=more✦alias cdmydir=”cd /some/big/long/path/that/is/hard/to/remember/much/less/type”
Shell Variables✦Many variables are set for you
✦Two types of variables: environment and local
✦ local variables are visible only to the current shell
✦environment variables are also visible to child shells and processes
✦env shows environment variables
Defining Shell Variables✦my_var=some_value - bash syntax for creating a local variable
✦export my_var - makes my_var an environment variable✦export my_var=some_value - creates an environment variable in one step
✦ set my_var some_value - tcsh syntax for creating a local variable
✦ setenv my_var some_value - tcsh syntax for creating an environment variable
Displaying Shell Variables
✦The echo command can be used to display the values of variables
✦When referencing a variable, precede the name with a $
✦This example shows the difference between local and environment variables
Shell Initialization Files✦Any aliases or variables you define on the command line are only in effect for the duration of that session
✦To make them permanent, simply add them to your .bashrc or .cshrc file as appropriate
Command Substitution✦Any command enclosed in grave accents (not single quotes) is executed first and then its’ output is substituted in on the command line
✦Command substitution can be used with other commands (as the first example shows) or to assign a value to a variable (as the second example shows)
Input / Output Redirection
✦Any shell has 3 filehandles open by default:✦stdin - standard input, defaults to keyboard, file descriptor 0
✦stdout - standard output, defaults to screen, file descriptor 1
✦stderr - standard error, defaults to screen, file descriptor 2
✦And yes, if you open a file, the first one you open would have file descriptor 3, etc.
Input / Output Redirection
✦ Input redirection - e-mail yourself a file: mailx [email protected] < example1
✦Output redirection - myprogram > output.log✦Error redirection - myprogram 2> error.log✦Output and error redirection to different files - myprogram > output.log 2> error.log
✦Output and error redirection to the same file - myprogram > combined.log 2>&1
Pipes and Filters
✦Pipes take the output of one command and make it the input to another command
✦Analogous to plumbing pipes✦ Filters are commands which can accept input from another command and also produce output
✦Syntax is command | filter_command✦Multiple pipes and filters can be strung together on one command line - command | filter_command1 | filter_command2 | filter_command3
A Few Useful Filters
✦wc - word count✦grep - (g)et (r)egular (e)xpression and (p)rint (i.e. pattern matching)
✦ sort - sort alphabetically or numerically✦uniq - filter duplicate lines✦ cut - cut specific fields or columns✦ sed - stream editor (i.e. search and replace)
So let’s combine filters and output redirection...
✦Oops! What happened?✦The shell did the output redirection first and we asked for a new file called example3.
✦We must use a different name for the output file.
Compressing / Archiving Files
✦ If you have a Windows client, use zip / unzip✦Example: zip mydata.zip ./mydatadir/*✦Then use scp to copy the data to your client and unzip it there: unzip mydata.zip
✦ If you have a Linux or Mac client, use tar and gzip✦Example: tar cvf mydata.tar ./mydatadir/*; gzip mydata.tar
✦Then use scp to copy the data to your client and uncompress / untar it there: gzip -d mydata.tar.gz; tar xvf mydata.tar
Getting Help from ACCRE✦ ACCRE website FAQ and Getting Started pages: http://
www.accre.vanderbilt.edu/support✦ ACCRE helpdesk: http://www.accre.vanderbilt.edu/support/contact/
submit_RT.php✦Tickets default to non-rush queue, which is for tickets about an issue which only impacts you and / or can wait until the next business day.
✦Rush tickets are for issues which would impact the cluster as a whole.
✦Rush queue pages ACCRE personnel (we normally only have staff on site during normal business hours), so please think before opening up a rush ticket at 3 AM!
✦ If you open a rush ticket - day or night - please plan to be available to work with ACCRE personnel to resolve the issue.
✦Once the issue (rush or non-rush) is resolved we would appreciate it if you would let us know.
Top Related