Unix Day1 v1.3
Transcript of Unix Day1 v1.3
Tech Mahindra Limited confidential© Tech Mahindra Limited 2008
UNIX
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 2
Objectives
At the end of the session, you will be able to: Understand Unix File System Use Unix basic and filter commands Read and write shell scripts Read and write awk scripts Understand process management in UNIX Understand the general utilities e.g. gzip, ftp & tar
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 3
Agenda
Introduction to Unix Navigating the File System General Purpose Commands Pipes and Filters Shell The vi Editor Shell scripting AWK scripting Process Management General Unix Utilities
Tech Mahindra Limited confidential© Tech Mahindra Limited 2008
UNIX
Day 1
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 5
Objectives
At the end of the session, you will be able to: Define UNIX Operating System Understand UNIX O.S. Components e.g. Kernel & Shell Understand Unix File System Execute Unix file related general commands Understand file access permissions Understand Pipe and redirection Understand filter command Edit a file using vi editor
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 6
Agenda: Day 1
Unix Overview
Unix File System
Unix Commands
Vi editor
Tech Mahindra Limited confidential© Tech Mahindra Limited 2008
Unix Overview
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 8
History of Unix
Developed in AT&T Bell Labs by Ken Thomson as a single user OS in 1969
Initially written in assembly language
Developed as multi-user OS later
Rewritten in C in 1973
Licensed to university for educational purposes in 1974
POSIX (Portable Operating System for Unix) was developed
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 9
Unix Flavors
AIX by IBM
Solaris by Sun Microsystems
HP-UX by Hewlett-Packard Company
IRIX by Silicon Graphics, Inc.
FreeBSD by FreeBSD Group
GNU/Linux by Open Source Movement
MacOS by Apple Computer, Inc.
SCO Unix by The Santa Cruz Operation Inc.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 10
Features of Unix
Multi-user
Multi-tasking
Portable
Interactive
Shell
Security
Hierarchical File System
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 11
Unix Architecture
Hardware Kernel
Shell & Unix Utilities
Other Application Program
Tech Mahindra Limited confidential© Tech Mahindra Limited 2008
Unix File System
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 13
Root Directory Structure
cat
ls
user2 user3user1
progrm1
progrm2
bin boot home etc usr dev lib
Desktop
start
/
Directory
File
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 14
File Basics
Everything on Unix is a file. File structure is hierarchical like an upside down tree.
File is just a sequence of bytes.
The meaning of the bytes depends solely on the programs that interpret the file.
The format of a file is determined by the programs that use it.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 15
File Attributes
Name (such as “test.log”)
An Owner (such as “sandhya”)
Access Rights (such as read, write, execute)
Other attributes (such as date of creation)
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 16
File Types
Every item in a UNIX file system belongs to one of the three possible types:
1. Ordinary/Regular files
2. Directory files
3. Device/Special files
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 17
Ordinary File
Contains text, data, or program information
Cannot contain another file or directory
Can be thought of as one-dimensional array of bytes
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 18
Directory File
Contains directory(s) and/or file(s) within it.
Has one line for each item contained within the directory.
Each line in a directory file contains only the name of the item, and a numerical reference to the location of the item, called inode number.
Inode number is an index to a table known as the inode table. Inode stores all information about the file except its name.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 19
Device File
Physical devices (printers, terminals etc) are represented as “files”.
Two types of device files:1. Character Special2. Block Special
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 20
Links
Links make the same file available in multiple directories at the same time Two types of Links:
1. Hard Link2. Soft Link or Symbolic Link
1. Hard Link A hard link is another name given to the existing file These names share the same inode
2. Soft Link A soft link to a file has a separate inode than the file It stores the target file’s path in its inode
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 21
Inode
Inode is a data structure containing useful information about an item in the Unix File System.
Inodes reside on disk and do not have names. Instead, they have indices (numbers) indicating their positions in the array of inodes as shown in the next slide.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 22
Inode
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 23
Pathname
Every item in the file system with a name can be specified with a pathname.
Pathname represents the path to the entry from the root of the file system. By following this path, the system can find the inode of the referenced entry.
Pathnames can be absolute or relative.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 24
Unix Users
Super User
Owner
Group
Others
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 25
Unix Users
Superuser Can also be referred to as a System Administrator Has an overall authority on Unix OS Responsible for OS maintenance, backup and recovery, user
management etc. Superuser login is root and prompt is #
Owner Is a user who creates a file For every Unix file there can be only one owner File owner can assign the file permissions to group and other
users
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 26
Unix Users
Group In Unix, groups can be formed based on area of work Superuser can create a group and assign members to it Owner of a file can decide what permissions to be given to
group members
Others User who is not a owner and does not belong to any specific
group is referred to as other user Owner of a file can decide what permissions to be given to
other users
Tech Mahindra Limited confidential© Tech Mahindra Limited 2008
Unix Commands
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 28
Basic Commands
$ pwdShows working i.e current directory
$ whoShows who is logged on
$ who am iShows the login name of the current user
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 29
Basic Commands
$ man command formats and displays online manual pages
The manual pages are divided into logical grouping of the commands called the sections. Sections are numbered from 1 through 9. For example, commands are 1, system calls are 2, library function are 3, file formats are 5, & management commands are 8.
If section is specified, man only looks in that section of the manual.
The command man 2 open displays the manual for the system call open.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 30
Basic Commands
$ date: command prints or sets the system date and time
For example,$ date$ date –r TestFileDisplays the last modification time of the file “TestFile”
This command can be used with suitable format specifiers as argument. Each format is preceded by a + symbol, followed by the % operator, and a single character describing the format.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 31
Basic Commands
Sequence Interpretation
%a abbreviated weekday name (Mon .. Sun)
%b or %h abbreviated month name (Jan .. Dec)
%d day of month (01 .. 31) %r time (12- hour) %T time (24- hour) %y last two digits of year (00 .. 99) %D date (mm/dd/yy)
e. g. $ date “+Today is %a %m %Y”
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 32
Basic Commands
echo “Hi $USER \n Welcome to Unix”
echo ‘Hi $USER \n Welcome to Unix’
echo Hi $USER \n Welcome to Unix
clear
tput clear
tput cup <row_coordinate> <column_coordinate>
tput cup 15 20
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 33
Basic Commands
$ which <command_name> Shows the path of the specified command
$ type <command_name> Shows the type of the specified command – shell built-in
or file path if it is external
$ file list1 Shows the type of the file whether regular file or
directory file or some other file
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 34
Basic Commands
$ cal Shows the calendar of the current month/year
$ bc 2.3+5.4
Shows the mathematical calculations involving integers as well as floats
bc is a filter command
$ write user1 Lets to write a message to the other user; user1 in this
example
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 35
File Management Commands
$ ls command to list files and directories
Option Description-l list in long format
-C multicolumn output -F indicates type of file by /, * -R recursive listing of all subdirectories
encountered -a list all files including hidden files -i List all files along with i-node number
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 36
Wild Card Characters
* - zero or more characters – e.g. ls m* lists all the files starting with m
? – single Character – e.g. ls m? lists all the files starting with m and having one character after that
[]- any character from all the characters within [] e.g. ls [aeiou]* lists all files starting with a or e or i or o or u
- Specifies range
! – works as not operator e.g. ls [!x-z]* will list all the files not starting with any character from the range x-z
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 37
Redirection
Redirection
Output Redirection
Error Redirection
Input Redirection
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 38
Input Redirection
Input Redirection specifies that the standard input, instead of coming from the standard input file, comes from a disk file
$cat < filename e.g. cat < data1 e.g. cat 0< data1
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 39
Output Redirection
The output redirection specifies that the standard output of a given command is written to the disk file, instead of monitor
$ cat filename1 > filename2 e.g. cat data1 > data2 e.g. cat data1 1 > data2
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 40
Error Redirection
The error redirection specifies that an error is written on the standard error-msg file, instead of the monitor
$ cat filename 2>error-msg file e.g. cat data4 2>error-msg
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 41
File Management Commands cat
To concatenate files To display contents of one or more ordinary files To create an ordinary file
$ cat file1 file2 … It displays contents of all files specified on the command line
one below the other
$ cat > file It creates a new file by accepting text from the standard input Press CTRL-d to save and exit the file
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 42
File Management Commands
mkdir [-p] dirname Makes a directory of a given dirname The dirname can contain the full path prefix of the directory to
be created More than one directories can be created at the same time When executed with –p option, it does not give any error if the
directory dirname already exists When executed with option –p, it makes parent directories in
the path, if needed (if any parent directory in the path is not available)
$ mkdir dir1 dir2 dir3 Makes directories dir1 dir2 and dir3 in the current directory
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 43
File Management Commands
rmdir dirname
Removes an empty directory
Avoids the chances of accidental removal of a directory with some useful data files.
$rmdir dir1 dir2 dir3 removes directories dir1 dir2 and dir3 from the current
directory, given that they are empty
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 44
File Management Commands
cd [directory] changes working directory to the directory, if specified;
otherwise to the home directory
cd .. moves to the parent directory
cd or cd ~ changes to the home directory
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 45
File Management Commands
cp command copies files and directories
$ cp –i file1 file2 Copies file1 to file2 -i - informs user before overwriting, if file2 exists
$ cp file1 file2 … dest_directory Copies multiple files in the specified existing directory
$ cp -r directory1 directory2 … dest_directory Recursively copies files from directory1, directory2 etc. to the
dest_directoryNote: Shell Meta-characters can also be used with “cp”
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 46
File Management Commands
mv command changes name of the file or moves the file to the specified destination path
$ mv file1 new-file Renames file1 as new-file
$ mv file1 file2 … dest_directory Moves multiple files to the specified existing directory
$ mv directory1 directory2 … dest_directory Moves one or more directory subtrees to an existing or new
dest_directory Note: Shell Meta-characters can also be used with “cp”
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 47
File Management Commands
rm command is used for deleting unwanted files/directories
$ rm [-i] file … It is interactive removal (option –i) of specified files
$ rm -r directory … It is recursive deletion of all the files within the specified
directories and also the directories themselves
Note: Shell Meta-characters can also be used with “cp”
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 48
Creating Links ln file1 file2
Creates a hard link to existing file file1 with the name file2 in the current directory
Both names point to the same inode
ln –s file1 file3 file3 is a soft / symbolic link to file1 here Removal of file1 affects file3
ls –i file1 file2 file3 Inode number of the file file1 and file2 are same but inode of
the file3 is different
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 49
Finding Files and Directories
find command is used for finding files and directories in the specified file tree based upon certain criteria
Command Syntax:$ find path criteria actione.g.
$ find /usr -name “ymess*” –print
Finds files recursively in the directory in path The path can be absolute or relative Multiple directories can be written in the path In the criteria, wild cards are allowed
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 50
Finding files by name:
find /home/user1 -name “*.sh” -print
finds all files with .sh extension in /home/user1 and its subdirectories
Finding files by type:
find /usr –type d –print
finds all directories in /usr and its subdirectories
find /var –type f –print
finds all regular files in /var and its subdirectories
Finding Files and Directories
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 51
Finding Files and Directories
Find files and take some other action than printing
find . –name “*.sh” –exec cp {} {}.bak \;
Finds all files with the extension .sh, in the current directory and its subdirectories, and then prepares a copy of each file with .bak extension
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 52
File Comparison
cmp command compares two files and reports the location of first mismatch
$ cmp file1 file2file1 file2 differ: byte 1, line 1
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 53
File Comparison
comm compares two sorted files and produces result in three colums:Lines: Unique to file1 Unique to file2 Common
$ comm file1 file2 Anil 12 2000
Ajit 12 2000 Dinesh 27 1400
Sunil 12 2000
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 54
File Comparison
diff command compares two files and proposes the changes in first file to make two files identical
$ cat file1Anil 12 2000Ajit 12 2000Sunil 12 2000
$ cat file2 Anil 12 2000Dinesh 27 1400
$diff file1 file2 2,3c2< Ajit 12 2000< Sunil 12 2000---> Dinesh 27 1400
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 55
File Access Permissions
chmod command is used to change the permissions on a file for owner, group and otherschmod <permission><filename>
Symbolic Method: Code Meaning
a allu userg groupo other+ add- remove= assign
chmod u+x, g-w, o+r, o-wx sample
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 56
File Access Permissions
Absolute value Method: Code Meaning 4 Read 2 Write 1 Execute
$ chmod 754 sample
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 57
File Access Permissions
umask command changes initial permission of newly created file
The value of argument can be calculated by subtracting the mode you want as default from the current default mode
Assume that the current default mode is 0666 and you want it as 0644 then 666 – 644 = 022 will be the parameter which we have to pass with “umask” command
$umask 0 – sets default mode which is 0666
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 58
Pipes
It is used to pass the output of one Unix command as an input to another Unix command
e.g. $ ls >temp $ more temp
ls |
moreInput Output
| $ ls | more
pipe
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 59
Text Manipulation Commands: Filters
A filter is a SHELL command which takes an input ( either from a standard input device or a file ) , processes it and send the processed out put to a standard output device or a file
At run time, the system supplies data to the filter as standard input. This standard input file can not be altered by the program.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 60
Filters
wc: counts lines, words and characters $ wc -[wlc] [filename]
head: displays first ‘n’ lines, horizontal slicing $ head -[n] [filename]
tail: displays last ‘n’ lines, horizontal slicing$ tail -[n] [filename]
split: divides files horizontally$ split -[n] [filename]
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 61
Filters
cut: cuts file vertically either column wise/field wise
$ cut -[cfd] [filename] -c columns/characters -f field number -d field delimiter/separator
$ cut -c2-5 sample cuts columns 2 to 5 from the file sample
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 62
Filters
$ cut -c1,2 file Cuts 1st and 2nd characters from file
$ cut -d” “ -f1-2 file Cuts first 2 fields of the file
$ cut -c2- names Starts cutting from 2nd char to the end of line
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 63
Filters
paste: merges lines of specified files and display onto standard output
$ paste -d[field separator] [list of files]
sort: ordering text files $ sort filename
e.g. $ sort +3 -4 emp Sorts emp file as per 4th column
$ sort –t”:” +3 -4 emp Sorts emp file as per 4th field $ sort –t”:” +3.3 -4.5 file Sorts emp file as per 4th and 5th columns of 4th field
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 64
Filters
uniq: removes duplicate lines from a sorted file $ uniq –[dcf] [file]
-d only print duplicate lines-c prefix lines with number of occurences-f2 avoid comparing first two fields
nl: Displays file contents along with line nos. $ nl –[options] [files]
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 65
Filters
tee: takes input from some command and generates two outputs. One is redirected to a file and other to standard output or next command
e.g. $who | tee tmp
who tee
tmp
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 66
Filters
tr: translating Characters$ tr [options] < [file]-d: deletes specified characters-cd: do not delete specified characters-s: substitute multiple occurrences of a character
by single occurrence.
$ tr “abc” “ABC” < samp: replaces all occurrences of a with A, b with B, c with C
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 67
Command Substitution
One command can be used as an argument to another command
$echo The date today is `date` $echo “There are `ls |wc –l` files”
Tech Mahindra Limited confidential© Tech Mahindra Limited 2008
vi Editor
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 69
The vi editor
The vi editor is a screen-based editor which lets a user create new files or edit existing files
A key concept in vi is combining a certain action with a movement
vi is extremely powerful in moving around within (or between) files
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 70
The vi editor
A vi session begins by invoking the command “vi” with a filename
$ vi [filename]
You can start vi without a filename, but when you want to save your work, you will have to tell vi which filename to save it into.
The last line in the screen is reserved for some commands that you can enter to act on the text. This line is also used by the system to display messages.
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 71
Modes of Operation
The three different modes of operations are: Command mode: This is the default mode where you can pass
the commands to act on the text, using most of the keys of the keyboard
You can switch to this mode using “Esc” key
Insert (Input) mode: To enter the text, you have to enter into input mode. Press key “i” to enter into insert mode from command mode
You can switch to command mode by pressing “Esc” key
ex mode or line mode: You have to save your file or switch to another file or make a global substitution in the file. You then have to use ex mode, where you can enter the instruction in the last line of the screen. To enter into this mode, press “Esc” key followed by “:”
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 72
Text Insertion Commands
Command Description
i inserts text before cursor position a Appends text after cursor position I inserts text at beginning of line A Appends text after end of line o opens line below current line to insert text O opens line above current line to insert text
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 73
Cursor Movement Commands
Command Description
h left by one characterl right by one characterk up by one linej down by one linew right by one wordb left by one word0 or ^ beginning of line$ end of line
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 74
Cursor Movement Commands
Command Description
ctrl+u move up half window
ctrl+b move up full windowctrl+d move down half
windowctrl+f move down full
windowctrl+e scroll downctrl+y scroll up
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 75
Text Search Commands
Command Description
/text searches and highlights the text downwards
n moves between highlighted text * searches the identical text on which the
cursor was ?text searches the text upwards
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 76
Text Deletion Commands
Command Description
x character under cursorX character before cursor[n]dw delete n wordsd0 beginning to cursor positiond$ or D cursor position to end of line[n]dd n lines from current linep p will put (paste) deleted lines to current cursor position
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 77
Text Yanking Commands
Command Description
y character
y0 beginning to cursor position y$ cursor position to end of line [n]yw copy n words [n]yy copy n lines from current cursor position
p p will put (paste) copied lines tocurrent cursor position
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 78
Text Manipulation Commands
Command Description
nc [space] overwrites next n characters with space
c0 overwrites the portion between beginning of line to cursor position
c$ overwrites the portion between cursor to end of line
cw overwrites current word:%s/pattern1/pattern2/g globally replaces pattern1 with pattern2 on
the specified lines:%s/pattern1/pattern2/gc globally replaces pattern1 with
pattern2 on the specified lines interactively
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 79
Text Related Commands
Command Description
ab <abb> <longword> set abbreviation for a long word
una string unset abbreviation>> right shifting a line<< left shifting a lineR replace characters starting with
current character till Esc is pressed
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 80
File Related Commands
Command Description
ZZ or :wq save and exit:w save & continue editing:q! quit without saving:r filnam I nsert contents of file filnam
:[addr1,addr2]w filname write the lines between line number addr1 and line number
addr2 in the file filename
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 81
File Related Commands
Command Description
“a4yy copy 4 lines into buffer “ap paste contents of buffer a to current
cursor position. Maximum 26 buffers are available buffer having names “a” to “z”
ctrl-v select particular columns “byy copy selected into buffer b
“bp paste contents of buffer b to current cursor position
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 82
File Related Commands
Command Description
1,$s/source/target/ substitute string source by string target from line number 1to las line
u undo last change on the lineU undo last changes on the lineCtrl-R redo the undone changes:e edit multiple files :e# return to previous file
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 83
Visual Mode commands
Command Description
v_u converts the character to lower case v_U converts the character to upper case :sp splitting window Ctrl-w toggle between windows <Ctrl-w>j moves to lower window <Ctrl-w>k moves to upper window
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 84
Customizing vi
set commands
Command Description
:set all displays all set option:set autoindent/ai does automatic indentation:set number/nu shows all line duly numbered:set showmatch helps to locate matching brackets:set tabstop=5 sets tab=5 for display:set ic ignore case while pattern matching
When the string “no” is prefixed to any option, it indicates that the option is inoperative
CONFIDENTIAL© Copyright 2008 Tech Mahindra Limited 85
Summary
In this session, we have covered: Unix Overview Unix File System Unix Commands Vi editor
Tech Mahindra Limited confidential© Tech Mahindra Limited 2008
Thank You