UNIX ™ /Linux Overview Unix/LINUX Intro. UNIX/Linux History.
Unix
-
Upload
viraat-gadde -
Category
Documents
-
view
37 -
download
0
description
Transcript of Unix
© Geometric Limited | www.geometricglobal.com
Unix
PLMi
Unix History
• Evolved at AT&T Bell Labs in late sixties.• Ken Thomson, Rudd Canaday, Doug McIilroy, Joe Ossanna and Dennis
Ritchie - writers of Unix.• Originally written as OS for PDP-7 and later for PDP-11.• Liberal liscensing: Various versions.• System V in 1983 - Unification of all variants.
Features of UNIX
• Multi-User & Multiprocessor System
• Time Sharing Operating System
• Written in “C” (HLL)
• Consistent File Format - the Byte Stream
• Hierarchical File System
Features of UNIX
• Simple User Interface
• Supports Languages - FORTRAN, BASIC, PASCAL, Ada, COBOL, LISP,
PROLOG, C, C++ etc
Architecture of UNIX System
Hardware
Kernel
CompilersDatabasePackages
Other Applicationand systemsoftware
Unix commands
Shell
shell
shell
user
user
Features of Unix
Services Provided :
• Process Management
• Creation, Termination, Suspension and Communication
Features of Unix
Services Provided :
• File Management
• Aspects related to files like Creation & Deletion, File security etc.
File System
• Hierarchical Structure• Consistent Treatment of Data: Lack of file format• The Treatment of Peripheral Devices as Files • Protection of File Data
File System
Root (/)
bin dev etc mnt tmp unix usr
<Files> <Files> <Files> <Files> bin Kumar Sharma
<files> prog safe
<files> <files>
File System
/ bin commonly used UNIX Command like /usr/bin who, ls, cat, wc etc. are stored here/dev - contains device files of all hardware
devices/etc - contains those utilities mostly used by
system administrator e.g. passwd, chmod, chown
File System
/tmp - used by some UNIX utilities especially vi and by user to store temporary files
/usr - contains all the files created by user, including login directory
/unix - kernel
Release V - does not contain / bin- contains / home instead of /usr
File System
• File Types
• Regular File
• Directory File
• Device File
File System
• Consists of Sequence of Logical Blocks(Each block: Multiple of 512 k bytes)
• An Installation may have• several physical disk units• each containing one or more file system
• Data from another File System can be Accessed by Mounting that File System
File System Layout
Boot Block
SuperBlock
Inode List
DataBlocks
File System
• Boot Block (numbered 0)
• Beginning of File System (1st Block)
• Contains Bootstrap Code
• Every File System has (possibly empty) Boot Block
File System
• Super Block (numbered 1)
• Size
• # of Files that can be Stored
• Free Space Information
• Inode Information
File System
• Inode blocks (numbered 2 onwards)
• Information like access permissions, data/time, phy.
Addr. etc. about each inode
File System
• Data Block : File Data & Administrative Data:stores actual contents of file
• Inode Table Fields :• File Owner Identifier - Owner & Group id• File Type - Regular, Directory, Character / Block / Pipe
File System
• File Access Permissionsr w x r w x r w x
user group othersDate & Time - Last File/Inode Modified/Accessed• No. of Links• Disk Addresses• File Size
File Permissions
• Associated with every file, useful for security.• 3 categories of users
• Owner (u)• Group (g)• Others (o)
• Access permissions• Read (r)• Write (w)• Execute (e)
Basic Unix Commands
• Logging in & out
• Logon name & password required
• Successful logon places user in home directory
• Changing password with passwd command
• file /etc/passwd stores username, password & other
login information
Basic Unix commands
• On line help with man command
• includes brief description, options & examples.
$man <command>
Commands related to Files & directories
• Checking current directory - pwd command
$ pwd
/usr/Kumar
Commands related to Files & directories
• Changing directories - cd commands
$ pwd
/usr/Kumar
$ cd Prog
$ pwd
/usr/Kumar/Prog
Commands related to Files & directories
• Moving one level up
$ cd ..
• Switching to home directory
$ cd
Commands related to Files & directories
3. Checking login directory - logname command
$ cd /usr/Sharma
$ logname
Kumar
$
Commands related to Files & directories
• Listing directory contents - ls commande.g. $ ls
a.outchap1chap2testtest.c$
Commands related to Files & directories
Options available in lsOption Description
-x Displays multi columnar output (prior to Release 4)-F Marks executables with *and directories with /-r Sorts files in reverse order (ASCII collating sequence
by default)-l The long listing showing seven attributes of a file-dForces listing of a directory-a Shows all files including ., .. And those beginning with
a dot
Commands related to Files & directories
Options available in lsOption Description
-t Sorts files by modification time-R Recursive listing of all files in sub-directories-u Sorts files by access time (when used with the -t
option)-i Shows i-node number of a file-s Displays number of blocks used by a file
Commands related to Files & directories
E.g. $ ls - ltotal 8-rw-rw-rw- 1 Kumar group 44 May 9 09:08 dept.h-rw-rw-rw- 1 Kumar group 212 May 9 09:08 dept.q-rw-rw-rw- 1 Kumar group 154 May 9 09:08 emp.h
Commands related to Files & directories
Consider first entry -Field1 --> mode
- r w x r w x r w x
--> user permissions --> group permissions --> others permissions
Commands related to Files & directories
• File type
- --> regular file
d --> directory file
c --> character - read
b --> block read
Commands related to Files & directories
Field2 - indicates number of linksField3 - File owner idField4 - Group idField5 - File size in bytesField6 - Date/time last alteredField7 - Filename
Commands related to Files & directories
• Displaying and Creating Files - cat command
$ cat dept.lst
01|accounts|6213
02|admin|5423
:
06|training|1006
Commands related to Files & directories
• Scrolling of large files can be freezed by pressing <ctrl s> and resumed by
<ctrl q>$ cat chap1 chap2
- display contents of chap2 immediately after displaying chap1
Commands related to Files & directories
$ cat > testType here the text contentsAfter typing the entire textPress <ctrl d> to save and exit<ctrl d>
$ cat chap1 chap2- display contents of chap2 immediately after displaying chap1
Commands related to Files & directories
• Copies a file or group of filese.g. $ cp chap1 temp/chap2Option - i (interactive)
$cp - i chap1 chap2cp: overwrite chap2 ? y
Option -r (recursive) to copy entire directory$cp - r temp newtemp
Commands related to Files & directories
• To delete files$ rm chap1 chap2 chap3$ rm * Are you sure? y
Option - i (interactive delete)$ rm - i chap1 chap2chap1 : ? y
Commands related to Files & directories
• Option - r (recursive delete)$ rm -r *(Warning: Pl. do not use this option)
Commands related to Files & directories
• To rename: mv command
• Used to rename file or group of files; as well as directories
$ mv chap1 man1
• The destination file, if existing, gets overwritten
Commands related to Files & directories
e.g. $ mv temp doc
e.g. $ mv chap1 chap2 chap3 man1
will move chap1, chap2 & chap3 to man1
directory
Commands related to Files & directories
• To alter file permissionsFormat :chmod <category> <operation> <permission> <filenames>Category Operations Attributeu-user +assigns permission r-readg-group -remove permission w-writeo-others =assigns absolute permission x-executea-all
Commands related to Files & directories
e.g. $ chmod u+x note$ ls - l note -rwx r-- r -- 1 …… note
e.g. $ chmod ugo+x note$ ls - l note
-rwxr-xr-x …… note
Commands related to Files & directories
e.g. $ chmod u-x, go+r note$ chmod u+x note note1 note2$ chmod o+wx note$ chmod ugo=r note
Commands related to Files & directories
• Octal notation• describes both category and permission• similar to =operator (absolute assignment)• read permission - assigned value
is 4
write permission - assigned value is 2
Execute permission - assigned value is 1
e.g. $ chmod 666 notewill assign read & write permission to all
Commands related to Files & directories
e.g. $ chmod 777 notewill assign all permissions to all
e.g. $ chmod 753 note?
Commands related to Files & directories
To create a directory - mkdir command• Creates a directorye.g. $ mkdir doce.g. $ mkdir doc doc/example doc/datae.g. $ mkdir doc/example doc
- will give error - Order important
Commands related to Files & directories
rmdir command• Used to remove directory• Only empty dir can be deleted• More than one dir can be deleted in single command• Command should be executed from at least one level above in the
hierarchy
Commands related to Files & directories
e.g. $ rmdir doc$ rmdir doc/example doc$ rmdir doc doc/example- will give error
Some other Unix commands :wc
• Counts lines, words and character depending on option• takes one or more filename as arguments
e.g. $ wc infile
3 20 103 infile
Some other Unix commands :wc
$ wc infile test 3 20 103 infile 10 100 180 test 13 120 283 total$ wc - l infile
3 infile$ wc - wl infile
20 3 infile
Some other Unix commands : more
• Allows user to view a file one screen at a timeSyntax :
more <options <+linenumber> <+/pattern> <filename>
more continued
• : appears after displaying screen• <spacebar> or <enter> - next screen• q - quit• n - switch to next file on command• p - switch to previous file• f - displays filename and line no.• /pattern - to locate line with pattern• ! - to run command
more continued
Command Descriptionk:n Skips to kth next file that has been
specified in the command linek:p Skips to kth previous file that has been
specified in the command line:f Displays current filename and line
number= Displays current line number
more continued
Command Descriptionspacebar Displays next screenks Skips k lines forwardkf Skips k screens forward, Reverts to beginning of file if no search
made, or to point where last search wasmade
/pattern Searches for pattern forward
more continued
Command Descriptionn Repeats last search forwardv Starts up the vi editor. Repeats previous command! cmd Executes the Unix command cmdq Exits from more
Some other Unix commands:cmp
• To compare two files & echo first mismatch• option -l gives detailed list of byte no. and different byte in octal
e.g. $ cmp note1 note2 note1 note2 differ : char3, line1$ cmp - l note1 note2 3 143 145 6 170 167 7 171 170
Some other Unix commands:comm
• To compare two sorted files• displays 3 - columnar output• first column - lines unique to first file
second column - lines unique to second filethird column - line common to both files
• option -1, -2, -3 can be used to display specific column only
Some other Unix commands:diff
• Display file differences
• Display which lines in one file have to be changed to make both files identical
Some other Unix commands:od
• Lists the octal value (ASCII) of any file content• used to display non-printing character• options
• b to display 16 byte of data on each line• c to display character above octal value
e.g. $ od - bc odfile
Some other Unix commands:od
• Lists the octal value (ASCII) of any file content• used to display non-printing character• options
• b to display 16 byte of data on each line• c to display character above octal value
e.g. $ od - bc odfile
Some other Unix commands: banner
• Creates poster by blowing up its arguments• maximum ten characters in a line
e.g. $ banner UNIX$ banner ‘UNIX’ ‘IS HERE’
Some other Unix commands:cal
• Displays calendar of a particular month/ year• any calendar from year 1 to 9999
e.g. $ cal 1991$ cal 01 1991
Some other Unix commands:date
• Displays system date• number of options available
option m - monthh - month named - day of monthy - last two digit of the yearH - hourM - minuteS - second
Some other Unix commands:date
option ( cont.. )T - time in hh:mm:ssa - day of week (Sun to Sat)r - time in AM/PM
e.g. $ date Fri Dec 7 15:00:21 EST 1990$ date +%m 12$ date +”%h%m” Dec 12
Some other Unix commands:who
• Displays current users of the system• displays logname, terminal no., login date and time
$ who$ who am i
logname of the user
Some other Unix commands
• Printing with lp command
• Spooling
• Job No
$ lp <file>
Some other Unix commands
• tr command
• translation of characters
• squeeze spaces ( -s )
$ tr –s “ ” < file1
$ tr “[a-z]” “[A-Z]” < file1
Some other Unix commands
• find command• locate files
find <path list> <selection criteria> <action>• The following command will locate all the .profile files in the system
$ find / -name .profile -print
Some other Unix commands
• nl command• This command elaborates schemes for numbering lines.
$ nl file1.txt• This command starts numbering lines from 40 increment value is 7 and
width of numbers is 2. ( max 6 allowed )$nl -w2 -v40 -i7 file1.txt
Terminal Related Commands
• tty - displays device name of the terminal• stty -a - displays current settings• stty can also be used to change the setting
To use <ctrl c> as interrupt key instead of <Del> key
$stty intr \^c
Introduction to Bourne Shell
• Shell - The agency that sits between user and UNIX System
- much more than command processor
• Bourne Shell- named after its founder Steve Bourne
- widely used
• C Shell - product from Univ. of California, Berkeley
- advanced user interface and enhanced features
• Korn Shell - by David Korn of Bell Lab.
Introduction to Bourne Shell
Executables in /bin directory
• sh indicates - Bourne Shell
• csh if present indicates - C Shell
• ksh if present indicated - Korn Shell
Working of shell
Continous sleep-waking-waiting cycle• Performs following activities:
• Issues a $ prompt & waits for user to enter a command• After user enters command, shell scans & processes the
command.• The command is passed on to the Kernel for execution
& the shell waits for its conclusion.• The $ prompt appears so that the user can enter next
command.
Bourne shell metacharacters
• * : To match any number of characters• ? : To match with a single character• [] : Character class; Matching with any single character specified within
[]• ! : To reverse matching criteria of character class.
Bourne shell metacharacters
• \ : To remove special meaning attached to metacharacters• ; : To give more than one command at the same prompt
Shell redirections
• Every Unix command has access to:• Standard input• Standard output• Standard error
• Shell can redirect I/p, o/p or error to any physical file using meta characters “<“, “>” & “2>”
Shell redirections
Examples:
$ ls > temp
$ wc < file1.txt > result
$ cat nonexistantfile 2> err
Building block primitives
• Pipe - allows stream of data to be passed between reader & writer process
• The data that the first processor writes into the pipe is input to second process$ who | wc-l$ ls | wc -l$ ls | wc -l > fcount
Building Block Primitives
tee - standard output can be saved in a file as well as displayed - can be placed anywhere in pipeline
e.g. $ who | tee user.lstkumar tty01 May 18 09 : 32sharma tty02 May 18 11 : 18tiwary tty03 May 18 13 : 21
e.g. $ who | tee user.lst | wc-l3
Command Substitution
• Command enclosed in backquotes (`), the shell executes the command first, and the enclosed command text is replaced by the output of the commande.g.$echo The date today is `date`
The date today is Fri 27 00:12:55 EST 1990$echo The date today is; date- issues command echo and date sequentially
Command Substitution
e.g. $echo “There are `ls | wc - l` files in current directory”$cal `date “+%m 19%y”`
echo Command
• Displays argumente.g. $echo GOOD Morning
GOOD Morning$echo \**$echo ‘*’*
echo Command
• escape sequence to manipulate cursor position\t : tab\f : formfeed\n : newline\c : cursor immediately after argumente.g. $echo ‘Enter Your Name : \c’
Enter Your Name : $_
echo Command
• accepts ASCII Octal valuese.g. $echo ‘\007’
<… beep hear…:
Shell Variables
• User variables can be defined• no type declaration or initialization• Format : variable=value• Value is of string type• No space preceding and following =operator
Shell Variables
• Evaluated by prefixing variable name with $• multi word assignment must be quoted
e.g.1 $x = 37$echo $ x37
Shell Variables
e.g.2 $msg = You have mailhave not found
e.g.3 $msg = ‘You have mail’$echo $msgYou have mail
Shell Variables
$msg = You\ have\ mail
$echo $msg
You have mail
$x = aa
$y = bb$echo $x$y
aabb
Shell Variables
$echo ‘Amount = $1000’$echo Amount = \$1000- both will display : Amount =$1000$echo “Amount = $1000”Amount=000- here shell evaluated variable $1 as double quoted
Shell Variables
$p = `pwd`$echo $p$echo ${fname}emp.sh$echo ${fname}xemp.shx
Shell script: an example
$ cat script1.sh
date
echo ‘calender for the current month is’
cal ‘date”+%m 20%y”’
Shell script: an example
$ cat script2.shecho ‘enter your name.’read unameecho “hi $uname”
Shell script: an example
$script2.shenter your name:xyzHi xyz
Simple Filters
• Central tools of Unix toolkit
• useful text manipulators
• both horizontal & vertical filters available
head Command
• Displays the beginning of one or more file se.g. $head -3 emp.lst
• will display first 3 lines• if number not specified (default) 10 lines
$head -2 emp.lst dept.lst
tail command
• displays the end of a file• default - display last 10 lines• $tail -3 emp.lst
• displays last 3 lines• $tail +50 emp.lst
• displays till end of file starting from line no. 50• OR• tail -n +2 emp.lst
tail command
$tail -15c emp.lst• display last 15 characters
$tail -1b emp.lst• block is used as unit of measurement
Simple filters
Cut :• Allows slicing file vertically• identifies both column and fields
e.g. $cut -c6-22, 24-32 emp.lst• cuts the file emp.lst vertically from column 6 to 22 and
from column 24 to 32$cut -c-5, 10-15, 25 - emp.lst implies 1-5 implies 25 - end of file
Simple filters
Cut :• -d option to specify delimiter can be used with -f option
which specifies field no.• default delimiter is tab• cut selects entire line if not properly delimited
$cut -d “|” -f 2,3 emp.lst$cut -d “|” -f 1,5- empl.lst | tee temp
Simple filters
Paste :paste more than one file laterally$paste file1 file2<…file1> <file2>
• delimiter can be inserted between two files
$paste -d “|” file1 file2
Sorting a file
• Reorders a line in a ASCII collating sequence• sorting can be done on a field• delimiter can be specified, -t option
$sort -t “|” +1 emp.lst
+1indicates sort after skipping first field i.e.on 2nd fieldOr$sort –t “|” –k 2 emp.lst
• sorting in reverse order -r option$sort -r +1 emp.lst
Sorting a file
• output filename can be specified as argument$sort -0 sortedf emp.lst
• -c (check) option to check if sort successful. No message indicates sort successful
• char position can be specified$sort -t “|” +4.6 - 4.9 emp.lst• will sort 7th & 8th column of field 5• Or$sort –t “|” –k4.6, 4.9 emp.lst
• sorting on multiple fields$sort -t “|” +2-3 +1 file1
$sort -m file1 file2 file3
• merges sorted files• Numeric Sort
$sort -n file1
Sorting a file
Uniq Command
• removes duplicate records (-d option)
$uniq dept.lst
$sort dept.lst | unique - uniqlst• -u option if only unique records to be listed• -d if only one copy of repeated records to be selected
Editors
• Three Types• Line Editor - ed, ex• Screen Editor - vi• Stream Editor - sed
vi Editor
• Three Modules :
CommandMode
exMode
Input command
Mode
i, I, a, A,r, R, o, O,s, S
<Esc>
<Enter>
sh
:
<ctrl d>or exit
Input Mode Commands
Command Functioni Insert text to left of cursorI Inserts text at beginning of linea Appends text to right of cursorA Appends text at the end of lineo Opens line belowO Opens line above
Input Mode Commands
Command Functionrch Replaces single character under
cursor with character ch (no<Esc>)R Replaces text from cursor to rights Replaces single character under
cursor with any number ofcharacters
S Replaces entire line
vi Editor - Save & Quit
• From input mode ZZ• From input mode to command mode press <Esc> and from command
modeTo Save :wTo Quit :qTo save & quit :wq
or :x
vi Editor - Command Mode
Deletionx - single char4x - deletes 4 characters from cursor to rightdd - line delete4dd - deletes 4 lines
Cursor Movement Commands
Command Functionh or backspace Moves cursor leftj Moves cursor downk Moves cursor upI or spacebar Moves cursor right^ Moves cursor to beginning of first word
of line (no repeat factor)
Cursor Movement Commands
Command FunctionO or l Moves cursor to beginning of line (no
repeat factor with O)$ Moves cursor to end of lineb Moves cursor backwards to beginning
of worde Moves cursor forward to end of wordw Moves cursor forward to beginning of
word
Paging Functions
Command Function<Control-f> Full page forward<Control-b> Full page backward<Control-d> Half page forward<Control-u> Half page backward<Control-l> Redraw page screen (no repeat factor)
Search and Repeat Commands
Commands Functions/pat Searches forward for pat?pat Searches backward for pattern patn Repeats search in the same direction along
which the previous search was made (norepeat factor)
N Repeats search in a direction opposite tothat which the previous search was made(no repeat factor)
Search and Repeat Commands
Commands Functions
fch Moves cursor forward to first occurrence of character ch in the current line
tch Moves cursor forward onto but before first occurrence of character ch in the current line
; Repeats search in the same direction along which the previous search was made with f or t
, Repeats search in a direction opposite to that along which the previous search was made with f or t
vi Editor - Other Features
• Joining lineJ - to join current line with next line4J - to join 4 lines from current line
• Marking Textm <single char label> - to mark line<single char label> - to locate marked line
vi Editor - Other Features
• Repeating last Instruction - . (dot key)• Undo last Instruction - u• Reverse all changes made to current line - U
A Few Specimen Operator-Command Combinations
Command Function
d$ or D Deletes from cursor to end of line5dd Delete five linesd/endif Deletes from cursor up to the first occurrence
of the string endif in the forward directiond30G Deletes from cursor up to line number 30df. Deletes from cursor to first occurrence of a
dotc0 Changes from cursor to beginning of linec$ or C Changes from cursor to end of line
A Few Specimen Operator-Command Combinations
Command Function3cw or c3w Changes three wordscc Changes current linecG Changes from cursor to end of filey$ or Y Yanks from cursor to end of line3yw or y3w Yanks three words from cursor position5yy Yanks five linesy?case Yanks from cursor up to the first occurrence
of the string case in the reverse!Gsort Sorts all lines from current position to end of
file
ex mode commands
Command Function:r note Reads file note into present cursor location:w saves buffer and remains in editing mode:w note1 Creates file note1 and writes buffer contents
to it:w! note1 Overwrites contents of file note1 with
contents of buffer:w>> note1 Appends buffer contents to file note1
ex mode commands
Command Function:x Saves buffer and quits editing mode:wq Saves buffer and quits editing mode:q Quits editing mode without saving:q! As above but after abandoning changes:e note1 Stops editing current file, and edits file note1“e! note1 As above, but after abandoning changes
made to current file
ex mode commands
Command Function:e! Loads last saved edition of current file:e # Returns to editing the most recently edited
file:n Edits next file (on the vi command line):rew Rewinds file list to start editing from first file
specified in the command line100 Cursor at start of 100th line
Customising editing environment
• set• map• abbreviate
Set Options Used by ex/vi
Option Abbreviation Significance
autoindent ai Next line starts at the previous indented level
autowrite aw Writes current file automatically when-ever
switching files with :n and escaping to the
shell with :sh
ignorecase ic Ignores case while searching for patterns
magic Treats the characters of the regular
expression set as special while searching
for patterns
Set Options Used by ex/vi
Option Abbreviation Significance
number nu Displays line numbers on screen
showmatch sm Shows momentarily the mach to a ) & }
showmode Displays a message when vi is in input
mode
tabstop ts Sets tab for display (default : 8 spaces)
wrapscan ws Continues pattern search by moving to
the other end of a file
Map & umap commands
• Map : used to assign undefined key or reassign defined ones.:map v “ayyto display current settings:mapto cancel a mapping:umap v
Abbreviate
• For mapping of characters in input mode ab se standard error
Options to vi
• To salvage a file after crash $ vi -r <file>
• to begin from specified line$ vi +<line> <file>
• to begin with specified pattern$ vi +/<pattern> <file>
grep Command - Advanced Filter
• Searches for a given pattern in the files specifiedgrep <options> <pattern> <filename(s)>
• Options-c displays count of occurrences -l displays list of filenames only-n displays line number along with lines-v displays all but the line matching pattern
grep Command - Advanced Filter
• Options ( cont..)--i ignores case-h omits filename-f fname reads expression/pattern from file
(egrep & fgrep option)-x displays lines match in entirety
(fgrep option)
grep Command
e.g. $grep sales emp.lst- lists records with sales as dept$grep ‘sales’ emp.lst$str = ‘sales’$grep -c “$str” emp?.lst• will count no. of directors in all emp?.lst• display will be for each file separate
Regular Expressions with grep
• Used to specify pattern• specified in single quotes
Expressionch* - 0 or more occurrence of char ch[pqr] - matches single char p, q or r[^pqr] - matches single char which is not p, q
or r[c1-c2] - match single char within ASCII range
specified
Regular Expressions with grep
• Expression.(dot) -matches with any single character^pattern - matches with pattern at beginning of linepattern$ - matches with pattern at end of linech \ {m\} - matches m occurrence of char chch \ {m,n\} - matches min. m & max. n occurrence
of char chch \ {m,\} - matches min. m occurrence
Regular expression with grepConsider following R.E.(i) ‘[cC]ho [wu]dh*ury’(ii) ‘[7-9] …$’ char [7-9] followed by 3 more
chars rep by … at end of line(iii) ‘director.*[7-9]…$’(iv) ‘^[\t]*$’(v) ‘^ ……… 03’
.\{10\}(vi) [0-9] [0-9] [0-9] [0-9]
egrep Command
egrep - Extending grep• extends grep pattern matching capabilities
• Extending R.E. features
ch+ matches 1 or more char ch
ch? Matches or 1 char ch
exp1/exp2 matches exp1 or exp2 expression
(x1/x2)x3 matches expression x1x3 or x2x3
x1/x2 matches expression x1 or x2
egrep Command
e.g. $ egrep ‘(sen/das)gupta’ emp.lst
$ egrep ‘[aA]gg?[ar]+wal’ emp.lst
$ egrep -f pat.lst emp.lst
fgrep Command
fgrep - multiple string searching• only simple (pattern) string or group of strings can be searched• faster than grep and egrep
e.g. $fgrep -f pat.lst emp.lst$cat pat.lst sales admin
Some limitations of grep family
• No options to identify & work with fields• Output formatting, computations etc. Not possible• Extremely difficult to specify patterns/ R.E always.
Advanced Filter - awk
• Report writer : very powerful text manipulation utility• Named after Aho
WeinbergerKernigham
• As powerful as any programming language• It can access, transform and format individual fields in a record• Generally awk programs are slow.
Advanced Filter - awk
• Syntaxawk <options> ‘line specifier {action}’ <files>
Advanced filter awk
Simple awk filteringawk ‘/director/ {print}’ emp.lst
line specifier Action(selects line (as has an effect of for processing) printing all selected lines)
• $0 variable stores entire line
Advanced Filter - awk
• Line specifier and action option are optional, either of them needs to be specified
• line specifier not specified indicates all lines to be selected• {action} omitted indicates print (default)• fields are identified by special variable
$1, $2, ….; default delimiter is contiguous string of spaces
Advanced Filter - awk
• Explicit delimiter can be specified using -F optione.g. awk -F “|” ‘/sales/{print $3, $4}’ emp.lst
• Regular expression of egrep can be used to specify pattern• Line numbers can be selected using NR built-in variable
$awk -F “|” ‘NR ==3, NR ==6 {print NR, $0}’ emp.lst
Advanced Filter - awk
• Output can be formatted using printf• similar to printf in ‘c’ but not a function
e.g. $awk -F “|” ‘/[aA]gg?[ar]+wal/{>printf “%3d%-20s%-12s%d\n”, NR, $2,$3,$6}’>emp.lst
Record no./Line number will be printed
• , (comma) to be used as delimiter in printf
Advanced Filter - awk
• Logical and Relational operators• Logical operator &&, ||
$awk - F “|” ‘$3 ==“director” || $3 == “chairman”{>printf “%-20s”,$2}’ emp.lstRelational Operators : <, <=, ==, !=, >=, >, ~, !~e.g. $awk -F “|” ‘$6>7500 {printf “%20s”, $2}’
emp.lste.g. $awk -F “|” ‘$3 == “director” || $6>7500 {
>print $0}’ emp.lst
Advanced Filter - awk
• == tries to find perfect match but string may have trailing spaces to overcome this ~ and !~ (match & negate of match) can be used with R.E.
$awk -F “|” ‘$2~/director/||$~/g.m/{>printf $0}’ emp.lst
$3 ~/^g.m/
Advanced Filter - awk
Number Processing• Computation on numbers can be done
+, -, *, /, % operator available• no type declaration for variables• variables initialized to zero
$awk -F “|” ‘$3 == director” || $6>7500 {>kount = kount+1>printf “%3d%-20s\n”, kount,$2}’ emp.lst
awk Command
• -f option• awk program can be written in a separate file and used in awke.g. $awk -F “|” -f emp.awk emp.lst
single quoted contents are written in this file without quotes
awk Command
• BEGIN and END SectionFormat: (i) BEGIN {action}
(ii) END {action}
awk<options> ‘BEGIN {action} line specifier {action} END {action}’ <files>
awk Command
e.g. $cat emp.awkBEGIN {printf “\n\t Employee details \n\n”}$6>7500{# increment sr. no. and sum salarykount++; tot+=$6printf “%d%-20s%d\n”, kount, $2, $6}END {printf “\n The Avg. Sal. Is %6d\n”, tot/kount}$_&awk -F “|” -f emp.awk emp.lst
awk Command
• Positional parameters and shell variable
• requires entire awk command should be in the shell script• to differentiate positional parameter and field identifier,
the positional parameter should be single quoted in awk program
e.g. $3 > ‘$1’
nawk
Enhancements to awk available as new command - nawk.• Its possible to use getline statement to accept I/p from the user or from a file.• Its possible to use the built-in variables ARGC & ARGV to check arguments
passed.• Its possible to define own functions.• Its possible to use the system function to run unix commands or any other
executable within nawk.• The ?: operator as in “C” is available within nawk. getline x < “Emp.lst”
getline x < “/dev/tty”
Built-in Variable
NR - Cumulative no. of records readFS - The i/p field separatorOFS - The output field separatorNF - No. of fields in current recordFILENAME- The current i/p fileARGC - No. of arguments in command lineARGV - List of arguments
Built in functions
Functionsint(x) - Returns integer value of xsqrt(x) - Returns square root of xindex(s1,s2) - Returns the position of string s2 in
s1length() - Returns length of the argument
Built in functions
Functionssubstr(s1,s2,s3) - Returns portion of string of length
- s3,starting position s2 in string s1split(s,a) - Split string s into array a optionally returns
no. of fields
Arrays
• Can handle single dimension array • index can be anything even string• no declaration required, initialized to zero automaticallye.g. totl[1] +=$6; tot[2]+=da; tot[3]+=HRA; temp[$3]
Interface with Shell
• Redirection > , piping | possible• restriction - filenames, commands that follows these symbol must be in
double quotesProgramming constructs :(I)if … then …
if (conditions is true)<statements>
else<statements>
Interface with Shell
Programming constructs :(II) For statement
for (i=1; i <= 9; i++){
}
Interface with Shell
(iii) for loop for arraysfor (k in array)
<commands>(iv) While loop
while (condition is true){}
(v) break and continue
Shell Variables
• System Variables
• Set during - booting- login
• script executed during login time - profile• profile alters operating environment of a user• to display list of system variable $set
Shell Variables
• list of variable
PATH - list of dir. In order of precedence that have to be scanned for hunting command
Home - dir. Named after Login name
IFS - string of char which are used as word separator in the command line
MAIL - mail directory
PS1 - Two prompts - primary and secondary
PS2 - (used by sed, awk)
TERM - Terminal type
Shell Variables
DATA in the same file - The here document• < redirection symbol, redirects the standard input to come from a file• << here document symbol signifies data is here rather than in a Separate
file
Shell Variables
e.g. $grep director <<mark - delimiter to indicate end of document
>1001 | Sanjay | g.m. | 5 >5001 | Rahul | director | 7
::
>mark
Read Statement
• Takes input from standard i/p• can be used to read one or more variables
e.g.echo “\n Enter the Pattern : \c”read patnameecho “\n Enter file name : \c”read fname grep “$patname” $fname
Command Line Arguments
• Shell procedures arguments in command• Arguments are specified along with the name of shell procedure• Arguments are assigned to special variables (positional parameters) $1 - First
parameters
$2 - Second parameters
:
:
Command Line Arguments
• Special parameters also available$0 - Name of executed command$* - Complete set of positional parameters$# - No. of arguments$$ - PID of current shell$! - PID of last background job$? - Exit status of last command$@ - Same as $* except when enclosed in double quotes
E.g.1 if [$# -nc 3]then
::
fi2 $kill - 9 $$3 grep “$1” $24 grep director emp.lst >/dev/null;echo$?
Exit Status indicates successful1 indicates failure2 indicates unreadable file
Command Line Arguments
Conditional Execution
• Logical Operators && and ||• && operator delimits two commands, the second command is executed only if
first succeeds• || operator delimits two commands, the second command is executed only if
first fails
e.g. $grep `director` emp.lst && echo “patern found”
$grep `manager` emp.lst || echo “pattern not found”
Conditional Execution
Script Termination - exit statement
• premature termination of program• control returned to the calling program
e.g. grep “$1” $2 || exit 2
echo “pattern found”
Return value
default zero
if Statement
• Format :(i) if <condition is true>
then<execute commands>
else(execute commands>
fi(ii) if <condition is true>
then<execute commands>
fi
if Statement
(iii) if <condition is true>then
<execute commands>elif <condition is true>then
<execute commands><…>
<…>else
<execute commands>fi
if Statement
• Condition can be specified either using test or [cond]• Relational Operator used
0 eq Equal to0 nc Not equal to0 gt Greater than0 gc Greater than or equal to0 lt Less than0 lc Less than or equal to
if Statement
• File related tests with test
-a <file> Trace if <file> exists-f <file> if exists & regular file-r <file> if exists & readable-w <file> if exists & writable-x <file> if exists & executable-d <file> if exists & is dir file-s <file> if exists & size > 0
Test - String Comparison
• String tests used by test
0 n str True, if str not a null string0 z str True, if str is a null string0 S1 = S2 True, if S1 = S20 S1! = S2 True, if S1 S20 str True, if str is assigned and not null
• Logical Operators
0 a .AND.0 O .OR.0 ! Not
Test - String Comparison
e.g. If test $x -eq $y if [$x -eq $y]
e.g. If [! -f fname]then
echo “file does not exists”fi
The Case Statement
• Format :case <expression> in<pattern 1> ) <execute commands> ;;<pattern 2> ) <execute commands> ;;
<…><…>
esac
The Case Statement
e.g. echo “\n Enter Option : \c”read choicecase $choice in1) ls -l ;;2) ps -f ;;3) date ;;4) who ;;5) exit ;;
esac
The Case Statement
E.g.Case `date | cat -d “ “ -fi ` inMon > <commands> ;;Tue > <commands> ;;
::
esac
The Case Statement
E.g.
Case “$ans” in[yY] [eE] [sS] ;;
[nN] [oO] exit ;;*) “invalid option” ;;
While Statement
• Format :while <condition is true>do
<execute statements>done
While Statement
e.g. while [$x -gt 3]do
ps -fsleep 5
donee.g. while true
dops -fsleep 5
done
While Statement
E.g. Script to edit, compile & execute a programwhile truedo
cc $1case $? In
o) echo “Compilation Successful” echo “Executing a.out” a.out ; exit ;;
*) echo “Compilation Error” echo “Press <Enter> to edit” read pause vi $1 ;;esacdone
Break & Continue Statement
• Continue : Suspends execution of all statements
following it, and switches control to
top of loop for next iteration• Break : causes control to break out of the loop
Break & Continue Statement
e.g. Validation of designation codefname = desig.lstwhile echo “Designation code : \c”do
read desigcase “$desig” in[0-9] [0_9] if grep “^$desig” $fname
>/dev/nullthen
echo “code exists”continue
fi ;;*) echo “invalid code”
continue ;;esacdone
Until Statement
• Complement of while stm• loop body is executed repeatedly as long as the condition remains false• e.g. until false
dops -fsleep 5
done
for Statement
• Format :for variable in listdo
<execute commands>done
for Statement
e.g. for x in 1 2 3do
echo “The value of x is $x”done
e.g. for var in $PATH $HOME $MAILdo
echo “$var”done
e.g. for file in *.cdo
cc $filedone
While Statement
E.g. Script to edit, compile & execute a programwhile truedo
cc $1case $? In
o) echo “Compilation Successful” echo “Executing a.out” a.out ; exit ;;
*) echo “Compilation Error” echo “Press <Enter> to edit” read pause vi $1 ;;esacdone
Break & Continue Statement
• Continue : Suspends execution of all statements
following it, and switches control to
top of loop for next iteration• Break : causes control to break out of the loop
Break & Continue Statement
e.g. Validation of designation codefname = desig.lstwhile echo “Designation code : \c”do
read desigcase “$desig” in[0-9] [0_9] if grep “^$desig” $fname
>/dev/nullthen
echo “code exists”continue
fi ;;*) echo “invalid code”
continue ;;esacdone
Until Statement
• Complement of while stm• loop body is executed repeatedly as long as the condition remains false• e.g. until false
dops -fsleep 5
done
for Statement
• Format :for variable in listdo
<execute commands>done
for Statement
e.g. for x in 1 2 3do
echo “The value of x is $x”done
e.g. for var in $PATH $HOME $MAILdo
echo “$var”done
e.g. for file in *.cdo
cc $filedone
Process
• Instance of program in execution
• Many processes can run at the same time
• Every process identified by Process Identifier
• PID is allocated by kernel
Parent & child processes
• On logging to system, a process is set up due to execution of shell
• Shell is the parent process for every other process setup due to execution of commands
• Every process has a parent process
• Parent process waits for death of child process before resuming execution.
Process Status
• ps command to display characteristics of a process• options
-f - full form -u - details of only users processes-a - all processes details-l - detailed listing-e - system processes
Background Process
• To run a process in background, use & operator• $sort -0 emp.lst emp.lst &
• nohup (no hangup) - permits execution of process even if user has logged off$nohup sort emp.lst &
sends output to nohup.out
Process Related Commands
• wait• waits for child process to complete $wait 138 - waits for background job with pid 138
• kill$kill 1005 (default signal 15)• kills job with pid 1005$kill -9 1005
sure killing of job$kill 0 - kills all background process
Process Related Commands
• nice - job execution with low priority• prefix the command with nice
$nice wc -l xxx$nice -15 wc -l xxx
reduce priority by 15 units
Communication & scheduling
• News command invoked by user to read any message sent by administrator• Messages stored in /usr/news• options: -n,-s,-a
Communication & scheduling
• motd• message od the day- a text file in /etc/motd• readable by all users
Communication & scheduling
• write command:• allows 2-way communication between users who are logged on.• Message occurs on screen anytime it is sent.• The mesg command with parameter n prevents disturbances.
Communication & scheduling
• mail command:• single channel communication.• User need not be logged on : message gets saved in users mailbox.• Many internal commands available within mail.
Communication & scheduling
• wall commandused by system administrator to send messages to all logged on users.
Communication & scheduling
• Some commands for scheduling:
• sleep : To introduce delay in script
• at : To schedule job execution at later date/time
• batch: To run the job when system load permits.
• cron : To schedule jobs for periodic execution.
While Statement
E.g. Script to edit, compile & execute a programwhile truedo
cc $1case $? In
o) echo “Compilation Successful” echo “Executing a.out” a.out ; exit ;;
*) echo “Compilation Error” echo “Press <Enter> to edit” read pause vi $1 ;;esacdone
Break & Continue Statement
• Continue : Suspends execution of all statements
following it, and switches control to
top of loop for next iteration• Break : causes control to break out of the loop
Break & Continue Statement
e.g. Validation of designation codefname = desig.lstwhile echo “Designation code : \c”do
read desigcase “$desig” in[0-9] [0_9] if grep “^$desig” $fname
>/dev/nullthen
echo “code exists”continue
fi ;;*) echo “invalid code”
continue ;;esacdone
Until Statement
• Complement of while stm• loop body is executed repeatedly as long as the condition remains false• e.g. until false
dops -fsleep 5
done
for Statement
• Format :for variable in listdo
<execute commands>done
for Statement
e.g. for x in 1 2 3do
echo “The value of x is $x”done
e.g. for var in $PATH $HOME $MAILdo
echo “$var”done
e.g. for file in *.cdo
cc $filedone
Program Development
• CC
• Make
• SCCS
CC Command
• The UNIX system compiler for C. • cc compiles the file if no errors are found an executable version of the
file is created. The default name of the executable file is a.out• Invokes a series of programs – the preprocessor, the compiler and the
linker.• Subprograms can be compiled together.
Make Utility
• Make utility and makefiles help automate building of an application. • Modifying any one or more of the source files requires recompilation
of that part of the program and relinking.• Automate this process by specifying the interdependencies between
files that make up the application along with the commands needed to recompile and relink each piece.
•
Make Utility
• During program development a number of subroutines may be developed separately and linked at the end. However keeping a track of the updated versions is difficult. Further, files that are changed may not be recompiled.
Make Utility
• These instructions are placed in make file. The command make is used to execute instructions defined in make file.
$ make myprog
Make Utility
• A make file consists of a series of entries of the following form:
[...targetfile..]: [.....dependencies....][One Tab Space] [.....commands........]
Make Utility
main.o: main.c cc -c main.c
• Makefile and all files referenced by MAKE should be in the same directory.• Sees the dependencies upon program.Then checks to see if program is current
SCCS-Source code control system
• SCCS is a group of programs that help in controlling the various versions of a program.
• Multiple copies are not stored. Only incremental changes to the program are stored.
SCCS-Source code control system
• The following commands are used with SCCSAdmin : used to create initial versions of the fileGet : used to retreive the SCCS filesComb : used to combine two or more versions and remove redundant versionsRemdel : used to remove a versionPrs : removes information about a SCCS file