Command Line Variations Selected Readings in Chapters 3, 5, 6
Project 1 kickoff Streams & Redirection: Sections 5.2, 5.3 Pipes and Tees: Sections 5.4, 5.5 Filenames & Wildcards: Section 3.1 Job Control: Section 5.10 Filter Commands: details in Chapter 6
cat, tr, head, tail, cut, paste, sort,
wc (word count)
Figure 5-4
Standard Streams
Standard I/O Standard Output (stdout) labelled (1) above
– default place to which programs write
Standard Input (stdin) labelled (0) above– default place from which programs read
Standard Error (stderr) labelled (2) above– default place where errors are reported
To demonstrate -- cat– Echoes everything you typed in with an <enter>– Quits when you press Ctrl-d at a new line -- (EOF)
Figure 5-6
Redirecting Standard Output
Redirecting Standard Output cat file1 file2 > file3
– concatenates file1 and file2 into file3– file3 is created if not there– Any pre-existing file3 is clobbered if there
cat file1 file2 >> file3– file3 is created if not there– file3 is appended to if it is there
cat > file3– file3 is created from whatever user provides from
standard input
Figure 5-7
Print who Output
Redirecting Standard Error
In bash (for shell scripts), standard error is redirected using 2> – cat myfile > yourfile 2> yourerrorfile
A better example that generates errors:ls –R / recursively list all files from rootyou will get many screens of output
Redirect output into a file:ls –R / > allfiles save output from aboveyou will get many “permission denied errors”more allfiles but you can look at listing in file now
Redirect errors to a second file:ls –R / > allfiles 2> errors errors go into file errors
If you don’t care about errorsls –R / > allfiles 2> /dev/null /dev/null is a ‘bit bucket’
where bits go to die!
Figure 5-5
Redirecting Standard Input
Redirecting Standard Input
terra[1] $ cat < oldfile > newfile A more useful example:
– terra[2] $ tr string1 string2 Read from standard input. Character n of string1 translated to
character n of string2. Results written to standard output.
– Example of use:terra[3] $ tr aeoiu eoiuaterra[4] $ tr a-z A-Z < file1 > file2
/dev/null
/dev/null– A virtual file that is always empty.– Copy things to here and they disappear.
cp myfile /dev/null mv myfile /dev/null
– Copy from here and get an empty file. cp /dev/null myfile
– Redirect error messages to this file cat myfile > yourfile 2> /dev/null Basically, all error messages are discarded.
Figure 6-1
Concept Of A Filter
Basic Filters Filters are programs that:
– Read stdin.– Modify it.– Write the results to stdout.
Filters typically do not need user input. Example:
– tr (translate): Read stdin Echo to stdout, translating some specified
characters Many filters can also take file names as
operands for input, instead of using stdin.
Filtering using tr (translate command) Using tr to translate what you type into upper case
tr [a-z] [A-Z] (enter, type, enter etc. ctrl-D to quit)
Using tr to translate a file using input redirection
tr [a-z] [A-Z] < foods (now foods will appear UC)
tr [a-z] [A-Z] foods (also OK, tr accepts filename args)
Using tr to translate a file using input redirection and store results using output redirection
tr [a-z] [A-Z] < foods > foodsCAP
tr [a-z] [A-Z] foods > foodsCAP (also OK, see above)
(now foodsCAP will hold UC output from foods)
More filters: grep, wc, sort (see Ch6) grep patternstr
– Read stdin and write lines with patternstr to stdout
terra[1] $ grep "unix is easy" < myfile1 > myfile2
terra[1] $ grep "unix is easy" myfile1 > myfile2 (also OK)– Write all lines of myfile1 containing phrase unix is easy
to myfile2 wc
– Count the number of chars/words/lines on stdin– Write the resulting statistics to stdout – Example: (wc < mars.txt OR wc mars.txt )
sort– Sort all the input lines in alphabetical order and write to
the standard output.
Figure 5-8
Piping Output to Next Command
Pipes The pipe:
– Connects stdout of one program with stdin of another
– General form:command1 | command2
– stdout of command1 used as stdin for command2– Example:
terra[1] $ cat foods | grep apples | wc -l A pipe works the same as (but more efficient than)
terra[2] $ cat foods > tmp1
terra[3] $ grep apples < tmp1 > tmp2
terra[4] $ wc -l < tmp2 Actually cat is redundant: grep apples foods | wc -l
Redirecting and Pipes (2) Note: The name of a command always comes
first on the line. There may be a tendency to say:
terra[1] $ foods > grep apples | wc -l – This is WRONG!!!– Your shell will go looking for a program named
foods To do it correctly, many alternatives!
terra[1] $ cat foods | grep apples | wc -l
terra[2] $ grep apples < foods | wc -l
terra[3] $ grep apples foods | wc -l
terra[4] $ grep -c apples foods
The tee Command (don’t worry about this) tee - replicate the standard output
– cat readme.txt | tee myfile
tee
stdin
stdout myfile
Unix Filename Rules Almost any character is valid in a file name
– all the punctuation and digits– the one exception is the / (slash) character– the following are not encouraged
? * [ ] “ ” ’ ( ) & : ; !– the following are not encouraged as the first
character - ~
– control characters are also allowed, but are not encouraged
UPPER and lower case letters are different– A.txt and a.txt are different files
Unix Filename Extensions No enforced extensions
– The following are all legal Unix file names a a. .a … a.b.c
Remember files beginning with dot are hidden– ls cannot see them, use ls -a
. and .. are reserved for current and parent directories
Unix Filename Extensions Even though Unix doesn't enforce extensions,
– “.” and an extension are still used for clarity .jpg for JPEG images .tex for LaTeX files .sh for shell scripts .txt for text files .mp3 for MP3’s
– some applications may enforce their own extensions Compilers look for these extensions by default
– .c means a C program file– .C or .cpp or .cc for C++ program files– .h for C or C++ header files– .o means an object file
Unix Executable Files Executable files usually have no extensions
– cannot execute file a.exe by just typing a– telling executable files from data files can be difficult
“file” commandUse: file filenameResult: print the type of the fileExample: terra[1] $ file ~/.bashrc
.bashrc: executable bash-shell script
Filenames and pathnames have limits on lengths– 1024 characters typically– these are pretty long (much better than MS-DOS
days and the 8.3 filenames)
Fixing Filename Mistakes It is very easy to get the wrong stuff into filenames
– Say you accidentally typed
terra[3] $ cp myfile -i– What if you type
terra[4] $ rm -i The shell thinks -i is an option, not a file Getting rid of these files can be painful
There is an easy way to fix this... – You simply type
terra[5] $ rm -- -i– Many commands use “--” to say there are no more options
Creates a filewith name -i
Filename Wildcarding (See section 3.1) Wildcarding is the use of “special” characters to represent
or match a sequence of other characters– a short sequence of characters can match a long one– a sequence may also match a large number of sequences
Often use wildcard characters to match filenames– filename substitution – generally known as “globbing”
Wildcard characters* matches a sequence of zero or more characters
– Example: a*.c* matches abc.c, abra.cpp,
? matches any single character– Example: a?.c matches ab.c, ax.c, but not abc.c
[...] matches any one character between the braces– Example: b[aei]tmatches bat, bet, or bit, not baet
Filename Wildcarding (2) Wildcard sequences can be combined
terra[6] $ mv a*.[ch] cfiles/ mv all files beginning with a and ending with .c or .h into
the directory cfilesterra[7] $ ls [abc]*.?
list files whose name begins with a, b, or c and ends with . (dot) followed by a single character
Wildcards do not cross "/" boundaries– Example: csnow*c does not match csnow/codec
Wildcards are expanded by the shell, and not by the program– Programmers of commands do not worry about searching
the directory tree for matching file names– The program just sees the list of files matched
Filename Wildcarding (3) Matching the dot
– A dot ( . ) at the beginning of a filename, or immediately following a /
must be matched explicitly.– Similar to the character /– Example:
terra[8] $ cat .c* As mentioned earlier, [....] matches any one of the
characters enclosed– Within “[...]”, a pair of characters separated by “-” matches
any character lexically between the two Example:
terra[9] $ ls [a-z]*
cat all files whose namesbegin with .c
lists all files beginningwith a character betweenASCII ‘a’ and ASCII ‘z’
Processes and Job Control
Figure 5-14
Job States
Foreground and Background (1)
Unix is a multi-tasking operating system– some of these tasks are being done by other
users logged in– some are being done by you in the
background e.g. watching for incoming mail
When you run a task (a Unix command, like ls or vi) it executes in the foreground of your shell– it has the “control" of your screen and
keyboard
Foreground and Background (2) If you still want to use your shell command line
terra[1] $ a_heavy_task & [1] 13607 terra[2] $
When you put a task in background– task keeps running, but you continue to work at the
shell in the foreground– if any output is done, it appears on your screen
immediately (can be confusing)– if input is required, process prints a message and
stops– when it is done, a message will be printed
Example ls –R / (again) Kick off a long command ls –R / > listall Suspend, Ctrl-Z Send to background bg
[1] ls –R / >listall& Oops, still get errors! Now you have to kill a background
job: kill %1 (or kill -9 %1)
[1] terminated Restart it the right way, in background, with err redirect:
ls –R / > listall 2> /dev/null &
Foreground and Background (3) Explicit background processes are needed
less often with windowing systems– Just go to another window and run the
command But explicit background processes are
used often ...– A command needs a long time, you do not
want to close that window by accident
– Run a job at the background and logout
– netscape& will open a new window, but leave the current shell window still available to use
A Simple Script We use the following shell script to illustrate job control Edit a file make_noise
terra[1] $ cat > make_noise#!/bin/shwhile truedo date sleep 1done^Dterra[2] $ chmod u+x make_noise
make_noise then is a shell script repeats to print the time for every second, until you terminate it using Ctrl-c.
Job Control – Suspending Jobs
csh, tcsh, and bash allow you to manage the running of different processes
Suspending jobs– the Ctrl-z special character stops the job
terra[1] $ make_noiseFri May 16 14:14:43 EDT 2003……^ZSuspendedterra[2] $ vi readme^Z
Job Control - Monitoring Jobs The "jobs" command shows which of your
jobs are running and/or stopped.
terra[3] $ jobs
[1] + Suspended make_noise
[2] + Suspended vi readme
Here there are two suspended processes, the make_noise and a vi process.
Job Control – Resuming Jobs Putting jobs back into the foreground:
– Use the "fg" command to move a job into the foreground.
terra[4] $ fg %2
– Puts job number 2 into the foreground.
– Works with either a background or stopped job.
Putting jobs into the background:terra[5] $ bg %1
Job Control – Killing Jobs Jobs can also be killed
– Use the Unix "kill" command
terra[6] $ kill %1
or if it won't die ...
terra[7] $ kill –9 %1
Jobs can be stopped and continuedterra[8] $ a_heavy_task &
terra[9] $ stop %1
terra[10] $ bg %1
Using ps (1) Jobs are really just a special case of Unix
processes ps can list your current processes
terra[11] $ ps
PID TT S TIME COMMAND
2312 pts/0 T 0:00 vi
2296 pts/0 R 0:00 tcsh
2313 pts/0 R 0:00 ps
Using ps (2) The ps command takes a number of
options -l gives you a long listing of what is going
on -u loginid tells you about loginid's
processes use man ps to see more options
kill pid kills the process pid– TERM signal will be sent to the process pid– kill -9 or kill -KILL will send the KILL signal– Use man –s 5 kill to find out more signals
Top Related