Introduction to Unix – CS 21

Lecture 13

Lecture Overview Finding files and programs

which whereis find xargs

Putting it all together for some complex programs

What Do You Do When You Want To Locate A File? Option 1: ls

Can be very time consuming Option 2: Other programs

which Used to see what you are running

whereis Used to see where programs are located

find A general tool to find all sorts of files in the

directory

which Usage: which PROGRAMNAME Tells you what executable is being

run when you run a program Useful to tell when more than one

executable exists (i.e. one in /bin and one in /sbin)

Only searches your PATH

Example Of which

whereis Usage: whereis PROGRAMNAME Locates the source, executable,

and man page for a command Searches set paths

Example Of whereis

find Locate files based on any number

of factors More than just the name

When it was modified What permissions it has What size it is

Running find Usage: find [path] [expression] A little different from all other

programs you’ve run Instead of a letter, options are the

complete word Example:

Instead of –n, you use -name

Expression Options To find -name “pattern”

Matches the name of a file -iname

Matches the name of a file but is case-insensitive -empty

Matches empty files -type

d f l

Examples

More Expression Options -atime N

File was last accessed N days ago -mtime N

File was last changed N days ago +N, -N

+N = More than N days ago -N = Less than N days ago

Examples

Combining Expression Options -o

Or -a

And \( \)

Grouping

Action Commands -print

Simply prints out the name of the file that was found

Most common action -exec

Executes a command -ok

Executes a command, but prompts the user first

The exec Option – A Powerful But Dangerous Option -exec COMMAND \; {} gets replaced with the name of

the file that was found Example:

find . –name “*.txt” –exec echo {} \; Same as find . –name “*.txt” -print

Examples

When Would You Ever Do This? The –exec option allows you to

perform some command on all files that match a certain criteria

Example: Someone cracked into your system and

you want to check all recently modified files for any changes against a backup

find –mtime –2 –exec diff {} .snapshot/{}

Problem The exec command will run the

command once for every file that matches A lot of overhead May take a lot longer than you’d like

Solution: xargs

The xargs Command Take values from standard in and

convert them into command line parameters

Usage: xargs COMMAND Example:

xargs grep pattern

What?

File1.txtFile2.txt

…

Program

stdin xargs Program File1.txt File2.txt …

Why? When a program outputs a list that

will be used as input to another program

Used mainly in pipes with find Example:

find . –name *.txt | xargs grep ‘Jason’ find . –name *.txt –exec grep ‘Jason’ {} \; Calls grep many, many times less than

the second call and takes less time

Another xargs example Problem: You want to kill all of your

processes First, list all your processes

ps aux | grep login Next, select all the PIDs

cut –d’ ‘ –f2 Now pass all those PIDs to kill ps aux | grep login | cut –d’ ‘ –f2 | xargs kill -9

A Simple Cleanup Script

#!/bin/bash ps aux | grep login | cut –d’ ‘ –f2 | xargs

kill -9exit 0

>./exitScript.sh &

When Would You Use Bash Versus Awk? What is bash good at?

Finding and manipulating files and directories What is bash bad at?

Manipulating data inside files What is awk good at?

Manipulating data inside files What is awk bad at?

Finding and manipulating files and directories

Building A Complex Program Start small

Always try to get something working before you get something working well

Build up Add functionality one part at a time

and test as you go

Building A Complex Program In Bash Starting small – #!/bin/bash exit 0 Slowly add functionality #!/bin/bash read input echo $input exit 0

Building Up A Complex Program – In awk Start small { print $0 } Slowly add functionality/Pattern/ { print $0 }

Hints Break the problem down into smaller

components Try printing first – always print as you go Insert temporary files or variables if needed Example:

If you are required to process any number of command line parameters one at a time, start by handling just one

After you can handle one, try two Generalize after you have gotten the first part

down

Example

Computing Problem Find all files we haven’t changed in

2 days and move them to a folder called oldStuff

Should we use bash or awk? Bash shell programming would be

better

Break It Down Find all files we haven’t changed in

two days find ~ –mtime +2 -print

Move each file to the oldStuff folder mv file ~/oldStuff/

Put It Together We need some way to connect the

first part with the second part How about a variable?

Store the list of file names and then go through them one by one

var=`command`

Script Solution

#!/bin/bashfilesToMove = `find ~ -mtime +2 –

print`for f in $filesToMovedo mv $f ~/oldStuff/doneexit 0

Another Computing Problem Remove all files that have the

phrase “Remove Me” located on the first line

Which should we use, bash or awk? We are looking inside files, but we

aren’t manipulating the data, so we should use bash.

Break It Down Find all files

find . –name “*” -print Get the first line

head -1 Look for “Remove Me”

grep “Remove Me” Remove the file if “Remove Me” was

there rm file

Put It Together Variables to pass information along

List of all files Temporary files or variables to

store results and test Results from grep

One Script Solution #!/bin/bash allFiles=`find . –name “*” –print` for f in $allFiles do head –1 $f | grep “Remove Me” > .tmpFile if [ -s .tmpFile ] ; then rm –i $f fi done rm –f .tmpFile exit 0

Another Computing Problem Tax calculator

Given a receipt, calculate the tax Receipt is a file containing the name

of the product and price of the product on one line

Pork Tenderloin:10.85

Should we use bash or awk? awk would be better in this case

Break It Down Calculate the tax on an individual

item Get the item

$2 Multiply it by the tax

$2 * .0775

Sum up all the taxes

Put It Together To keep a sum, we need to store

the running total in some location How about a variable?

totalTax = totalTax + newTax We need to initialize the tax in the

beginning and print out the tax at the end

Awk ScriptNF == 2 { totalTax = totalTax + $2 * .0775}BEGIN { totalTax=0}END { print totalTax}

Problem 4 Given a file that contains

inventory, print out all items that are out of stock File format: ITEM/PRICE/QUANTITY Example: Needlenose pliers/2.50/4

Should we use bash or awk? Awk is better this time since we are

checking the data

Script Solution

{ if ($3 == 0) {

print $1 }

}

Next Time Quiz #2

Bash and awk programs sed

Perl Programming Introduction