Tips and Tricks in Unix Shell scripting DB2 Database ...
Transcript of Tips and Tricks in Unix Shell scripting DB2 Database ...
![Page 1: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/1.jpg)
Tips and Tricks in Unix Shell scripting DB2 Database Administration
Speaker: James SobieskiFourth Millennium Technologies
Session Code: D14 -May 26, 2016 8:00 AM – 9 AM| Platform: DB2 for Linux, UNIX and Windows
Photo by Steve from Austin, TX, USA
![Page 2: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/2.jpg)
Agenda
• Scripting Uses
• Template Scripts
• Building Shell Functions
• Common Unix Commands used in scripts
• Awk usage
• Examples
2
![Page 3: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/3.jpg)
Shell Scripting Uses
• Automating DBA Activities
• Scheduling Complex activities via Cron
• Developing a library of “How I Did Something” scripts
• If you are going to do something 2 or more times, script it!
• General Advice• Ensure Scripts are Documented
• Be Consistent – this encourages reuse
• Write re-usable code snippets
• “Borrow generously” from others’ scripts
• This presentation will deal with mainly Korn Shell techniques
• We will review many powerful Unix commands used in shells
3
![Page 4: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/4.jpg)
Templates Script
• Recommend that you create a template script
• Include• Heading Comments
• Name, Description, Usage, Author, Create Date, Mod Date + Description
• Standard Functions
• Help or Usage
• Command Line Parsing
• Exit Cleanup
• DB2 Command post processing
• Common Debugging Code
• Standard Variables containing names of directories for TEMP files, outputs, scripts, etc
• Common scripting structures
4
![Page 5: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/5.jpg)
Temporary Files
• Many scripts use temporary files
• Recommend putting filenames in a variable
• Include the PROGRAM name in the filename
• Include some clue to its use in the filename
• Use “$$” to include the scripts’ Process Id in the file name
TEMP_SQL_OUT=“$TEMP_DIR/$PROGNAME.sql_outfile.$$”
db2 –tvf $MySqlFile >$TEMP_SQL_OUT
RC=$?
Run_SQL_Post_Processing_Function $TEMP_SQL_OUT
rm $TEMP_SQL_OUT
5
![Page 6: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/6.jpg)
Common Shell Variables :
• $0 - the program name, fully qualified
• $* - all the arguments passed on the invoking scripts
• $# - the number of arguments provided on the command line
• $n - “n” is an integer – each successive arg on command line
• $$ - the variable containing the Process Id of the script
6
![Page 7: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/7.jpg)
Common Shell Variables :
• When referencing variables, one can • Test if variable is null and assign a default value. Use the :- construct
DBNAME=${1:-”Sample”}
• Test if the variable is null and print an error message and exitDBNAME=${1?”$usage”}
• Invoking a script with a dot “.” in front of it, runs the script in the caller’s shell. Note that all variables and changes made in the script remain in effect when it returns control. ./.profile
7
![Page 8: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/8.jpg)
Building a Shell Script Funtion:
• Functions have two declaration syntaxes:• function <funcname> {
stmts}
• <funcname> ( parameter_vars) {stmts}
• Both allow parameters passed in• Parameters are provided the same way a script is provided
parms• Variables are global – available to calling script• Local variables can be declared
• <xxx>
• Inside function, $ variables are local
8
![Page 9: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/9.jpg)
A Usage Function:
• Functions need to be defined prior to invoking them
• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }
if [ $# != 2 ]
then Usage
fi
9
![Page 10: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/10.jpg)
Typical Template Variables:
• Variables in many Template Scripts
# Get start time in db2 format
START_TIME=$( date '+%Y-%-m-%d-%:H:%M:%S )
# Scripts, Temp and Output directories: SCRIPT_DIR=/dbamaint/scripts
TEMP_DIR=/TMP/script_temp
OUTPUTS_DIR=$HOME/work/outputs
10
![Page 11: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/11.jpg)
A Usage Function:
• Functions need to be defined prior to invoking them
• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }
if [ $# != 2 ]
then Usage
fi
11
![Page 12: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/12.jpg)
Processing Command Line Options:
• GETOPTS is a function that will search the command line for a character preceded by a minus or plus and allow you to process the next argument following that character:
while getopts i: c
do
case $c in
i) instnme=$OPTARG;;
\?) Usage
exit;;
esac
done
shift `expr $OPTIND - 1`
12
![Page 13: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/13.jpg)
DB2 CLP Command
• The db2 command, a.k.a, Command Line Processor is powerful
• Commonly used options
• adfsf
13
Option Descriptiont Terminators in SQL/Statements so they can be multi-linedf Filename follows it - reads SQL/Statements from the file
m Returns the number of rows modified by the last SQL stmta Returns the SQLCA areas Stops the execution if an Error or Severe Error occursp Toggles prompting for input +p turns off promptc Toggles Autocommit. -c Auto commits after each Stmt. +c does notv Verbose - echos statements to std out as they are executed
![Page 14: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/14.jpg)
The DB2 SQLCA
• When –a is provided on the db2 command the SQLCA is displayed after every statement executed
• Fields in the SQLCA include
14
Option DescriptionSQLCAIDSQLCABCSQLCODE
SQLERRMLSQLERRMC
SQLERRP
SQLERRD1SQLERRD2SQLERRD3SQLERRD4SQLERRD5SQLERRD6SQLWARNSQLSTATE
![Page 15: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/15.jpg)
Post Processing DB2 SQLCA example
15
![Page 16: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/16.jpg)
Function to run a DB2 SQL Statement
• Run_db2 is a function which will run a db2 command and set standard variables for subsequent interrogation
16
![Page 17: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/17.jpg)
AWK Programs and common uses
• AWK is a programming language well suited for parsing, math and command processing
• Common uses include piping output into an Awk program to:• Select the Nth token from the command line:
grep “$( hostname –s ) db2nodes.cfg | awk ‘{ print $1}’
• Find the line number of a string in a fileawk ‘/James Sobieski/ {print NR }’ my_contacts
• db2 get dbm cfg | grep ‘DIAGPATH’ | awk -F"=" '{print $2}'
17
![Page 18: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/18.jpg)
Common Simple GREP usage
• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file
• Common grep options
18
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
![Page 19: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/19.jpg)
Common SORT command usage
• SORT is powerful tool to post-process output from a command or a file and deduplicate output
• Common sort options
19
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
![Page 20: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/20.jpg)
Common HEAD unix command usage
• Head is powerful tool to pull lines of data from a file or stdoutstarting at the beginning of the file
• Common head options
20
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
![Page 21: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/21.jpg)
Common TAIL unix command usage
• Tail is powerful tool to pull lines of data from a file or stdoutreading backwards through the file
• Common tail options
21
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
![Page 22: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/22.jpg)
Common XARGS unix command usage
• XARGS is powerful tool which will execute a command substituting variables in positions of the command
• XARGS options
22
Option Description
![Page 23: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/23.jpg)
Common Simple GREP usage
• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file
• Common grep options
23
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
![Page 24: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/24.jpg)
Example Snippets – Archiving the DB2 Diag log
• Archiving the db2 diag.log
24
![Page 25: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/25.jpg)
Example Snippets – Post Processing db2pd -hadr
25
![Page 26: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/26.jpg)
Example Snippets – Examining an Application Snapshot
• Snapu script:
26
![Page 27: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/27.jpg)
Example Snippets – making list applications readable
• listu
27
![Page 28: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/28.jpg)
Example Snippets – Displaying and Summarizing Tablespace status
• Show_ts.ksh
28
![Page 29: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/29.jpg)
Example Snippets – Combining System Catalog Tables
• Show_tab_ix
29
![Page 30: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/30.jpg)
Example Snippets – Counting SQL messages in a file
• show_all_sql_msgs.ksh
30
![Page 31: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/31.jpg)
Example Snippets – using AWK to summarize vmstat
• vmstatt.ksh
• iostatt.ksh
31
![Page 32: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/32.jpg)
Example Snippets – Capture Single SQL Execution Stats
• Stmt_execute.ksh
32
![Page 33: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/33.jpg)
Example Snippets – Run an explain on SQL statement
• Jexpln.ksh
33
![Page 34: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/34.jpg)
Example Snippets – Capture runaway Applheapsz
• Capture_applheapsz_too_large.ksh
34
![Page 35: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/35.jpg)
Example Snippets – Add Partition to RP table
• Add_next_partition.ksh
35
![Page 36: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/36.jpg)
Example Snippets – Convert Snapshot cmd output to script
• Gen_all_ss_variables.ksh
36
![Page 37: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/37.jpg)
Example Snippets – Generate Views for Catalog Tables
• make_one_shortcat.ksh
37
![Page 38: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/38.jpg)
Example Snippets – Generate Views for Catalog Tables
• make_all_shortcats.ksh
38
![Page 39: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/39.jpg)
Example Snippets – Report on backups
• Report_all_backups_taken.ksh
39
![Page 40: Tips and Tricks in Unix Shell scripting DB2 Database ...](https://reader031.fdocuments.in/reader031/viewer/2022012101/6169eb1f11a7b741a34ccff9/html5/thumbnails/40.jpg)
Insert Name HereInsert Company Name HereInsert E-mail here
SessionTitle
Please fill out your session
evaluation before leaving!
Photo by Steve from Austin, TX, USA