DB-3: Tips for Database Administrators Gus Björklund Wizard, Progress Software Corporation.

DB-3: Tips for Database Administrators

Gus BjörklundWizard, Progress Software Corporation

© 2007 Progress Software Corporation2 DB-3: Tips for Database Administrators

Ask questions as we goif I am not clear about something.

Warning: there is a mistake in these slides.


Topics

Tip Nr. 0x00 … Tip Nr. 0x16


Tip Number 0x00

If it is not broken, don’t fix it !


Tip Number 0x01

If you do fix something, test your fix !


Tip Number 0x02

Read the man pages


Read the man pages

man man man [ -s section ] name Read the man pages!


Tip Number 0x03

SSH is your friend!

Like Linux, more great stuff fromFinland


SSH

SSH replaces telnet, rlogin, rcp, rsh, etc. All connections are encrypted Use for remote access over Internet


SSH

Don’t need username, password login• Can also use certificates for authentication

Can send output of• local commands to remote host

• remote commands to local host

Can do secure file transfer• scp and sftp

Can tunnel other protocols (e.g. X windows)


Simple remote command execution

gus@dirac:gus $ ssh bespin lsgus@bespin's password: 101arkt90ajdk.txtGSIMailNewsa.outa7adb.mangus@dirac:gus $

Nothing much to see here


Remote command output to local host

gus@dirac:gus $ ssh bespin "ls; echo DONE” >ls.datgus@bespin's password: gus@dirac:gus $ cat ls.dat101arkt90ajdk.txtGSIMailNewsa.outa7adb.manDONEgus@dirac:gus $


Local output piped to remote host

gus@dirac:gus $ cd 4gl; tar -cf - *.p | ssh bespin "cd foo; tar -xf -"gus@bespin's password: gus@dirac:4gl $


Remote backup piped to local file

gus@dirac:4gl $ ssh bespin "cd pdr; tar -cvf - ." >backup.tar gus@bespin's password: a ./ 0Ka ./Makefile 1Ka ./pdrcache.c 131Ka ./pdrproc.c 23Ka ./pdrtrig.c 175Ka ./dsmcon.c 94Ka ./dsmrec.c 17Ka ./dsmseq.c 15Kgus@dirac:4gl $


Turning off sshd password authentication

in the file: /etc/ssh/sshd_config,

do this:

# Protocol 1,2Protocol 2 . . .# Authentication:PasswordAuthentication no

then save changes.

restart daemon:

# service sshd restart


Enabling SSH certificate authentication

gus@dirac:gus $ ssh-keygenGenerating public/private rsa key pair.Enter file in which to save the key (/Users/gus/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/gus/.ssh/id_rsa.Your public key has been saved in /Users/gus/.ssh/id_rsa.pub.The key fingerprint is:3f:a9:2c:a6:74:5e:36:59:39:7f:ed:3f:5d:e3:a6:ee [email protected]@dirac:gus $ gus@dirac:gus $ ssh-copy-id -i id_rsa gus@bespingus@dirac:gus $


Enabling SSH certificate authentication 2

gus@dirac:gus $ cat /Users/gus/.ssh/id_rsa.pub.ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEApwswruyeETRbQ/T7ZHdNxtXIC5J41XpnJwWND4HB3WTuXPLx/Qf2S83Y3VPSgGoc+9YqmLOc/hfs/gQry+TQC9jGIPVYtW/E5PPvhh/HxTIKVc0eYMmpKlxFCjfcv0KyDcAwnd7r/wczPqw2TpurzZcXfA0c2upqufJZbnweXZtDrcs7bUwdQXskrqDj1EDBxACuH1/omZa2M/PdfFzyrRtbRMeyrfIWAfoL5RNOFsPYYOxWJxfp11tuSF7OiBUQtcbXAfS8cdHsQkJNNzSZqM6B69LvK28mlHZ8m2b5+cXICA27aUyDZfsSlMLR6rl+Ao3UByL8G+ZQBLUmhMsuRQ== [email protected]

gus@dirac:gus $


More about SSH

www.openssh.com

http://www.ietf.org/rfc/rfc4251.txt


Tip Number 0x04

Windows can do SSH too


SSH for Windows

Client: use Putty

http://www.chiark.greenend.org.uk/~sgtatham/putty/

Server: get Cygwin

http://www.cygwin.com


Tip Number 0x05

After-image journalling is your friend!


After Imaging

Do you use after-image journalling?• YOU SHOULD

AI records all database changes Intended for

• disaster recovery

• hot standby on another machine


AI Basics

Add AI extents to database Back up the database Enable after-imaging Archive ai extents as they become full

• 10.1 has an automatic archiver

OpenEdge DBA guide has details


AI basics 2

A failure occurs and you lose your database …..

Fix the failure• replace disk, memory, machine, etc.

Restore the backup copy of your database Roll forward all ai extents you archived

• You do know where you put them, don’t you?• I hope they weren’t on the failed disk

When roll forward done, start database Now you are back to where you were


Tip Number 0x06

Specify scratch space for index rebuild


Index rebuild scratch space

Command line looks like this:

proutil dbname -C idxbuild table customer \ -SS dbname.srt -TB 31 -TM 32 -B 1000

Scratch space spec file (dbname.srt) looks like this:

300 /user2/db1/first/400 /user3/junk/0 /user4/last/

note multiple drives in this example


Tip Number 0x07

Tools you should know about


Tools

uptime find top lsof ifconfig vmstat iostat netstat bzip2


Tools: uptime

$ uptime 11:26am up 17 day(s), 9:16, 163 users, load average: 3.09, 3.36, 3.76$


Tools: top

gus@dirac:gus $ top -l 1Processes: 65 total, 3 running, 62 sleeping... 183 threads 10:51:24Load Avg: 0.04, 0.06, 0.07 CPU usage: 11.1% user, 44.4% sys, 44.4% idleSharedLibs: num = 170, resident = 37.0M code, 5.30M data, 6.44M LinkEditMemRegions: num = 6460, resident = 166M + 16.3M private, 164M sharedPhysMem: 161M wired, 758M active, 879M inactive, 1.76G used, 248M freeVM: 9.47G + 102M 230499(0) pageins, 16746(0) pageouts

PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE 606 top 0.0% 0:00.05 1 17 19 276K 684K 748K 26.9M 586 bash 0.0% 0:00.02 1 14 16 224K 1.05M 844K 27.1M 585 login 0.0% 0:00.01 1 16 40 172K 768K 636K 26.9M 583 Terminal 0.0% 0:01.56 5 130 197 2.64M 11.9M 8.93M 364M 563 TextEdit 0.0% 0:00.23 1 67 149 2.05M 8.94M 5.80M 355M 558 TeXShop 0.0% 0:03.18 2 109 238 4.49M 20.2M 25.6M 365M 467 firefox-bi 0.0% 1:31.55 6 96 389 48.1M 29.4M 67.4M 428M


Tools:find

find files modified more than 30 days ago• find . -mtime +30 -print

delete files modified more than 30 days ago• find . -mtime +30 -exec rm {} \;

find files bigger than 16k• find . -size +16384c -print• find . -size +32 -print

delete files bigger than 16k• find . -size +16384c -print | xargs -i rm \;

Read the man page!


Tools:lsof

_progres 26240 bleicher /usr/lib/librt.so.1_progres 26240 bleicher /usr/lib/libsocket.so.1_progres 26240 bleicher /usr/lib/libintl.so.1_progres 26240 bleicher /usr/lib/ld.so.1_progres 26240 bleicher /usr/lib/libdl.so.1

. . . _progres 26240 bleicher /tools/solaris/dlc100b/promsgs_progres 26240 bleicher /tools/solaris/dlc100b/bin/_progres

ftp://vic.cc.purdue.edu/pub/


Tools:ifconfig

1-bespin-gus-> /usr/sbin/ifconfig -alo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1inet 127.0.0.1 netmask ff000000 ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2inet 172.16.7.168 netmask ffff0000 broadcast 172.16.255.255


Tools:ifconfig

gus@dirac:gus $ ifconfig en0en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 tunnel inet --> inet6 fe80::217:f2ff:fecc:b686%en0 prefixlen 64 scopeid 0x4 inet 172.16.114.99 netmask 0xffff0000 broadcast 172.16.255.255 ether 00:17:f2:cc:b6:86 media: autoselect (1000baseT <full-duplex,flow-control>) status: active supported media: autoselect 10baseT/UTP <half-duplex> 10baseT/UTP<full-duplex>10baseT/UTP <full-duplex,hw-loopback> 10baseT/UTP <full-duplexflow-control> 100baseT<half-duplex> 100baseTX <full-duplex> 100baseTX<full-duplex,hw-loopback> 100baseTX<full-duplex,flow-control> 1000baseT <full-duplex>1000baseT <full-duplex,hw-loopback>1000baseT <full-duplex,flow-control> none vlan: 0 parent interface: <none> bond interfaces: <none>


Tools:netstat

1-bespin-gus-> netstat

TCP: IPv4 Local Address Remote Address Swind Send-Q Rwind Recv-Q State-------------------- -------------------- ----- ------ ----- ------ -------bespin.1023 rdlserv.nfsd 26280 0 49640 116 ESTABLISHEDbespin.32880 bespin.6015 49152 0 49152 0 ESTABLISHEDbespin.32879 bespin.6014 49152 0 49152 0 ESTABLISHEDbespin.6014 bespin.32879 49152 0 49152 0 ESTABLISHEDbespin.6015 bespin.32880 49152 0 49152 0 CLOSE_WAIT

. . .


Tools: netstat -s

gus@dirac:4gl $ netstat -stcp: 794419 packets sent 37468 data packets (22991001 bytes) 51 data packets (11242 bytes) retransmitted 0 resends initiated by MTU discovery 430311 ack-only packets (196294 delayed) 0 URG only packets 0 window probe packets 322074 window update packets 4516 control packets 1308690 packets received 33675 acks (for 22967429 bytes) 2270 duplicate acks 0 acks for unsent data 1248341 packets (1521010141 bytes) received in-sequence 385 completely duplicate packets (343882 bytes) 0 old duplicate packets 41 packets with some dup. data (28512 bytes duped) 33202 out-of-order packets (38094369 bytes) 1 packet (376 bytes) of data after window


vmstat

vmstat 3 50procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 7 3656 226920 16528 5294120 0 0 360 636 281 823 4 4 49 42 0 6 3656 205504 16584 5302288 0 0 2688 903 1066 4675 7 6 11 75 2 0 3656 186088 16584 5310512 0 0 2656 684 1010 4687 6 6 8 8022 28 3656 165196 16600 5319748 0 0 2887 871 1096 4623 8 6 8 78 0 94 3656 138524 16648 5331008 0 0 3909 815 1292 3455 8 6 6 79

. . .


iostat

gus@dirac:gus $ iostat 5 5 disk0 cpu KB/t tps MB/s us sy id 18.07 4 0.07 7 3 90 0.00 0 0.00 2 1 97 0.00 0 0.00 4 1 95 0.00 0 0.00 2 2 96 0.00 0 0.00 3 1 95


Tools:bzip2

$ ls -l tips_for_dbas_00.ppt -rw-r--r-- 1 gus staff 644096 Apr 23 09:18 tips_for_dbas_00.ppt

$ bzip2 tips_for_dbas_00.ppt

$ ls -l tips_for_dbas_00.ppt.bz2-rw-r--r-- 1 gus staff 246356 Apr 23 09:18 tips_for_dbas_00.ppt.bz2


Tools: Windows tabbed console


Tools: windows tabbed console

http://sourceforge.net/projects/console/


Tools: More Windoze Stuff

http://www.microsoft.com/technet/sysinternals

Some useful stuff you can get there:

Handle: Who has what files openProcess Monitor: Lots of information about processes

and disk activityAutorun: what programs get started automaticallyRegmon: monitors registry activityTcpview: shows all active TCP and UDP endpointsPsTools: list processes, system info, who is logged onZoomit: zoom in on parts of screen

Lots more


Tools: Still More Windoze Stuff

http://www.baremetalsoft.com/baretail


Tools: Solaris

pldd List the dynamic libraries linked into each process

pfiles Report fstat(2) and fcntl(2) information for all open filesin each process.

pmap print information about the address space of a process

plimit get or set the resource limits of running processes

pargs print process arguments, environment variables

preap force a defunct process to be reaped by its parent

pkginfo list what packages are installed


Tools: AIX svmon

svmon -G -I size inuse free pin virtualmemory 1048576 425275 623301 66521 159191pg space 262144 31995

work pers clntpin 46041 0 0in use 129600 275195 0

PageSize PoolSize inuse pgsp pin virtuals 4 KB - 404795 31995 46041 159191L 16 MB 5 0 0 5 0

List top 15 memory consumers

svmon -Pt15 | perl -e ’while(<>){print if($.==2||$&&&!$s++);$.=0 if(/^-+$/)}’


Tools: AIX vmstat

vmstat -v 1048576 memory pages 1002054 lruable pages 478136 free pages 1 memory pools 95342 pinned pages 80.1 maxpin percentage 20.0 minperm percentage 80.0 maxperm percentage 36.1 numperm percentage 362570 file pages . . .

Use vmo command to adjustminperm, maxperm, numperm as needed


Tip Number 0x08

Windows scripting


Windows scripting

Cygwin• provides lots of Unix tools and a shell for

Windows. Also SSH server• Download from: www.cygwin.com

Windows Services for UNIX Version 3.5• From Micro$oft, but free• Lots of goodies: Korn, C, bash shells, 350 UNIX

commands, NFS, NFS gateway, telent, telnetd• Download from http://technet.microsoft.com/en-

us/interopmigration/bb380242.aspx

Windows .bat scripts are brain damaged


Tip Number 0x09

how to turn off Windows indexing service


Stop Windows indexing service

Go to the Start menu, pick run.• Type services.msc, click OK.• Services dialog will show.

Right-click on indexing service to show properties dialog,

if service running, click stop. select disabled. click apply close.


Tip Number 0x0a

creating Windows services


User SRVANY to run Windows Services

Lets you create “user defined” Windows services

Instructions and download fromhttp://www.tacktech.com/display.cfm?ttid=197

Warning:• This is not for n00bs.

• Study before using

• Read http://support.microsoft.com/kb/137890


Tip Number 0x0b

How to telnet to Windows


Enable Windows telnet login by name/passwd

NTLM authentication by default To fix:

• run TLNTADMN

• take menu pick no 3 (display/change …)

• pick 7 (NTLM)

• set value to 1

• take menu pick no 4 (restart service)

BUT: don’t use telnet


Tip Number 0x0c

scripting Progress backup


Scripting backup

bkupdev="${DB_bkupdir}/${DB_name}.bkup"rm -f ${bkupdev}* 2>&1#echo "Backing up database $DB_dir/$DB_name to $bkupdev"#$DLC/bin/probkup online $DB_dir/$DB_name ${bkupdev}01 \-vs 250000 -bf 40 <<EOF${bkupdev}02${bkupdev}03 . . .${bkupdev}12${bkupdev}_overflowEOF#

source: PEG, from Jared Middleton


Tip Number 0x0d

scripting promon


Scripting promon

$DLC/bin/promon $DBNAME > promon.log \ <<- "EOF" 2> /dev/nullR&D519999p32xEOF

Set page size, capture list of IO operations by process


Tip Number 0x0e

scripting after imaging


Scripting ai archiving

EXTENT_NAME=`_rfutil $DB_DIR/$DB -C aimage extent full`#EXTENT_DATE=`date +%Y%m%d-%H%M%S`# like this: 20070516-140856

tar -czvf /anotherdisk/aiarchives/$EXTENT_DATE $EXTENT_NAME

rfutil $DB_DIR/$DB -C aimage extent empty $EXTENT_NAME


Tip Number 0x0f

How much space is being usedby storage areas?


How much space is being used?

for each _AreaStatus where ( not _AreaStatus-Areaname matches "*After Image Area*" ) no-lock:

display _AreaStatus-Areanum format ">>>" column-label "Num" _AreaStatus-Areaname format "x(20)" column-label "Area Name" _AreaStatus-Totblocks column-label "Tot blocks" _AreaStatus-Hiwater column-label "High water mark" _AreaStatus-Hiwater / _AreaStatus-Totblocks * 100 column-label "% use" _AreaStatus-Extents format ">>>" column-label "Num Extents" .end.

from PEG: Dmitri Levin


Tip Number 0x10

List tables by storage area



for each _Area, each _Storageobject where (_Storageobject._Area-number = _Area._Area-number), each _File where (_File._File-Number = _Storageobject._Object-number) and (_File._File-Number > 0) break by _File._File-name:

display _Area._Area-name _File._File-name.end.


Listing of tables by storage area

Area-name File-Name

Schema Area agedarSchema Area agedarSchema Area customerSchema Area customerSchema Area itemSchema Area itemSchema Area monthlySchema Area monthly



for each _Area, each _Storageobject where (_Storageobject._Area-number = _Area._Area-number), each _File where (_File._File-Number = _Storageobject._Object-number) and (_File._File-Number > 0) break by _File._File-name:



List tables by storage area 2

for each _Area, each _Storageobject where (_Storageobject._Area-number = _Area._Area-number), each _File where (_File._File-Number = _Storageobject._Object-number) and (_File._File-Number > 0) and (_StorageObject._Object-type eq 1) break by _File._File-name:



Tip Number 0x11

List indices by storage area and table


List indexes by storage area and table

for each _Area, each _Storageobject where (_Storageobject._Area-number = _Area._Area-number), each _Index where (_Index._Idx-num = _Storageobject._Object-number) and (_StorageObject._Object-type eq 2) : find _File of _Index. if (_File._File-number > 0) then display _Area._Area-name _File._File-name _Index._Index-name.end.


Tip Number 0x12

Learn to use vi


vi cheat sheet


Tip Number 0x13

Tom Bascom’s protop


protop

Tom Bascom’s Open Source Progress 4GL VST based performance monitoring program

Displays tons of useful information Study source code to learn how it is done Download from

http://www.greenfieldtech.com/articles/protop.shtml


Tip Number 0x14

Adam Backman’s DBA scripts


Adam Backman’s dba scripts

Available on PEG: http://www.peg.com/utilities.html Been aroud awhile, but very good Stuff they do:

• backup

• check ai status

• check db status

• look for blocked clients

• more …

Study code to learn how it is done


Tip Number 0x15

Where to get more tips


Where to learn more

OpenEdge Database Administration Guide PSDN: www.psdn.progress.com

Classes (not the only ones)• Bravepoint: bravepoint.com

• White Star Software: wss.com

• Progress DBA classeshttp://www.progress.com/openedge/services/education/index.ssp

PSC Tech Support Knowledge Center


Tip Number 0x16

Join the PEG

www.peg.com

Ask questions there


Questions

DB-3: Tips for Database Administrators Gus Björklund Wizard, Progress Software Corporation.

Documents

Transcript of DB-3: Tips for Database Administrators Gus Björklund Wizard, Progress Software Corporation.