DB2 „Deep dive into DB2“ - eXstor · · PDF fileIs reorg needed? db2...
Transcript of DB2 „Deep dive into DB2“ - eXstor · · PDF fileIs reorg needed? db2...
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
„Deep dive into DB2“
Bruno Friess
DB2
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Agenda:
TSM/DB2 installationDownload, installation, and then: where is it?DB2 instance creation
DB2 process overviewMultithreaded engineProcess and threads
Settings for dbm and dbTuning for performance, scalabilityMemory and virtual memory manager
Database reorganizationDB2 logs
db2diaglogFODC, trace...
HADR – high availability disaster recovery
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Download and installation
Where to download?http://ftp.software.ibm.com/storage/tivolistoragemanagement/ is still there.
● You can use IBM FixCentral as well.● Never use doublezeros (6.3.0.0 etc.)● There are maintenance and patch levels● Packages including the license files are available for paying customers
It's one file. Extract the file and start with installation over CLI or Java.
Installation logfiles are located in /var/tivoli/tsmDefault DB2 Install Path is /opt/IBM/db2/V9.7 but TSM uses /opt/tivoli/tsm
If the installation is not successful:Use the uninstaller located in /opt/tivoli/tsm/_uninstIf DE hangs: delete /var/ibm/common/acsi and /usr/ibm/common/acsi
TSM installation is using db2_install, which will install all needed packages, but will not perform user and group creation, instance creation, or configuration.TSM uninstallation is using db2_deinstall.
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2? Where is it?
After a successful installation, there is no rpm or lpp inventory.rpm -qa or lslpp -l doesn't show anything about DB2.
Useful: /usr/local/bin/db2ls
With version 6.2:> db2lsInstall Path Level Fix Pack Install Date Installer UID ---------------------------------------------------------------------------------/opt/tivoli/tsm/db2 9.7.0.2 2 Sun Feb 27 17:58 2011 CET 0
With version 6.3:> db2ls
Install Path Level Fix Pack Install Date Installer UID ---------------------------------------------------------------------------------/opt/tivoli/tsm/db2 9.7.0.5 5 Wed Mar 20 22:16 2013 CDT 0
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 instance creation
A DB2 instance is an environment in which you store data and run applications(= a database manager, a virtual server environment or „container“). Run the db2icrt command.
DB2DIR/instance/db2icrt a AuthType u FencedID InstName
Using the user tsm01 and instance tsm01:> db2icrt -a SERVER -u tsm01 tsm01
Three remaining commands...> db2 update dbm cfg using diagpath <any_path>> db2 update dbm cfg using dftdbpath <home_of_user>> db2set -i tsm01 DB2CODEPAGE=819
...end of TSM installation guide.(Other instance commands: db2idrop, db2ilist...)
→ Now? What is going on?
fenced user ID instance name
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 process overview
FIREWALL
DB2 Data Serverdb2sysc
Application Level Database Level per Request
Instance Level db2start
dbsysc
db2ipccm
RemoteListeners
dbaiothr
RemoteClientAgent
LocalClient
Application IdleAgentPool
db2agent
db2agent
db2agntpdb2agntp…db2agntp
db2loggwdb2pfchrdb2pclnrdb2loggrdb2dlock
db2lmrdb2med
db2wdog
db2vend
db2fmp
db2acd
tiny hole for the NSA
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 common threads
db2syscSystem controller thread, which is responsible for startup, shutdownand management of the running instance
db2agentCoordinator agent that performs database operations on behalf of applications
db2agntpActive subagent spawned if INTRA_PARALLEL is set to YES
db2pfchrAsynchronous bufferpool prefetcher (NUM_IOSERVERS)
db2pclnrAsynchronous bufferpool cleaner/writer (NUM_IOCLEANERS)
db2loggrManipulates log files to handle transaction processing and recovery
db2loggwWrites log records to the log files
db2dlockThread responsible for deadlock detection
db2panic„panic agent“ writes dump/log files during a crash (FODC) or when a limit is reached
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 common processdb2sysc
Main DB2 system controller or enginedb2acd
Autonomic computing daemon. Performs client side autonomic tasks, such as healthMonitor, maintenance (reorg) and scheduling
db2wdogDB2 watchdog daemon cleans up resources if db2sysc abnormally terminates
db2vendFenced vendor process runs outside of the engine
db2fmpFenced processes that run user code outside the firewall for stored procedures and user defined functions
> ps -ef | grep db2root 10193 1 0 Aug06 ? 00:07:59 /opt/tivoli/tsm/db2/bin/db2fmcd root 12360 1 0 Aug06 ? 00:00:00 db2wdog 0 rlin01 12373 12360 0 Aug06 ? 02:27:38 db2sysc 0 rlin01 12389 12360 0 Aug06 ? 00:10:22 db2acd 8a6678,14,1e014,2,0,1,11fc0,388009,2,6800crlin01 13287 12360 0 Aug06 ? 00:11:20 db2fmp (8a6678,14,1e014,2,0,1,31fc0,388009,2,77802frlin01 24860 12360 0 Aug06 ? 00:00:08 db2fmp (8a6678,14,1e014,2,0,1,91fc0,388009,2,ab68039
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12373 rlin01 18 0 2164m 300m 258m S 0.3 7.9 147:40.70 db2sysc 17448 root 15 0 12764 1148 824 R 0.3 0.0 0:00.20 top
Linux top
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 settings for dbm and db
Settings for the database manager> db2 get dbm cfg
Settings for the database (you have to connect to TSMDB1)> connect to TSMDB1 > db2 get db cfg> disconnect TSMDB1
IBM: „No change needed“
Before changing, please consider the TSM 6.4 performance guide.Read the DB2 manuals, especially the overview of the performance impact.Some of them needs a stop/start.
How to change/update:> db2 update dbm cfg using <parameter> <value>> db2 update db cfg using <parameter> <value>
To reset all (!!) values to their default:> db2 reset dbm cfg> db2 reset db cfg
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 settings for dbm and db
Potentially parameters of dbm to change:
name online impact explanation
dftdbpath yes none Path to the database meta-data
diaglevel yes low Sets the diag level, default is 3, range is 0-4
diagpath yes none Path to the DB2 diag files
intra_parallel no high Enables/Disables Intrapartition parallelism IBM Lab recommends „NO“, default is -1
max_querydegree yes high Max number of parallel threads in one SQL statementIBM Lab recommends 1, default is ANY (or -1)
notifylevel yes medium Admin notification level (file instance.nfy), default is 3, 1 is enough
Potentially parameters of db to change:
name online impact explanation
auto_maint yes medium Maintenance values (Reorg, runstats, auto db backup..)
dft_degree yes high Switch off intrapartition parallelism, default is 1 IBM Lab recommends „1“
longindexbuild yes none Needed for HADR with ON, default is OFF
and: newlogpath, mirrorlogpath, overflowlogpath, failarchpath...
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 memory
STMM – Self tuning memory manager
Default setting for TSM is to configure DB2 memory usage to AUTOMATIC.
DBMemPercent Auto
Within „get db cfg“ you will see:Self tuning memory (SELF_TUNING_MEM) = ONSize of database shared memory (4KB) (DATABASE_MEMORY) = AUTOMATIC(135180)Database memory threshold (DB_MEM_THRESH) = 10Max storage for lock list (4KB) (LOCKLIST) = AUTOMATIC(4120)Percent. of lock lists per application (MAXLOCKS) = AUTOMATIC(97)Package cache size (4KB) (PCKCACHESZ) = AUTOMATIC(29586)Sort heap thres for shared sorts (4KB) (SHEAPTHRES_SHR) = AUTOMATIC(268)Sort list heap (4KB) (SORTHEAP) = AUTOMATIC(53)
In the db2 diaglog you will see: CHANGE : CFG DBM: "Instance_memory" From: "20" <automatic> To: "780850" <automatic>
Log information is written into: [DIAGPATH]/stmmlog
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 memory
What does this mean?
If you have one TSM server on one machine, then automatic is ok.If not, then it is difficult: TSM allows only a range from 10 to 99% of the memory.
What if you have one machine with 32 GB RAM and three instances?What happens when you have 512 GB RAM and one instance? Or 20 instances??
TSM is using:> db2pd -dbptnmemDatabase Partition 0 -- Active -- Up 1 days 17:07:21 -- Date 09/13/2013 08:25:21Database Partition Memory Controller StatisticsController Automatic: YMemory Limit: 119984660 KBCurrent usage: 66347392 KBHWM usage: 74097600 KBCached memory: 6702912 KB
Individual Memory Consumers:
Name Mem Used (KB) HWM Used (KB) Cached (KB)========================================================APPL-TSMDB1 58048 160000 10624DBMS-tsmp01 126464 126464 48448FMP_RESOURCES 22528 22528 0PRIVATE 157376 409280 32320DB-TSMDB1 65967104 73406400 6611520
Total memory
TSM + DB2
db2sysc
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Database reorganization (see IBM technote 1452146 or the presentation of N.Pott)Is reorg needed? db2 reorgchk current statistics on table all >db2reorgchk.out
Settings of the database:Automatic maintenance (AUTO_MAINT) = ON Automatic database backup (AUTO_DB_BACKUP) = OFF Automatic table maintenance (AUTO_TBL_MAINT) = ON Automatic runstats (AUTO_RUNSTATS) = ON Automatic statement statistics (AUTO_STMT_STATS) = ON Automatic statistics profiling (AUTO_STATS_PROF) = OFF Automatic profile updates (AUTO_PROF_UPD) = OFF Automatic reorganization (AUTO_REORG) = OFF
Is Reorg on?
TSM options:(6.1.5/6.2.3 or higher)
all are onlinechangeable(setopt)
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Database reorganization ….now it's reorganized, but I still have no more space in the filesystem.To get the free space which is at the end of the tables now, try this:
You can reduce the space of the database in the filsystem by „reducing“ it.This is for DB2 9.7 and only if this feature is switched on.
> db2 connect to tsmdb1> db2 set schema tsmdb1> db2 "select reclaimable_space_enabled from \table(mon_get_tablespace('',-1)) where tbsp_id in (2,4,5,6)"
If all tablespaces show, that reclaimable_space_enabled=1 then you can start with reducing:
> db2 ALTER TABLESPACE USERSPACE1 REDUCE MAX> db2 ALTER TABLESPACE IDXSPACE1 REDUCE MAX> db2 ALTER TABLESPACE LARGESPACE1 REDUCE MAX> db2 ALTER TABLESPACE LARGEIDXSPACE1 REDUCE MAX
Remarks:● All those ALTER process are slow, please start one after another● Can be done during TSM operating (online)● But consider I/O on your disks / filesystems
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 log files
Set the diaglog size and location:> db2 update dbm cfg using diagpath <path_to_diaglog>> db2 update dbm cfg using diagsize 1024
If you have more space, then >1024 is ok.
Inside this log, there are a lot of messages and most of them are difficult to read.IBM needs this file for error analysis.
Content of the [DIAGPATH]-rw-rw-rw- 1 rlin02 tsmsrv 28765146 Sep 13 14:52 db2diag.0.log-rw-rw-rw- 1 rlin02 tsmsrv 58231 Oct 15 2011 db2diag.log-rw-r----- 1 rlin02 tsmsrv 6291316 Aug 6 15:29 db2eventlog.000drwxrwxrwx 2 rlin02 tsmsrv 8192 Sep 12 20:55 eventsdrwxr-x--- 3 rlin02 tsmsrv 8192 Aug 6 10:31 FODC_Panic_2013-08-06-10.30.49.236060-rw-rw-rw- 1 rlin02 tsmsrv 67042 Sep 13 14:41 rlin02.0.nfy-rw-rw-rw- 1 rlin02 tsmsrv 763 Oct 15 2011 rlin02.nfydrwxrwxr-t 2 rlin02 tsmsrv 8192 Sep 12 08:27 stmmlog
DB2 memory logs
Admin notification
Automatic FODC
FODC – First occurence data collection(a kind of dump files collection)Old FODC directories can be deleted / archived
0 to 9
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
DB2 log files
Strange files in [DIAGPATH] or in sqllib/db2dump:Hundreds or thousands of *.dump.bin, *.stack.txt, *.apm.bin and *.trap.txt.For more information see IC72708: „Dump files fills up filesystem“TSM will report „ANR1545W Add more space to the file system.“Check this place and delete the files (as a workaround)Old files can be deleted while TSM / DB2 is runningFiles are needed by manual FODC command
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
HADRHigh availability disaster recovery.
It replicates data in real time from a primary database to one or more standby databases.Multiple standbys are introduced with DB2 10.1 (see TSM 7.1).
To use HADR with TSM no additional licensing is necessary!!!
Easy solutionIs integrated into DB2Only needs TCP (no additional software / hardware)Setup only requires a few configuration parameters
(primary/standby network address, and hadr_syncmode)Only 3 commands to manage: start/stop/takeover HADRChoose the best sync mode for your business: sync, nearSync, async, superAsyncAutomatic reconfiguration after failover or role switch in multiple standbys
But...Multiple standbysDelayed replay
starts with DB2 version 10
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
HADRQuick installation...
1. Priming the database● Backup of the primary database to disk ● Drop database on second node● Restore database on second node
2. Use the same ports for this instance in /etc/services on both machines
3. Configure HADRdb2 update db cfg for tsmdb1 using hadr_local_host exaix01.exstor.dedb2 update db cfg for tsmdb1 using hadr_local_svc 60010db2 update db cfg for tsmdb1 using hadr_remote_host exaix02.exstor.dedb2 update db cfg for tsmdb1 using hadr_remote_inst tsm01db2 update db cfg for tsmdb1 using hadr_remote_svc 60010db2 update db cfg for tsmdb1 using hadr_syncmode SYNCdb2 update db cfg for tsmdb1 using hadr_timeout 120
4. Start HADRdb2 start hadr on db TSMDB1 as standby (on standby server)db2 start hadr on db TSMDB1 as primary (on primary server)
5. Failover with HADRShutdown DB2 on primary serverdb2 takeover hadr on db TSMDB1 by force (on standby server)
Swap this on primary and secondary
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Bruno [email protected]: 52.511992,13.392902