What‘s Log Got to Do, Got to Do With It? - neodbug · Session code: What‘s Log Got to Do, Got...

Session code:

What‘s Log Got to Do, Got to Do With It?

David BaldonBMC Software, Inc.

1

Agenda

• Db2 Logging System Overview• Basic log record types and RBA/LRSNs• Parts and pieces of the logging system

• Configuring and Managing Db2 Log Resources• Log Management Utilities and Commands

• Forensic analysis using the Db2 Log• Who did what and when did they do it?

Db2 Logging System Overview

• Basic Log Record Types

Basic Log Record Types

• Unit of Recovery records – primarily for reversing data changes• Begin UR

• RBA of this record is known as the URID• Will be in the LRH (log record header) of all related records

• UNDO / REDO• Describes a change to a database (insert, update, delete)• Used to rollback a change when necessary

• REDO / UNDO• Known as Compensation Records• Used to reverse the rollback

Basic Log Record Types – cont.

• End UR (end phase 2)• Indicates successful COMMIT or ROLLBACK and if Db2 completed all the

associated work• Database Exception Table (DBET) records

• Exception states and image copy information of “special table spaces” i.e. DSNDB01.SYSUTILX

• Exception states of any database, table space, index space, or partition• Data pages that are logically in error (logical page list or LPL entries)• Data pages that are physically in error (write error page or WEPR entries)

Basic Log Record Types – cont.

• Checkpoint records – primarily to reduce restart time• When a predefined number of log records have been written or after a

predetermined number of minutes has elapsed • Switching between active logs• End of a successful restart• End of normal termination• Many log records can be written for a single checkpoint

• Minimum of Begin_Checkpoint and End_Checkpoint

• Database Page Set Control records - primarily deals with page sets• Allocation, Open, and Close

Db2 Logging System Overview – cont.

• Part and Pieces of the Db2 Logging System

Parts and Pieces of the Db2 Logging System

• Allows for recovering from hardware, software and application errors

• Combination of VSAM (KSDS & LDS) and sequential data sets• Bootstrap – KSDS• Actives – LDS• Archive – sequential

• Dual logging provides redundancy

Parts and Pieces of the Db2 Logging System – cont.

• Boot Strap Data Set (BSDS) Content• Records Active and Archive Logs

• Time and RBA range each log covers

• Information about other members in a Data Sharing group

• Additional Information• ARCHIVE LOG command• CHECKPOINT records• Conditional Restart Records

9


• Active Logs• Default is 3 with a maximum of 93• Db2 V11

• Maximum size is 4GB • Db2 V12

• Increased maximum size from 4GB to 768GB (must be at FL M500 or higher)• 192 times larger!• Increased workloads result in Db2 far more log records• E.g. one customer created over 4 billion log records in a 12 hour period!

10


• Db2 V12 – cont.• Expanded RBA / LRSN is required• Must convert prior to migrating from Db2 V11 to Db2 V12

11


• Archive Logs• Sequential data set on DASD or tape

• Range of log data on each archive is recorded in the BSDS

• Maximum number of archive logs is 10,000

• Virtual tape can speed up the offload process

12


• ZPARM Settings• DSN6LOGP – Log Initialization Parameters

• Max archives to be stored in the BSDS• Single or Dual logging• Log Pool Buffer size• Read COPY2 Archive

• DSN6ARVP – Archive Log Parameters• Names and attributes of archive log data sets

• DSN6SYSP – System Initialization Parameters• Checkpoint Frequency

Parts and Pieces of the Db2 Logging System – cont.NOT LOGGED Implications• NOT LOGGED was introduced in Db2 v6 for LOBs only

• Extended to all spaces in Db2 v9

• Good for limited circumstances• When the data can be recreated/recovered from a different source• As a temporary means for a “mass” operation• Materialized Query Tables usually created in a Table Space (TS) defined as NOT

LOGGED

Parts and Pieces of the Db2 Logging System – cont.NOT LOGGED Implications – cont.• Causes potential problems• TS goes into ICOPY (informational copy pending) when table updated• If a ROLLBACK (or negative SQL code) occurs, TS goes in RECP (recover pending)• If you ALTER to LOGGED, the TS goes immediately into COPY (copy pending)• Abnormal termination of Db2 and restart might cause problems

• Pages not externalized• No log records available for restart

• Use NOT LOGGED only after careful consideration!

Parts and Pieces of the Db2 Logging System – cont.LOB Logging• LOGGED LOBs do not log everything• All space map changes are logged regardless of NOT LOGGED setting• After data images are LOGGED

• INSERT data• UPDATE after data

• Before data images are not LOGGED• UPDATE before images• DELETE images

• ROLLBACK is accomplished by compensating the space map changes

• BACKOUT RECOVERY cannot be used on LOBs

Db2 Logging System Overview

• RBA / LRSNs

Db2 V11 Expanded RBA / LRSN

• Basic format RBA / LRSN are 6 bytes• 12 hex digits – x’FFFFFFFFFFFF’• Ever increasing from the point of initialization

• Expanded RBA / LRSN are 10 bytes• 20 hex digits – x’FFFFFFFFFFFFFFFFFFFF’• MVS STCK (store clock) instruction• MVS STCKE (store clock extended) instruction

• Expanded is optional in Db2 V11 but is mandatory in Db2 V12!

Db2 V11 Expanded RBA / LRSN – cont.

• RBA increases from 6 to 10 bytes• FULLWORD added in front (left) of the existing RBA• Range increases from 2(48) to 2(80)

• From 281 TRILLION bytes – 281,474,976,701,656• Terabytes

• To 1 Septillion – 1,208,925,819,614,629,174,706,176• Yottabye

• x’00000000rrrrrrrrrr’

Db2 V11 Expanded RBA / LRSN – cont.

• LRSN also increased from 6 to 10 bytes• One byte added before the existing LRSN

• Covers approximately 36K years!!!• Existing 6 byte LRSN• Three bytes of precision added at the end (right)

• Current LRSN has ~16 microsecond precision (10-6)• Current z/OS STCK has precision to ~244 picosecond (10-12)• STCKE and Expanded LRSN has precision to ~953 femtoseconds (10-15)

• x’00111111111111000000’

Logging implications with Extended Format

• 16 fewer data bytes on each physical log page• Seems to be a wash between the additional CI size versus free space• Approximately 1.3% to 1.5% combined of each page

• 42 more bytes logged for EVERY record in the LRH (log record header)

• 6 more bytes logged for every DM (data management) record

• More bytes for Save Point and Check Point log records

• About a 22% increase in overall logging

• But…no worries, we have a YOTTABYTE of log!

Configuring and Managing Db2 Log Resources

• Log Management Utilities

Log Management Utilities

• DSNJCNVT – Convert to expanded RBA / LRSN format• Must be run before starting Db2 v12

• DSNJLOGF – Preformat an active log file• Save time by preformatting new active log data sets

• DSNJU003 – Change Log Inventory• Manage the BSDS – add or delete archive log data sets, add or delete checkpoint

records, create “conditional restart” records, etc…

Log Management Utilities – cont.

• DSNJU004 – Print Log Map• Print a wide variety of information about the content of the BSDS

• DSN1LOGP – Print Log Data• Two types of reports: recovery log report and summary report


• DSNJU003 – Change Log Inventory• Add or remove active or archive data sets

• Add or remove checkpoint records

• Create or cancel a conditional restart control record (CRCR)

• Db2 Utility Guide and Reference contains the complete list

25


• DSNJU004 – Print Log Map• Use to get time, RBA range information and data set names of ACTIVE and

ARCHIVE log files• Formats additional information stored in the BSDS

• ARCHIVE LOG command history• Checkpoint Records• Conditional Restart Records• Data sharing group member records• Quiesce history• BACKUP SYSTEM utility history• System Coded Character Set ID (CCSID)

Log Management Utilities – cont.DSNJU004 – Sample JCL//DSNJU004 JOB (9999),'DBALDON',CLASS=A,

// MSGCLASS=X

//*

//STEP01 EXEC PGM=DSNJU004

//STEPLIB DD DISP=SHR,DSN=CSGI.DLJ1.DSNEXIT

// DD DISP=SHR,DSN=CSGI.DB2V12M.DSNLOAD

//SYSPRINT DD SYSOUT=*

//SYSUT1 DD DISP=SHR,DSN=DSNDLJ.DLJ1.BSDS01

//SYSIN DD *

MEMBER *

//

//* MEMBER DDNAME - Requires MxxBSDS DD statements

//* MEMBER (sys1,sys2,sys4) – Requires GROUP DD statement

Log Management Utilities – cont. DSNJU004 – Basic OutputDSNJCNVT CONVERSION PROGRAM HAS RUN DDNAME=SYSUT1

LOG MAP OF BSDS DATA SET COPY 1, DSN=DSNDJQ.DJQ1.BSDS01 LTIME INDICATES LOCAL TIME, ALL OTHER TIMES ARE GMT.

DATA SHARING MODE IS ON SYSTEM TIMESTAMP - DATE=2017.237 LTIME=15:28:38.89 UTILITY TIMESTAMP - DATE=2015.040 LTIME=22:36:11.41 VSAM CATALOG NAME=DSNDJQ HIGHEST RBA WRITTEN 0000000001B96EB4381C 2017.237 20:28:38.8HIGHEST RBA OFFLOADED 0000000001B965716FFF

... (more RBA info here)STCK TO LRSN DELTA 00000000000000000000

THIS BSDS HAS MEMBER RECORDS FOR THE FOLLOWING MEMBERS: HOST MEMBER NAME: DJQ1 MEMBER ID: 1 GROUP NAME: DSNDJQ BSDS COPY 1 DATA SET NAME: DSNDJQ.DJQ1.BSDS01 BSDS COPY 2 DATA SET NAME: DSNDJQ.DJQ1.BSDS02 ENFM START RBA/LRSN: 00000000000000000000

... (other members listed here)**** DISTRIBUTED DATA FACILITY ****

COMMUNICATION RECORD 20:57:41 AUGUST 25, 2017

LOCATION=DJQ IPNAME=(NULL) PORT=7223 SPORT=NULL RPORT=7224ALIAS=(NULL) IPV4=NULL IPV6=NULL GRPIPV4=NULL GRPIPV6=NULL LUNAME=DJQ1LU PASSWORD=(NULL) GENERICLU=(NULL)

Log Management Utilities – cont.DSNJU004 – Sample ACTIVE Log OutputACTIVE LOG COPY 1 DATA SETS

START RBA/LRSN/TIME END RBA/LRSN/TIME DATE/LTIME DATA SET INFORMATION

---------------------- ---------------------- ---------- --------------------

0000000001B934937000 0000000001B94D026FFF 2014.082 DSN=DSNDJQ.DJQ1.LOGCOPY1.DS03

00D30A318C7575A70200 00D30A3E815ED46A3A00 23:10 STATUS=REUSABLE

2017.237 12:09:10.7 2017.237 13:07:08.7

0000000001B94D027000 0000000001B965716FFF 2014.082 DSN=DSNDJQ.DJQ1.LOGCOPY1.DS01

00D30A3E815ED46A3A00 00D30A3FEC4B39474800 23:10 STATUS=REUSABLE

2017.237 13:07:08.7 2017.237 13:13:29.3

0000000001B965717000 0000000001B97DE06FFF 2014.082 DSN=DSNDJQ.DJQ1.LOGCOPY1.DS02

00D30A3FEC4B39474800 .................... 23:10 STATUS=NOTREUSABLE

2017.237 13:13:29.3 ........ ..........

• GMT versus Local Time

• DATE/LTIME of an Active Log is from when the file was created and added to the BSDS using the DSNJU003 utility

Log Management Utilities – cont.DSNJU004 – Sample ARCHIVE Log OutputARCHIVE LOG COPY 1 DATA SETS

START RBA/LRSN/TIME END RBA/LRSN/TIME DATE/LTIME DATA SET INFORMATION

---------------------- ---------------------- ---------- --------------------

0000000001B91C247000 0000000001B934936FFF 2017.237 DSN=DSNDJQ.DJQ1.ARCHLOG2.A0005331

00D30A313BDD5CFD7000 00D30A318C7575A70200 7:09 VOL=761901 UNIT=CART

2017.237 12:07:46.2 2017.237 12:09:10.7

CATALOGUED

0000000001B934937000 0000000001B94D026FFF 2017.237 DSN=DSNDJQ.DJQ1.ARCHLOG2.A0005332

00D30A318C7575A70200 00D30A3E815ED46A3A00 8:07 VOL=574024 UNIT=CART

2017.237 12:09:10.7 2017.237 13:07:08.7

CATALOGUED

0000000001B94D027000 0000000001B965716FFF 2017.237 DSN=DSNDJQ.DJQ1.ARCHLOG2.A0005333

00D30A3E815ED46A3A00 00D30A3FEC4B39474800 8:13 VOL=779881 UNIT=CART

2017.237 13:07:08.7 2017.237 13:13:29.3

• GMT versus Local Times

• DATE/LTIME for Archive Logs are from when the file was created

Log Management Utilities – cont.DSN1LOGP – Print Log Data• Log Recovery Report Formats• Detail Report – details about individual log records.• Summary Report – summary of the content of log(s)• If possible use DATAONLY(YES) and SUMMARY(YES) to avoid large output

• Syntax can be used to limit output to certain record types• TYPE(hex-value) – causes most other keywords to be ignored• SUBTYPE(hex-value) of UR UNDO (type 200) UR REDO (type 400)

• Optional VALUE/OFFSET used for additional criteria• TYPE and SUBTYPE are mutually exclusive

• Log record formats can be found in hlq.DSNMACS(DSNDQJ00)

Log Management Utilities – cont.DSN1LOGP Sample JCL//DSN1LOGP JOB 9999,DBALDON,CLASS=Q,MSGCLASS=X

//PLOGMAP EXEC PGM=DSN1LOGP

//STEPLIB DD DISP=SHR,DSN=SYS3.DJQ1.DSNEXIT

// DD DISP=SHR,DSN=CSGI.DB2V11M.DSNLOAD

//ARCHIVE DD DISP=SHR,DSN=DSNDJQ.DJQ1.ARCHLOG2.A0005332

//SYSIN DD *

STARTRBA(0000000001B934937000) ENDRBA(FFFFFFFFFFFFFFFFFFFF)

TYPE(0020)

DATAONLY(NO)

SYSCOPY(NO)

/*

//SYSSUMRY DD SYSOUT=*

//SYSPRINT DD SYSOUT=*

//

Log Management Utilities – cont.DSN1LOGP – Sample UR Control Record Output0000000000A307A5BF05 URID(0000000000A307A5BF05)

LRSN(00D0881CF8B14E304400) TYPE(UR CONTROL) SUBTYPE(BEGIN UR)

*LRH* 000000BA 00500008 03A00000 00000000 00000000 00A307A5 BF050000 00000000

00000000 00000000 00005000 00200001 00000000 00000000 00000000 000000D0

881CF8B1 4E304400 00010000 00000000

0000 00300000 0000D000 00000000 00000700 0000C4D9 C4D6F6C9 40404040 4040D4E5

0020 E2C5E9D4 F140D088 1CF8B14D E788C1D3 D7E3C9C1 C440C2C1 E3C3C840 4040C2C1

0040 E3C3C840 40400000 00000000 0000001A 0001E4E2 C2D4C3D5 F0F1C4D1 D8F1D3E4

0060 4040D088 1CF8AEC7 0001

Log Management Utilities – cont.DSN1LOGP – SUBTYPE Heading Values• UR CONTROL records – partial list• SUBTYPE(BEGIN COMMIT1) / SUBTYPE(PHASE 1 TO 2)

• Or • SUBTYPE(END COMMIT1) / SUBTYPE(BEGIN COMMIT2)

• SUBTYPE(END COMMIT2)• SUBTYPE(BEGIN ABORT) / SUBTYPE(END ABORT)• SUBTYPE(END UNDO) Rolled back INFLIGHT at DB2 Startup• SUBTYPE(END REDO) Committed INFLIGHT at DB2 Startup• SUBTYPE(END TODO) Terminated from Phase 1 location

Log Management Utilities – cont.DSN1LOGP – Sample UR-UNDO Record Output0000000001BE2B1F7864 TYPE( UNDO ) URID(0000000001BE291749CC)

LRSN(00D317E029EA1C692A00) DBID(0211) OBID(0019) PAGE(000000FB)

SUBTYPE(TYPE 2 INDEX UPDATE) CLR(NO) PROCNAME(DSNKISPL)

*LRH* 00001068 00B34008 0EA00000 00000000 00000000 01BE2917 49CC0000 00000000

00000000 01BE2B1F 77B15000 02000019 00000000 000001BE 2B1F77B1 000000D3

17E029EA 1C692A00 00010000 00000000

*LG** 00021100 19000000 FB0240D3 17E0296A 9BEA4800 65000000

0000 1000020E 800000E3 00000000 00000000 00000000 00000000 00000000 FB7C0000

0020 00000000 00000018 40780094 00120EB2 00000000 00130000 00020106 00E30094

0040 00000000 040A0000 06960092 00030881 6D6F5080 7E8CB6B7 269401E7 46384040

Log Management Utilities – cont.DSN1LOGP – SUBTYPE Heading Values• DM (data management) Records – partial list• SUBTYPE(INSERT IN A DATA PAGE)• SUBTYPE(UPDATE IN-PLACE IN A DATA PAGE)• SUBTYPE(UPDATE NOT IN-PLACE, DATA PART ONLY IN A DATA PAGE)• SUBTYPE(TYPE 2 INDEX UPDATE)• SUBTYPE(FORMAT PAGE OR MODIFY SPACE MAP)

Log Management Utilities – cont.DSN1LOGP Oddities• Log Records tied together• Often times within a Unit of Recovery• Compensation record points to original record being compensated• Page set activity linked

• Spanned Log Records• Log pages are 4K

• 32K records must span multiple 4K pages• DSN1LOGP does paste together segments when it presents them• “Documented Interface” presents them pasted as a single record too

Log Management Utilities – cont.DSN1LOGP Oddities – cont.• Multi-record updates• For large updates, undo and redo segments can be different records• Overflow Records

• INSERT into new page, UPDATE changing original row to a pointer record• DELETE from original page, INSERT into new page, UPDATE to pointer record

Configuring and Managing Db2 Log Resources

• Log Management Commands

Log Management Commands

• -ARCHIVE LOG – close current active and open next available active

• -DISPLAY LOG – displays information about an offload task

• -DISPLAY ARCHIVE – displays archive log information

• -SET ARCHIVE – set maximum number of tape units for archive log

Log Management Commands – cont.

• -DISPLAY LOGDSNJ370I *DJQ2 DSNJC00A LOG DISPLAY 439

CURRENT COPY1 LOG = DSNDJQ.DJQ2.LOGCOPY1.DS01 IS 78% FULL

CURRENT COPY2 LOG = DSNDJQ.DJQ2.LOGCOPY2.DS01 IS 78% FULL

H/W RBA = 0000000001BD2D973410

H/O RBA = 0000000001BD1AA40FFF

FULL LOGS TO OFFLOAD = 0 OF 6

OFFLOAD TASK IS (AVAILABLE)

SOFTWARE ACCELERATION IS DISABLED

DSNJ371I *DJQ2 DB2 RESTARTED 04:03:50 AUG 12, 2017 440

RESTART RBA 0000000001BA3B42A000

CHECKPOINT FREQUENCY 500000 LOGRECORDS

LAST SYSTEM CHECKPOINT TAKEN 07:09:36 AUG 25, 2017

DSN9022I *DJQ2 DSNJC001 '-DISPLAY LOG' NORMAL COMPLETION

Log Management Commands – cont.

• -DISPLAY ARCHIVEDSNJ322I -DISPLAY ARCHIVE REPORT FOLLOWS

COUNT TIME

(TAPE UNITS) (MIN,SEC)

DSNZPARM 2 0,00

CURRENT 2 0,00

===============================

ADDR STAT CORR-ID VOLSER DATASET_NAME

03B0 BUSY SHEDDEN A00001 DSNT2AR1.DT25.D04169.T1328583.A0012701

RCAL 03RCRSC MIGRAT DSNT2AR1.DT25.D04169.T1334426.A0012704

A99B BUSY 14DRSTRT ARN690 DSNT2AR1.DT25.D04169.T1346176.A0012705

BDDD BUSY 10LPLALR ARN738 DSNT2AR1.DT25.D04170.T1506437.A0012743

END OF DISPLAY ARCHIVE REPORT.

Forensic Analysis using the Db2 Log

• Who did what and when did they do it?• Log Utilities• Log Tools

Who did what and when did they do it?

• Log Utilities

Log UtilitiesDSN1LOGP – Deleted Row0000000001BE440F8EC4 TYPE( UNDO REDO ) URID(0000000001BE41BD2203)

LRSN(00D317E8E2114109B800) DBID(0211) OBID(0002) PAGE(000726E9) 04:58:37 17.248

SUBTYPE(DELETE IN A DATA PAGE) CLR(NO) PROCNAME(DSNIDILS)

*LRH* 00000427 00744008 0EA00000 00000000 00000000 01BE41BD 22030000 00000000 *

00000000 01BE440F 8E505000 06000001 00000000 000001BE 440F8E50 000000D3 * && & L

17E8E211 4109B800 00010000 00000000 * YS

*LG** 08021100 02000726 E9000000 00000000 00000000 39000000 * Z

0000 03BF2001 00031015 0003B700 030100C8 D6E7D7E8 E2D6D5D1 D6E5C4C5 D4C1D5D9 * HOXPYSONJOVDEMANR

0020 D9404040 40404040 00200011 03102713 10271300 F0000000 00000000 42066992 *R 0 k

0040 00000000 00D4C1D5 D9D9F1F9 F6F360F1 F160F0F3 F2F0F0F0 60F1F160 F0F3F4F2 * MANRR1963-11-032000-11-0342

0060 F0F6F6F9 F9F2F0F0 F0404040 40404040 40404040 40404000 D9D90080 03276B00 *066992000 RR ,

0080 F0F060F1 F160F0F3 60F1F04B F2F74BF1 F34BF140 40404040 40404040 40404040 *00-11-03-10.27.13.1

00A0 40404040 40404040 40404040 40404040 40404040 40404040 40404040 00F1F0F2 * 102

00C0 F7F1F3F1 F1F2F5F2 4BF4F6F2 40404040 40404040 40404040 40404000 F0000011 *71311252.462 0

• UNDO REDO Record for a delete from table Q283779D.Q83779T1• Requires looking up the DBID and OBID in the Db2 Catalog to get the table name

Log UtilitiesDSN1LOGP – Deleted Row – Compressed Data0000000001BE438EC218 TYPE( UNDO REDO ) URID(0000000001BE41BD2203)

LRSN(00D317E8D541044DA400) DBID(0211) OBID(0002) PAGE(000581C1) 04:58:23 17.248

SUBTYPE(DELETE IN A DATA PAGE) CLR(NO) PROCNAME(DSNIDILS) COMPRESSED DATA

*LRH* 000001DD 00740008 0EA00000 00000000 00000000 01BE41BD 22030000 00000000 *

00000000 01BE438E C1A45000 06000001 00000000 000001BE 438EC1A4 000000D3 * Au& Au L

17E8D541 044DA400 00010000 00000000 * YN (u

*LG** 08021100 02000581 C1000000 00000000 00000000 39000000 * aA

0000 01752001 00035015 04016D00 030111C0 E80D70D3 7B90D20E 80C40E38 7FB0F3A7 * & _ Y L# K D 3x

0020 07700900 90090330 41009033 04117609 20790140 710976F7 789F785F DF9EF015 * 7 ^ 0

0040 FDF7BD57 F77E25CA EF36113B 34004042 06B122F0 BF83F83F DEDD0D4E 32EF9AE1 * 7 7= 0 c8 +

0060 ED0DD574 48E68340 01001207 20510771 6D025017 07207911 C0E80D70 D37B90D2 * N Wc _ & Y L# K

0080 0E80C40E 387FB0FC B1D74F9E F015FDF7 BD57F77E 25CAEF0B F83F83FD EDD0D4EF * D P| 0 7 7= 8 c M

00A0 9AE1ED0D D57448E6 8F2BF00D 28D57F7B F3510409 20091F85 FDF83FDE DD0D4EF9 * N W 0 N #3 e 8 +9

00C0 AE633241 76002051 07702709 10420790 9711B0E8 0C40E387 F2450890 D5000083 * p Y Tg2 N c

• UNDO REDO Record for a delete from table Q283779D.Q83779T1• Requires finding the corresponding compression dictionary to understand the data. • Not an easy thing to do!

Log Utilities – cont.DSN1LOGP – Database Stopped0000000001B939E6FECF LRSN(00D30A3B6F5AC01C5200) TYPE(SYSTEM EVENT) 07:53:24 17.237

SUBTYPE(TRACE RECORD)

*LRH* 000001AE 01B64008 10A00000 00000000 00000000 00000000 00000000 00000000 *

00000000 00000000 00005000 00100041 00000000 00000000 00000000 000000D3 * & L

0A3B6F5A C01C5200 00010000 00000000 * ?!

0000 015E0000 0000004C 01120001 00000014 00370001 002F60E2 E3D6D740 C4C24DC7 * ; < -STOP DB(G

0020 E5F1F1C2 F0F0F15D 40E2D7C1 C3C54DE7 E3C1F0F7 F9F1F15D 40C1E34D C3D6D4D4 *V11B001) SPACE(XTA07911) AT(COMM

0040 C9E35D29 4720D000 00000040 00560117 005A02B1 1C4D43A0 C4D1D8F1 D30A3B6F *IT) ! ( DJQ1L ?

0060 5AB65362 0000B914 0000B914 00000000 C4D1D840 40404040 40404040 40404040 *! DJQ

0080 E4E2C2D4 C3D5F0F1 C4D1D8F1 D3E44040 D30A3AB5 A83D0059 00000000 0000C4C2 *USBMCN01DJQ1LU L y DB

00A0 F2C200A8 0200C3E2 E3C3E7D5 4040C7E5 F1F1C2F0 F1F74040 4040C4C2 F2C3C1D3 *2B y CSTCXN GV11B017 DB2CAL

00C0 D340C3C4 C2D7D3C1 D540C3E2 E3C3E7D5 40400000 00020000 00000000 00000000 *L CDBPLAN CSTCXN

00E0 00000000 00000000 00000000 0000C3E2 E3C3E7D5 40404040 40404040 4040C7E5 * CSTCXN GV

0100 F1F1C2F0 F1F74040 40404040 40404040 40404040 40404040 40404040 4040C4C2 *11B017 DB

0120 F2C3C1D3 D3404040 40404040 40404040 00000000 00000000 00000000 00000000 *2CALL

0140 00001C4D 43A00000 00000014 2000C4D1 D8F14040 4040C4E2 D5C4D1D8 4040 * ( DJQ1 DSNDJQ

• SYSTEM EVENT trace record of a command

Log Utilities – cont.DSN1LOGP Sample Output0000000000A306A7520A TYPE( UNDO REDO ) URID(0000000000A306A75150)

LRSN(00D0881CBAC4077C1C00) DBID(0006) OBID(02D5) 14:37:38 16.092

SUBTYPE(DBE TABLE WITH EXCEPTION DATA)

*LRH* 00000180 00BA0008 0EA00000 00000000 00000000 00A306A7 51500000 00000000 * t x &

00000000 00A306A7 51505000 2600001D 00000000 000000A3 06A75150 000000D0 * t x && t x &

881CBAC4 077C1C00 00010000 00000000 *h D @

0000 05020000 C4E2D5C4 C2F0F640 E2E8E2C1 D3E3C5D9 000602D5 02000000 00000000 * DSNDB06 SYSALTER N

0020 001A0000 00000000 00000000 00000000 00000000 00000052 00000000 00000000 *

0040 80000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *

0060 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *

0080 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *

00A0 00000000 00000000 00000000 00000000 00000000 00000052 00000000 00000000 *

00C0 04000000 00010000 00000000 00000000 00000000 00000000 00000000 00000000 *

00E0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *

0100 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 *

0120 00000000 00000000 00000000 00000000 *

UNDO: DSNDB06 .SYSALTER

NO EXCEPTION STATES SET

REDO: DSNDB06 .SYSALTER Utility: QUIESCE

UTRO

NO. OF UTRO UTILITIES: 1

Who Broke My Application?

• Log Tools

Log Tools

• Read the log

• Provide context to log records• Associate units of recovery information to DM activity• Combine multiple records into single entities• Manage committed versus rolled back activity

• Allow for robust selection (filtering) criteria

• Authorization checking for access to log records

• Allow for “ONGOING” processes

• Produce different outputs: SQL, DDL, LOAD, COMMANDs, reports

Log Tools – cont.

• UNDO SQL / DDL• Extract the bad changes within the specified range and filter criteria• Reverse the changes made in error• Surgical updating of table(s) while the data remains online

• REDO SQL• Targeted for when there are many bad updates scattered among few good

updates• Requires a Point In Time Recovery to remove the bad updates• Extract the relatively small number of good updates to apply after the RECOVER

is complete.

Log Tools – cont.

• Externalization of Data• Completion of partially logged updates

• Can be costly unless DATA CAPTURE CHANGES is on for the TS• LOB completion

• Before images of updates/deletes are not logged even if LOG YES• Decompression using dictionaries from various locations• Invoke Edit and Field Proc Decode functions when necessary• Decode Db2 internal formats

• Numeric, Date & Time, Timestamps• Normalize log records to current version• Serialize XML

Log Tools – cont.

• Output Files • SQL

• UNDO / MIGRATE / REDO• DDL

• UNDO / MIGRATE• COMMAND

• BIND, REBIND, FREE• UNDO / MIGRATE• Not to be confused with Db2 –COMMANDs

Log Tools – cont.

• Output Files – cont.• LOAD

• File of Insert/Update/Delete activity within the range• Used by some Log Tools as a reusable format• Used by some customers to post-process• Different formats such as CSV or SDF may be available

• Reusable Format• LLOG – or Logical Log• DATA File has the Insert/Update/Delete activity• CNTL File has Context Schema information• Can reprocess this file even if the schema is dropped or has changed

Summary

• Use the Db2 Log to your advantage

• Think out of the box when using the log

• Questions?

Session code:

Please fill out your session

evaluation before leaving!

David BaldonBMC Software, [email protected]

56

Please fill out your session

evaluation before leaving!

What‘s Log Got to Do, Got to Do With It? - neodbug · Session code: What‘s Log Got to Do, Got...

Documents

Transcript of What‘s Log Got to Do, Got to Do With It? - neodbug · Session code: What‘s Log Got to Do, Got...