©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to...
-
Upload
brandon-henderson -
Category
Documents
-
view
215 -
download
2
Transcript of ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to...
![Page 1: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/1.jpg)
1©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
An Expert System
designed to evaluate
IBM z/OS systems
![Page 2: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/2.jpg)
2©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
Product Overview
• Helps analyze performance of z/OS systems.
• Written in SAS (only SAS/BASE is required).
• Runs as a batch job on mainframe (or on PC).
• Processes data in a standard performance data base (either MXG, SAS/ITRM, or NeuMICS).
• Produces narrative reports showing results from analysis!
• Product is updated every six months
• 45-day trial is available (see license agreement for details).
![Page 3: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/3.jpg)
3©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
Components Delivered• SRM Component * March 1991
• TSO Component * April 1991
• MVS Component * June 1991
* These legacy components apply only to Compatibility Mode
• DASD Component October 1991
• CICS Component May 1992
• WLM Component April 1995
• DB2 Component October 1999
• WMQ Component June 2004
![Page 4: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/4.jpg)
4©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
Product Documentation
Each component has an extensive User Manualavailable in hard-copy or CD, and web-enabled
• Describes the likely impact of each finding
• Discusses the performance issues associated with each finding
• Suggests ways to improve performance and describes alternative solutions
• Provides specific references to IBM or other documents relating to the findings
• More than 4,000 pages for all components
![Page 5: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/5.jpg)
5©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WLM Component•Checks for problems in service definition
•Identifies reasons performance goals were missed
•Analyzes general system problems:
• Coupling facility/XCF
• Paging subsystem
• System logger
• WLM-managed initiators
• Excessive CPU use by SYSTEM or SYSSTC
• IFA/zAAP, zIIP, and IOP/SAP processors
• PR/SM and LPAR problems
• Intelligent Resource Director (IRD) problems
![Page 6: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/6.jpg)
6©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WLM Component - sample reportRULE WLM103: SERVICE CLASS DID NOT ACHIEVE VELOCITY GOAL
DB2HIGH (Period 1): Service class did not achieve its velocity goal during the measurement intervals shown below. The velocity goal was 50% execution velocity, with an importance level of 2. The '% USING' and '%TOTAL DELAY' percentages are computed as a function of the average address space ACTIVE time. The 'PRIMARY,SECONDARY CAUSES OF DELAY' are computed as a function of the execution delay samples on the local system.
‑‑‑‑‑‑LOCAL SYSTEM‑‑‑‑‑‑‑‑ % % TOTAL EXEC PERF PLEX PRIMARY,SECONDARY MEASUREMENT INTERVAL USING DELAY VELOC INDX PI CAUSES OF DELAY 21:15‑21:30,08SEP1998 16.6 83.4 17% 3.02 2.36 DASD DELAY(99%)
RULE WLM361: NON‑PAGING DASD I/O ACTIVITY CAUSED SIGNIFICANT DELAYS
DB2HIGH (Period 1): A significant part of the delay to the service class can be attributed to non‑paging DASD I/O delay. The below data shows intervals when non‑paging DASD delay caused DB2HIGH to miss its performance goal:
AVG DASD AVG DASD ‑‑AVERAGE DASD I/O TIMES‑ MEASUREMENT INTERVAL I/O RATE USING/SEC RESP WAIT DISC CONN 21:15‑21:30,08SEP1998 31 1.405 0.010 0.003 0.004 0.002
![Page 7: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/7.jpg)
7©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WLM Component - sample reportRULE WLM601: TRANSPORT CLASS MAY NEED TO BE SPLIT
You should consider whether the DEFAULT transport class should be split. A large percentage of the messages were too small, while a significant percentage of messages were too large. Storage is wasted when buffers are used by messages that are too small, while unnecessary overhead is incurred when XCF must expand the buffers to fit a message. The CLASSLEN parameter establishes the size of each message buffer, and the CLASSLEN parameter was specified as 16,316 for this transport class. This finding applies to the following RMF measurement intervals:
SENT SMALL MESSAGES MESSAGES TOTAL MEASUREMENT INTERVAL TO MESSAGES THAT FIT TOO BIG MESSAGES 10:00‑10:30,26MAR1996 JA0 4,296 0 57 4,353 12:00‑12:30,26MAR1996 Z0 2,653 6 762 3,421 12:30‑13:00,26MAR1996 Z0 2,017 0 109 2,126
RULE WLM622: THE NUMBER OF OUTBOUND PATHS MAY NEED TO BE INCREASED
The PATH BUSY (when selected for transfer) was high relative to the PATH AVAILABLE for the C605 path on System JB0, sending messages to the C611 path on System JA0 in transport class DEFSMALL. This usually means that you need to add more OUTBOUND paths to the transport class. This finding applies to the following RMF measurement intervals:
TOTAL PCT OUTBOUND PCT PATH MEASUREMENT INTERVAL MESSAGES PATH BUSY AVAILABLE 12:00‑12:30,26MAR1996 2562 21.1 78.9
![Page 8: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/8.jpg)
8©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WLM Component - sample reportRULE WLM705: STAGING DATA SETS NOT EFFICIENTLY USED FOR LOG STREAM
GRP1.CICS1T9A.DFHLOG: The bytes deleted after data was offloaded from staging data sets to DASD log data sets was high relative to the bytes deleted before data was offloaded. More than 10% of the bytes deleted were deleted after being offloaded to DASD, with associated unnecessary I/O operations. This indicates that the space allocated for the DASD staging data sets was not efficiently used for the log stream data. If this finding is consistently produced, you should consider increasing the size of the DASD staging data set for log stream GRP1.CICS1T9A.DFHLOG. This finding applies to the following SMF measurement intervals:
BYTES BYTES DELETED BYTES DELETED PERCENT MEASUREMENT INTERVAL DELETED BEFORE OFFLOAD AFTER OFFLOAD AFTER 14:45,03OCT2002 844K 0 844K 100.0
RULE WLM537: ZAAP-ELIGIBLE WORK HAD HIGH GOAL IMPORTANCE
Rule WLM530 or Rule WLM535 was produced for this system, indicating that a relatively large amount of zAAP-eligible work was processed on a central processor. One possible cause of this situation is that the zAAP-eligible work was assigned a relatively high Goal Importance (the Goal Importance was either Importance 1 or Importance 2). Please see the discussion in the WLM Component User Manual for an explanation of this issue.
![Page 9: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/9.jpg)
9©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 Component
•Analyzes standard DB2 interval statistics
•Applies analysis from DB2 Administration Guide
and DB2 Performance Guide (with DB2 9.1)
•Analyzes DB2 Versions 3, 4, 5, 6, 7, 8, and 9
•Evaluates overall DB2 constraints, buffer pools,
EDM pool, RID list processing, Lock Manager,
Log Manager, DDF, and data sharing
•All analysis can be tailored to your site!
![Page 10: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/10.jpg)
10©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 Component
• There might be insufficient buffers for work files
• There were insufficient buffers for work files in merge passes
• Buffer pool was full
• Hiperpool read requests failed (pages stolen by system)
• Hiperpool write requests failed (expanded storage not available
• Buffer pool page fault rate was high
• Data Management Threshold (DMTH) was reached
• DWQT and VDWQT might be too large
• DWQT, VDWQT, or VPSEQT might be too small
Typical DB2 local buffer constraints
![Page 11: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/11.jpg)
11©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 ComponentTypical DB2 I/O prefetch constraints
• Sequential prefetch was disabled, buffer shortage
• Sequential prefetch was disabled, unavailable read engine
• Sequential prefetch not scheduled, prefetch quantity = 0
• Synchronous read I/O and sequential prefetch was high
• Dynamic sequential prefetch was high (before DB2 8.1)
• Synchronous read I/O was high
![Page 12: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/12.jpg)
12©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 Component
• Parallel groups fell back to sequential mode
• Parallel groups reduced due to buffer shortage
• Prefetch quantity reduced to one‑half of normal
• Prefetch quantity reduced to one‑quarter of normal
• Prefetch I/O streams were denied, shortage of buffers
• Page requested for a parallel query was unavailable
Typical DB2 parallel processing constraints
![Page 13: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/13.jpg)
13©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 ComponentTypical DB2 EDM pool constraints
• Failures were caused by full EDM pool
• Low percent of DBDs found in EDM pool
• Low percent of CT Sections found in EDM pool
• Low percent of PT Sections found in EDM pool
• Size of EDM pool could be reduced
• Excessive Class 24 (EDM LRU) latch contention
![Page 14: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/14.jpg)
14©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 ComponentTypical DB2 Lock Manager constraints
• Work was suspended because of lock conflict
• Locks were escalated to shared mode
• Locks were escalated to exclusive mode
• Lock escalation was not effective
• Work was suspended for longer than time-out value
• Deadlocks were detected
![Page 15: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/15.jpg)
15©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 ComponentTypical DB2 Log Manager constraints
• Archive log read allocations exceeded guidance
• Archive log write allocations exceeded guidance
• Waits were caused by unavailable output log buffer
• Log reads satisfied from active log data set
• Log reads were satisfied from archive log data set
• Failed look-ahead tape mounts
![Page 16: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/16.jpg)
16©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 ComponentTypical DB2 Data Sharing constraints
• Group buffer pool is too small
• Incorrect directory entry/data entry ratio
• Directory reclaims resulting in cross-invalidations
• Castout processing occurring in “spurts”
• Excessive lock contention or false lock contention
• GBPCACHE ALL inappropriately specified
• GBPCACHE CHANGED inappropriately specified
• Conflicts between applications
![Page 17: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/17.jpg)
17©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 Component - sample report
RULE DB2-208: VIRTUAL BUFFER POOL WAS FULL Buffer Pool 2: A usable buffer could not be located in virtual Buffer Pool 2, because the virtual buffer pool was full. This condition should not normally occur, as there should be ample buffers. You should consider using the -ALTER BUFERPOOL command to increase the virtual buffer pool size (VPSIZE) for the virtual buffer pool. This situation occurred during the intervals shown below:
BUFFERS NUMBER OF TIMES MEASUREMENT INTERVAL ALLOCATED POOL WAS FULL 10:54-11:24, 15SEP1999 100 12 11:24-11:54, 15SEP1999 100 13
RULE DB2-216: BUFFER POOLS MIGHT BE TOO LARGE Buffer Pool 1: The page fault rates for read and write I/O indicated that the buffer pools might be too large for the available processor storage. This situation occurred for Buffer Pool 1 during the intervals shown below:
BUFFERS PAGE-IN FOR PAGE-IN FOR PAGE MEASUREMENT INTERVAL ALLOCATED READ I/O WRITE I/O RATE 11:15-11:45, 16SEP1999 25,000 36,904 195 41.2 11:45-12:15, 16SEP1999 25,000 30,892 563 35.0 12:45-13:15, 16SEP1999 25,000 23,890 170 26.7
![Page 18: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/18.jpg)
18©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DB2 Component - sample report
RULE DB2-601: COUPLING FACILITY READ REQUESTS COULD NOT COMPLETE Group Buffer Pool 6: Coupling facility read requests could not be completed because of a lack of coupling facility storage resources. This situation occurred for Group Buffer Pool 6 during the intervals shown below:
GROUP BUFFER POOL TIMES CF READ MEASUREMENT INTERVAL ALLOCATED SIZE REQUESTS NOT COMPLETE 11:01-11:31, 14OCT1999 38M 130
RULE DB2-610: GBPCACHE(N0) OR GBPCACHE NONE MIGHT BE APPROPRIATE Group Buffer Pool 4: This buffer pool had a very small amount of read activity relative to write activity. Pages read were less than 1% of the pages written. Since so few pages were read from this group buffer pool, you should consider specifying GPBCACHE(NO) for the group buffer pool or specifying GBPCACHE NONE for the page sets using the group buffer pool. This situation occurred for Group Buffer Pool 4 during the intervals shown below:
GROUP BUFFER POOL PAGES PAGES READ MEASUREMENT INTERVAL ALLOCATED SIZE READ WRITTEN PERCENT 10:34-11:04, 14OCT1999 38M 14 18,268 0.07%
![Page 19: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/19.jpg)
19©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
CICS Component
•Processes CICS Interval Statistics contained inMXG Performance Data Base (standard SMF 110)
•Analyzes all releases of CICS (CICS/ESA,CICS/TS for OS390, and CICS/TS for z/OS)
•Applies most analysis techniques contained inIBM’s CICS Performance Guides
•Produces specific suggestions for improving CICS performance
![Page 20: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/20.jpg)
20©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
CICS Component(Major areas analyzed)
•Virtual and real storage (MXG/AMXT/TCLASS)•VSAM and File Control (NSR and LSR pools)•Database management (DL/I, IMS, DB2)•Journaling (System and User journals)•Network and VTAM (RAPOOL, RAMAX)•CICS Facilities (temp storage, transient data)•ISC/IRC (MRO, LU61., LU6.2 modegroups)•System logger•Temporary Storage •Coupling Facility Data Tables (CFDT)•CICS-DB2 Interface•Open TCB pools•TCP/IP and SSL
![Page 21: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/21.jpg)
21©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
CICS Component - sample report
RULE CIC269: EXCESSIVE GETMAIN/FREEMAIN ACTIVITY FOR MRO SESSIONS
The IOAREALEN parameter specifications for the below connections might cause excessive GETMAIN/FREEMAIN activity. The GETMAIN/FREEMAIN rate was greater than 25 per second to the Storage Domain SMTP Subpool. This high GETMAIN/FREEMAIN activity normally means that the IOAREALEN parameter was set to the default (zero), or was set to a smaller value than the TIOA (Terminal I/O Area) required to support the MRO traffic. The information shown below describes the total GETMAIN and FREEMAIN activity, the Connection ID, the percent of message activity to each Connection ID, and TIOA size (and the percent of message activity at that TIOA size for the Connection ID) for connections with the highest total function shipping activity.
GETMAIN+ CONN PERCENT APPLID FREEMAIN ID ACTIVITY TIOA--PCT TIOA--PCT TIOA--PCT CICS1TAA 191,502 1AAA 34.2 624 96.2% 120 3.7% CICS1TAA 191,502 1AAB 33.0 600 97.0% 264 2.9% CICS1TAA 191,502 1AAC 32.7 600 97.1% 344 2.8%
![Page 22: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/22.jpg)
22©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
CICS Component - sample report
RULE CIC267: INSUFFICIENT SESSIONS MAY HAVE BEEN DEFINED
CPExpert believes that an insufficient number of sessions may have been defined for the CICS DAL1 connection, or the application system could have been issuing ALLOCATE requests too often. The number of ALLOCATE requests returned was greater than the value specified for the ALLOCQ guidance variable in USOURCE(CICGUIDE). CPExpert suggests you consider increasing the number of sessions defined for the connection, or you should increase the ALLOCQ guidance variable to cause CPExpert to signal a potential problem only when you view the problem as serious. For APPC modegroups, this finding applies only to generic ALLOCATE requests. This finding applies to the following CICS statistics intervals:
STATISTICS ALLOCATE REQUESTS COLLECTION TIME APPLID RETURNED TO USERS 10:00,26MAR1996 CICSDTL1 335 11:00,26MAR1996 CICSDTL1 12 12:00,26MAR1996 CICSDTL1 27
![Page 23: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/23.jpg)
23©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DASD Component•Processes SMF Type 70(series) to automaticallybuild model of your I/O configuration.
•Identifies performance problems with deviceswhich have most potential for improvement.
• PEND delays• Disconnect delays• Connect delays• IOSQ delays• Shared DASD conflicts
•Analyzes SMF Type 42(DS) and Type 64 toidentify VSAM performance problems.
![Page 24: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/24.jpg)
24©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DASD Component - sample report
RULE DAS100: VOLUME WITH WORST OVERALL PERFORMANCE
VOLSER DB2327 (device 2A1F) had the worst overall performance during the entire measurement period (10:00, 16FEB2001 to 11:00, 16FEB2001). This volume had an overall average of 56.8 I/O operations per second, was busy processing I/O for an average of 361% of the time, and had I/O operations queued for an average of 1% of the time. Please note that percentages greater than 100% and Average Per Second Delays greater than 1 indicate that multiple I/O operations were concurrently delayed. This can happen, for example, if multiple I/O operations were queued or if multiple I/O operations were PENDing. The following summarizes significant performance characteristics of VOLSER DB2327:
I/O --- AVERAGE PER SECOND DELAYS--- MAJOR MEASUREMENT INTERVAL RATE RESP CONN DISC PEND IOSQ PROBLEM 10:00-10:30,16FEB2001 59.1 1.308 0.316 0.004 0.988 0.000 PEND TIME 10:30-11:00,16FEB2001 57.2 3.792 0.300 0.004 3.483 0.006 PEND TIME 11:00-11:30,16FEB2001 54.2 5.769 0.279 0.004 5.464 0.023 PEND TIME
![Page 25: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/25.jpg)
25©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DASD Component - sample report
RULE DAS130: PEND TIME WAS MAJOR CAUSE OF I/O DELAY.
A major cause of the I/O delay with VOLSER DB2327 was PEND time. The average per-second PEND delay for I/O is shown below:
PEND PEND PEND PEND PEND TOTAL MEASUREMENT INTERVAL CHAN DIR PORT CONTROL DEVICE OTHER PEND 10:00-10:30,16FEB2001 0.492 0.000 0.000 0.000 0.495 0.988 10:30-11:00,16FEB2001 1.927 0.000 0.000 0.000 1.556 3.483 11:00-11:30,16FEB2001 2.840 0.000 0.000 0.000 2.624 5.464
RULE DAS160: DISCONNECT TIME WAS MAJOR CAUSE OF I/O DELAY.
A major cause of the I/O delay with VOLSER DB26380 was DISCONNECT time. DISC time for modern systems is a result of cache read miss operations, potentially back-end staging delay for cache write operations, peer-to-peer remote copy (PPRC) operations, and other miscellaneous reasons.
--PERCENT-- DASD CACHE ----CACHE---- READ WRITE TO TO MEASUREMENT INTERVAL READS WRITES HITS HITS CACHE DASD PPRC BPCR ICLR 8:30- 8:45,22OCT2001 14615 932 19.2 100.0 11825 903 0 0 0 8:45- 9:00,22OCT2001 14570 921 20.7 100.0 11567 907 0 0 0
![Page 26: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/26.jpg)
26©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DASD Component - sample report
RULE DAS300: PERHAPS SHARED DASD CONFLICTS CAUSED PERFORMANCE PROBLEMS
Accessing conflicts caused by sharing VOLSER DB2700 between systems might have caused performance problems for the device during the measurement intervals shown below. Conflicting systems had the indicated I/O rate, average CONN time per second, average DISC time per second, average PEND time per second, and average RESERVE time to the device. Even moderate CONN, DISC, or RESERVE can cause delays to shared devices. .. I/O MAJOR OTHER -------OTHER SYSTEM DATA-------- MEASUREMENT INTERVAL RATE PROBLEM SYSTEM I/O RATE CONN DISC PEND RESV 8:30- 8:45,22OCT2001 31.3 QUEUING SY1 35.0 0.041 0.001 0.455 0.000 SY2 88.2 0.100 0.003 0.714 0.000 SY3 109.0 0.123 0.003 0.723 0.000 TOTAL 232.2 0.264 0.006 1.892 0.000 8:45- 9:00,22OCT2001 25.7 QUEUING SY1 46.4 0.054 0.001 0.565 0.000 SY2 98.2 0.112 0.003 0.836 0.000 SY3 119.0 0.136 0.003 0.846 0.000 TOTAL 263.5 0.303 0.007 2.247 0.000
![Page 27: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/27.jpg)
27©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DASD Component - sample reportRULE DAS607: VSAM DATA SET IS CLOSE TO MAXIMUM NUMBER OF EXTENTS
VOLSER: RLS003. More than 225 extents were allocated for the VSAM data sets listed below. The VSAM data sets are approaching the maximum number of extents allowed. The below shows the number of extents and the primary and secondary space allocation: .. TOTAL EXTENTS ---ALLOCATIONS--- SMF TIME STAMP JOB NAME VSAM DATA SET .. EXTENTS THIS OPEN PRIMARY SECONDARY 10:30,11MAR2002 CICS2ABA RLSADSW.VF01D.DATAENDB.DATA................. 229 4 30 CYL 1 CYL
RULE DAS625: NSR WAS USED, BUT LARGE PERCENT OF ACCESS WAS DIRECT
VOLSER: MVS902. Non-Shared resources (NSR) was specified as the buffering technique for the below VSAM data sets, but more than 75% of the I/O activity was direct access. NSR is not designed for direct access, and many of the advantages of NSR are not available for direct access. You should consider Local Shared Resources (LSR) for the below VSAM data sets (perhaps using System Managed Buffers to facilitate the use of LSR). The I/O RATE is for the time the data set was open. The SMF TIME STAMP and JOB NAME are from the last record for the data set. .. I/O OPEN -ACCESS TYPE (PCT)- SMF TIME STAMP JOB NAME VSAM DATA SET .. RATE DURATION SEQUENTIAL DIRECT 13:19,19SEP2002 NRXX807. SDPDPA.PK.MVSP.RT.NDMGIX.DATA............... 8.4 0:07:08 0.0 100.0 13:19,19SEP2002 NRXX807. SDPDPA.PR.MVSP.RT.NDMGIXD.DATA.............. 11.2 0:06:42 0.0 100.0 13:33,19SEP2002 TSJHM... SDPDPA.PR.MVSP.RT.NDMRQFDA.DATA............. 0.3 2:21:58 0.0 100.0 13:33,19SEP2002 TSJHM... SDPDPA.PR.MVSP.RT.NDMRQF.DATA............... 2.8 3:37:53 0.0 100.0 13:33,19SEP2002 TSJHM... SDPDPA.PK.MVSP.RT.NDMTCF.DATA............... 11.1 6:24:10 0.1 99.9
![Page 28: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/28.jpg)
28©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
DASD Component(Application Analysis)
•Requires simple modification to MXG or MICS
• Modification collects job step data while processing SMF Type 30 (Interval) records
• Typically requires less than 10 cylinders
• Data is correlated with Type 74 information
•CPExpert associated performance problems tospecific applications (jobs and job steps)
•CPExpert can perform “Loved one” analysis ofDASD performance problems
![Page 29: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/29.jpg)
29©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WMQ ComponentAnalyzes SMF Type 115 statistics, as processedby MXG or NeuMICS and placed into performancedata base.
• MQMLOG - Log manager statistics
• MQMMSGDM - Message/data manager statistics
• MQMBUFER - Buffer Manager statistics
• MQMCFMGR - Coupling Facility Manager stats
Type 115 records should be synchronized withSMF interval recording interval.
IBM says overhead to collect accounting data is negligible.
![Page 30: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/30.jpg)
30©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WMQ ComponentOptionally analyzes SMF Type 116 accounting
data, as processed by MXG or NeuMICS and
placed into performance data base.
• MQMACCTQ - Thread-level accounting data
• MQMQUEUE- Queue-level accounting data
Type 116 records should be synchronized withSMF interval recording interval.
IBM says overhead to collect accounting data is 5-10%
![Page 31: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/31.jpg)
31©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQTypical queue manager problems
Assignment of queues to page sets
Assignment of page sets to buffer pools
Queue manager parameters
Index characteristics of queues
Characteristics of messages in queues
Characteristics of MQ calls
CPExpert analysis uses SMF Type 116 records
![Page 32: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/32.jpg)
32©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQTypical buffer manager problems
Buffer thresholds exceeded for pool
Buffers assigned per pool (too few/too many)
Message traffic
Message characteristics
Application design
CPExpert analysis uses SMF Type 115 records
![Page 33: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/33.jpg)
33©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQTypical log manager problems
Log buffers assigned
Active log use characteristics
Archive log use characteristics
Tasks backing out
System paging of log buffers
Excessive checkpoints taken
CPExpert analysis uses SMF Type 115 records
![Page 34: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/34.jpg)
34©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQTypical DB2-interface problems
Thread delays
DB2 server processing delays
Server requests queued
Server tasks experienced ABENDs
Deadlocks in DB2
Maximum request queue depth was too large
CPExpert analysis uses SMF Type 115 records
![Page 35: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/35.jpg)
35©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQTypical Shared queue problems
Structure was full
Large number of application structures defined
MINSIZE is less than SIZE for CSQ.ADMIN
SIZE is more than double MINSIZE
ALLOWAUTOALT(YES) not specified
FULLTHRESHOLD value might be incorrect
CPExpert analysis uses SMF Type 115 records and Type 74 (Coupling Facility) records
![Page 36: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/36.jpg)
36©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQ – sample reportRULE WMQ100: MESSAGES WERE WRITTEN TO PAGE SET ZERO
More than 0 messages were written to Page Set Zero during the intervals shown below. Messages should not be written to Page Set Zero, since serious WebSphere MQ system problems could occur if Page Set Zero should become full. This finding relates to queue SYSTEM.COMMAND.INPUT
MESSAGES WRITTEN STATISTICS INTERVAL TO PAGE SET ZERO 13:16-14:45, 28AUG2003 624
RULE WMQ122: DEAD.LETTER QUEUE IS INAPPROPRIATE FOR PAGE SET ZERO
Buffer Pool 0. The DEAD.LETTER queue was assigned to Page Set Zero. A dead-letter queue stores messages that cannot be routed to their correct destinations. If the DEAD-LETTER queue grows large unexpectedly, Page Set Zero can become full, and WebSphere MQ can enter a serious stress condition. You should redefine the DEAD.LETTER queue to a page set other than Page Set Zero. This finding relates to queue SYSTEM.DEAD.LETTER.QUEUE
![Page 37: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/37.jpg)
37©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQ – sample reportRULE WMQ110: EXPYRINT VALUE IS OFF OR TOO SMALL
Buffer Pool 3. There were more than 25 expired messages skipped when scanning a queue for a specific message. Processing expired messages adds both CPU time and elapsed time to the message processing. With WebSphere 5.3, the EXPYRINT keyword was introduced to allow the queue manager to automatically determine whether queues contained expired messages and to eliminate expired messages at the interval specified by the EXPYRINT value. This finding applies to queue: DPS.REPLYTO.RCB.IVR04
GET BROWSE EXPIRED MESSAGES STATISTICS INTERVAL SPECIFIC SPECIFIC PROCESSED 13:41-13:41, 03JUL2003 0 0 313
RULE WMQ320: APPLICATIONS WERE SUSPENDED FOR LOG WRITE BUFFERS
Applications were suspended while in-storage log buffers are being written to the active log. This finding normally means that too few log buffers were assigned. However, the finding could mean that there is an I/O configuration problem and the log buffer writes to the active log are delayed for I/O reasons. This finding applies to the following statistics intervals.
NUMBER OF SUSPENSIONS STATISTICS INTERVAL WAITING ON OUTPUT BUFFERS 14:19-14:44, 12SEP2003 139
![Page 38: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/38.jpg)
38©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQ – sample reportRULE WMQ201: BUFFER POOL ENCOUNTERED SYNCHRONOUS (5%) THRESHOLD
Buffer Pool 0. This buffer pool encountered the Synchronous Write threshold (less than 5% of the pages in the buffer pool were "stealable" or more than 95% of the pages were on the Deferred Write queue). While the Synchronous Page Writer is executing, updates to any page cause the page to be written immediately to the page set (the page is not placed on the Deferred Write Queue, but is written immediately to the page set as a synchronous write operation). This situation harms performance of applications, and is an indicator that the buffer pool is in danger of encountering a Short on Storage condition.
BUFFERS TIMES AT IMMEDIATE STATISTICS INTERVAL ASSIGNED 5% THRESHOLD WRITES 17:08-17:09, 07OCT2003 1,050 19 19
RULE WMQ205: HIGH I/O RATE TO PAGE SETS WITH SHORT-LIVED MESSAGES
Buffer Pool 0. This buffer pool had short-lived messages assigned. The total I/O rate (read and write activity) to page sets for the short-lived messages was more than 0.5 pages per second. Writing pages to the page set and subsequently reading the pages from the page set cause I/O overhead and delay to the application. This finding applies to the following intervals:
BUFFERS PAGES PAGES I/O RATE STATISTICS INTERVAL ASSIGNED WRITTEN READ WITH DASD 11:32-11:32, 24JUL2006 50,000 101 0 50.5
![Page 39: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/39.jpg)
39©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
WebSphere MQ – sample reportRULE WMQ300: ARCHIVE LOGS WERE USED FOR BACKOUT
WebSphere MQ applications issued log reads to the archive log file for backout more than 0 times during the WebSphere MQ statistics intervals shown below. Most log read requests should come from the output buffer or the active log. Using archive logs for backout purposes often indicates that either the active log files were too small or long-running applications were backing out work. NUMBER OF LOG READS STATISTICS INTERVAL FROM ARCHIVE LOG 4:30- 5:00, 12SEP2003 192
RULE WMQ611: LARGE NUMBER OF APPLICATION STRUCTURES WERE DEFINED
SMF TYPE74 (Structure) statistics showed that more than 5 application structures were defined to a coupling facility. IBM suggests that you should have as few application structures as possible. Having multiple application structures in a coupling facility can degrade performance.
WEBSPHERE MQ COUPLING FACILITY STRUCTURES DEFINED CF1 8 CF2 9 CF3 8
![Page 40: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/40.jpg)
40©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
License fees(Site license)
Components First Year Additional year
WLM Component 7,500 5,000
DB2 Component 7,500 5,000
CICS Component(see note) 5,000 3,000
WMQ Component 5,000 3,000
DASD Component 3,000 1,500
Note: Fees shown for the CICS Component are for analyzing no more than 50 CICS regions.
![Page 41: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/41.jpg)
41©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
CPExpert Release 17.1(Issued April 2007)
Major enhancements with this update:• Provided email alert feature for serious problems
• Provided additional analysis of paging problems
• Provided additional analysis of XCF and coupling facility problems
• Redesigned CICS Component to use wild-card feature (with significant reduction in processing time for large CICS users of CICS Component)
• Provided additional analysis of WebSphere MQ performance problems
![Page 42: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/42.jpg)
42©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
Major enhancements with this update:• Provided support for z/OS Version 1, Release 9
• Provided additional analysis of z/OS performance problems (in WLM Component), including “blocked workload” analysis
• Provided support for CICS/TS Release 3.2 and added significant additional analysis of CICS performance problems
• Provided support for DB2 Version 9.1 and additional analysis of DB2 performance problems
• Provided support for NeuMICS in WebSphere MQ
CPExpert Release 17.2(Issued October 2007)
![Page 43: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/43.jpg)
43©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
Summary
• The major objective is to share solutions and provide insight into new z/OS features.
• CPExpert is updated every six months; support for new versions of z/OS has been available within 30 days after General Availability of the new z/OS release.
• CPExpert is offered at a low cost (affordable by all z/OS shops).
• 45-day trial is available (see license agreement for details).
![Page 44: ©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA 1 An Expert System designed to evaluate IBM z/OS systems.](https://reader030.fdocuments.in/reader030/viewer/2022032804/56649e585503460f94b513bc/html5/thumbnails/44.jpg)
44©Copyright 1998, Computer Management Sciences, Inc., Alexandria, VA www.cpexpert.com
For more information, please contact
Don DeeseComputer Management Sciences, Inc.634 Lakeview DriveHartfield, VA 23071-3113
Phone: (804) 776-7109Fax: (804) 776-7139email: [email protected]
Visit www.cpexpert.com for more information, to review sample output, to review documentation in SAS ODS “point-and-click” format, to download license agreements in .pdf “form” mode, etc.