Hsm 34

52
SHARE 109 Session 2817 San Diego, CA 17 August 2007 Ed Jaffe, [email protected] Brian Peterson, [email protected] Sam Knutson, [email protected] Skip Robinson, [email protected] Bit Bucket x Bit Bucket x 22 22

Transcript of Hsm 34

Page 1: Hsm 34

SHARE 109Session 2817San Diego, CA17 August 2007

Ed Jaffe, [email protected] Peterson, [email protected]

Sam Knutson, [email protected] Skip Robinson, [email protected]

Bit Bucket xBit Bucket x’’2222’’

Page 2: Hsm 34

The “Weakest” Link

Page 3: Hsm 34

IBMLink Access From Web – Problematic

• Access to VM IBMLink – 3270 aka “environmentally-friendly (green) screen”interface – was scheduled for removal after March 2007.

• Meanwhile, the web-based IBMLink interface suffered many lengthy and frustrating outages.

• Search IBM-MAIN archives if you want some of the gory details.• http://bama.ua.edu/archives/ibm-main.html• The biggest issues appear to be with IBM’s web authentication system.

• IBM extended VM IBMLink availability through May 2007.

• Meanwhile, the web-based IBMLink interface suffered many lengthy and frustrating outages (see above).

• IBM “temporarily” re-activated VM IBMLink access sometime betweenJune 3 and June 10.

• Meanwhile, the web-based IBMLink interface suffered many lengthy and frustrating outages (see above).

• Last word was that VM IBMLink availability will continue indefinitely.• This probably means until the web stuff can be made to work reliably.

Page 4: Hsm 34

Accessing VM IBMLink

• Many people mistakenly believe that when their SNA-based access to the IIN (later ATT) infrastructure disappeared they could no longer access VM IBMLink.

• IBMLink has had a TCP/IP presence for many years.• Small companies, like the one for which I currently work,

used it to replace the old “Passport” dial-up access.

• Access is via ibmlink.advantis.com (204.146.168.209) using standard telnet port 23

• Valid clients are any PC-based emulator supporting TN3270 or built-in z/OS telnet client invoked from TSO/E command prompt.

Page 5: Hsm 34

Parameters for PC-based TN3270 Client (In this case PCOMM)

Page 6: Hsm 34

Connected via Pc-based TN3270 Client

Page 7: Hsm 34

Connecting via TELNET Command under TSO/E

Page 8: Hsm 34

Connecting via TELNET Command under TSO/E

Page 9: Hsm 34

Connecting via TELNET Command under TSO/E

Page 10: Hsm 34

Connecting via TELNET Command under TSO/E

Page 11: Hsm 34

Required Credentials for IBMLink Logon

• VM IBMLink:• Account (8 byte value)• Userid (8 byte value)• Password (8 byte value)

• Web-based IBMLink:• Userid (long value – usually your email address)• Password (long value)

• IBM’s identity mapping products (Tivoli?) used under the covers to map these credentials together.

• If you’ve forgotten your original VM IBMLink credentials (or never had them to begin with), you must contact IBM to get them re-assigned. Your designated site administrator for IIN may need to be involved.

Page 12: Hsm 34

Unicode for the Smart and LazyRedux

Page 13: Hsm 34

Unicode for the Smart and Lazy Redux

• At SHARE in Tampa, explained Unicode on Demand

• Earlier Unicode update provided prebuilt Unicode image in SYS1.SCUNIMG by APAR OA04069 (January 2004)

• Used by DB2 V8 if • SYS1.SCUNIMG in link list • Unicode not already initialized

• This is good but it seems to waste some real storage

• z/OS 1.7 introduces "Unicode on Demand" (but, TCB mode only)

• APAR OA14231 (July 2006) extended z/OS 1.7 support to SRB mode - "the rest of the story"

• "Unicode on Demand" builds the exact translation tables a calleruses so it is no longer necessary to correctly predict which translation tables your applications will need

Page 14: Hsm 34

Using pre-built Unicode conversion image

D UNI,ALL CUN3000I 12.53.03 UNI DISPLAY 944 ENVIRONMENT: CREATED 08/12/2007 AT 10.18.26

MODIFIED 08/12/2007 AT 10.21.55 IMAGE CREATED 06/15/2004 AT 13.46..49

SERVICE: CHARACTER CASE NORMALIZATION COLLATION STORAGE: ACTIVE 10352 PAGES

LIMIT 524287 PAGES CASECONV: NONE NORMALIZE: DISABLED NORM VER: NONE COLLATE: DISABLED

COLL RULES: NONE CONVERSION: 25520-01200-R 00367-05123-R

25520-01208-R 05353-01208-R 33268-01200(13488)-R 01200(17584)-01100-E

40M of storage 2640 conversion tables loaded in

the Unicode pre-built conversion

image

Page 15: Hsm 34

Unicode for the Smart and Lazy Redux

On IBM-MAIN a couple of weeks ago, Kenneth Tomiak pointed out an obvious answer

BROWSE SYS1.PARMLIB(CUNUNI00) - 01.00 Command ===>

***************************** Top of Data ** ADD,FROM=37,TO=1208,TECH=R **************************** Bottom of Data

• add UNI=00 to SYS1.PARMLIB(IEASYS00)

Page 16: Hsm 34

Unicode for the Smart and Lazy Redux

D UNI,ALL CUN3000I 12.52.14 UNI DISPLAY 875 ENVIRONMENT: CREATED 08/12/2007 AT 12.18.54

MODIFIED 08/12/2007 AT 12.23.14 IMAGE CREATED --/--/---- AT --.--..--

SERVICE: CHARACTER CASE NORMALIZATION COLLATION STRINGPREP BIDI

STORAGE: ACTIVE 74 PAGES LIMIT 524287 PAGES

CASECONV: NONE NORMALIZE: DISABLED NORM VER: NONE COLLATE: DISABLED

COLL RULES: NONE STRPROFILES: NONE CONVERSION: 01200(13488)-00037-E 01208-00037-E

00037-00367-E 00037-01208-R 00367-00037-E 00037-01200(13488)-R

6 conversion tables which are actually used are loaded! Only 300K of

storage used

Page 17: Hsm 34

No Reservations Required

Page 18: Hsm 34

Blue Cross GRS Conversion of all RESERVE

• Conversion for Blue Cross of Minnesota took place in early 2004 from third party serialization product, to GRS Star

• In our "from" environment, we were already converting SYSIGGV2, SYSVTOC and SYSVVDS to cross-system enqueues to resolve lockouts

• Very expensive to convert busy reserves to enqueues in anything other than GRS Star

• However, made conversion to GRS Star easySysplex = GRSPlex = DASDPlex = SMSPlex More complex, or not possible, if not "="

• PARMLIB GRS RNL List (GRSRNL00) • RNL(EXCL) convert SCOPE=SYSTEMS to SCOPE=SYSTEM • RNL(INCL) convert SCOPE=SYSTEM to SCOPE=SYSTEMS • RNL(CON) convert RESERVE to SCOPE=SYSTEMS

Page 19: Hsm 34

GRS RNL List today all RESERVEs converted

D GRS,ALL ISG343I 09.33.03 GRS STATUS 584

LIST TYPE QNAME RNAME INCL GEN SIBIXFP INCL GEN SYSDSN INCL GEN SYSIKJBC INCL GEN SYSZVOLS EXCL SPEC SYSDSN SYS1.LOGREC EXCL GEN SYSDSN SYS1.MAN EXCL SPEC SYSDSN SYS1.STGINDEX CON PATT *

Page 20: Hsm 34

The Conversion

• All systems IPL • Stop using third party serialization product • IPL one LPAR in GRS Star mode • Verify serialization works • Verify CF structure allocated • IPL second LPAR, verify... • IPL rest of Sysplex

Page 21: Hsm 34

GEICO GRS Conversion of all RESERVE

• Why convert all RESERVEs?• IBM best practice reflected in Health Checker for z/OS• RESERVE is an inhibitor to using GDPS, Hyperswap, LDMF, and

likely other things I don’t know about• RESERVE against large volumes (3390-9, 3390-27, 3390-54)

can be a problem and is an inhibitor to volume consolidation.CHECK(IBMGRS,GRS_CONVERT_RESERVES) ISGH0307E Not all RESERVEs are being converted. Explanation: Check found an exception. No entry such as

"RNL(CON) TYPE(PATTERN) QNAME(*)" was found, therefore RESERVEs are not currently being converted by default. For improved performance, availability, and serviceability, IBM suggests that a star complex convert all RESERVEs to global ENQs using the RESERVE Conversion Resource Name List. Note that it is not always possible to convert all RESERVEs.

System Action: None. Operator Response: Contact your system programmer.

Page 22: Hsm 34

GEICO GRS Conversion of all RESERVESystem Programmer Response: Investigate whether all RESERVEs can

be converted to ensure data integrity is maintained. This check should be disabled if certain RESERVEs cannot be converted, for example when the volume is being shared outside of the sysplex. For futher information on converting RESERVEs and for suggestionon what RESERVEs should or should not be converted see z/OS MVS Planning:Global Resource Serialization. Also check with individual applications and components to see if the RESERVEs they issue can safely be converted.

The Global Resource Serialization ENQ/DEQ/RESERVE Monitor can help determine what RESERVEs are being issued on the system. See z/OSMVS Planning: Global Resource Serialization for more information on the Global Resource Serialization ENQ/DEQ/RESERVE Monitor. Source: IBM Global Resource Serialization Reference Documentation: z/OS MVS Planning: Global Resource Serializationz/OS MVS Initialization and Tuning Reference

Check Reason: When in STAR mode, converting RESERVEs can help improve performance and avoid deadlock.

Page 23: Hsm 34

GEICO GRS Conversion of all RESERVE

• Conversion for GEICO completed in July 2007 after two years of interim steps planned to reduce risk

• Highly risk averse environment• GRS used since the 1980’s long since converted to GRS Star • Like Blue Cross our Sysplex = GRSPlex = DASDPlex = SMSPlex• GRS RNL lists had many artifacts including some definitions for

QNAMEs no longer in use or added for reasons no longer clear• Each QNAME was researched and the results documented• A canonical source was sought for every change to confirm that

the related product or component would operate correctly• Changes were then planned with the involvement of stakeholders• Changes implemented to convert RESERVE for specific products

and remove exceptions from RNL lists• IBM GRS Monitor used for data gathering• Leverage two annual DRP exercises to fully vet significant changes

in advance of implementation in the real world

Page 24: Hsm 34

GEICO GRS Conversion of all RESERVE

• IBM and OEM product teams do not all understand or concur on this recommendation to convert all RESERVE to global enqueue

• A number of mainstream OEM product support teams and developers do not understand the way that serialization works onz/OS

• Caveat: Findings and actions taken to change our serializations settings are shared for your information only. They are NOT intended for you to take as technical advisement. You alone areresponsible for the integrity and correctness of serialization in your environment. You should carefully research, plan, and testany changes to your environment. Incorrectly configured GRS parameters can results in an outage or data corruption.

• ManagementSpeak: This is a "best practice.“Translation: This is something I want to do. Source : Bob Lewis http://www.issurvivor.com/kjrArchive.asp

• I think that the effort to document and cleanup your serialization configuration and possibly convert all RESERVEs is worthwhile and will likely provide more benefits in the future

Page 25: Hsm 34

GRS RNL Lists circa 2006 INCLUDES

D GRS,ALL ISG343I 12.01.00 GRS STATUS 014

LIST TYPE QNAME RNAME INCL GEN CTLIELEM INCL GEN CTLIMSTR INCL GEN CTLIPROC INCL GEN ENDEVOR INCL GEN IAMENQ INCL GEN IGWSYSZ0 INCL GEN IGWSYSZ1 INCL GEN SPFEDIT INCL GEN SYSDSN INCL GEN SYSIGGV1 INCL GEN VLAM-MDB

Investigating with CA-ENDEVOR support if these

can be removed.

Artifacts of original IBM PDSE implementation no longer required technote created to document this

"SYSZIGW0 and SYSZIGW1: PDSE Serialization“ at

www.ibm.com/support

Page 26: Hsm 34

GRS RNL Lists circa 2006 INCLUDES

D GRS,ALL ISG343I 12.01.00 GRS STATUS 014

LIST TYPE QNAME RNAME INCL GEN CTLIELEM INCL GEN CTLIMSTR INCL GEN CTLIPROC INCL GEN ENDEVOR INCL GEN IAMENQ INCL GEN IGWSYSZ0 INCL GEN IGWSYSZ1 INCL GEN SPFEDIT INCL GEN SYSDSN INCL GEN SYSIGGV1 INCL GEN VLAM-MDB

These we needed to keep

Page 27: Hsm 34

GRS RNL Lists circa 2006 INCLUDES

D GRS,ALL ISG343I 12.01.00 GRS STATUS 014

LIST TYPE QNAME RNAME INCL GEN CTLIELEM INCL GEN CTLIMSTR INCL GEN CTLIPROC INCL GEN ENDEVOR INCL GEN IAMENQ INCL GEN IGWSYSZ0 INCL GEN IGWSYSZ1 INCL GEN SPFEDIT INCL GEN SYSDSN INCL GEN SYSIGGV1 INCL GEN VLAM-MDB

IAM current levels do not require documentation

updated. Removed during CICS/BATCH outage

Kept old Image Sciences document software still

researching

Page 28: Hsm 34

GRS RNL Lists circa 2006 EXCLUDES

EXCL SPEC $AVRS SERIALIZE.$AVRS.LOCK.FILE EXCL GEN ARCAUDIT EXCL GEN ARCBACK EXCL GEN ARCGPA EXCL GEN ARCGPAL EXCL GEN ARCSDPDS EXCL GEN ARCUPDT EXCL GEN SYSCTLG EXCL SPEC SYSDSN HSM.JRNL EXCL GEN SYSDSN IMSPR1V.PROD.DBRC EXCL GEN SYSDSN IMSPR2V.PROD.DBRC EXCL GEN SYSDSN IMSPR3V.PROD.DBRC EXCL SPEC SYSDSN PASSWORD EXCL SPEC SYSDSN SYS1.DCMLIB

Updated per SEA documentation

Page 29: Hsm 34

$AVRS QNAME researched

• $AVRS install guide and sample definitions clearly documented the current recommendations

• $AVRS QNAME was an artifact no longer used HW RESERVE was being done under different QNAME

• Removed RNLDEF RNL(EXCL) TYPE(SPECIFIC) QNAME($AVRS) RNAME(SERIALIZE.$AVRS.LOCK.FILE)

• Removed RNLDEF RNL(CON) TYPE(GENERIC) QNAME($AVRS) • Added these four entries from $AVRS.INSTLIB(GRSDEF)/* ADDS THE RNAMES. NOTE: THERE ARE 4 ENQUES FOR 5.1B

AND HIGHER. */ RNLDEF RNL(CON) TYPE(GENERIC) QNAME(ENQ$AVRS) RNLDEF RNL(CON) TYPE(GENERIC) QNAME($AVRACTV) RNLDEF RNL(CON) TYPE(GENERIC) QNAME($AVMAINT) RNLDEF RNL(CON) TYPE(GENERIC) QNAME($ELRCLEN)

Page 30: Hsm 34

GRS RNL Lists circa 2005 EXCLUDES

EXCL SPEC $AVRS SERIALIZE.$AVRS.LOCK.FILE EXCL GEN ARCAUDIT EXCL GEN ARCBACK EXCL GEN ARCGPA EXCL GEN ARCGPAL EXCL GEN ARCSDPDS EXCL GEN ARCUPDT EXCL GEN SYSCTLG EXCL SPEC SYSDSN HSM.JRNL EXCL GEN SYSDSN IMSPR1V.PROD.DBRC EXCL GEN SYSDSN IMSPR2V.PROD.DBRC EXCL GEN SYSDSN IMSPR3V.PROD.DBRC EXCL SPEC SYSDSN PASSWORD EXCL SPEC SYSDSN SYS1.DCMLIB

DFHSM had many entries in GRS RNL list which we were eventually able to

remove

Page 31: Hsm 34

DFSMShsm QNAMEs researched

• DFSMShsm has several different well documented options for serialization see Chapter 11. DFSMShsm in a Multiple-Image Environment in the z/OS DFSMShsm Implementation and Customization Guide.

• CDSQ=YES|NO SERIALIZE CDSs WITH GLOBAL ENQUEUES• CDSR=YES|NO SERIALIZE CDSs WITH VOLUME RESERVES• CDSSHR=YES|NO|RLS SPECIFY CDS SHARABILITY• We were already running DFHSM with CDSQ=YES, CDSR=NO,

CDSSHR=YES so we were puzzled to see RESERVE activity from DFHSM recorded

• Consulting IBM we found that DFHSM would still perform RESERVE on the DFHSM Journal

• Most of what was in GRSRNL00 was artifact and so all the old unneeded customization was removed from GRSRNL00 and conversion for QNAME ARCGPA was added during a DFHSM outage prior to the Sysplex outage to convert all RESERVEs

Page 32: Hsm 34

DFSMShsm QNAMEs researched

ARCZCDQR is a general purpose cleanup module that checks for reserves and shared enqueues that might still exist after HSM has completed a process. It first checks for a reserve and releases it if one is held.

It then checks for a shared enqueue and releases it if it's held. We check both conditions because a CDSR=Y, CDSQ=Y environment is permissable.

You will still see these TEST RESERVE no matter you are using CDSQ or CDSR. The only true RESERVE when running CDSQ is when the CDS backup is running, RESERVE will put on the journal dataset in this situation.

Regarding your journal definition: RNLDEF RNL(EXCL) TYPE(SPECIFIC) QNAME(SYSDSN)

RNAME(HSM.JRNL) This is not required since HSM will use resource MAJOR(ARCGPA)

MINOR(ARCJRNL). When CDS backup runs, this resource is getting RESERVE by default but you can change to global ENQ if you want.

Page 33: Hsm 34

GRS RNL Lists circa 2006 EXCLUDES

EXCL SPEC $AVRS SERIALIZE.$AVRS.LOCK.FILE EXCL GEN ARCAUDIT EXCL GEN ARCBACK EXCL GEN ARCGPA EXCL GEN ARCGPAL EXCL GEN ARCSDPDS EXCL GEN ARCUPDT EXCL GEN SYSCTLG EXCL SPEC SYSDSN HSM.JRNL EXCL GEN SYSDSN IMSPR1V.PROD.DBRC EXCL GEN SYSDSN IMSPR2V.PROD.DBRC EXCL GEN SYSDSN IMSPR3V.PROD.DBRC EXCL SPEC SYSDSN PASSWORD EXCL SPEC SYSDSN SYS1.DCMLIB

z/OS R7 GRS Planning Guide mentioned as having been removed obsolete long

before that

Removed as we did not have .DCMLIB

Page 34: Hsm 34

GRS RNL Lists circa 2006 EXCLUDES

EXCL SPEC $AVRS SERIALIZE.$AVRS.LOCK.FILE EXCL GEN ARCAUDIT EXCL GEN ARCBACK EXCL GEN ARCGPA EXCL GEN ARCGPAL EXCL GEN ARCSDPDS EXCL GEN ARCUPDT EXCL GEN SYSCTLG EXCL SPEC SYSDSN HSM.JRNL EXCL GEN SYSDSN IMSPR1V.PROD.DBRC EXCL GEN SYSDSN IMSPR2V.PROD.DBRC EXCL GEN SYSDSN IMSPR3V.PROD.DBRC EXCL SPEC SYSDSN PASSWORD EXCL SPEC SYSDSN SYS1.DCMLIB

PRODUCTION IMS RECONSRemoved after discussion with

Systems IMS team

Page 35: Hsm 34

GRS RNL Lists circa 2006 more EXCLUDES

EXCL GEN SYSDSN SYS1.DUMP EXCL SPEC SYSDSN SYS1.HASPCKPT EXCL SPEC SYSDSN SYS1.LOGREC EXCL GEN SYSDSN SYS1.MAN EXCL SPEC SYSDSN SYS1.NUCLEUS EXCL GEN SYSDSN SYS1.PAGE EXCL SPEC SYSDSN SYS1.STGINDEX EXCL SPEC SYSDSN SYS1.SVCLIB EXCL SPEC SYSDSN SYS1.TCPPARMS EXCL SPEC SYSDSN SYS1.VTAMLST EXCL SPEC SYSDSN SYS1.VTAMNCP EXCL GEN SYSVSAM IMSPR1V.PROD.DBRC EXCL GEN SYSVSAM IMSPR2V.PROD.DBRC EXCL GEN SYSVSAM IMSPR3V.PROD.DBRC

JES2 Checkpoint for test JESPlex

converted to CF and then this was removed and SYSZJES2 converted

Removed

Kept for now while we work with our

network team to change their

implementation procedures

Page 36: Hsm 34

GRS RNL Lists circa 2006 more EXCLUDES

EXCL GEN SYSDSN SYS1.DUMP EXCL SPEC SYSDSN SYS1.HASPCKPT EXCL SPEC SYSDSN SYS1.LOGREC EXCL GEN SYSDSN SYS1.MAN EXCL SPEC SYSDSN SYS1.NUCLEUS EXCL GEN SYSDSN SYS1.PAGE EXCL SPEC SYSDSN SYS1.STGINDEX EXCL SPEC SYSDSN SYS1.SVCLIB EXCL SPEC SYSDSN SYS1.TCPPARMS EXCL SPEC SYSDSN SYS1.VTAMLST EXCL SPEC SYSDSN SYS1.VTAMNCP EXCL GEN SYSVSAM IMSPR1V.PROD.DBRC EXCL GEN SYSVSAM IMSPR2V.PROD.DBRC EXCL GEN SYSVSAM IMSPR3V.PROD.DBRC

Removed SMF data sets have used a different naming scheme for some

time

PRODUCTION IMS RECONSRemoved after discussion with

Systems IMS team

Page 37: Hsm 34

GRS RNL Lists circa 2005 yet more EXCLUDES

EXCL GEN SYSZJES2

EXCL GEN TDMFRESV

EXCL GEN UCC7CMDS

Removed and CONVERSION of SYSZJES2 added

Removed we no longer have TDMF installed

Removed and added to CONVERSION after

checking with CA

Page 38: Hsm 34

GRS RNL Lists circa 2005 CONVERSIONS

CON GEN $AVRS CON GEN ADRPRDCT CON GEN ARCBACV CON GEN ARCMIGV CON GEN CTLIELEM CON GEN CTLIMSTR CON GEN CTLIPROC CON GEN DSPURI01 CON GEN ENDEVOR CON GEN IGDCDSXS CON GEN LMRK CON GEN MIMGLOBL

CON GEN SPFDSN CON GEN SPFEDIT CON GEN SPZAPLIB CON GEN SYSIEWLP CON GEN SYSIGGV2 CON GEN SYSVTOC CON GEN SYSZRACF CON GEN SYSZVVDS CON GEN TMSQNAME

“*” will replace all don’t worry

about cleanup too much here

Page 39: Hsm 34

GRS Monitor

• Shipped with z/OS • Documented in MVS Planning: Global Resource Serialization

Chapter 3. Using the ENQ/RESERVE/DEQ monitor tool.• SYS1.SAMPLIB(ISGRUNAU) and other ISG* members• Data was gathered on a recurring basis using IBM GRS Monitor • SOFTEK supplies a RESERVE monitor with LDMF not used in this

exercise.• GRS Monitor can record a LOT of data • A USERMOD is available from GRS Level 2 to limit the data

recorded to RESERVE information• It is possible that this function will be rolled into the GRS

Monitor in a future release of z/OS as a filter option• I use WLM service class SYSSTC to run this because IBM says

“IBM suggests that you run the job at the highest possible dispatching priority in your system. If the monitor is not at a high dispatching priority, virtual storage can become exhausted and ENQ/DEQ processing might impacted.” TIME=1440 too!

Page 40: Hsm 34

Health Check GRS_EXIT_PERFORMANCE

• Guideline: The monitor’s intent is to assist with planning and tuning of RNLs for the implementation of global resource serialization. The monitor uses system resources in order to collect data. IBM recommends that you do not run the monitor continuously during normal production work. While running the monitor you might observe high CPU in address spaces that issue many ENQ/RESERVE/DEQ requests.

• Using the USERMOD I ran the monitor for days at a time with no impact on all LPARs

• Health Check will trip when the GRS Monitor is running this may drive automation at your shop

CHECK(IBMGRS,GRS_EXIT_PERFORMANCE) The Global Resource Serialization ENQ/DEQ/RESERVE Monitor is in use. This could negatively impact system performance. Explanation: Check found an exception. The Global Resource

Serialization ENQ/DEQ/RESERVE Monitor is in use. This could negatively impact system performance.

Page 41: Hsm 34

GRS Monitor Reports

TIME JOBNAME PROGRAM AUT SVC RNL SCOPE VOLSER MAJOR MINOR DATE 2006 316 11 05.39.84 ICOMPROD SASSICSM AL ENQ E*RESERVE 01 UCC701 UCC7CMDS:CMDS 11 05.40.49 CA7PROD UCC7 AL ENQ E*RESERVE 01 UCC701 UCC7CMDS:CMDS 11 05.41.71 TMSS DTMCFMV6 L ENQ RESERVE 01 SYS030 TMRMCF

:SYSOP.THRUPUT.CONTROL 11 05.43.42 TMSS DTMJMQV6 L ENQ RESERVE 01 SYS008 DTMJBMQ

:DSS.THRUPUT.SPOOL SYS00823 07.03.97 DFHSM ARCILOG AL ENQ E*RESERVE 02 SHSM03 ARCGPA :ARCRJRN23 07.04.00 DFHSM# NOT-PRB ENQT E*RESERVE 01 SHSM00 ARCGPA :ARCOCDS

//* SEQUENTIAL TRACE REPORT //REPORT EXEC PGM=ISGAMED1 //INPUT DD DISP=SHR,DSN=U06T03.AUDIT.ASYS.OUT1.A110806//OUTPUT DD SYSOUT=*, // DCB=(LRECL=132,RECFM=FB,BLKSIZE=18348) //SYSUDUMP DD SYSOUT=*

ENQT == RET=TEST

Page 42: Hsm 34

GRS Conversion of all RESERVE implemented

• Sysplex-wide IPL July 22 we completed the last stage removed ALL individual CONVERSION entries and added one pattern entry to convert all RESERVES.

• RNLDEF RNL(CON) TYPE(PATTERN) QNAME(*) /* CONVERT ALL! */

CHECK(IBMGRS,GRS_CONVERT_RESERVES) START TIME: 07/22/2007 00:48:39.576689 CHECK DATE: 20050105 CHECK SEVERITY: LOW All RESERVEs are being converted while in STAR mode. Converting RESERVEs to global ENQs can help avoid deadlocks and

improve reliability, availability, and serviceability. Note thatthere could be some RESERVEs that should not be converted for your installation. For further information see z/OS MVS Planning: Global Resource Serialization.

END TIME: 07/22/2007 00:48:39.578338 STATUS: SUCCESSFUL

Page 43: Hsm 34

GRS Tidbits

Page 44: Hsm 34

GRSQ CONTENTION

CHECK(IBMGRS,GRS_GRSQ_SETTING) GRS dump settings not ideal: GRSQ=LOCAL Explanation: The current setting of GRSQ=LOCAL while in Global

Resource Serialization STAR mode. IBM recommends having a GRSQ=CONTENTION, because this will collect all ENQ resources on the local system, as well as all global ENQ resources that are in contention throughout the entire sysplex. This may also significantly reduce the amount of time and space required for the dump.

Operator Response: Notify the system programmer. System Programmer Response: Change the GRSQ setting to

CONTENTION by issuing SETGRS GRSQ=CONTENTION on all affected systems and update GRSCNFxx GRSQ(x) for subsequent IPL.

Check Reason: IBM recommends a GRSQ setting of contention. • Previously we had set GRSQ to LOCAL to optimize SVC dump

performance for CICS made this change with no perceptible problems

Page 45: Hsm 34

GRS APAR OA11382 DYNAMIC CNS, GRSQ, ENQ MAXIMUMS

APAR Identifier ...... OA11382 Closed ......... 07/03/09NEW FUNCTION - DYNAMIC SETTINGS FOR THE GRS CONTENTION

NOTIFICATION SYSTEM (CNS), GRSQ, AND ADDRESS SPACE ENQ MAXIMUMS.

• PTFs UA33111, UA33112, UA33113 • It is difficult for installations to manage the Contention

Notification System(CNS) for global ENQ resources in Star mode. It is also difficult for installations to manage the concurrent ENQ maximums by address space.

• SPE rolls down new function from z/OS V1R8• Before in z/OS V1R7, you could zap the GVTCREQA and

GVTCREQ in the GVT now you cannot use the new support• GRSQ keyword was added to the GRSCNFxx parmlib member in

z/OS V1R4 via APAR OA0797 and the default was ALL now it is CONTENTION

Page 46: Hsm 34

SETGRS CNS Example

D GRS,ALL <SNIP> GRS STAR MODE INFORMATION LOCK STRUCTURE (ISGLOCK) CONTAINS 16777216 LOCKS. THE CONTENTION NOTIFYING SYSTEM IS ASYS SYNCHRES: YES ENQMAXU: 16384 ENQMAXA: 250000 GRSQ: CONTENTION

SETGRS CNS=BEND,NOPROMPTIEE712I SETGRS PROCESSING COMPLETE

ISG364I CONTENTION NOTIFYING SYSTEM MOVED FROM SYSTEM ASYS TO SYSTEM BEND. OPERATOR COMMAND INITIATED.

Page 47: Hsm 34

INCONSOL(e)ABLE

Page 48: Hsm 34

Living in a one-dimensional world

• In 1.8, one-byte console ids have rolled off the edge of the earth after many dire warnings from IBM

• We’ve had an alert mechanism in place for several releases to unearth miscreants: D OPDATA,TR

• The idea was to locate and fix—or expunge—any application still using one-byte ids

• ISV code or RYO code (never IBM code of course!)• So you think you have everything covered• Then one day you can’t start JES (lonnng story)• You call on your trusty rescuer: S PDSE,SUB=MSTR• Started task version of StarTool (Serena) or the old

shareware PDS command• Oops: you don’t get back any command responses on the

console: major bummer

Page 49: Hsm 34

Living in a one-dimensional world

• S PDSE,SUB=MSTR• ? CONNECTED • ?LISTD 'sys1.proclib’• The sound of silence is deafening…• What we see in syslog (hmm, no JES running)SKIPPER 00000210 ?LISTD 'sys1.proclib‘... UNKNOWN 00000000 IKJ56644I NO VALID TSO USERID…UNKNOWN 00000010 SYS1.PROCLIBUNKNOWN 00000010 --RECFM-LRECL-BLKSIZE-DSORGUNKNOWN 00000010 FB 80 27920 POUNKNOWN 00000010 --VOLUMES--UNKNOWN 00000010 SYSS00

Page 50: Hsm 34

Living in a one-dimensional world

• Ancient code is called upon and found wanting

• This task is used seldom but with great expectation

• You may have this or some other rescue product

• When you need it most is a bad time to hit the one-byte wall

• But 1.8 also provides help for this conundrum

• In PARMLIB CONSOLExx the CONSOLE statement:

• INTIDS{(Y)|(N)} UNKNIDS{(Y)|(N)}

• Can also be set dynamically:

• V CN(skipper),UNKNIDS=Y

• By the by, Serena has fixed this problem

• To be included in 7.6.3 or by PTF in 7.6.2

Page 51: Hsm 34

Acknowledgements

• Christopher Brooker (IBM)

• Nick Matsakis (IBM)

• Scott Fagen (IBM)

• Keith Martens (GEICO)

• Daryl Hoblitz (GEICO)

• Ken Tomiak (Cornerstone)

• Ed Finnell (University of Alabama)

Page 52: Hsm 34

See you in Orlando …