Jobtrac Conversion – The good, the bad & the ugly.

Presented by Paul Robins

Reason for choosing Control-M

Single Enterprise GUICross datacentre schedule integrationFlexible exception alertingConditional scheduling (e.g. RC based triggers)Server Sysout analysisServer Sysout lifecycle managementViewing sysout via GUI

Volumes…

2,500 Production JCL members converted7,900 Tech (DBA) JCL members converted3,700 zOS job schedules converted600 x-platform (MS-Win) job schedules converted

Peripheral Challenges…Operator TrainingUser Training (~50 users at remote site)Source Control management (SCLM)Lack of testing environmentsInstability in conversion codeIncorrect conversion manualsNo gateway from Non-Prod mainframe to DTLConversion of ‘JCL Build’ automationEvent Management setupRACF setup – User administrationClient rolloutOPSMVS to Control-OChange Moratorium

Tech Challenges – Bugs & EnhancementsISS03523214 – XML output fails validation by deftable utilityISS03534321 – Jobs with DUMMY overlib and INSTREAM JCL=Y are

submitting JCLISS03534344 – Conversion has not produced required MAYBE conditionsISS03542351 – X-Platform command line statements with invalid variable

concatenationISS03548180 – Conversion PREVENT-NCT2 parm overrides product

defaultISS03551469 – JCLLIB and DOCLIB concatentation featureISS03499256 – JOB1 DAXML DD missing, DEFAULTS member

&DEFTYPE parm missingISS03501128 – Multiple abends. Unsupported new features.ISS03505113 – conditions not in JOBNAME-OK format

Tech Challenges – Alternate Member NameCA-Jobtrac provides a feature to provide an alternate

member name to the scheduler event name. This allows JCL member names to be consistent across environments while the event name reflects the different environment prefixes.

e.g.

This is catered for by the conversion by using the INSTREAM JCL feature:

Tech Challenges – JCLLIB ConcatenationCA-Jobtrac offers a JCL and DOC library concatenation

feature to allow non-prod jobs to search up through the development lifecycle for JCL members.

To our surprise this can be replicated in Control-M!Specify the libraries and sequence in IOA PARM

member IOADSNL. This assigns a logical DD which is used as your MEMLIB reference in job schedules. (Wish\exit mod required).

Tech Challenges JCL TruncationIn many cases Control-M auto-edit concatentation is more

verbose than CA-Jobtrac JMF concatenation.Manual intervention may be required to remediate truncation

issues.Use a JCL reformatter to convert JCL to

well-formed JCL prior to conversion (e.g. ASG-JOB/SCAN).

Beware IOA Col 72 – 80 processing changes introduced in 6.3.04

Tech Challenges – Additional JCL changesThe CA-Jobtrac JCL members did not have any JOBLIB

statements. This was inserted by Jobtrac during submission.

The CA-Jobtrac CA11NR DD statements were not converted to CTRNORST by the conversion utility.

ASG-JOB\SCAN was used to make these changes

Tech Challenges – Temporary JCL overridesCA-Jobtrac provides an excellent JCL override feature

whereby a temporary member is allocated for changes to be applied to, which is only associated with the instance of the job on the AJF. This reduces risk that temp changes may be referenced by subsequent job orders.

Due to use of the INSTREAM JCL feature the standard OVERLIB is not available. This could however be built into the JCLLIB concatenation if required.

Tech Challenges: DON’T CORRUPT THE DATELIB!Jobtrac variables are held in a PDS

member which can be converted do a Control-M variable member.

Jobtrac variable members can contain referrals to other variable members.

e.g. Member JMFSYM01 contains the statement: JLOC002:JOBTRAC.PROD.DATELIB(JMFSYM02)

The CONTROL-M conversion utility will convert ALL MEMBERS referred to in this manner.

If you take a copy of the member for conversion, remove all referrals to other members.

Tech Challenges – CPM supportCA-Jobtrac has a facility called Critical Path Manager (CPM)

which is similar to Control-M\BIMConversion of CPM alerting is not included in the BMC

utilities. This is a manual task.

Tech Challenges: NCT2 PreventionJobtrac\CA11 has the capability to perform NCT2 prevention

by deleting duplicate DSNs prior to job execution.IMHO, NCT2 prevention is Bad.If NCT2 prevention has been in place for an extended period

of time, you may find that you have a large number of failures post-conversion unless you turn it on in Control- M.

Tech Challenges: Reserved Variable UsageThere is a possibility that Jobtrac JMF variables, when

converted to autoedit variables, will match Control-M reserved system variables.

Any such variables will need to be remediated manually.

e.g. JMF variable &:RDATE is a user defined variable. Autoedit %%RDATE is a reserved system variable.

Tech Challenges – MAXWAIT valueThe conversion will only provide for a default MAXWAIT

value.This may not be suitable for your entire job library (e.g.

Cyclic jobs will probably be MAXWAIT 0).Select the most prevalent value for your jobs and retrofit any

exceptions.

Tech Challenges – DUMMY jobs default to MFIn Jobtrac there is nothing to identify what platform a Dummy

job should run on.The conversion tool defaults Dummy jobs to mainframe

definitions.This can result in x-platform job streams having mainframe

jobs in the middle.If only converting x-platform jobs, this could result in jobs

being lost.

Tech Challenges – Cross Data Centre job streamsThe conversion tool is unable to identify conditions that need

to be passed between data centres.Because of the prevalence of cross data centre job streams

in our schedules we chose to make ALL conditions Global conditions.

This will need to be reviewed at a later date due to the amount of gateway traffic it creates.

Tech Challenges – Jobtrac commentsIf a Jobtrac job definition contains a comment in the middle

the conversion utility will produce two schedule definitions. One definition was usually found to be invalid and would fail.

Tech Challenges: Shout Severity RetrofitJobtrac does not provide a facility for shout severities.A REXX intercepted all job alerts and downgraded alerts

based on an ‘exclusion list’.The conversion process only provides for 1 default severity.The default shout severity was set to ‘V’ and manually

downgraded according to the exclusion list.

Tech Challenges – Inconsistent Start Day Of WeekDue to a legacy setup issue, our site has a different Start

Day Of Week for Control-M\zOS and Control-M\Server.When converting Windows jobs the conversion tool

references the Mainframe IOA parms for the Start Day of Week.

This was overcome by setting up an alternate IOA Parm and alternate conversion JCL with the Server date setting.

Tech Challenges – Missing GROUP statementsWhen converting Jobtrac schedules that contain Non-

Mainframe job definitions, all definitions will need to be converted to XML and loaded through Enterprise Manager.

The resultant XML file did not have the required GROUP statements. These were added in manually via a mass update of the JOB tag.

Tech Challenges – Condition naming standardsThe Jobtrac conversion utility will probably not deliver

condition names that meet your site standard.These will have to be modified.BMC provided a PTF to produce conditions in the format

JOBNAME-OK which we easily modified to %%GROUP- JOBNAME-OK during the XML phase of the conversion.

When performing mass changes of conditions be sure to consider Maybe conditions and Globals (if exist).

Tech Challenges – !MANUAL conditionsAny IN conditions that are not matched to OUT conditions

during the conversion process are flagged with a !MANUAL prefix.

These conditions need to be remediated.For this reason it is advisable not to break down the

schedule conversion too much.

Tech Challenges – LATESUB shoutsFor any job that has a FROM TIME value, the conversion

will produce a LATESUB shout on the job.The LATESUB value will match the FROM TIME value.This can result in erroneous alerts and should be modified.

Tech Challenges – ctmdefine errorThere is a known bug that the EM incorrectly flags

definitions containing both DATES and WDAYS/WCAL parameter as invalid.

Contact BMC Support for recommended workaround.

Tech Challenges: Agent installationIf your Jobtrac installation is running x-platform jobs you

must identify all Nodes that jobs are submitted to and install a Control-M Agent.

Ensure that the Agent has the same permissions, or runs under the same user, as the Jobtrac jobs.

Tech Challenges – Newday timeIn order to standardise your site you may need to run

Control-M with a different ‘Newday’ time to Jobtrac.If this is the case the conversion process should cater for

any shifts in scheduling days.However, you will need to carefully consider the running of

jobs at cutover time between the end of the last Jobtrac day and the start of the first Control-M day.

Tech Challenges – APIsWe have a number of ISPF and REXX processes using the

Jobtrac API that have been converted to Control-M.Discovery of these processes can be problematic.During the discovery phase of your conversion, ensure that

Control-M can replicate all of the functions that Jobtrac allows via API.

In some cases API functions were converted to Control-O functions.

Tech Challenges – Interact with DesktopWe have a number of legacy Windows jobs which have

been designed to raise ‘pop-up’s to the server console when an error is encountered.

Under our default install of Control-M\Desktop these errors are not visible via MS RDP.

We are currently investigating implementing the Control-M ‘Interact With Desktop’ capability to provide this function.

Tech Challenges – MASTER conversionAs well as providing a ‘User Daily’ style facility, Jobtrac also

provides a ‘MASTER’ schedule definition which can contain scheduling data for job scheduling tables.

The contents of MASTER are converted into STC schedules which submit the CTMJOBPR utility to order the tables.

We chose to deschedule these and add the tables to standard User Daily jobs.

Tech Challenges – Scheduling with no single point of truthWe discovered with our Jobtrac definitions, that the individual job

definition did not reflect the truth about scheduling of the job.e.g.JOBA might have a definition that is scheduled 7 days per week.However this job might be added to the AJF by User Daily 1 on

weekdays, and by User Daily 2 on Saturdays, and not run on Sundays.

This situation required significant remediation.

Tech Challenges – Job Schedule CorruptionISS03649976Due to the previous mentioned usage of INSTREAM JCL we

encountered a bug in EM whereby mainframe schedules can be corrupted during Download to EM.

This resulted in jobs failing with NOMEM errors.This has the potential to impact every job in a schedule

table.See BMC Support for the fix.

Tech Challenges – X-Platform docmemsAs Jobtrac schedules x-platform jobs from the mainframe,

the documentation members also reside on the mainframe.

Members will need to be downloaded to a location where the Control-M\Agent can read them.

The conversion will populate the DOCLIB field with a single default value specified at conversion time.

Tech Challenges – X-platform Maybe conditionsIt is important to remember that

when you are running Maybe condition processing on any given platform, it will not ‘see’ jobs on the AJF of another data centre.

This was overcome by defining dummy jobs on the predecessor platform to action the Maybe dependency and pass a standard condition to the successor data centre.

Recommendations

• Run a Proof of Concept first• Ensure that you have sufficient Jobtrac and Control-M

skills engaged with the project. Perform additional training of project staff prior to conversion if required.

• Engage with BMC. Ensure you have the correct levels of software and manuals. Request any known issues. Ensure that the conversion team are aware of the project.

• New BMC ‘amigo’ initiative.• Get Control-M\Forecast!