Post on 22-Nov-2014
description
www.bphx.com
Mainframe Technology Overview
March 2008
2
Mainframe Technology Overview
1. BluePhoenix Mainframe Architecture
2. C-Scan
3. Toolbox and IT Discovery
4. Repository Files
5. IT Discovery Database and Query Facility
3
BluePhoenix Mainframe Architecture
Integration of the following:
Methodology – Designed to solve the problem systematically with little disruption to the client.
C-Scan – Conversion engine with its own interpreted language components and exits.
ISPF Panels and REXX – Workflow engine that enforces the methodology and controls processing (batch and online).
Mini-Scheduler – Controls batch job flows.
Toolbox – JCL and programs in C-Scan and REXX for analysis and conversion.
Repository – Set of detailed tables, describing the environment and containing rules for conversion.
4
BluePhoenix Mainframe Architecture
InventoryReports
Generationof Tools
Conversion
Unit Test
System Test
Implementation
Repository Enhancement
Repository FilesSpecific Cluster Metadata
Libraries
Generation Parameters
Libraries
ConvertedLibraries
Implementation Parameters
5
C-Scan
Assembler Interpreter – Provides some of the assembler abilities, such as dynamic file allocation and edit of PDS directory.
Event Driven Engine – Driven by record-by-record automated reading, with logic controlled by events or keys.
Writes Records – Creates records with implicit and explicit location setting of fields; is able to write and execute its own code.
Flexible – Has COBOL-like structures for multiple file processing; Record Merge; Subprogram.
Easy to Learn – Simple and consistent syntax.
6
The Toolbox
Global Assessment/Impact Analysis – Performs inventory definition and analysis for migration or field adjustment projects.
FieldEnabler – Performs conversions of field lengths and formats automatically while protecting the corporate knowledge asset.
COBOL/LE-Enabler – Performs COBOL/LE conversions.
DBMSMigrator – Performs migration from non-relational to relational databases and from their access language to SQL (DBMSMigrator is a PC-based tool).
7
Converting Programs, Files, Control Statements, and JCL
Only C-Scan produced C-Scan scripts are used.
The toolbox is self-documenting and controlled, and comments converted source.
8
Global Assessment and Field Adjustment
Global Assessment
IT DiscoveryProduct made up of C-Scan programs (and others) used for the static understanding of the application components within a full system environment.
Field Adjustment
The ToolboxTools made up of C-Scan programs (and others) used for the analysis and eventual conversion of specific system components through dynamic propagation of fields tracking data within and between programs.
Tools and ProductsTools and Products
C-ScanC-Scan
9
Toolbox Selection Screen (1 of 3)
10
Toolbox Selection Screen (2 of 3)
11
Toolbox Selection Screen (3 of 3)
12
Some Repository Files
DBJCL
DBDSN01
DBSOURCE
DBDTSCAN
DBPGROUT
DBLOADDBTRAN
DBCALL
INVLIST
13
Mainframe Technology Overview
1. BluePhoenix Mainframe Architecture
2. C-Scan
3. Toolbox and IT Discovery
4. Repository Files
5. IT Discovery Database and Query Facility
14
Entry-points (Events)
INIT - Before the first record of the input.
TERM - After the end of the input file.
INITMEM - Before reading the first record of a member of alibrary.
TERMMEM - After the last record was read from a member of a library.
OUTREC - After reading each record.
HEADER - At the beginning of a page when generating reports.
TRAILER - At the end of a page when generating reports.
KEYS - When the defined keys are changed.
15
Power of C-Scan
Some of the Available Variables
$$WORK Work area for that C-Scan step$$GLOB Work area that bridges multiple C-Scan steps$$RECORD The input record$$OUTREC The output record $$DSN The DSN on the input DD statement - PDS or SEQ$$MEMBER The member name within a PDS$$PNVMEM The member name of a PANVALET data set$$VOLSER The VOLSER of the input dataset $$DATE The current date in installation format$$DATER The current date in format “mmm, dd yyyy”$$TIME The current time (HH:MM:SS)$$JOB The name of the job specified on the JOB statement$$STEP The name of the step from the EXEC statement$$UID The UserID that last updated the member
16
Power of C-Scan
INDEX Function
Searches for a string in the specified part of the source record, can use specialized wildcards.
Returns 0 to indicate false (string not found) or position of the found string.
Creates the following variables:
$$IDXA The text specified by the INDEX function.
$$IDXB The text following the text that was found.
$$IDXC The text preceding the text that was found.
17
Power of C-Scan
Some Tables
Used to delimitate fields (set pointers)
Continue as long as…
ALU Uppercase Alphabetic
ALN Alphanumeric (Uppercase)
NUM Number
DSN Characters acceptable for dataset names
COB Characters acceptable for COBOL field names
N… Continue as long as it is not …
Also specified characters and wildcards
18
//SYSCIN DD * S010_RPT BUILD,DD1=IN010,DD2=OUT010,PARMDD=P010, D=|#+?|,SORTO='(1,8,CH,A,10,32,CH,A)' //P010 DD * OUTREC ( IF (INDEX(6,72,' ?PIC ?X(50)'),EQ,0) EXIT(LEAVEREC) IF ($$IDXC(1,COB,32),EQ,'FILLER') EXIT(LEAVEREC) BUILD($$MEMBER,10:$$IDXC(1,COB,32),/) )
Power of C-Scan
Use of Index, Tables, and Wildcards
From a source library, create a list of fields with a PIC of X(50), along with the member name
19
C-Scan in Production – JCL
20
C-Scan in Production – Control Statements
21
C-Scan in Production – Control Statements
22
C-Scan in Production – XTBLDREC
23
Mainframe Technology Overview
1. BluePhoenix Mainframe Architecture
2. C-Scan
3. Toolbox and IT Discovery
4. Repository Files
5. IT Discovery Database and Query Facility
24
The Mainframe Tool Libraries
JOB (JFIX; JUSER) – JCL and Mini-Scheduler lists
PARM (PFIX; PUSER) – Programs and control tables written in C-Scan
REXX – Mini-Scheduler and Screens
JCLFIX – JCL templates for situation specific jobs
25
Controlling the Flow – Screens
26
Controlling The Flow – Batch: The Mini-Scheduler
All jobs are submitted with the same JOBNAME and JOBCLASS
Jobs have two steps added, STEP00 as the first step and STEPFF as the last:
STEPFF writes to the LOG that the JOB successfully completed (based on Condition Code less than 5)
STEP00 causes a failure (System 806-4) if the previous JOB’s STEPFF did not write to the LOG (based on Run Type)
This permits:
High Level of Automation
High Number of Jobs – easy to debug
27
What We Need to Know About a Program
Language
Files
Other programs with same files
JCL (or OLTP) used
Routines called
Routines calling
Copies
Loads
DBMS structure
Etc.
28
What We Need to Know
29
DBJCL
Source JCL ITD
ExternalReaderOPJCL PROCLIB
ITD
What We Need to Know: Collect JCL with PROCs
30
Cross Referencing Components
Cross-referencing loads, JCL, CICS, and source
Identifying “missing” entities – sources not run, JCL or TRANS without source
There is a turnaround screen to link load and source where names differ
31
Repository ReportsDate Fields
xxxx sssssxxxx sssss
xxxx sssss
Records
xxxx sssssxxxx sssss
xxxx sssss
PL/I
Easytrieve
Source and CopyCOBOL
MasterRepository
Meta-COBOL
#SRV22
Inventory
JCL; PROCs;PARMs; CSD;DBMS; Load
Assembler
Other (4th GL)
Multi-Language Interface
32
Revise Commands, Conflicts, and Fixes Screen
33
IT Discovery
Provides understanding of the mainframe operational and application environments, their inventory and interrelationships.
Accesses, analyses, cross-references software components on an MVS mainframe.
Makes sense of the spaghetti that is an MVS application environment. Finds redundancies, missing entities, and inefficient interconnections.
Creates a DB2 database on the components reportable through SQL.
Includes a network based query facility (QF).
34
OLTP and Batch SOURCE
and COPIES
StaticInformation Collection
IT DiscoveryRepositoryFlat Files
JCL; PROC; Control
Statements
LOAD Modules
CICS Tables (CSD)
IMS/DC
Database Definitions
DB2; IMS; IDMS
Job Schedulers
IT DiscoveryRepository
DB2
Reports
Queries
NetworkQuery Facility
WebQueries
BluePhoenix IT Discovery
35
OutputProcess StageInput
1. DB2 RepositoryProcess Relational Database41. Database parameter definitions2. Repository Files
1. Updated Repository2. Reports
Turnaround: • Resolve variables containing
routine names • Resolve variables containing
DD names
Analyze Source Entities31. Production source programs and copy libraries
2. Repository Files
1. Populated Repository2. Reports
Turnaround: • Resolve Load without
corresponding source
Analyze Inventory Components
21. Production source programs and copy libraries
2. Production JCL, PROC and Control statement libraries
3. Production Load libraries4. Production OLTP control data5. Production DB formats
1. Customized environment2. Empty Repository datasets
Build Environment11. Product libraries2. Customer parameters
Process Stages
36
Analyze Source Entities
This step does the in-depth static analysis of the source programs.
ITD builds a Repository file that lists and details entities and relationships from within COBOL programs to the “outside world” including flat files, VSAM, DB2, IMS, IDMS and routines.
Other languages are analyzed without routine calls.
The Repository file includes detailed information on the interfaces between routines and between programs and data.
37
Analyze Source Entities
Turnaround Screens
“Variable Routines” Where a routine is called using a variable field for the name, and that variable is not defined as a constant in the Data Division.
“Variable DDNAMEs” Where a DDNAME is a variable field, and that variable is not defined as a constant in the Data Division.
38
IT DiscoveryRepository
(DB2)
QF ServerQF Client
QF Client
QF Client
QF Client
Query Facility Architecture
39
IT Discovery Collection
Scripts
Control Datasets
Catalog
IT DiscoveryRepository
C-Scan
Engine
Libraries
DB Definitions
Changes, Additions and Deletions ONLY
ITD – Incremental Running
40
Mainframe Technology Overview
1. BluePhoenix Mainframe Architecture
2. C-Scan
3. Toolbox and IT Discovery
4. Repository Files
5. IT Discovery Database and Query Facility
41
DBSOURCEDBSOURCE—Programs Dataset
Description The DBSOURCE dataset contains the details of all source programs in the project, and indicates whether or not the programs are active.
Source This dataset is generated from the source libraries and the DBDTSCAN, DBJCL, DBLNGLST, DBLOG, and DBPGROUT datasets.
Table 1 DBSOURCE Dataset Structure
SEQ.# FIELD NAME LENGTH POSITION NOTES / POSSIBLE VALUES
1 Library name 44 1
2 Member name 8 45
3 Language 8 53 APS ASSEMBLR C COBOL 1 COBOL 2 DYL280 EASYTRV IDEAL JCL PL/I SAS UNKNOWN
4 Compilation procedure 8 61 Name of the compilation procedure.
5 OLTP interface 8 69 Online environment. CICS
DBSOURCE Dataset (1 of 4)
42
SEQ.# FIELD NAME LENGTH POSITION NOTES / POSSIBLE VALUES
IDMS/DC IMS/DC
6 Special compile procedure 8 77 This field contains additional information about the language of the source program contained in Compilation procedure (seq# 4), e.g., BIBLOS is "dialect" which is used for COBOL ADABAS. ADAPREP BIBLOS IMS
7 Database interface 1 8 85 ADABAS DB2 IDMS IMS
8 Database interface 2 8 93 ADABAS DB2 IDMS IMS
9 Module type 8 101 Indicates if the module is a job, main program, MAP, JCL procedure, or routine. JOB MAIN MAP PROC ROUTINE
10 Lines of code LOC 5 109 Number of lines of source code.
11 Active indicator LOG 1 114 Based on DBLOG dataset. A Active (blank) Not active
DBSOURCE Dataset (2 of 4)
DBSOURCE
43
SEQ.# FIELD NAME LENGTH POSITION NOTES / POSSIBLE VALUES
12 Load module name 8 115
13 Cluster ID 8 123
14 System ID 8 131
15 Subsystem ID 8 139
16 FILLER 1 147
17 Active indicator JCL 1 148 Based on DBJCL dataset. A Active (blank) Not active
18 FILLER 2 149
19 Active indicator (META) 1 151 Based on METAOUT dataset. M Active (blank) Not active
20 Active indicator (cross reference) 1 152 Based on Cross Reference dataset. C Active (blank) Not active
21 FILLER 46 153
22 Conversion indicator 1 199 C Converted (blank) Not converted
23 Compile selection 1 200 Y Yes N No
24 Alias 5 201 Five letter name used to identify the library during the conversion process. All related files/processes are created using this ALIAS.
DBSOURCE Dataset (3 of 4)
DBSOURCE
44
SEQ.# FIELD NAME LENGTH POSITION NOTES / POSSIBLE VALUES
25 Output suffix 5 206 The convention used to name the library containing the converted programs, is the input library name plus a suffix (the value in this field). This library must be allocated manually.
26 Original library 44 211 The name of the original library containing the programs that will be converted.
DBSOURCE Dataset (4 of 4)
DBSOURCE
45
IT Discovery – DB2 Repository
46
IT Discovery – DB2 Repository
47
Tbinvlist
Lists the inventory that defines the scope of the project. It contains the contents of all the source libraries that are used as input for the IT Discovery process.
Table 22 Tbinvlist Table Structure
KEY COLUMN NAME COLUMN DISPLAY NAME DATA TYPE COLUMN DESCRIPTION
PK, FK Org_unit_id Unit ID Number SMALLINT(2) A generated primary key for each organizational unit identifying the different organizational units in the organization.
PK, FK Lib_cd Alias CHAR(5) Each entity is uniquely defined by the library in which it was found when the source members were scanned. Each library has been allocated an arbitrary five-digit code in order to save the space that a 44 letter DSN would require. The original library can be retrieved by joining this column with the table TBLIB and extracting the contents of the column Orig_lib_nm.
PK Mbr_nm Member CHAR(8) The name of the member that was scanned.
Object_typ Type CHAR(6) A distinction is made between a source and a copy. The source is the actual source code or program while copy is name of the copybook member that is included in the source by means of a COPY or INCLUDE statement. The two values that exist are SOURCE for the source and COPY for the copybook members. This list also includes JCL procedures and JCL include members.
Total_lines Total Lines INT(5) There are various ways of measuring the length of a source member. One is the number of lines that the member contains, including comments and blank lines, another is the actual number of "executable" lines of code. This column contains the total number of lines of code that were in the member when it was scanned. It includes blank lines and comment lines and is equivalent to the value contained in the ISPF statistics.
Tbinvlist Table (1 of 2)
48
KEY COLUMN NAME COLUMN DISPLAY NAME DATA TYPE COLUMN DESCRIPTION
Loc LOC INT(5) There are two ways of measuring the length of a member. One is the number of lines that the member contains, including comments and blank lines. The second is the number of lines of code. This column contains the actual number of lines of code that were counted when the member was scanned. It excludes blank lines and comment lines.
Used_in Selected CHAR(1) The inventory list is made up of all members that are contained in the input libraries. The user can choose which members are to be scanned. Members that have been selected have a value "S" while members that exist in the catalogued libraries but have not been selected (or have been deselected) have a value "N".
Creation_dt Creation Date DATE() Object creation date
Last_ch_dt Last Change Date DATE() ast change date of the object
Tbinvlist Table (2 of 2)
49
Mainframe Technology Overview
1. BluePhoenix Mainframe Architecture
2. C-Scan
3. Toolbox and IT Discovery
4. Repository Files
5. IT Discovery Database and Query Facility
50
Query Facility Architecture
Thin Client – Users have no software installed and access the server via a browser (intranet)
Server – Operates software that:
Provides screens and interfaces to user
Manages users, queries, and access to queries
Passes SQL queries from clients to mainframe (via DB2Connect) and returns results
Mainframe – Contains ITD Relational Database (DB2)
51
Query Facility Users
Administrator – Connects repositories; maintains Users, Public, and Shared query folders.
Power User – Can create and run own queries in Private folder. Can copy to Shared folder. Can run queries from Public and Shared, and copy to Private.
Regular User – Can only run queries from Public folder.
52
Query Facility Folders
Public – Categories and queries available; “Read Only” to everyone; maintained by the Administrator.
Shared – Categories and queries available; “Read, Write, Not Update” to Power Users (if enabled) and the Administrator.
Private – Queries only available; “Read, Write, and Update” to Power Users and Administrator.
53
Query Facility – Categories and Queries
Categories – Groupings of Queries
Categories belong in Folders
Queries – Use standard DB2 SQL
Queries can have drill down menus from specific results to other nested queries
54
Query Facility – Categories and Queries
55
Query Facility – Queries
56
Query Facility – Running and Drilling
Queries are run by hitting EXECUTE
Parameter is per SQL – avoid using only %
Drill down menus are available on underlined results
Nested Queries available through the drill down menus
57
Query Facility – Running Queries
58
Query Facility – Drilling Down
59
Query Facility – Nested Query
60
Query Facility – Filters and Export
Results columns can be filtered for viewing
Results can be exported to XML; CSV; Excel; HTML
61
Query Facility – Filtering Results
62
Query Facility – Exporting Results
63
Query Facility – Exporting Results
64
Query Facility – Creating Queries
Creation by Power Users and Administrators
Must be in Category
Can create drill-down menus
Can link drill-down menus
65
Query Facility – Query Creation
66
Query Facility – Drill-Down Linking
67
Query Facility – Online Documentation
68
Query Facility – Online Documentation
69
Thank You!