WPG PREPARED BY: M. Gilbreath DATE: SYSTEM NAME: … S - DQ9420PX... · 2016-05-12 · DQ9420PX -...

DQ9420PX - PROGRAM SPECIFICATION EXHIBIT S

Washington State INH EDE Data Integration

Administrative Office of the Courts of 28 ACQ-2016-0301-RFP

WPG PREPARED BY: M. Gilbreath DATE: 3/19/2013 SYSTEM NAME: DISCIS PROGRAM NAME: DQ9420PX PROGRAM NAME: Data Management Destruction of Records

Purge Active Cases for Removal and Reporting DORR and DOR reports by Court PURPOSE: This program will process selected cases and remove them from the on-

line DISCIS database as part of the DORR and DOR Destruction of Records Process.

CHANGE HISTORY: 3/19/2013 MDG ITG41 New program specification. 7/5/2015 MDG ITG41-2 Iteration 2 – Add new Case Purge Review Date, Permanent

Retention Flag, enhanced reporting for selection exceptions, NCCD case processing, and UTX table restart capability.

ENVIRONMENT: BATCH (X) ON-LINE ( ) LANGUAGE: COBOL (X) NATURAL/DB2 ( ) OTHER ( ) REQUIRED TURN-AROUND TIME: N/A RESTART INFORMATION: Depending on how this program dies, it may be a complex restart. *If program ends in CODE=99 with time exceeded, then check the UTX table for the last entry marked as DQ9420PX. This is the last case processed. Case should also be listed in either the JIS.PRDDISC.DOR.PRINT.WORK.CASE or JIS.PRDDISC.DOR.PRINT.WORK.NAME dataset. Restart on the DOR step, as the program will look for the last case processed and start from next case in the SELECTCS dataset - JIS.PRDDISC.DOR.Q9410SRT.WORK. Be sure the next case appearing in the JIS.PRDDISC.DOR.PRINT.WORK.CASE and/or NAME dataset follows the one you restarted from using the UTX table. For EIXISS errors, If you start with the same case as is listed as having had the error, it may clear itself up and run properly. If not, start with following case number. DO NOT ever assume the case reported in job output is the last case actually processed since this job always has two cases in process at the same time. Other issues will require research prior to restart. Code 99: Due to DQ9410PX/SELECT step found no cases to process. Research why this may have occurred. JCL should not allow it. SQL -532 Error on DELISS2: Deletion out of parent/child sequence error. Some of these errors will “self-clear” if you restart the job on same case as described in previous section above. This is a rare error involving Etickets: EIXISS01 on DELISS2 step. The EIX table purge prior to the ISS table purge step makes DB2 sometimes think it has a




hang-up out of sequence when it doesn’t. This is only for cases with Etickets. There is a history of Eticket table issues on old cases. It has never been proven to be an actual error on restart. If error ever re-occurs twice, consult programmer to diagnose situation further. It may be a pair of Etickets are holding the same key. Solution 1: Restart on the case that failed. It will likely run fine. Solution 2: Adjust JIS.PRDDISC.DOR.Q9410SRT.WORK to the appropriate case following the problem case number as specified above and restart DQ9420PX. Return Code = 2: Normal end. Program wrote errors to the Audit report dataset and an Email needs sent to the AOC group. Return Code = 3: Normal end. Program wrote nothing to the IRA table. This may or may not be an error condition and the return code will limit other programs from processing in the JCL. Datasets of concern:

1) JIS.${PARMLIB}..DOR.Q9410SRT.WORK - contains the cases to process. It may be necessary to create a backup of this file and delete records so as to start where program left off. Copy the records from the original dataset into a version of JIS.${PARMLIB}..DOR.Q9410SRT(+1) as an audit trail.

2) JIS.${PARMLIB}..DOR.COUNTS.C${YYMMDD}. – this dataset will need copied into a backup version and the original deleted for program restart.

3) JIS.${PARMLIB}..DOR.AUDITEX - this dataset holds the error messages from the run that AOC needs to be aware of, and email notification gets set the thte group.

4) JIS.${PARMLIB}..DOR.SELEXCP – Dataset from DQ9410PX step holding all exceptions messages from the run. These messages will output to the court.

5) JIS.${PARMLIB}..DOR.PRINT.WORK.CASE and JIS.${PARMLIB}..DOR.PRINT.WORK.NAME – these datasets are MOD,CATLG,CATLG so new records add to bottom of file. Be sure restart does NOT add same case twice.

6) OPTIONAL: In UTX table there are a series of records labelled ‘DOR-XXX ’ where XXX is the court initials. (ie: ‘DOR-THD ‘ for Thurston County District Court). If DQ9420PX dies, and it is decided NOT to restart it, then programmer might consider looking into the dataset in #1 above and finding each court whose cases are not going to be processed. Take the beginning case number for that court and enter it into the UTX table UTX_TX field in the first 9 spaces ZERO FILL LEFT (ie: XXXXXXXXXCTCASETOKENHERE) This serves to force DQ9410PX to start at that case on next run. If you do not do so, the cases not processed in this run, will come around again eventually in future. Entering LOW VALUES in the case number will force court to start over on next run.

More restart information: This step uses datasets and IRA Table that were initialized above. We need to keep the data! * JIS.PRDDISC.DOR.AUDITEX – for AOC review * JIS.PRDDISC.DOR.SELEXCP – list of exception messages for the court. * JIS.PRDDISC.DOR.COUNTS.CYYMMDD - contains counts for DBA's - SAVE DATASET TO BACKUP! Copy to JIS.PRDDISC.DOR.COUNTS.CYYMMDDBK Delete original prior to restart. * JIS.PRDDISC.DOR.PRINT.WORK.CASE - by LEA/CASE




contains a list of the records going into the DORR/DOR Print routine. DO NOT LOSE PROCESSED CASES LISTED IN THIS FILE. * JIS.PRDDISC.DOR.PRINT.WORK.NAME - by Name contains a list of the records going into the DORR/DOR Print routine. DO NOT LOSE PROCESSED CASES LISTED IN THIS FILE. * JIS.PRDDISC.DOR.Q9410SRT.WORK You may need to alter this file, and remove all cases that have already been processed before restarting. Suggest you copy file to: JIS.PRDDISC.DOR.Q9410SRT(+1) prior to altering original dataset. For COMMIT or NON-COMMIT runs, this is advice for these datasets. Run Type: DATASET: NON-COMMIT: JIS.PRDDISC.DOR.Q9410SRT.WORK COPY ORIG TO BACKUP JIS.PRDDISC.DOR.COUNTS.CYYMMDD DELETE - NOT NEEDED All others: LEAVE AS IS - RESTART DQ9420PX STEP COMMIT : JIS.PRDDISC.DOR.Q9410SRT.WORK COPY ORIG TO BACKUP. JIS.PRDDISC.DOR.COUNTS.CYYMMDD COPY ORIG TO BACKUP & DELETE ORIG All others: LEAVE AS IS - RESTART DQ9420PX




INPUT DATASET NAMES AND DESCRIPTIONS: Input work file SELECTCS: Selected Records from DQ9410PX SELECTCS work file 1. Physical locations begin in column 1 of the work file record. The record should be a 168 columns card image. Format is #DQ9401LX-SELECTED-CASE-REC from DQ9401LX. DB2 Table access All access as indicated below. READ only: CRT Court Write to IRA Table only: AD Address CAX CBA AP Cross Reference ICH Individual Case History INV Individual CBA Court bank Activity CHK Check CMT Comments (may or may not be written to IRA due to line size) EML Email PHN Phone PIN Personal Identification Number PNH Person Name Change History Write to IRA Table and Update Record Timestamp: PAD Person Address PCM Person Comment PEM Person Email PER Person PPH Person Phone Write to IRA Table and Purge from DB2 active tables: ADJ Adjustment AP Accounts Payable AR Accounts Receivable BND Bonds CIT Citation CJ Civil Judgment CJI Civil Judgment Item CJP Civil Judgment parties CLH Collection Status History CPO Case Person Obligation CRX Case Relationship Cross Reference CS Case CSC Case Cause Code




DCM Document DRR Destruction of Records Restore DKT Docket DVO Domestic Violence Order DVP Domestic Violence Order Party FWO Failure to Appear Warrant Order GRN Garnishment ISS Issue JVO Journal Voucher NCC non-Civil Judgment Cost Fees NCJ Non-Civil Judgment PAR Participant PCN Process Control Number PKC Parking Citation PKI Parking Issue PPR Proceeding Participant PRC Proceeding PRK parking PRX Person relationship Cross reference PSH Proceeding Reschedule History RLN Relationships RPS Representation SPH Schedule Payment History VVC VRV Source Document Case Cross Reference VXR Vehicle Cross Reference Purge from DB2 active tables (no backup): Uses external call to subprogram. EIX Electronic Ticket Issue Message GUID Cross Reference ELC Electronic Ticket Case Log (removed) ELI Electronic Ticket Issue Log ELK Electronic Ticket Lock EMX Electronic Ticket Message GUID Cross Reference ETB Electronic Ticket Binary Object ETC Electronic Ticket Binary object Cross Reference ETD Electronic Ticket Defendant ETK Electronic Ticket ETL Electronic Ticket Location ETO Electronic Ticket Officer ETP Electronic Ticket Person ETS Electronic Ticket Statute ETV Electronic Ticket Vehicle ETW Electronic Ticket Witness ETX Electronic Ticket Case Cross Reference (removed) MGD Message GUID MTR Message Transcription *ETT ETT table is not part of ITG41. ETT table created in June 2014 and will be cascade deleted when ETK records are purged. We do not save or process ETT table records.




Write to UTX Table: UTX Audit Messages for processing records OUTPUT DATASET NAMES AND DESCRIPTIONS: Output file COUNTS: Counts for the DB2 database administrators Use work file to record counts of records by table that were processed by the run. The DB2 database administrators will use this data to balance run load. Output file AUDIT1: Exception/Counts report for AOC Use to identify the cases that encounter exceptions to the purge processing. Output file MESSAGE1: Exception report for courts Use to identify the cases that encounter exceptions to the purge processing. Output file PROGRES1: Count of records processed Optional Output file TESTIRA: IRA Table test file. Use for testing only in a non-production environment. Output file PRINT1/PRINT2: DORR/DOR Destruction of Records report for courts Destruction of Records Report listing all cases meeting DORR and DOR criteria. These records are used to create the DORR and DOR Report that is later printed on court CPF domains. Uses DQ9405LX format #DQ9405LX-DESTROYED-CASE-RCD. CS-CRT-ITL-NU SORT-KEY /* sort by LEA/CS/TYP/Name SORT-KEY-1-LEA-ITL-NU SORT-KEY-1-CS-NU SORT-KEY-1-CS-TYP-CD SORT-KEY-1-NM SORT-KEY /* sort by NAME/LEA/CS/TYP SORT-KEY-2-NM SORT-KEY-2-LEA-ITL-NU SORT-KEY-2-CS-NU SORT-KEY-2-CS-TYP-CD /* end sort key definition CS-NU /* Case number




LEA-ITL-NU /* LEA CS-TYP-CD /* Case Type Code CSC-CAU-CD /* CV Cause Code CS-TK /* Case Token CS-CIV-CS-FL /* Civil Flag PER-NM /* Name CRT-SHT-NM /* Short Court Name SORT-SEQ-CD /* Court Sort Preference CS-PRINT-CLASS /* (C)riminal Case /* (I)nfraction /* (P)arking /* (V) Small Claims /* (V) Civil /* (D)eleted Case NCCD CS-OUTRDB-FL /* (R)pt/(D)elete/(B)oth /* (Y) Rpt and Upd PRG /* (Z) Upd PRG RVW Dt CS-OUT-TYPE-FL /* (E)lectr or (M)annual CS-ETK-FL /* Y or N CS-VRV-FL /* Y or N CS-DV-FL /* Y or N CS-CSL-CD /* Caseload Code CS-PRMNT-RTNTN-FL /* Y or N Auxiliary Output: ALL records that are reported on the DORR Destruction of Records Report have an entry made on the case docket stating that the case paperwork is eligible to be destroyed. NCCD deleted cases are NOT given a Docket Entry. INPUT PARAMETER DESCRIPTION: 1. Case Base Aging Date. Input this date in MM/DD/YYYY format from the COBOL JCL

input stream. This will allow AOC staff manual control of the removal process for testing, processing pilot sites, and special runs.

2. Elapsed Run-time Limit. Input this value from the COBOL JCL input stream. The design

intent is that timing will be done with COBOL time facility. Therefore it should be input as a six digit number representing HHMMSS (hours, minutes, seconds). When the elapsed run time exceeds the time limit the program will stop. When the time limit expires the program will stop regardless of whether all the courts specified have been processed or the max number of rows for the run has been selected.

3. COMMIT FLAG: Y or N for database update. If marked N, then cases will be written to

output dataset and set on to DQ9470PX for test reporting. 4. Maximum Number of Records to Process for the Run. Input this number from the

COBOL JCL input stream. The program will count the total number of records processed for all courts processed. When the number reaches the maximum limit the




program will stop. When the maximum is reached the program will stop regardless of whether all records have been processed or the time limit has expired.

5. Write Test IRA file flag. (NOT USED ANYMORE) Input this value from the parameter file. The design intent is that use of the test file will be combined with setting the Commit Changes flag to No during testing to preview the data that would be inserted into the IRA table. An edit will force the Commit flag to No when the Test IRA flag is Yes. Value Y = write each IRA row to the Test IRA file. This option is for rolled back tests. Value N = do not write to the test IRA file. This option is intended for production use. 6. DEBUG Flag: Y or N. If ever marked Y, then additional messages will appear in

reporting for the purpose of troubleshooting process. It is not recommended that DEBUG Flag = Y if the COMMIT Flag = Y as there will be messages in the reports that users will not expect to see.

OUTPUT DATASET NAMES AND DESCRIPTIONS: Output File 1: Run Counts file COBOL DDNAME: COUNTS. COBOL FILE NAME: FILE-COUNTS. Write one record to this file for every table for every court processed in the run. Even if a given table had no deletions or insertions for a given court a record should be written with a count of zero. At the end of the run write a separate set of records containing the overall totals for all courts involved in the run. Use the dummy court initials listed below to identify the overall totals records. A new run counts dataset will be created by each offline removal run. The run counts data will be used by AOC Database Administration staff to do database capacity planning. The file will be fixed length sequential format. Physically the first field begins in column 1 of the record. Each successive field begins immediately after the preceding field.

FIELD Source of the data COUNTS-RUN-DATE Current date at beginning of program run in

YYYYMMDD format. COUNTS-COURT-INITIALS Use CS_CRT_ITL_NU for court level counts. For overall

run counts use "999" so that the overall counts sort to the end of the group for a given run.

COUNTS-DB2-TABLE-NAME DB2 table name. COUNTS-DELETED-ROW-COUNT Count of rows deleted from or inserted into the table. In

signed zero-filled character format, COBOL PIC +9(9). Tables with deletions will have a negative sign. Tables with insertions will have a positive sign. Zero will show a positive sign.

Output file 2: Audit file COBOL DDNAME: AUDIT1. COBOL FILE NAME: FILE-EXCEPTION-AUDIT.




This is the exception report print file. See the report specifications for details.

Output file 3: Messages file COBOL DDNAME: MESSAGE1. COBOL FILE NAME: FILE-EXCEPTION-MESSAGE. This is the exception report print file. See the report specifications for details. PROGRES1 Output file – records counter to run sheets. Output File 3: Test IRA file COBOL DDNAME: TESTIRA. COBOL FILE NAME: FILE-TEST-IRA. When the Write Test IRA file flag input parameter is Yes write every IRA row to this file after it is successfully inserted. Because the program lets DB2 assign the value for IRA_UPD_TS there will be no values in the IRA_UPD_TS field in the output file. Output File 4: DORR/DOR Print records file COBOL DDNAME: PRINT1 or PRINT2. COBOL FILE NAME: FILE-PROCD-CASE or FILE-PROCD-NAME. This is the output file for all records processed in this step that are bound for print on the DORR/DOR Destruction of Records Report. Record format can be found in DQ9405LX or in this program under W-OUTPUT-CASE-RECORD.

Specifications for access and update of other DISCIS DB2 tables. All rows for a selected case in every one of the tables listed below will be selected and written to the offline storage file, except as noted below.

AD Address Rows remain active, NO deletions. Special rules. Paired with PAD.

ADJ Adjustments delete rows AP Accounts Payable delete rows AR Accounts Receivable delete rows BND Bond delete rows CAX CBA AP Cross Reference rows remain active, NO deletions. CBA Court Bank Activity rows remain active, NO deletions. CHK Check rows remain active, NO deletions. CIT Citation delete rows CJ Civil Judgment delete rows CJI Civil Judgment Item delete rows CJP Civil Judgment Participant delete rows CLH Collection Status History delete rows. CMT Comment Rows remain active, NO deletions. Paired with

PCM. See special notes. CPO Case Person Obligation delete rows. Special handling for nullable TS. CRX Case Relationship Xref Special rules govern deletion CS Case delete rows. Special handling for nullable times.

Update rows for CS_DTN_NTF_DT.




CSC Civil Small Claim delete rows DCM Document delete rows DKT Docket delete rows DRR Destruction of Records

Restore delete rows

DVO Domestic Violence Order delete rows. Special handling for nullable time. DVP Dom Viol Order Participant delete rows EML Email address Rows remain active, NO deletions. Special rules.

Paired with PEM. FWO FTA / Warrant Order delete rows GRN Garnishment delete rows ICH Individual Change History Rows remain active, NO deletions. Child of INV. INV Individual rows remain active, NO deletions. ISS Issue delete rows JNE Journal Entry ignored by this program. JNE's are removed from

the JIS database by a separate process. JVO Journal Voucher most rows are deleted; those associated with court

checks and case level investment transfers, earnings, and charges remain active.

NCC Non-civil Cost/fee delete rows NCJ Non-civil Judgment delete rows PAD Person address Rows remain active, NO deletions. Paired with AD. PAR Participant delete rows PCM Person comment Rows remain active, NO deletions. Paired with CMT.PCN Process control number Delete rows. PER Person rows remain active, NO deletions. PEM Person email address Rows remain active, NO deletions. Paired with EML. PHN Telephone number Rows remain active, NO deletions. Special rules.

Paired with PPH. PIN Personal Identification

Number rows remain active, NO deletions.

PKC Parking Citation delete rows PKI Parking Issue delete rows PNH Person Name Change Rows remain active, NO deletions. Child of PER. PPH Person phone number Rows remain active, NO deletions. Paired with PHN.PPR Proceeding Participant delete rows PRC Proceeding delete rows. Special handling for nullable times. PRK Parking case delete rows PRX Person Relationship Xref Special rules govern deletion PSH Proceeding Schedule History delete rows. Special handling for nullable times. RDS Restitution Recipient

Distribution Schedule delete rows

RIX Restitution Interest Xref delete rows RLN Relationship Special rules govern deletion. Goes with CRX, PRX. RPS Representation delete rows SPH Schedule Payment History delete rows.

UTX Utility Text Save case status messages for audit trail.




VEL Vehicle rows remain active, NO deletions. VXR Vehicle Cross Reference delete rows

UTX table is updated throughout run for restart capability and for error tracking. PROGRAM NARRATIVE: This program performs the purge database processing step for the data management active table Destruction of Records process. It reads in the records selected by DQ9410PX to be reported or removed or both, or alternatively, to have just the case purge review date updated. See the DISCIS Data Management Destruction of Records ITG41 documentation and related analysis documentation for background on this program. Input: This program will read the records to be processed from the input dataset. COMMIT FLAG: If COMMIT = N, all cases will be processed normally but no updates will be made. This allows testing and reporting to happen without risk of mistakenly updating or purging case records. The program will pass all records marked as REPORT #DQ9401LX-CS-OUTRDBYZ-FL = ‘R’ or ‘Y’ directly through to the output dataset. These records will be checked and updated if the flag #DQ9401LX-CS-DTN-NTF-UPD = ‘Y’ or the Purge Review Date needs updated. The program will read through the input file until it reaches the end of the file or hits a maximum time or maximum records limit. All records NOT marked as REPORT #DQ9401LX-CS-OUTRDBYZ-FL = ‘R’ or ‘Y’, and are not marked for Purge Review Date updates #DQ9401LX-CS-OUTRDBYZ-FL = ‘Z’, will be processed to be purged from the active DB2 tables and then written to the output dataset. Should an exception case occur while purging a record, the specific record will be rolled back and reported to the exception report and on the UTX table. The program will always stop when the elapsed time limit is exceeded. If the cases in the input dataset have not been completely read, program will end with Code=99 and list last case processed. Restart on DOR step! All cases MUST be completed prior to next run of this job. The program will always stop when it reaches the maximum number of rows to process for the run as specified by the input parameter. The program will always record an audit record on UTX table for every record that passes through DQ9420PX. >>>Control: In case of DB2 errors or other severe errors this program will either warn the programmer via the exception report, or terminate the run with an appropriate condition code. This program produces a purge process exception report for AOC use. The exception report is not routed to the court(s). The following lists show the tables to read for each case type: Case Type: IT, IN, CT, CN, CF, PC




Tables: ADJ, AP, AR, BND, CAX, CBA, CHK, CIT, CLH, CPO, CRX, CS, DCM, DKT, DVO, DVP, FWO, ISS, JVO, NCC, NCJ, PAR, PPR, PRC, PRX, PSH, RLN, RPS, VEL, VXR.

Case Type: PR Tables: ADJ, AP, AR, BND, CAX, CBA, CHK, CIT, CLH, CPO, CS, DKT, DCM, ISS, JVO, NCC,

NCJ, PAR, PKC, PKI, PPR, PRC, PRK, PSH, RPS, VEL, VXR. Case Type: CV, SC Tables: ADJ, AP, AR, BND, CAX, CBA, CHK, CJ, CJI, CJP, CLH, CPO, CRX, CS, CSC, DCM,

DKT, DVO, DVP, FWO, GRN, JVO, PAR, PPR, PRC, PRX, PSH, RLN, RPS, VEL, VXR. For persons (some of these table are not used with CV (civil) and PR (parking) persons): Tables: AD, CMT, EML, ICH, INV, PAD, PCM, PCN, PER, PHN, PNH, PPH, PIN, RLN. PRX is unusual. Even though it contains no case token it cannot exist unless connected to a case by a CRX row. Therefore PRX is treated as a case-oriented table. The JNE table is not included in the lists above. A separate process will remove JNE's from the database. Exception: Cases that have been deleted via DISCIS delete screen (NCCD) will be fully

deleted. Their rows will be deleted from the active DISCIS tables but will not be saved in the IRA table.

Special selection problem: shared JVO's for court refund checks. JVO's (and the associated CBA's) for payments that involve money from one or more

cases usually do not contain a case token. These JVO rows can be selected by using the closing JVO key data from each case's AP rows.

Special selection problem: shared JVO's for trust transfers from one case to another. The JVO contains the case token for the target case. The source case has no JVO

associated with it by case token. To link this kind of JVO row to the source case the JVO must be selected by using the closing JVO key data from the source case's AP rows.

Solution for both of the JVO special selection problems listed above. All JVO's related to a case can be selected by using the following SQL. SELECT all columns of JVO FROM JVO WHERE JVO_CRT_ITL_NU = the court being processed AND JVO_CS_TK = CS_TK of the case being processed UNION SELECT all columns of JVO FROM AP, JVO WHERE AP_CRT_ITL_NU = the court being processed AND AP_CS_TK = CS_TK of the case being processed AND AP_CLO_JVO_PST_ST = JVO_PST_DT AND AP_CLO_JVO_SEQ_NU = JVO_PST_SEQ_NU AND AP_CLO_JVO_NU = JVO_NU AND JVO_CRT_ITL_NU = the court being processed Explanation of the SQL: This SQL does a UNION of two selects:




1) JVO's that contain the case token, 2) JVO's that closed AP's for the case (trust refund checks and trust transfers do

this.) This part of the SQL selects those JVO's that do not contain the case token of the case being processed.

Remember that a UNION does not include duplicate rows in the answer set. Special selection problem: FWO's may not have a DCM row. Background: When an FTA or warrant is ordered the FWO row is created. The

associated DCM row is not created until the FTA or warrant is actually produced by DISCIS. If the FTA or warrant is cancelled before being printed then no DCM row will ever exist.

Solution: Process FWO's separately from DCM's. This turns out to be more efficient than trying to access FWO's via the DCM table because, at the time of this writing, the production FWO table has no index that includes FWO-DCM-ITM-TK. The FWO table indexes are all based on case token.

Exceptions to the selection rules. These rows will not be selected. These rows will not be written to the IRA storage file. 1. Person data will not be included for the following: a. Officials (PER_MJR_TYP_CD = "OF") b. Organizations (PER_MJR_TYP_CD = "OR"). c. DISCIS special persons (PER_MJR_TYP_CD = "SP") 2. LAW table rows associated with a case will not be included in the IRA storage

file. 3. Codes table data will not be included. Special handling for the CMT table - item 1.

CMT table has a maximum CMT row length of 4056 bytes. At this time no JIS application supports the full length. Therefore this project will implement the following:

1) Because the CMT_TX column is not the final column in the CMT row we will have to store CMT rows in a special IRA record layout.

2) the length of the IRA row will NOT be changed to accommodate the long CMT rows.

3) no design will be implemented to split long CMT rows for storage in multiple IRA rows or in multiple offline storage file records.

Special handling for the PAR table.

For each case select PAR rows in ascending PAR_TYP_CD, PAR_TYP_SEQ_NU, PAR_UPD_TS sequence. PAR_UPD_TS is included so that any duplicate PAR_TYP_SEQ_NU’s appear from oldest to newest. Newer duplicates will be renumbered. Check the PAR_TYP_CD plus PAR_TYP_SEQ_NU against the values of previous PAR rows for the case. If a match is found and PAR_PER_TK is different (and it better be) then increment PAR_TYP_SEQ_NU. Store the “before” and “after” PAR_TYP_CD, PAR_TYP_SEQ_NU for each PAR_PER_TK. Replace the “before” value PAR_TYP_SEQ_NU with the “after” value in the PAR, PPR, RPS tables and in the header of the offline storage file records. The VXR design assumes that PAR_TYP_CD plus PAR_TYP_SEQ_NU is unique within a case. The VXR table contains PAR_TYP_CD and PAR_TYP_SEQ_NU but not PAR_PER_TK. So VXR needs no repair. Be sure to initialize the list of PAR data for each case.




Control: Data integrity checking requirements. Insert an IRA row for every row of every case being processed. 1. If the system goes down we must not lose any case data. DB2 recovery will handle this. 2. If the case has -911 trouble or any other minor errors that caused the program to skip

the problem case but not to abort the run then we do not want to include the case in the archive. Executing a DB2 rollback will do this.

Use the SQL like the following to select the CS row: Select (all columns listed by name) from CS where CS_TK = case token from the selected case file and CS_UPD_TS = the timestamp value from the selected case file If the timestamp values are different then the case was updated between the time the selection program picked it and the time this program processed it. Mark the record. Write an appropriate error message. Skip any further processing for the case. Rollback all DB2 activity on the case, thus restoring all deleted rows and removing any incomplete entries. Continue processing with the next case from the selected case file. Control: Data integrity checking. Do not check whether JVO's being processed have been posted. The required retention period is long enough that the JVO's should be posted by the time a case is eligible for processing. Note that this is a potential problem in testing when AOC staff force cases to be processed. Control: Severe MVS errors, such as running out of file space, will be allowed to cause an abend. Process: Delete the rows for each case from the DISCIS database as they are selected. See the table of rules for exceptions. Insert appropriate entries in the IRA table (see Appendix 2 for details). Exceptions to the deletion rules. These rows must remain in the active database. 1. For trust refund checks where an AP RD or AP OT JVO created CBA and CHK row, do

not delete the JVO, the CBA nor the CHK because all checks are required to appear permanently on the DISCIS check processing screens. Consolidated checks will result in an AP CD JVO and will also create a CAX row in addition to the CBA and CHK. These JVO's will not be deleted.2

2. Delete void check JVO's. They are not required by the CKR screen to display checks. Do not archive the CBA nor the CHK row for void check JVO's. The CBA will have been archived when the JVO that created it was archived. Archiving the CBA with the void check JVO would cause the CBA to be duplicated in the archive storage file.

3. Delete case-oriented NSF check reversal DP DB JVO's. Copy the CBA associated with the DP DB to the archive. Do not delete the CBA.

4. For trust transfers from one case to another, the transfer JVO contains the receiving case's case token, but the AP's associated with the JVO come from both the source and receiving cases. The JVO will be deleted when the receiving case is deleted. Each AP will be deleted when its corresponding case is deleted.

5. Do not delete case level JVO’s with type/subtype of ‘DA/BT’, ‘IA/BT’, and ‘CR/IA’ since these are used to create CBA rows. The CBA/JVO pairing is required to display a permanent record of the check register.

Technical note: this design for trust transfers will leave the source case with an "orphan" AP row, if the source case is not archived in the same run as the receiving case. This is not a problem. Once the transfer JVO is removed, no on-line DISCIS function can




access the AP because the JVO is missing. Batch reports access trust data via open AP or BND rows with positive amounts. This cannot occur on a closed case.

6. The program should process the CRX for the case plus the associated RLN and PRX rows so that all of the case’s relationships are written to the IRA table. However deletion of the PRX and RLN rows from the active database follows a special rule. The rule is that family RLN’s are deleted when there are no cases associated with the relationship. Therefore do not delete the RLN and PRX rows if there are any CRX rows connecting the RLN to cases other than the case being processed.

All person rows referenced by the PRX rows for a relationship are supposed to be participants in the case. Therefore we will assume that the normal participant processing for the case will process all the person rows associated with all relationships for the case.

Process: Unit of Work for DB2 commit / rollback processing. Treat each case as an individual DB2 unit of work. J&S-grouped cases are not handled in this processing. If the Commit Changes flag is YES, perform a DB2 commit after each case or group of cases is processed. After the DB2 commit each case or group of cases will be considered to be purged as far as the DISCIS application is concerned. Cases or groups of cases that do not receive a successful commit will not be purged. If the Commit Changes flag is NO, perform a rollback instead of a commit. Control: Add special handling to make sure a commit or rollback is issued for each case or group of cases. If this occurs skip the bad case or group of cases but do not stop the run. Control: DB2 contention processing. In case of DB2 contention error SQLCODE -911 the program must skip the case or group of cases, rollback the unit of work, and proceed to the next case or group of cases. Control: Case data error and DB2 error processing. In case of errors at the individual case level or group of cases level, print a message, rollback the unit of work, and attempt to proceed to the next case or group of cases. In case of overall fatal errors print a message, rollback the unit of work, and end the program run. Keep counts of all rows deleted for each table for each court and for the entire program run. Keep count of cases rolled back. At the end of each court's cases write the row counts for the court to the run counts file. At the end of the program run write the overall counts to the run counts file. Also print start, end, and elapsed times. This program does not produce any report required by the courts. This program prints a standard exception report. The report is not routed to the courts.




Appendix 1 - Sort Codes The sort code numbers indicate relative sequence within a single IRA table dataset. The design intent of the sort code is to put the tables into relative sequence for insertion processing following the relational integrity rules. If new codes are needed the list can simply be renumbered. By not tying a specific sort code value to a specific DB2 table on a permanent basis, offline storage files containing old sort codes will not become obsolete if new tables are added to the DISCIS archiving system. The CS row comes first so it gets sort code 00. Each parent table must have a lower sort code number than any of its child tables. To put the rows for one case into the proper sequence for insertion sort the rows in ascending sequence on the sort code. The diagram below shows the DB2 tables in referential integrity insertion sequence, i.e. child tables are shown below and offset from their parent.

Sort Code

CS CS child Lev 1

CS child Lev 2

CS child Lev 3

Xrefs Other Entity

Other child Lev 1

Notes

00 CS independent entity 02 VVC Child of CS, DCM 04 ELI Child of ISS 05 ETB Independent (uses ETK_KY) 06 ETL Independent (uses ETK_KY) 07 ETK Child of ETB, ETL (in theory it is

parent of all eTicket tables) 08 ETP Child of ETK 09 ETW Child of ETB, ETP 10 ETD Child of ETB, ETL, ETP 11 ETO Child of ETB, ETL, ETP 12 ETS Child of ETB, ETK 13 ETC Child of ETB, ETK 14 ELK Child of ETK 15 ETV Child of ETK, ETL 16 EIX Child of ISS, MGD 17 MGD independent 18 EMX Child of ETK, MGD 19 MTR Child of MGD, MPT, MSE, MTY 25 PER independent entity 26 INV Child of PER. 27 ICH Child of INV. 28 PNH Child of PER. 29 PIN Child of PER. 30 CMT Independent entity, parent of every

row with an embedded CMT_TK. 31 PCM Child of PER, CMT. 32 AD Independent entity.




33 PAD Child of PER, AD, CMT. 34 EML Independent entity. 35 PEM Child of PER, EML,CMT. 36 PHN Independent entity. 37 PPH Child of PER, PHN,CMT. 38 PAR child of CS, PER 39 PCN Child of CS, PER. 40 VXR child of CS 42 RLN independent entity 43 CRX child of CS, RLN 44 PRX child of PAR, PER, RLN 54 CJ child of CS 55 CJP child of CJ, PAR, PER 56 CJI child of CJ 57 DCM child of CS 58 FWO Special situation: An FWO gets

DCM row at printing time. A cancelled FWO has no DCM.

59 CSC child of CS, DCM 60 CIT child of CS, DCM 61 PKC child of CIT 62 GRN child of CS 63 DKT child of CS 64 DVO child of CS 65 DVP child of DVO, PAR, PER 66 SPH child of CS, PER 67 CPO child of CS, PER 68 CLH child of CS, PER 69 ISS child of CS 70 PKI child of ISS 71 NCJ child of ISS 72 NCC child of CS 73 JVO child of CS 74 BND child of CS, JVO 75 AP Child of JVO 76 AR Child of CS, JVO, PER 77 ADJ Child of JVO 83 CBA child of JVO 84 CHK child of JVO, CBA 86 CAX child of AP,CBA 87 PRK child of CS 88 PRC child of CS 89 PSH child of PRC 90 PPR child of PAR, PER, PRC 91 RPS child of CS, PAR, PER




NA DRR Independent NA IRA Independent This table attempts to show the parent to child relationships that have case and various other entities as the parents. Follow the columns down and right to go from CS to its children. As the primary parent with respect to processing, the case table (CS) appears at the top of the diagram. Children of the same parent are shown in alphabetical order from top to bottom. Tables and relationships for person-oriented or for the other minor parent entities are shown on the right side of the table. Proceed down and left to go from parent to child. The other independent parent entities are: PER and RLN Some complex indirect relationships, such as the relationship of CBA and JVO to multiple cases through the cases' AP rows, cannot be shown here.

Tables and sort codes in order by DB2 table name

DB2 Table Code 32 AD 77 ADJ 75 AP 76 AR 74 BND 86 CAX 83 CBA 84 CHK 60 CIT 54 CJ 56 CJI 55 CJP 68 CLH 30 CMT 67 CPO 43 CRX 0 CS

59 CSC 57 DCM 63 DKT 64 DVO 65 DVP 16 EIX 4 ELI

14 ELK 34 EML 18 EMX 5 ETB

13 ETC 10 ETD 7 ETK




6 ETL 11 ETO 8 ETP

12 ETS 15 ETV 9 ETW

58 FWO 62 GRN 27 ICH 26 INV 69 ISS

73 JVO 17 MGD 19 MTR 72 NCC 71 NCJ 33 PAD 38 PAR 31 PCM 39 PCN 35 PEM 25 PER 36 PHN 29 PIN 61 PKC 70 PKI 28 PNH 44 PRX 40 VXR 2 VVC




Sort codes and tables in order by sort code.

DB2 Table Code 0 CS 2 VVC 4 ELI 5 ETB 6 ETL 7 ETK 8 ETP 9 ETW

10 ETD 11 ETO 12 ETS 13 ETC 14 ELK 15 ETV 16 EIX 17 MGD 18 EMX 19 MTR 25 PER 26 INV 27 ICH 28 PNH 29 PIN 30 CMT 31 PCM 32 AD 33 PAD 34 EML 35 PEM 36 PHN 37 PPH 38 PAR 39 PCN 40 VXR 42 RLN 43 CRX 44 PRX 54 CJ 55 CJP 56 CJI 57 DCM 58 FWO 59 CSC 60 CIT 61 PKC




62 GRN 63 DKT 64 DVO 65 DVP 66 SPH 67 CPO 68 CLH 69 ISS 70 PKI 71 NCJ 72 NCC 73 JVO 74 BND 75 AP 76 AR 77 ADJ 83 CBA 84 CHK 86 CAX

Row Archive Table (IRA) Insert an IRA row for each row selected for each case processed. NO IRA rows are to be inserted for rows for DISCIS deleted cases.

FIELD NAME Source of the data IRA_UPD_TS current timestamp IRA_CRT_ITL_NU CS_CRT_ITL_NU IRA_CS_NU CS_NU IRA_LEA_ITL_NU CS_LEA_ITL_NU IRA_CS_TYP_CD CS_TYP_CD IRA_PAR_TYP_CD Only present on AD, INV, PER, PAR rows. For rows associated with

a case participant use PAR_TYP_CD; blank on all other rows. IRA_PAR_TYP_SEQ_NU Only present on AD, INV, PER, PAR rows. For rows associated with

a case participant use PAR_TYP_SEQ_NU; zero on all other rows. IRA_TBL_NU The three character DB2 table name to identify the row data stored in

the ROW IMAGE field. IRA_SRT_CD A two character sort code that will put the DB2 tables in referential

integrity sequence, i.e. child rows after their parent row for easy insertion or before their parent for easy deletion. See Appendix 1 for the code values.

IRA_ROW_TX_01 IRA_ROW_TX_02 Together these text fields form a single 500 byte field to hold the data

from one row. It was split into parts to get around DB2 and NATURAL column length restrictions.

Special handling for tables with nullable columns. Examples: The PRC, PSH, CPO and DVO tables contain nullable columns that hold time of day variables. CPO contains a nullable timestamp column. This list does not show all tables with nullable columns.




The standard layout generated by the DB2 DCLGEN program does not include null indicator variables. Rows with nullable columns will be stored in special IRA layouts that include storage locations for null indicator variables. The data stored for each nullable column and its null indicator will be whatever comes back from DB2. Each null indicator will be the standard 2 byte binary number provided by DB2.

The DB2 documentation claims that the value of a null column is unpredictable. Test runs show that they usually contain low-values (hex ’00’). The null indicator variable contains either binary zero (hex ’00’) for a column that has a value or binary -1 (hex ’FF’) for a column that is null.

For each nullable column the program will store the null indicator at the end of the row image area. The null indicators will be stored in the reverse of the order that the columns appear in the row. The null indicator for the first nullable column goes at the end of the row image area, in positions 499 and 500; the null indicator for the second nullable column goes in the two bytes ahead of the first indicator, in positions 497 and 498; etc. When a new nullable column is added to the row its null indicator will will be stored in the next available location. This means that null indicator storage locations will work forward from the end of the row image area. Obviously care must be taken to make sure that an extremely long row does not overlap its null indicators. The rationale for storing the null indicators at the end of the image area is to minimize data conversions. By storing the null indicators at the end of the row image we make it possible to add new columns and new null indicators without having to move or rearrange existing row data or existing null indicator variables. Removal of a nullable column will require reorganization of the null indicators.

Appendix -- Analysis Decision History 3/19/2013 MDG ITG41 New Program. 7/05/2015 MDG ITG41-2 Iteration 2 – add new purge review date, NCCD deleted case

processing, enhanced reporting, and UTX table restart capability.


Washington State INH EDE Data Integration Administrative Office of the Courts of 28 ACQ-2016-0301-RFP

DQ9420PX - Data Management Destruction of Records (DORR/DOR) Exception Report 1 2 3 4 5 6 7 8 9 10 11 12 13 123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012 ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ¿ 01DQ9420PX Data Management Destruction of Records (DORR/DOR) Exception Report Page XXXXX 01 02MM/DD/YYYY HH:MM:SS 02 03 03 04 Input parameters: Removal Date: XX/XX/XXXX 04 05 Time Limit..: XX:XX:XX 05 06 Commit Flag.: X 06 07 Processed Row Limit: 9999999 07 08 09 09 10 ERROR Input parameter file is empty. 10 11 ERROR Removal date parameter is missing. 11 12 ERROR Invalid time limit (999999) 12 13 ERROR Elapsed time limit parameter is missing. 13 14 ERROR Commit flag (X) Value must be Y or N. 14 15 ERROR Commit flag parameter is missing. 15 16 ERROR Processed row limit parameter is missing. 16 17 ERROR Invalid archived row limit (9999999). 17 18 19 20 20 21 XX:XX:XX elapsed time. Initialization complete. Beginning DOR process. 21 22 22 23 XX:XX:XX elapsed time. Processing time limit exceeded. 23 24 24 25 26 26 27 Court XXX Case XXXXXXXXX XXX XX NOT processed because update timestamp changed during the run. 27 28 Timestamp when selected for processing.: XXXX-XX-XX-XX:XX:XX.XXXXX 28 29 Timestamp when processing was attempted: XXXX-XX-XX-XX:XX:XX.XXXXX 29 30 30 31 ERROR. Input selected case file is out of sequence. 31 32 Previous record. Court: XXX Case: XXXXXXXXX XXX XX 32 33 New record. Court: XXX Case: XXXXXXXXX XXX XX 33 34 34 35 Processed row limit exceeded. The case being processed will be rolled back and skipped. The run will stop. 35 36 36 37 ERROR End of case reached without commit or rollback. Skipping this case. 37 38 38 39 Court XXX Case XXXXXXXX XXX XX Token HHHHHHHHHHHHHHH (case token in hex) 39 40 Case was not processed. 40 41 (followed by standard SQL error output) 41 42 The processing run continued processing. 42 43 (or) 43 44 The processing run will stop now. 44 45 45 46 Row not found for PAR-PER-TK hhhhhhhhhhhhhhhh 46 47 Error on PAR-TYP-CD xxx 99 47 48 48



49 The active row table (IRA) contains processed case data. 49 50 (or) 50 51 No case data has been written to the active row table (IRA). 51 52 52 53 ERROR This program cannot handle document type code (X). 53 54 54 55 ERROR. Input selected case file is empty. 55 56 56 57 XX:XX:XX elapsed time. End of processing for court XXX XXXXXXX cases processed. XXXXXXX cases deleted. 57 58 58 59 --------------------------------------------------------------------------------------------------------------------------------- 59 60 End of processing run. XXXXXXX courts processed. 60 61 61 62 XXXXXXX Selected case records read. 62 63 XXXXXXX Cases were skipped. 63 64 XXXXXXX Cases were processed. 64 65 XXXXXXX DISCIS deleted cases were fully deleted. 65 66 66 67 For table XXX XXXXXXX processed. 67 68 (or) 68 69 For table XXX XXXXXXX deleted, XXXXXXX processed. 69 70 (or) 70 71 For table XXX No rows were processed. 71 72 (or) 72 73 For table XXX XXXXXXX inserted. 73 74 ... (print the appropriate counts for each table) 74 75 75 76 Total records written to the IRA table XXXXXXX (total lines up with detail lines) 76 77 77 78 XXXXXXX database statistical count records written. 78 79 79 80 Average number of cases processed per minute: 9999999 80 81 81 82 Average number of rows processed per minute: 9999999 82 83 83 84 XX:XX:XX elapsed time. Successful completion. 84 85 (or) 85 86 XX:XX:XX elapsed time. ERROR END 86 87 ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÙ87 12345678901234567890123456789012345678901234567890123456789012346789012345678901234567890123456789012345678901234567890123456789012 1 2 3 4 5 6 7 8 9 10 11 12 13



Detailed Program Discussion DQ9420PX – ITG41 This program reads an input dataset from DQ9410PX selection routine that is then sorted by court and key fields. Any cases marked as REPORT only, meaning that #DQ9401LX-CS-OUTRDBYZ-FL = 'R', will be written to the output file immediately. All other cases will be processed. This process deletes cases from the DISCIS database, and inserts all deleted case rows from all processed tables into the IRA table for later copy to an audit GDG dataset. The ETK table sets are NOT copied to the IRA table. In the case of the ETK table, one IRA record is created that contains the case data. This is a place marker. This program also logs audit messages on the UTX table if any cases fail the purge criteria in this program. NOTE: THIS PROGRAM DOES SQL UPDATES, DELETES, and INSERTS. Input format: DQ9401CX Output format: DQ9405LX (263 char) SPECIAL PROGRAMMER NOTE: This program does a commit AFTER reading the next record from the input file, so you always have TWO records in play. The current record in W-SELECTED-CASE-RECORD and the previous record in WO-SELECTED-CASE-RECORD. In order to write out the case data from the COMMIT process step I am using a temporary hold variable WO-SELECTED-CASE-RECORD. All output from this job MUST come from WO-SELECTED-CASE-RECORD or you will be referring to the new unprocessed record instead of the records just processed in DB2. The 040-COMMIT call happens in 022-CTL-BRK-UNIT-OF-WORK-END called from 008-CONTROL-BREAK-TEST. W-DB2-U-OF-W-STATUS-STARTED flag controls COMMIT/ROLLBACK. At this point in the code, you have the changes to the record in WO-SELECTED-CASE-RECORD ready to COMMIT and that case exists in the UTX table and possibly the Output Dataset. The new



record in W-SELECTED-CASE-RECORD is next to be processed. If program does ROLLBACK, be careful to note what last record was actualy processed as COMMIT/ROLLBACK happens after subsequent read on input file. 029-PROCESS-REPORT-ONLY routine updates only CASE and DOCKET. 028-PROCESS-1-CASE routine updates/deletes all other tables. If COMMIT = 'N', then all cases are written to output file with no updates done. If program ends in CODE=99 with time exceeded, then check JIS.PRDDISC.DOR.PRINT.WORK.CASE and/or JIS.PRDDISC.DOR.PRINT.WORK.NAME dataset for the last record processed. Match last COURT processed to find right dataset to check. Restart from same or following record in JIS.PRDDISC.DOR.Q9410SRT.WORK dataset. If this is an EIXISS error, it sometimes clears up on rerun. DO NOT assume the case reported in job output is the last case actually processed. There are two cases in play at any given time. Record read but unprocessed: W-SELECTED-CASE-RECORD Record read/processed/ready to output: WO-SELECTED-CASE-RECORD Reporting only: Update CS table: CS_DTN_NTF_DT and timestamp Update DKT table: Docket and timestamp Purge Order of table deletions: Civil CJI CJP CJ Non-Civil NCC NCJ IF ‘PR’ CASE PKI PRK



ETK – (EMX, EIX, MTR, MGD, ETV, ELK, ETC, ETS, ETO, ETD, WTW, ETP, ETK, ETL, ETB, ELI, and system deletes ETT automatically.) ISS PCN BOTH CASE TYPES DO THESE: PKC CSC CIT DCM FWO RIX RDS AR JSA BND CLH CPO GRN SPH UTR CIX IVT BAA (not purged) ADJ CHK (not purged) CBA (not purged) CAX (not purged) JVO AP DVP DVO PRX CRX RLN PIN (not purged) PNH (not purged)



ICH (not purged) INV PHN PPH (not purged) EML (not purged) PEM (not purged) PCM (not purged) AD (not purged) CMT (not purged) PAD (not purged) PER PAR RPS PPR PSH PRC VXR DKT DRR CS

WPG PREPARED BY: M. Gilbreath DATE: SYSTEM NAME: … S - DQ9420PX... · 2016-05-12 · DQ9420PX -...

Documents

Transcript of WPG PREPARED BY: M. Gilbreath DATE: SYSTEM NAME: … S - DQ9420PX... · 2016-05-12 · DQ9420PX -...