Pain Free Duplicate Record Merging Denise Cordrey Emily Carr University of Art + Design...
-
Upload
marvin-boardley -
Category
Documents
-
view
217 -
download
0
Transcript of Pain Free Duplicate Record Merging Denise Cordrey Emily Carr University of Art + Design...
Pain Free Duplicate Record Merging
Denise Cordrey
Emily Carr University of Art + Design
Kelly NymanHEITBC/OA Solutions
Who We Are?
Consortium of eight schools in British Columbia
Collaborate on all aspects of ColleagueRange in FTE’s from 500 to 7,0006 Service Bureau schools2 Stand alone schools OA Solutions is our Service Provider
Where We Are?
Camosun College
College of New Caledonia
College of the Rockies
Emily Carr University of Art + Design
Nicola Valley Institute of Technology
North Island College
Northern Lights College
Northwest Community College
Developing a Duplicate Merge Utility
Duplicates!Worked with our Service Provider to
develop a way to deal with the problem of duplicate ID’s
First developed the XMGD Merge Utility in 2003
Further enhancements have been made over the last two years
Items To Consider
What hasn’t been developed yet: Financial Aid Colleague Advancement SQL server has not been tested
What it can’t do: Merge duplicate accounts when the ‘merged from’ ID is used in Payroll There is no restore method after process has been run
General Overview
• Identify Duplicate Records• Determine where the ID’s are used• Determine which ID to keep• Pre-steps and Report to run• Running the Merge utility• Post steps and Clean up
Identifying Duplicate Records
Reports XIBD (bday) and XIDL (name)One person coordinates all student
duplicatesConfirms if it is a true duplicateIdentifies the source – Credit/Continuing
Studies and forwards them to be mergedOrganizations and Vendors are handled
by our Finance folks
Where ID’s Are Used
Determine where the ID’s are usedUse the Lookup resolution screens to
determine the “Where Used” information NAE ORGP VEND
This also helps to determine which records to keep and which to merge
NAE Screen
Determining Which ID To Keep
Duplicate ID’s can group themselves across a number of different applications:
• CF, ST and HR• CF and ST• CF only• ST only
ID Used: CF, ST, HR
Keep the ID that has any Employment history (EMP) or HR history (HRP)
No EMP of HRP activity? Then keep the one with the most “where used” codes
If both ID’s are equal in “where used” then keep the ID with the most Transactions
Note: If a ID has Employment history or has HR history you can not use this ID to “merge from”
ID Used: CF and ST
If both ID’s are equal in “where used” codes then keep the ID with the most Transactions.
For the number of CF transactions look on VENI
For the number of ST transactions look on STAC & ARAI
ID Used: CF or ST only
When an ID has been used in CF onlyYou have 2 options: Review the VENI form and keep the ID that has the most
transactions Keep the account with the latest CORE information
When an ID has been used in ST only Keep the ID that has the most transactions Review the STAC & ARAI forms for the number of ST
transactions
Retaining/Discarding
The ID that is to kept be will be referred as “Merge To” ID
The ID to be removed is referred as “Merge From” ID
Temporary Naming Of An ID
When the merge process cannot be dealt with in a timely manner
To prevent other users from adding more transactions to the “Merge From” ID. We need to rename the “Merge From” ID to
reference the ‘merge to ID’
In this example “Duplicate See 0002570”
Duplicate Look Up
Pre-Merge Preparation - ST• Fields that this utility does not merge• Capture Core information for both records
before merging the ID’s NAE Name and Address Entry ORGP Organization Profiles BIO Biographic information EMPL Employment information FINF Foreign Person Information EMER Emergency Information DADD Additional Demographics FNM Formatted Names ADSU Address Summary
Pre-Merge Preparation - ST
TRAN -Transcript for both IDs. Use the transcript grouping(s) that capture all credit types
ARSP - AR Statement Print report for both IDs. With a start year of 1960
STAL – Student Academic Level. For the “Merge From” ID, SACP Student Academic Program SACP. Resolution form only Check that there is no activity on the ’Merge From’ ID in the
Finance Application.
Pre-Merge XMGR
Run XMGR to provide comparison reports of the modified files
This will identify anything that was missed with the “Where used”
After the merge is completed you can match XMGR post list to verify that the merge was successful
XMGR
XMGR Report
XMGD
XMGD Report
Post-Merge XMGR
Run the XMGR report again and keep the report
Compare the Pre and Post XMGR report
Retaining the report for future use will help un-merge the ID’s if needed
Post-Merge XMGR
Manual Post–Merge Processes
NAE – Name and Address Entry ADSU – Address Summary (ensure that all of the address history has copied) EMAIL – XMGD does not merge email information. In the account you are going to keep,
ensure that the email types, addresses and preference flag are correct. BIO – Biographic Information FINF – Foreign Person Information (check that statuses have copied correctly) EMER – Emergency Information DADD – Additional Demographics TRAN - Transcript Print for both accounts. Use the transcript Grouping ‘ALL’ so that you
can see CS, GS and OFT courses. STAL – Student Academic Level SACP – Student Academic Program VENI –Vendor History (review the vendor forms if applicable) RTSL – Recalculate Term Status Load (recalculate if applicable)
Duplicate Record Delete
When you are sure that all of the information has copied over correctly, go to the ‘merge from’ ID record and delete it.
Merge/Convert ProcessThe Nuts and Bolts
XMGR
XMGR role in the Merge/Conversion• Ensures to the user that the files converted
properly• Provides debug info to illustrate what files
have been converted• Help if needed to unmerge properly
Note: XMGR is coded identical to XMGD
Pre Process
Load Pre – defined lists in XMGD that exist for files and fields.
One list is where ID is part of the Primary Key for files.
The other list is for the ID that is a field in a file.
Verify if “Merge From” ID used is not one of the following: HR, Financial Aid, Benefactor
Note: There are over 270 fields effected by this process
The Main Process
Loops through each list creating Select statements to gather records based on the “Merge From”
Changes the record to the “Merge to” IDSome files will call on a separate GOSUB so
that extra processing that can be doneBackup record in Backup file
Naming convention is filename*recordID
Post Process
Error reports is created when the file could not be changed or ID’s are equal to each other
Conversion report is createdDelete Web access to WebAdvisor on “Merge
From” account
NOTE: If an error occurs based on a file could not be read the process will stop immediately. You would have to do research of the cause and manually edit records to ensure conversion/merge can be done properly. This includes un-merging records already merged.
Unmerge Process
An IT or database admin will need to analyze each and every record that been effected by the process Review backup’s and reports Determine if the file has been touched since the date
of the merge Revert change either in colon prompt or use EDRC
form (although EDRC can be tricky and should be avoided)
Tips and Best Practices
Try to keep your data clean and teach Users to avoid duplicates
Keep all pre and post screen shots and reports of the merge
Ensure that the ID’s are correct before placing a Yes to proceed