How to Refine Your Data De-duplication Strategy

© Experian Limited 2008. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited.

Other product and company names mentioned herein may be the trademarks of their respective owners. No part of this copyrighted work

may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian Limited.

Confidential and proprietary.

How to Refine Your Data De-duplication Strategy

Thursday, August, 12th, 2010

Teleconference:

Dial-in: 1-800-214-0745

Passcode: 697685

© Experian Limited 2008. All rights reserved.

Confidential and proprietary. 2

Welcome!Introductions and Overview of Today’s Session

Experian QAS reviews the impacts of duplicate records in an organization

Today’s speakers:

Cait Porte

Product Manager, Experian QAS

Liz MacKenzie

Marketing Program Specialist, Experian QAS

Best practices for eliminating and merging records

Tutorial of QAS Unify

Questions from the audience



Where Does Removing Duplicate Records Fit in the Overall Data Quality Process?

Step 1: Understand your data

Step 2: Clean existing data

Step 3: REMOVE DUPLICATE RECORDS

Step 4: Enhance and update data

Step 5: Verify data during all capture processes

Step 6: Continue to enhance, update, and learn



Today’s Focus: Impacts of Duplicates



Tip 1: Understand Your Database

Ask the following questions:

What information are you taking in?

How often is information being taken in?

How is that information being formatted?



Tip 2: Define your Criteria

Decide what data you want to merge

What elements are you looking to match on

Will you be de-duping entire file vs. a segmented portion

Choose what level of de-duping is required for your organization



Tip 3: Pull Data to be Merged

Understand how your data is formatted

Is your information standardized when it is entered into your database?

Will you have to manipulate the data?



Tip 4: Assess Match Rules

Methods of matching

Manual/visual review

Comparing records one-by-one

DBA queries

Rules set by DBA to do one level matching

Software/Service

Multi-level matching capability

1 JOHN PENNY 123 MAIN ST

2 JONATHAN PENNIE 123 MAIN RD



Tip 4: Assess Match Rules

Matching Techniques

Phonetic

Ex: Dorhety = Doherty

Character Occurrence

Ex: Watson = Waston

Table-based

Ex: James=Jack=Jim=Jimmy

Element matching

Ex: Mr. S. Jones = Sam Jones = Jones Sam



Tip 5: Decide How to Import De-duped File

Now that duplicates have been identified you must:

Merge/purge de-duped data

Put “clean” information back into your database

Determine what channel is appropriate to import your data

.csv

.xls

.dat

.txt

SQL

Oracle

ODBC



Tip 6: Continue Regular Maintenance

Data is constantly changing

By regularly removing duplicates from your database enables you to:

Save money

Better understand your customer

Increase customer satisfaction

Efficiently use time



QASProducts & services

Real-time verification Clean & enhance

Clean

QAS Batch

QAS Bulk Processing

Phone & Email Batch

Enhance

QAS Unify

NCOALink®

Address

QAS Pro

QAS Pro On Demand

QAS Pro Web

QAS Pro API

Phone and Email

QAS Phone

QAS Email



QAS Unify Tutorial

QAS Unify



Today’s Focus: Impacts of Duplicates



Please visit www.qas.comfor more information.

http://www.qas.com/?tid=2326

http://www.qas.com/?tid=2326

How to Refine Your Data De-duplication Strategy

Technology

Transcript of How to Refine Your Data De-duplication Strategy