Supercharging Decisions through Additional Data Sources - … · 2017-10-04 · October 3, 2005...
Transcript of Supercharging Decisions through Additional Data Sources - … · 2017-10-04 · October 3, 2005...
October 3, 2005
Credit Scoring & Credit Control IX
8 September 2005
Supercharging Decisions through Additional Data Sources
Dr. Andrew JenningsGeneral Manager
Fair Isaac
2© 2005 Fair Isaac Corporation. All rights reserved. 2© 2005 Fair Isaac Corporation. All rights reserved.
Agenda�Motivation
� Solutions
� Challenges
� Case Studies
� Summary
3© 2005 Fair Isaac Corporation. All rights reserved.
Current banking industry landscape
The problem:
� Customers getting other financial products elsewhere
� Customers poorly understood
� Daily transactions not analyzed
� Attrition indicators not recognized
� Opportunity indicators ignored
� Customers at risk of over-indebtedness
4© 2005 Fair Isaac Corporation. All rights reserved.
Slow
Fast
High
Tim
elin
ess
of
Info
mat
ion
Where will the improvement come from?
BasicBasic
AdvancedAdvanced
Low
5© 2005 Fair Isaac Corporation. All rights reserved. 5© 2005 Fair Isaac Corporation. All rights reserved.
Agenda� Motivation
�Solutions
�Advanced technology
�Increased understanding
�Better decisions
� Challenges
� Case Studies
� Summary
6© 2005 Fair Isaac Corporation. All rights reserved.
Advanced technology in high-tech scientific and military applications can be migrated to commercial use
� Language and Text Mining from massive multi-lingual textual data
� Information retrieval and Knowledge extraction
� Automatic document categorization, trend discovery, and decisioning
� Advanced language representation, understanding, and question answering
� Bioinformatics Research
� Unique DNA ‘fingerprint’ identification
� Advanced Military Research
� Military target recognition
� Hierarchical reinforcement learning algorithms for war-game simulations and automatic strategy learning
� Neuroscience
� Sparse-coded brain-like associative memories
����
�������
��� ������ ������������� �������������������������������������������������� ����������������� !�� "
7© 2005 Fair Isaac Corporation. All rights reserved.
Visualizing trends in call center dataEarly warning of parts failure in automotive accident data
“The Ford-Firestone dispute blew up in August 2000 and is still going strong. In response to claims that their 15-inch Wilderness AT,
radial ATX and ATX II tire treads were separating from the tire core--leading to grisly, spectacular crashes--Bridgestone/Firestone
recalled 6.5 million tires, mostly original equipment on the Ford Explorer, the world's top-selling sport utility vehicle (SUV).” - Forbes
8© 2005 Fair Isaac Corporation. All rights reserved.
Example - Pairwise Co-occurrence Consistency (PCoC or “Peacock”)
Salon Products
Spa Services
Pharmacy
Supplements Books
Body Care
Mercantile
Bulk Herbs
Wine
Beer
Meat Seafood
Non-
Food Grocery
Grocery
CheeseBulk Dairy
Frozen Deli
Deli (Hot Foods)
Juice Bar
Bakery
Sushi
Floral
Produce
Salon Products
Spa Services
Pharmacy
Supplements Books
Body Care
Mercantile
Bulk Herbs
Wine
Beer
Meat Seafood
Non-
Food Grocery
Grocery
CheeseBulk Dairy
Frozen Deli
Deli (Hot Foods)
Juice Bar
Bakery
Sushi
Floral
Produce
Computer
Hardware
Video
Games
Joystick
Computer
Hardware
Video
Games
JoystickJoystick
9© 2005 Fair Isaac Corporation. All rights reserved.
Example - Pairwise Co-occurrence Consistency (PCoC or “Peacock”)
ClassicalClassical
CookwareCookware
Car RadioCar Radio
AppliancesAppliances
Latin/MexicanLatin/Mexican
LANLAN
TVTV
SEGASEGA
CTOCTO
SatelliteSatellite
DSLDSL
FitnessFitness
ServicesServices
A/CA/C
Internet TVInternet TV
ROCKROCK
Office
Furniture
Office
Furniture
RenewalsRenewals
Freezer
Washer/Dryer
Freezer
Washer/Dryer
10© 2005 Fair Isaac Corporation. All rights reserved. 10© 2005 Fair Isaac Corporation. All rights reserved.
Agenda� Motivation
�Solutions
�Advanced technology
�Increased understanding (additional information)
�Better decisions
� Challenges
� Case Studies
� Summary
11© 2005 Fair Isaac Corporation. All rights reserved.
Notion of “Transactions”
� Transaction is a time stamped record of an event or state associated with an entity or a unique key
� Sorted and merged transactions provide a complete time series history
� Characteristics of transactions
� Transactions are heterogeneous
� Transaction volumes can be enormous
� Transaction timing is irregular
� An opportunity to change the outcome – real time decisions
����
��������
� � #�$%���������&�%��������� ���� �������'��&����' ���$()�������������������� ���������������!� !�� "
12© 2005 Fair Isaac Corporation. All rights reserved.
The Key to Transaction Data is to Understand Each Field
� Who
� Account number, name, cardholder country, cardholder zip
� When
� Transaction date and time, velocity, shopping times
� Where
� Geographic region / metro classification, proximity, distance
� What
� Type of business (MCC/SIC), goods sold, services offered
� High-end / low-end, typical spend: Chain, brand, loyalty
� Product codes (SKUs) and brand preferences
� How much
� Cash usage, charge-backs, payments
� Spending compared to others by Merchant, MCC/SIC, Post Code
13© 2005 Fair Isaac Corporation. All rights reserved.
Understanding further enhanced by text
�The text that you rarely use in analytics
�� Collectors notes, Call center notes, phone call transcriptsCollectors notes, Call center notes, phone call transcripts
�� Notes in credit bureau reportsNotes in credit bureau reports
� Annotations on accounts, text in application forms (online/offline)
�� Merchant names in credit card transactionsMerchant names in credit card transactions
� Codes, annotations and other unstructured data that people that people understandunderstand, but is ignored in modeling & scoring.
14© 2005 Fair Isaac Corporation. All rights reserved.
What is the Problem with MCC Codes?
ATM
Debit
card
Credit
Card
Standing
Orders
Direct
Debits
Customer
View
MCC codes present
MCC codes NOT present
15© 2005 Fair Isaac Corporation. All rights reserved.
SICs simply cannot identify skiers
0.00%
0.25%
0.50%
0.75%
1.00%
1.25%
Text Analysis SIC
Percent of
All Cardholders
Identified as
Skiers
Method
Transaction Analytics with text processing finds the skiers
16© 2005 Fair Isaac Corporation. All rights reserved.
Example Infant PurchasersFair Isaac’s Targeted Solution
#3
#2
#1
MCC codes miss so many merchants
17© 2005 Fair Isaac Corporation. All rights reserved.
How much ? Balance Spin-down
Spin-down patterns
Same Aount
Within $100
Within $100-$500
$500+
Minimum
Payment Only
-100%
-50%
0%
50%
100%
150%
200%
250%
Payment Patterns
Re
lativ
e R
isk T
o A
vera
ge
���*
��������
��� +���&�������������'��'�� �������������� ����$��������������� ���������������!� !�� "
18© 2005 Fair Isaac Corporation. All rights reserved.
# C
ash
Ad
va
nces
# C
ash
Ad
va
nces
Revolving Balance Amount
HighHigh
LowLow HighHigh
John Doe’s Profile Two Weeks Ago
John Doe’s Profile Last Week
John Doe’s Profile This Week
Trending - Recognize Shifts in Individual Cardholder Behavior
19© 2005 Fair Isaac Corporation. All rights reserved.
HighHigh
HighHigh
John Doe’s Profile
Portfolios Aggregate “Good”
Profile
# C
ash
Ad
va
nces
# C
ash
Ad
va
nces
Revolving Balance AmountLowLow
Compare and Contrast Aggregated Behavior
Portfolios Aggregate “Bad”
Profile
��*�
��������
���� #�$%����'��$()���$���$�$$��������� ����������������� !�� "
20© 2005 Fair Isaac Corporation. All rights reserved. 20© 2005 Fair Isaac Corporation. All rights reserved.
Agenda� Motivation
�Solutions
�Advanced technology
�Increased understanding
�Better decisions
� Challenges
� Case Studies
� Summary
21© 2005 Fair Isaac Corporation. All rights reserved.
Better decisions and more timely decisions
665 620708 690
£500Cash adv
Overlimit
Min paymentmadeMin payment
madeNSF hits
Delinquent
CustomerActions
Jan 1 Feb 1 Mar 1 Apr 1
719 683 667 622
BehaviourScore
TransactionScore 600 580
22© 2005 Fair Isaac Corporation. All rights reserved.
Triggers for Marketing Communication
Home Home
ImprovementsImprovementsHome EquityHome Equity
LoanLoan
Car Broke Car Broke
DownDownAuto LoanAuto Loan
MarriageMarriageMortgage or Mortgage or
Rental InsuranceRental Insurance
College College
Graduation Graduation Card UpgradeCard Upgrade
New BabyNew Baby Life InsuranceLife Insurance
23© 2005 Fair Isaac Corporation. All rights reserved.
Cross-sell solutionsCar loan trigger
Trigger definition
� Identification of large valuemotor repair transactions
� Identification of towing andcrash repair transactions
� Identification of driving school and test fee transactions
� Trigger could includepre-approved decision criteria
Application
� Cross-sell for car loans
24© 2005 Fair Isaac Corporation. All rights reserved. 24© 2005 Fair Isaac Corporation. All rights reserved.
Agenda� Motivation
� Solutions
�Challenges
�Data volumes
�Variable reduction
�Stability
� Case Studies
� Summary
25© 2005 Fair Isaac Corporation. All rights reserved.
Cardholder Profiles
� Profiles utilize a wide array of information, examples
� Long term typical historical behavior of the cardholder
� Speed of spending
� SIC/MCC codes frequented
� Favorite shopping hours and days
� Distance from home
� How often the card is used
� Not just transaction data
� Profiles also look at masterfile data
26© 2005 Fair Isaac Corporation. All rights reserved.
Analytic methods for periodic trigger design
Comb or shaped filters
� Simple technique, easy to design
� Allows for the encoding of intuitive triggers or events
�Facilitates encoding of payroll detection
� Can increase complexity with Fourier / Cosine transforms
Deposits Withdrawals Cash
27© 2005 Fair Isaac Corporation. All rights reserved.
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Salary Biweekly fixed interval Fixed amount, $2166.98
Mortgage One month fixed interval Fixed amount, $1245.32
Car Loan payment One month fixed interval Fixed amount, $399.00
Pattern detectionBonus! Baby!
28© 2005 Fair Isaac Corporation. All rights reserved.
T = 3y / (c2 + w) * log (q)
Data Spiders find predictive variable combinations
Salary depositsOverdrafts
Vanity purchases
Home improvement
College spendingFamily vacationsCar repair
SSA deposits
Mortgage payments
BalancesCar p
ayments
To
ys
AdventureSportsFine dining
Health products
Branch visits
ATM WithdrawalsInternet
Electronics
Golf
Discount buying
Boating
Nascar
Casinos
Do
nat
ion
s
Opera
Paw
n S
hop
s
SkiingAntiques
Infant
purchases
Work travel
Sm
all b
usi
nes
s
Address change
Lan
dsc
apin
g
Gas bill
Pet buys
Graduation
WeddingRetirement
Ele
ctri
c bi
ll
Groceries
Cash Adv. Out-of-pattern
large deposit
Jewelry
Gas
Camping
Mutual Fund
TransfersTax Preparation
Boating
Furniture
Sta
y-at
-ho
me
par
ent
Co
mp
ute
rs
Fast food
V = z * n / log (q)Q = 2p / (c2 + w) * log (q)
29© 2005 Fair Isaac Corporation. All rights reserved.
Transaction Model Stability
Transaction-based models can be very powerful, but they can be less stable. Why?
������������
30© 2005 Fair Isaac Corporation. All rights reserved.
Decision Hysteresis
Applying simple decision thresholds can result in unstable decision making
�����
����
��������
���,
��������
� �� - ��� %������$������&����� ����$ �������$�#./�������������'���'���� ������������ ���������������!� !�� "
31© 2005 Fair Isaac Corporation. All rights reserved.
Applying Decision Hysteresis Solves This
Apply an upper and lower threshold, one to activate the decision, one to deactivate the decision
�����
����
�����������
������������
��������
32© 2005 Fair Isaac Corporation. All rights reserved.
724
702
651
663
707710
653
600
620
640
660
680
700
720
740
2/23 2/28 3/4 3/9 3/14 3/19 3/24 3/29 4/3
Time
Sc
ore
(B
eh
av
ior
Sc
ore
Sc
ale
)
Transaction Score Monthly Score
What about Stability?
� Real Data Example: How much do scores change during a month?
33© 2005 Fair Isaac Corporation. All rights reserved.
Long term stability – Transaction scores
� How much do TRIAD Transaction Scores change over the course of a few months?
0
10
20
30
40
50
60
70
80
90
100
Q1 Q2 Q3 Q4
% o
f A
ccounts
in S
am
e Q
uart
ile
Initial Month 2 Month 3 Month 6 Month 9
0
10
20
30
40
50
60
70
80
90
100
Q1 Q2 Q3 Q4
% o
f A
cco
un
ts i
n S
am
e Q
uart
ile
Initial Month 2 Month 3 Month 6 Month 9
� How much do Behaviour scores change over the course of a few months ?
34© 2005 Fair Isaac Corporation. All rights reserved.
Other Challenges
�Scalability
�Flexibility
�Deployment (models, triggers)
�Automation (test & learn)
�Cost
35© 2005 Fair Isaac Corporation. All rights reserved. 35© 2005 Fair Isaac Corporation. All rights reserved.
Agenda� Motivation
� Solutions
� Challenges
�Case Studies
� Summary
���!
��������
� � ( ����� ���-���������% ��������$�����$�������� ��������&������'��&'��&��'0��1������� ���������������02��� ���$�''����34��������� ����������������� !�� "
36© 2005 Fair Isaac Corporation. All rights reserved.
Credit Risk Model Performance Improvement
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
Percent Good Detected
Pe
rce
nt
Ba
d D
ete
cte
d
Transaction Based Behavior Score Master File Based Behavior Score
0
10
20
30
40
50
60
70
80
90
100
0 2 4 6 8 10
37© 2005 Fair Isaac Corporation. All rights reserved.
Young Account Segment Benefits More
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
Percent Good Detected
Pe
rce
nt
Ba
d D
ete
cte
d
Transaction Based Behavior Score Master File Based Behavior Score
0
10
20
30
40
50
60
70
80
90
100
0 2 4 6 8 10
38© 2005 Fair Isaac Corporation. All rights reserved.
What about Results?
� Real Data Example: Authorisations
Authorisations - In order Accounts Transaction Monthly
28% of overlimit authorisation attempts declined on current accounts Score Score
# of current accounts
# of attempted overlimit authorisations 188,052 188,052
% of transacting bads identified with score cut-off 67.6% 62.7%
% improvement in bads identified 7.8%
# of bads identified with score cut-off 6,764 6,272
Additional bads identified 492
Bads to charge-off ratio 65.0%
Additional future charge-off identified by transaction score 320
Average transaction amount £170
Balances saved from C/O £54,390
Monthly Authorisation Benefit for In order Accounts £54,390
Annual Authorisation Benefit for In order Accounts £652,676
39© 2005 Fair Isaac Corporation. All rights reserved.
What about Results?
� Real Data Example: Authorisations
Authorisations - One Cycle Accounts Transaction Monthly
27% of one-cycle accounts not eligible for authorsation Score Score
# of one cycle accounts
# of auth requests per one-cycle delinquent account 1.0 1.0
# of one cycle accounts requesting authorisation 107,500 107,500
% of total bads identified with score cut-off 63.7% 55.8%
% improvement in bads identified 14.2%
# of bads identified with score cut-off 10,496 9,191
Additional bads identified with transaction score 1,305
Bads to charge-off ratio 65%
Additional future charge-off identified by transaction score 849
# of future charge-off accounts requesting an overlimit authorization 849
Average transaction amount £170
Total balances saved from charge-off £144,247
Monthly Authorisation Benefit for Underlimit One Cycle Accounts £144,247
Annual Authorisation Benefit for Underlimit One Cycle Accounts £1,730,958
40© 2005 Fair Isaac Corporation. All rights reserved.
First party fraud for large UK retail bankThe elements of first party fraud
� Definition:
�When a customer takes credit with no intention of repayment
Manifestation
First payment defaults or very early defaults
Excessively over-limit
Behaviours
Open accounts with no intention of paying on them
Open accounts with false information
Boost credit limits – artificially and through manipulation of behaviour scores
False claims of fraud
41© 2005 Fair Isaac Corporation. All rights reserved.
First party fraud is big
First Party Fraud
� Perpetrated by customers
� Manifests as kiting, churning first pay default, skips
� No customer validation when it happens
� Very little control or detection of the problem
Third Party Fraud
� Perpetrated by others
� Manifests as lost/stolen, counterfeit, forgery
� Customer confirms the occurrence of fraud
� Many controls and fraud detection systems in place
Estimated loss ratio = 100 basis points
Estimated loss ratio = 10 to 12 basis points
First Party Fraud is significantly different than Third Party Fraud
42© 2005 Fair Isaac Corporation. All rights reserved.
(3) New Limits Displayed
Individual qualifies for new overdraft limits and more loans
A closer look revealed a common fraud scheme
First party fraud schemes:
(6) Write-off
Account is written off, typically as a bad debt, not fraud since no known victim
(5)Collections/Recovery
Account slips into collections, the individual in most cases skips or changes identity
(4) Draw Facilities
Individual takes as much as they can from the current account and other loans
(2) Transacts
Individual transacts heavily over time to give the appearance of creditworthiness
(1) Account Opening
Individual opens account with some true and some false information or documents
43© 2005 Fair Isaac Corporation. All rights reserved.
Solution
� Neural network models
�Use data from 48 sources (customer, account and transaction)
�Score each account daily if a new transaction was made
�Process millions of transactions a day
�Utilise a sophisticated profiling engine
� Case Manager
�Establish a holistic view into the customer relationship
�Provide prioritisation of cases based on risk
44© 2005 Fair Isaac Corporation. All rights reserved.
We reduced First Party Fraud losses by over 50%
First Party Fraud (Basis Point Losses)
20.0
40.0
60.0
80.0
100.0
120.0
140.0
Jan-
02Feb
-02
Mar
-02
Apr-02
May
-02
Jun-0
2Ju
l-02
Aug-02
Sep-0
2O
ct-0
2Nov-0
2Dec-
02Ja
n-03
Feb-0
3M
ar-0
3Apr-
03M
ay-0
3Ju
n-03
Jul-0
3Aug
-03
Sep-0
3O
ct-0
3Nov-0
3Dec-
03Ja
n-04
Feb-0
4M
ar-0
4
Advances
45© 2005 Fair Isaac Corporation. All rights reserved.
Cross-sell Home Equity Lines based on DDA (Current Account) Transactions and other data
� Goal: Derive Conversion Models, based upon customer transaction behavior, which can be applied to DDA customers for Home Equity Line of Credit Cross Sell Opportunities
� Challenges:
� Identify existing mortgage holders
� Timely deployment
� Use Data Spiders to find predictive transaction characteristics
46© 2005 Fair Isaac Corporation. All rights reserved.
Check Transactions
$0
$500
$1,000
$1,500
$2,000
$2,500
$3,000
$3,5000
22
44
61
80
10
0
11
7
13
5
16
4
18
2
20
3
22
5
24
2
Days
Ch
eck A
mo
un
t
Inferred as mortgage payment
Illustrating the Mortgage Detection Algorithm
47© 2005 Fair Isaac Corporation. All rights reserved.
Examples of transactions found by Data Spiders
� Proportion of all transactions done via ATM over the last 210 days
� Time since the most recent home improvement transaction in the last 240 days
� Ratio of the number of txns at MCC 5814 (restaurants) in the last 20 days to the last 210 days
� Percentage of transactions greater than $1,000 in the last 240 days
� # of checks greater than $50 in the last 240 days
� # of days since the most recent home improvement transaction in the last 180 days
� Maximum amount of all home improvement transactions in the last 240 days
48© 2005 Fair Isaac Corporation. All rights reserved.
Results
CheckingAccounts3 million
Home Equity Lines
3%
�Typical Home Equity Line worth $400 - $700
�Bank with 3 million checking accounts
�Increase by 2% the checking account customers with Home Equity LOC
�Increase profit $24 million
5%
49© 2005 Fair Isaac Corporation. All rights reserved.
Context Vector Technology Applied to Collections Modeling
� Once in collections beyond 1 cycle delinquent little structured transactional data
� Difficult to predict customer behavior in collections cycle
� Little or no differentiation in customer treatment
� Late stage collections tactics not well used by issuer
50© 2005 Fair Isaac Corporation. All rights reserved.
Mathematical representations of meaningLearning meaning from text data through analytics
“You shall know a word by the company it keeps”
--(Firth,1957)
51© 2005 Fair Isaac Corporation. All rights reserved.
Context defines meaning in a robust way
� What is this word?
� iprmoetnt
� Now guess what this word is:
� "the only iprmoetnt thing is that..."
� Now read this entire sentence:
� 'Aoccdrnig to a rscheearch at an Elingsh uinervtisy, it deosn'tmttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnttihng is taht frist and lsat ltteer is at the rghit pclae and the cotenxt. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae we do not raed ervey lteter by itslef but the wrod as a wlohe. ceehiro.'
52© 2005 Fair Isaac Corporation. All rights reserved.
Meaning = Statistical Context
Source Text
Customer undergoing divorce proceedings. Claims spouse is responsible.
Five Stem Window
N1j N2j Tj N3j N4j
Customer undergoing divorce proceedings claims
TARGET
NEIGHBORS
53© 2005 Fair Isaac Corporation. All rights reserved.
Collections Notes Vectors Cluster into segments with different payment potential
!"�������#�$��!"�������#�$��
%���"�������!������%���"�������!������!"��������������&��'���(�!"��������������&��'���(�
�)�"������*����)�"������*���
�������5�%&��/���������5�%&��/��
!+����������)����!+����������)����
�)�"������))���'��&��(�)�"������))���'��&��(
������,�-����������,�-����
��(�����������-�����(�����������-���
-�����(����)"���-�����(����)"���
��(��������(���+��&��(��������(���+��&
54© 2005 Fair Isaac Corporation. All rights reserved.
Concept Space for Collection NotesWhy do borrowers don’t pay?Text ���� “Reason Codes” ���� Appropriate Actions
55© 2005 Fair Isaac Corporation. All rights reserved.
Context Vector Technology Applied to Collections Modeling
��Portfolio Receivables Valuation pilotPortfolio Receivables Valuation pilot
� Medium sized bank
� 590K accounts with text notes (58% of all accounts)
� 1 to 280 comments per account with average=14
� Target:
�12 mo collection (as % of owed) on date of each comment
�First 3 months of comments used for modeling records
���"
��������
��� 6������������������ �����$������������������ ����������������� !�� "
56© 2005 Fair Isaac Corporation. All rights reserved.
Context Vector Technology Applied to collections Modeling
Dramatically Better Receivables Valuation Models
More accurate prediction leads to lower reserves:
Total actual dollars = $33,469K
Predicted with text = $42,021K (125% of actual)
Predicted with no-text = $67,460K (201% of actual)
Better lift:
Model KS with no-text =27.8
Model KS with text =52.5
Increase in model accuracy driven by text =88%
���,
��������
��� 7 %�������$���&�������$�3� ��� ������ �����$��������� ����������������� !�� "
57© 2005 Fair Isaac Corporation. All rights reserved.
Summary
� Increasing wallet share while managing risk and fraud requires new techniques for better decisions
� Decisions can be improved through better technology, more information, and more timely information
� Technology from a variety of sources can be used for commercial benefit
� Transaction data
� captures important events and identifies changing patterns of behavior
� is further enhanced through text or unstructured data
� allows for more timely action
� Working with transaction data is challenging, BUT
� The results are large!