Artificial Intelligence_ Rise of the Machines _ the Economist
Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)
-
Upload
vishal-bamba -
Category
Documents
-
view
100 -
download
1
Transcript of Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)
![Page 1: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/1.jpg)
Customer Intelligence –Harnessing Elephants at Transamerica
Stephen Lloyd BI Architect, Transamerica Life & Protection
#strataconf #hadoopworld
Vishal BambaChief Architect Transamerica Life & Protection
Dave BeaudoinAVP Transamerica Investments & Retirement
![Page 2: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/2.jpg)
2#strataconf #hadoopworld
Agenda
• About Transamerica
• Current State and Data Architecture
• Use Cases / Learnings
• POC Highlights
• Solution Design
• Lessons Learned
• Questions / Discussion
![Page 3: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/3.jpg)
3#strataconf #hadoopworld
Transamerica Org Structure
• Investments & Retirement– Retirement and Benefit Plan services to employers/employees
– Mutual funds and variable annuities
– Mission to help people save and invest wisely to secure their retirement dreams, the I&R business unit serves more than 3 million retirement plan participants across the entire spectrum of defined benefit and defined contribution plans.
• Life & Protection– Term and Perm insurance products
– Medicare supplement, long term care, accidental death, final expense
– Mission to protect what you’ve built, secure what’s next.
![Page 4: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/4.jpg)
4#strataconf #hadoopworld
Data Architecture
• Rich data environment across organizational business units, comprised of many source systems across various platforms
• A consistent enterprise view of data across business units is required
![Page 5: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/5.jpg)
5#strataconf #hadoopworld
Data Architecture
![Page 6: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/6.jpg)
6#strataconf #hadoopworld
Data ArchitectureAs in many industries, we are focused on leveraging technology to build data-driven customer relationships.
How can the current data architecturesupport this strategic direction?
![Page 7: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/7.jpg)
7#strataconf #hadoopworld
Data Architecture
Enterprise Data Management Crossroads
Another traditional warehouse project?
or
Enterprise data hub/lake/ocean with new
technologies?
![Page 8: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/8.jpg)
8#strataconf #hadoopworld
Data Architecture – New Direction?
![Page 9: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/9.jpg)
9#strataconf #hadoopworld
• 360 degree view of consumers for marketing, planning, and analytics• Discover and mine relationships• Create highly targeted and individualized marketing programs
The Vision
Enterprise Marketing and Analytics Platform
The How• Co-location, master data management, custom data quality and cleansing rules
and more• Which allow integration of data from across and outside of the company to
create 360 view of consumers
![Page 10: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/10.jpg)
10#strataconf #hadoopworld
Use Cases
Complete a Data Append ProcessAppend Data Elements from Enrichment sources to Customer/Prospect recordsTests: Data Integration, Processing Power
![Page 11: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/11.jpg)
11#strataconf #hadoopworld
Use Cases
Score Prospects with a Predictive ModelScore prospects with the current model to predict likelihood to respond to a direct marketing offerTests: Processing Power, Predictive Modeling
![Page 12: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/12.jpg)
12#strataconf #hadoopworld
Use Cases
Measure online contracts/sales, create visitor personas, create simple attribution
Match weblog visitor data to Salesforce leads and to policy holders to generate a sales pipeline. Join to enrichment sources to create personas. Join to Direct Mail solicitation history and test for correlation.Tests: Data Integration, Processing Power, Analytics
![Page 13: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/13.jpg)
13#strataconf #hadoopworld
Web Log
Salesforce
Epsilon
Direct Mail History
POC Success
Join weblogs > CRM > Demographics > mailings
Enabled:• Connected online activity to offline activity• Created time series of events• Created demographic profile
TOTAL ANALYSIS TIME = 2 hours
120k visits4,252 ppl
14% in niche35% income > $100k
83 sent DM after visit
**Use case figures are for illustration purposes only**
![Page 14: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/14.jpg)
14#strataconf #hadoopworld
Summary of Key BenefitsProvides a single platform to house key customer and
prospect data sources,
Establishes persistent keys across previously disparate data sources ,
Provides for rapid intake of new data sources (structured and unstructured),
Eliminates today’s data intake and append bottleneck,
Empowers Analysts to explore all data elements,
Increases processing power for statistical analysis and,
Improves recruiting and retention of Data Engineers and Data Scientists
![Page 15: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/15.jpg)
15#strataconf #hadoopworld
Solution Architecture
Pow
er C
ente
r B
ig D
ata
Edit
ion
HDFSD
ata
Qu
alit
y B
ig D
ata
Edit
ion
Iden
tity
Res
olu
tio
n
HB
ase
Hive
Map Reduce
Cleansed Files
Individual Household
Informatica Big Data Edition Cloudera Big Data Platform
Vis
ual
izat
ion
s
Big Data Analytics
Extract Load & Transform
Data Quality –Cleaning, Identity Resolution
CustomerData
Partner Files
Prospects
EnrichmentInp
uts
CRM
Solicitation History
Weblogs
Pre
dic
tive
An
alyt
ics
Visualizations
Consumption
Dat
ame
er
Predictive Analytics
![Page 16: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/16.jpg)
16#strataconf #hadoopworld
Product Value Proposition
Cloudera Enterprise Data Hub
– Strong commitment to community driven, open source platform
– Security & Governance : Authentication; Authorization; Auditing; Data lineage & Data discovery
– Strong presence in Financial Services Industry
Informatica
– Leverage existing skills; Visual development; Increased Productivity
– Prebuilt connectors: RDBMS, VSAM, OLAP, Salesforce, Social Media (Facebook, LinkedIn, Twitter)
– Data Profiling & data quality on Hadoop: Identify data issues earlier; score carding; cleanse, match and standardize on Hadoop; prebuilt data quality rules, data masking
– Natural Language Processing (NLP): Mine semi-structured & unstructured data (emails, twitter feeds, Facebook posts)
![Page 17: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/17.jpg)
17#strataconf #hadoopworld
Product Value Proposition
Datameer– Big Data Analytics: Analyze structured & semi-structured; data mining;
built for Hadoop
– Easy, Excel-like interface eliminates need to learn new programming languages… Would appeal to more broad analytic user community across the enterprise.
– Create, execute & test data pipelines natively on Hadoop
– Rapidly combine and enrich existing data sets; what-if scenarios
– Pre-stage data for Campaigns / Reporting
![Page 18: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/18.jpg)
18#strataconf #hadoopworld
POC Highlights
10 Node Cluster running CDH 5.0
718 Million Rows of data from 1200+ input files
Seven use cases with increasing complexity
30 TB of Data
![Page 19: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/19.jpg)
19#strataconf #hadoopworld
POC TimelineTask Participants Duration
Cloudera Install, Setup and cluster certification for Enterprise Data Hub, Security review
ClouderaCore Team
2 weeks
Installation, Data preparation, ingestion, using PowerCenterBigData Edition
Informatica /TA Core Team
2 weeks
Profiling, Data cleansing and standardization, aggregation, de-duping, customer identification using Data Quality and Identity Resolution
InformaticaTA Core Team
4 weeks
Visualizations, models, pmml, campaign files using Big Data Analytics using Datameer
DatameerTA Core Team
2 weeks
SAS integration TA Core Team 1 week
Wrap up 1 week
![Page 20: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/20.jpg)
20#strataconf #hadoopworld
POC Infrastructure
![Page 21: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/21.jpg)
21#strataconf #hadoopworld
Infrastucuture
• Cloudera• Enterprise Data Hub (5.0.1)• Hive (0.12), Hue (3.5), Impala, Hbase (0.96),
Pig (0.12), Spark (0.9)• 10 node cluster, 6 data nodes• RHEL 6.5, 20 cores, 128 GB RAM• 80TB usable space
• Informatica Big Data Governance Edition• BDE 9.6.1• Identity Match (MapIR)
• Datameer
![Page 22: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/22.jpg)
22#strataconf #hadoopworld
Team
• Business (Sponsor, Data, Analytics, Campaign)• PM• BA • IT (6)• Operations (3 part time)• Support staff
• Legal• Procurement• Security
![Page 23: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/23.jpg)
23#strataconf #hadoopworld
Why Hbase?
• Faster update processing• Leverage index to perform seek for upsert
processing instead of full scan against hdfs. Huge gains in update processing times.
• Faster Analytics• Can leverage Hbase index for improved
query performance
![Page 24: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/24.jpg)
24#strataconf #hadoopworld
Informatica Developer (Big Data Edition)
![Page 25: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/25.jpg)
25#strataconf #hadoopworld
Data Profiling on Hadoop
![Page 26: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/26.jpg)
26#strataconf #hadoopworld
Score carding
![Page 27: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/27.jpg)
27#strataconf #hadoopworld
Datameer – Workbooks
![Page 28: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/28.jpg)
28#strataconf #hadoopworld
Datameer - Visualizations
![Page 29: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/29.jpg)
29#strataconf #hadoopworld
Lessons Learned
• Invest in a PoC
– Tie to business use cases to demonstrate value
• Partner with key vendors
– Technology is changing rapidly
• Small team with the right skillset
– Naturally innovative individuals
– Can wear multiple hats
• Evangelize & sell your idea
– Partner with business
– Socialize the platform & the vision
• Big Data requires Data Governance
– Establish tools & processes to support data governance
– Data Stewards: Profile, validate, catalog, metadata creation, lineage
– Managed & Curated Data
• Align with larger enterprise strategy
![Page 30: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/30.jpg)
30#strataconf #hadoopworld
Questions
![Page 31: Customer Intelligence_ Harnessing Elephants at Transamerica Presentation (1)](https://reader034.fdocuments.in/reader034/viewer/2022042607/55a73f4c1a28ab7e698b4686/html5/thumbnails/31.jpg)
31#strataconf #hadoopworld
Stephen [email protected]://about.me/stephenlloyd
David [email protected]
Vishal [email protected]: @vishalbamba