Exclusive Verizon Employee Webinar: Getting More From Your CDR Data

Verizon & Big Data: Getting More from CDR Data

© 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555

The program will begin shortly. Please listen to the webinar with your computer speakers turned on.

John Michael Brack, Manager - Northeast Region, Pentaho Bo Borland, Head of System Engineers, Pentaho

Agenda

•  Project Profile

•  Pentaho Overview

•  Big Wireless Analytics

•  Call Detail Records (CDR) dashboard and analysis

•  Retail Sales Reporting, IR, Mobile and Analysis

•  DW Optimization

•  Rescuing CDR data from tape archive with Hadoop

•  Pentaho for Hadoop (ingestion, map reduce, orchestration, analysis)

2 © 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555

Verizon Project Profile

Audience: Business, Executives, Management, IT, Architects Company: Verizon Communications Inc. was founded in 1983 and is based in New York, New York and has over 180,000 associates and 115 B in revenue. Verizon Communications Inc., through its subsidiaries, provides communications, information and entertainment products and services to consumers, businesses, and governmental agencies worldwide. Its Verizon Wireless segment offers access to various wireless voice and data services comprising Internet access through smart phones and basic phones, and notebook computers and tablets; messaging services, which enable customers to send and receive text, picture, and video messages; and consumer-focused and business-focused multimedia applications. . Goals:

•  Self Service Business Analytics •  Better TCO •  Data Integration •  Self Service Powerful Reporting •  Usability and connectivity •  Self Service Dashboards •  Time to Market •  Access to Big Data and Analytical DB


A modern, unified embeddable platform built for the future of analytics, including big data and cloud-ready analytics


CENTRAL ADMINISTRATION, AUDITING & MONITORING

DELIVER When & Where Users Need It

STREAMLINE Information Delivery

VISUALIZE & Report Information In Any Style

ACCESS All Enterprise Data Sources

ISV & Packaged Applications

SaaS / Cloud Applications

EMBEDDED

Web

Mobile

Print

E-Mail

STANDALONE

‣  Advanced & Predictive Analytics

DATA MINING

‣  Interactive ‣  Operational

‣  Enterprise

REPORTING

‣  Ad hoc Exploration ‣  Multi-Dimensional

ANALYSIS

‣  Interactive Metrics ‣  Rich Visualizations

DASHBOARDS

ERP / CRM / Enterprise Apps (e.g. SAP, Oracle)

Hadoop & NoSQL Data

Unstructured & semi-structured (XML, Excel, Files, etc.)

Relational Data Sources

Cloud (e.g. Salesforce, Amazon, Dell)

‣  Direct Access

‣ Data Integration

‣  Hadoop Clustering

‣  Graphical ETL Designer

‣  Enterprise Scalability

INTEGRATE, CLEANSE, & ENRICH DATA

‣  In Memory Caching

‣  High Performance

‣  Relational OLAP Cubes

METADATA LAYER

Over 1500 Customers Across All Industries


Skype – Sales & financial reporting with plans to analyze user stats to improve performance monitoring.

Comcast – Data integration for “single version of the truth” and new BI initiative to enable self service reporting/analytics for business analysts.

Nuance – Use Pentaho to join and transform data from mobile device usage logs to perform complex data analysis.

Bell Canada – Optimize project and consulting resources by being able to view all project activity across multiple teams (and acquisitions –all different data sources).

Pentaho for Hadoop Large Financial Institution

6 © 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 6 © 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555

Why Pentaho •  Ability to load disparate data sources into

HDFS and Hbase

•  Ability to load post-processed data into DB2

•  Ability to interface with caching and message queue technologies via customer-specific Java libraries

Business Challenge To gain competitive advantage through intraday balance reporting for commercial customers.

Pentaho Benefits •  Lowers technical barriers by providing an easy to use ETL

environment for designing MapReduce jobs without having to write code

•  Provides a graphical orchestration environment for Hadoop, HBase and DB2 data integration workloads

•  Processes Client, Account, Reference, Transaction and Balance information at the lowest level of granularity possible

Forrester Enterprise Hadoop Solutions Wave Highest-Scored Analytics Vendor


Pentaho BIG Data & Data Integration Walkthrough

Bo Borland

Head of System Engineers


Pentaho Platform Design Drivers

1.  Big data is changing the world

2.  Open systems are more innovative

3.  Subscriptions models reduce cost and risk

4.  Simplicity empowers the masses

5.  Pluggable java architectures enables flexibility and competitive advantage

6.  Enterprise-wide integration reduce cost and complexity

7.  Predictive technologies are next big thing in analytics

9

Big Wireless


3 Calling Plans

•  Nationwide •  PAYG •  Prepaid 50

2 Business

Units

•  B2B •  B2C

7 Retail Stores

7 Product Lines

3 Websites

Big Wireless– Wireless Carrier

•  San Francisco •  Boston •  NYC •  Paris •  Tokyo •  Sydney •  London

•  Smartphones •  Home Phones •  Wifi Devices •  Modems •  Notebooks •  Tablets •  Accessories

•  Ecommerce Site •  Reseller Portal •  Manufacturer Portal

Store Managers

Executives & Product Managers

Operations and Store Employees

Marketing & Customer Support

B2B Sales Organization

Databases

Call Detail Records

Retail Sales

Website Clickstream

Website User Registration

2013 Performance Goals

12

Increase subscription revenue

Improve store profitability

Eliminate inventory stock outs

Leverage big data to maximize profits

Profile and target profitable customers

Improve supply chain visibility for partners

2013 Performance Goals

13

Goals Objectives Enablers Increase

subscription revenue

Analyze call data to upsell PAYG customers to subscriptions

Improve store profitability

Hold store managers accountable by pushing store income statements to

email

Eliminate inventory stock outs

Empower store employees with iPads and real-time inventory reports

Profile and target profitable customers

Profile mobile plan customers with high average call duration

Leverage big data to maximize profits

Analyze e-commerce clickstream data in MongoDB to profile purchasing

users and predict users propensity to purchase.

Improve supply chain visibility for

partners

Give phone manufacturers and resellers web access to secure sales

reports

3 Calling Plans

•  Nationwide •  PAYG •  Prepaid 50

2 Business

Units

•  B2B •  B2C

7 Retail Stores

7 Product Lines

3 Websites

Enterprise-Wide Analytics

10 Resellers

10 Phone Manf

Red River Mobile

•  San Francisco •  Boston •  NYC •  Paris •  Tokyo •  Sydney •  London

•  Smartphones •  Home Phones •  Wifi Devices •  Modems •  Notebooks •  Tablets •  Accessories

•  Ecommerce Site •  Reseller Portal •  Manufacturer Portal

EXTERNAL INTERNAL

IFrame Integration

Custom Widget Embedding

Big Data


Mobile Network Provider

Call Detail Records (CDR) •  Mobile networks generate vast amounts of daily call data •  CDR tracks every voice, SMS, or location service •  2 years of detailed CDR records in DW •  Archived to tape after 2 years

Data Sources Data Warehouse Architecture

Data Warehouse (Master & Transactional Data)

ERP

CRM

CDR

Analytic Data Mart(s)



Tape Archive

Current Data Warehouse Architecture

Data Sources Data Warehouse Architecture


ERP

CRM

CDR




Tape Archive

With Current EDW Architecture With Hadoop

EDW stores only 2 years of data à Hadoop active archive for all history

Infrastructure at capacity à Frees EDW capacity for high value data

Expensive to scale à Lowers cost and inexpensive to scale

ETL process complex and slow à Streamlined ingestion of raw data

Only analyze 2 years of data à Analyze 10 years of data

Data Warehouse Optimization

Data Sources Big Data Architecture


ERP

CRM

CDR




Logs Logs

Other Data

Raw Data

Parsed Data

Analytic Datasets

Master Data

Tape Archive

ORCHESTRATE

ERP DW

Processing

CRM

Pig, Oozie, Flume, Hive, Hbase, Sqoop

Raw Data

Parsed Data

Analytic Datasets

Pentaho Analytics for Hadoop

© 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 19

Master Data

Analysis & Reporting

ANALYZE

Unstructured Data

Structured Data

INGEST

Ingestion

VISUAL MAP REDUCE

Data Integration Analytics

Raw Data

Ingest Raw and Master Data


Master Data Unstructured

Data

Structured Data

INGEST

Ingestion

Processing

Raw Data

Parsed Data

Analytic Datasets

Visual Map Reduce


Master Data

VISUAL MAP REDUCE

1.  Map Reduce Input – calling data

2.  Calculate Month, Day, Day of Week

3.  Extract 3 digit area code

4.  Lookup geo master data in HDFS

5.  Filter for weekend and US only calls

6.  Create “Value” field for Key-Value Pair

7.  Create “Key “ field for Key-Value Pair

8.  Map Reduce Output – Key-Value Pair

Java Programing

Data Agnostic & Data Orchestration


Pentaho Data Integration

Parsed Data

Analytic Datasets

Hadoop Data Analysis


Analysis

ANALYZE

•  Analyze 10 years of call data by geography, time zone for US only calls made on the weekend. –  Understand annual growth rates

–  Which geographies are driving highest call volume growth rates?

ORCHESTRATE

Raw Data

Parsed Data

Analytic Datasets

Pentaho Big Data Demonstration


Master Data

ANALYZE

INGEST

VISUAL MAP REDUCE Ingest CDR data into Hadoop 1

Execute Map Reduce to enrich CDR data 2

Create and load a Hive table with the map

reduce results 3

Analyze 10 years of call data 4


Thank You

Join the conversation. You can find us on:

blog.pentaho.com @Pentaho Facebook.com/Pentaho Pentaho Business Analytics

Exclusive Verizon Employee Webinar: Getting More From Your CDR Data

Technology

Transcript of Exclusive Verizon Employee Webinar: Getting More From Your CDR Data