Business Intelligence: Using Data for More Than Analytics 2015... · ANNUAL EDUCATIONAL CONFERENCE...

IASA 87TH ANNUAL EDUCATIONAL CONFERENCE & BUSINESS SHOW

Business Intelligence: Using Data for More Than Analytics Session 672


Session Overview

Business Intelligence: Using Data for More Than Analytics What is Business Intelligence? Business Intelligence Solution Data Cleansing & Data Validation Data Consolidation Using Data for More Than Analytics Question & Answers


Introductions

Your Speakers Robert Clark

• Vice President of Development, 4Sight Business Intelligence Matt Carter

• Data Warehouse Analyst, Tower Hill Insurance Group Michael Paparatto

• Reporting and Business Intelligence Developer, Anchor General Insurance Group


What is Business Intelligence?

Definition:

“Business intelligence (BI) is the set of techniques and tools for the transformation of raw data into meaningful and useful information for business analysis purposes… The goal of BI is to allow for the easy interpretation of these large volumes of data. Identifying new opportunities and implementing an effective strategy based on insights can provide businesses with a competitive market advantage and long-term stability.” - Wikipedia


Business Intelligence Solution

Three Main Components Extract Transform and Load (ETL) – BI loading process Data Store – Data storage for analysis BI Tools – End-User analysis


Data Warehouse

Components of a BI Solution

Load

Transform

Extract

Business Intelligence Application(s)

Reports

Dashboards

Ad-hoc Analysis

Source Systems

Extract, Transform and Load

Data Store BI Tools

IOIOIOIOI IOIOIOIOI

Policy Admin

Claims Admin

Accounting

IOIOIOI

IOIOIOI

IOIOIOI



Load

Transform

Extract

Source Systems


IOIOIOIOI

Policy Admin

Claims Admin

Accounting



Extract, Transform and Load (ETL) Extract

• Pull data in raw format from the source system • Prepare it for transforming

Transformation • Data Cleansing (more on this shortly) • Code Translations • Calculations • Data Validation (more on this shortly)

Load • Loading the transformed/converted data into the target system or

database


Data Warehouse


Load

Transform

Extract

Source Systems


Data Store

IOIOIOIOI IOIOIOIOI

Policy Admin

Claims Admin

Accounting



Data Store Data Warehouse

• Star Schemas – comprising of fact tables (measures and values) connected to dimensions (values used to slice and dice the data)

• Often Contain Star Schemas with Different Levels of Aggregation (coverage, policy, claim, month, quarter, year, etc.)

Big Data • Large Datasets – datasets that are too large and complex that traditional

databases are inadequate to process the data

• Term Often Used Interchangeably with Predictive Analytics



Data Store (continued)

Cube (Online Analytical Processing – OLAP)

• MOLAP (Multidimensional Online Analytical Processing) – Data stored in N dimensional cube and calculations are generated ahead of time

• ROLAP (Relational Online Analytical Processing) – Similar to MOLAP but uses relational database to retrieve values


Data Warehouse


Load

Transform

Extract


Reports

Dashboards

Ad-hoc Analysis

Source Systems


Data Store BI Tools

IOIOIOIOI IOIOIOIOI

Policy Admin

Claims Admin

Accounting

IOIOIOI

IOIOIOI

IOIOIOI



Business Intelligence Tools Data Retrieval & Basic Analysis

• Reporting and Querying Software – basic report design tools and database query clients

• Spreadsheets Dashboards – graphical snapshot of data showing historical trends and analysis

Advanced Analysis • OLAP (Online Analytical Processing) – slicing and dicing, drill down,

drill up, aggregations, etc. • Data Mining – discovering patterns in large sets of data (groups,

anomalies and associations)


Data Warehouse


Load

Transform

Extract


Reports

Dashboards

Ad-hoc Analysis

Source Systems


Data Store BI Tools

IOIOIOIOI IOIOIOIOI

Policy Admin

Claims Admin

Accounting

IOIOIOI

IOIOIOI

IOIOIOI


Data Cleansing & Data Validation

Data Cleansing is the process of identifying and correcting inaccurate or invalid data

Data Validation

is the process of using rules or constraints to check the validity or correctness of the data


Data Cleansing

Data Cleansing Steps: Identification of Invalid Data Correcting Data or Removing It


Data Cleansing: Identifying Invalid Data

Identifying Invalid Data Using Tools or Queries to Identify

• Field Data Types • Uniqueness Constraints (e.g. duplicate records or field values)

• Formats and Patterns (e.g. mm/dd/yyyy MM:hh:ss.SSSS)

• Ranges (min / max)

• Accuracy (e.g. address verification)

• Set Membership (e.g. Male/Female, AR/AZ/OH/NJ/TX, etc.)


Data Cleansing: Correcting/Removing Data

Correcting or Removing Data Data is Corrected if Possible

• Invalid Values Researched to Identify Correct Values • Values Updated to “Correct” Values in Batch (using queries or tools)

• Not Possible to Bulk Update? • Correct data in the source system • Default to a value • Remove Record

Removing Data • Filter Out When Reading Data • Delete the Record


Data Validation

Data Validation Steps Define Validation Rules Execute Rules Against Data (as loaded or in batch during ETL)

Review Invalid Data & Decide Course of Action


Data Validation: Define Rules

Validation Rules Rules Are Written In ETL Tool, Using Queries or a Rules

Engines Example Rules Include

• All Coverage Effective/Expiration Dates Must Be Within the Policy Term

• Claim Must Be Associated with a Valid Coverage (especially if separate systems for policies and claims)

• Coverage In-Force at Time of Loss • Appropriate Coverage for Type of Loss • Sum of Loss Reserves is a Positive Number • Endorsement Prorated Premium Calculated Correctly


Data Validation: Execute Rules

Rule Execution Rules Are Executed By

• ETL Process During Transformation Phase • Manually Through Queries or Tools • Scheduled


Data Validation: Review & Decide Course of Action

Review the Results and Decide Course of Action to Correct Needs To Be Corrected In Source System

• Bulk Update of Data • Add Edits Source System to Prevent Issue

Needs to be Corrected Outside Source System • Same Process as Data Consolidation: Correcting or Removing Data


Importance of Data Cleansing & Data Validation

Key Business Decisions Are Being Made on the Data Simply Just Pushing Data From Source Systems Into a BI

Solution is Not Enough • Operational Systems Are Not Perfect • Often Legacy Data Has Been Ported From Older Systems • Data is Coming From Disconnected Systems (policy, claim, TPA)

• ETL Itself May Have Issues Garbage In, Garbage Out (GIGO)

“… in the field of computer science or information and communications technology [GIGO] refers to the fact that computers, since they operate by logical processes, will unquestioningly process unintended, even nonsensical, input data (‘garbage in’) and produce undesired, often nonsensical, output (‘garbage out’).” - Wikipedia


Data Consolidation

What is Data Consolidation? Collecting and Integrating Data From Disparate Systems

Into a Single Data Store.

Data Warehouse Policy Admin

(Personal Lines)

Policy Admin (Commercial Lines)

TPA Claims Extract

Legacy Policy Admin

Accounting

Billing System

10101010101010110101 10101010101010110101


Data Consolidation

What Are the Challenges? Missing Data Between Systems Different Systems, Different Codes Consolidating and Referencing Entities

• Agents / Producers • Third Parties (Loss Payee, Other Interests) • Insureds • Claimants • Underwriters • Adjusters


Data Consolidation

What are the Benefits? Centralized Reporting

• All Reports Come From One System • No Longer Need Spreadsheets to Merge Data

Single Version of the “Truth” • Calculations are Consistent Across the Organization

• Earned / Unearned Premium • Incurred Formulas • Loss Ratios • Many More

• Non-Redundant Data (one source) • Definitions and Included/Excluded Data are Consistent


Data Consolidation

What are the Benefits (continued)? Consistent Granularity of Data

• Earned Premium (Calculated at Policy Level vs. Coverage Level)


Using Data for More Than Analytics

You have cleaned up and consolidated all this data. Now what can you do with it?



Business Intelligence Standardized Reporting

Dashboards

Ad-hoc Analysis



Governmental Reporting Statistical Reporting

• Extracting Data for ISO, NISS, NAIC, NCCI, etc. • Third Party Submissions

State Data Calls • Extract Data in State Specific Formats



System Conversions Load Legacy Data Into Data Warehouse Load New System’s Data Into Data Warehouse on a Regular Schedule Provide Seamless Reporting

• No Need to Consolidate Reports • Single Version of the “Truth” • Ease Transition when Renewing into New System

Provides Coverage Verification • Coverages Can be Verified Even if Policy Is Not In New System



System Conversions (continued) Conversion of Data

• Data Has Been Cleansed and Validated • Convert Data Over to New System • Companies Offer Web Services To Extract Data

Conversion All At Once • Port All Data Over to New System

Conversion On Renewal • Port All Inforce Policies to New System • Then Port Policies Coming Up for Renewal



System Conversions (continued) Conversion of Open Claims

• Port Only Open Claims • Port Any Claim That Reopens When Needed

Conversion of Insurance Fund Assumptions (e.g. Citizens) • Loading Insurance Fund Assumptions Allows for Analysis • Conducive to Converting from DW to Operational Systems



Self Service and Web Portal Customer Portals

• View Policy and Coverages Agent Portals

• Ability to Query Upcoming Renewals • View Profitability • Of Course Check Commissions

Third Party Portals (e.g. TPAs, Reinsurers, Auditors, etc.)

• View and Query Data • Controlled Analysis of Data


Slide Title Goes Here in 28pt Arial Bold Flush Left

Predictive Analytics Machine Learning – algorithms that can learn and make predictions on the data (Open Source R)

Predictive Modeling – using statistics to forecast or predict outcomes

Data Mining – discovering patterns in large sets of data (groups, anomalies and associations) Examples of Predictive Analytics • Fraud Detection • Risk Management • Underwriting • Cross Selling



Other Uses CLUE & APLUS Claims Contributions and Submissions DMV Reporting OFAC Validation Medicare Regulations (e.g. 111) Reinsurance Reporting


Conclusion

Using Data for More Than Analytics Business Intelligence Solution

• Extract, Transform & Load • Data Store • Business Intelligence Tools

Importance of Data Cleansing & Data Validation • Key Business Decisions Are Being Made on the Data • Simply Just Pushing Data From Source Systems Into a BI Solution is

Not Enough. It Needs to be Validated and Cleansed. Consolidation of Data

• Single Version of the “Truth” • Seamless Reporting


Conclusion

Using Data for More Than Analytics Uses For the Cleaned & Validated Data

• Governmental Reporting • Statistical Reporting • State Data Calls

• System Conversions • Conversions of Data all at Once or On Renewal • Conversion of Open Claims and Reopen When Needed

• Self Service Web Portals • Customers • Agents and Third Parties


Conclusion

Using Data for More Than Analytics (Continued) Uses For the Cleaned & Validated Data

• Predictive Analytics • Machine Learning • Predictive Modeling • Data Mining • Fraud Detection, Cross-Selling, Risk Management, etc.

• Many Other Uses • CLUE & APLUS Claims Contributions and Submissions • DMV Reporting • OFAC Validation • Medicare Regulations (e.g. 111) • Reinsurance Reporting


Question & Answers

Questions?


Please Complete the Session Evaluation Form on the Conference App

Business Intelligence: Using Data for More Than Analytics 2015... · ANNUAL EDUCATIONAL CONFERENCE...

Documents

Transcript of Business Intelligence: Using Data for More Than Analytics 2015... · ANNUAL EDUCATIONAL CONFERENCE...