Information Quality and Metadata in Healthcare Management

The NCQA Baseline Assessment Tool

. . . and How Sharing Information Production Process Metadata Fosters Process Improvement

The BAT and Information Production Process Metadata

• What is the Baseline Assessment Tool?• What is this Metadata all about?

How it relates to:– Quality Management and Process Improvement– Information Production Processes– HEDIS/QARR

• Process Improvement Details Useful for the BAT

What is the BAT?

The NCQA Baseline Assessment Tool

• The BAT collects information that helps the NCQA assure that health plan information systems can generate reliable quality measures for HEDIS/QARR

• Supports the NCQA's HEDIS Compliance Audit, an audit for compliance with HEDIS specifications for measures.

Sections of the BAT• Data Processing Systems:

– Encounters– Members– Providers

• Completeness of Data• Integration of Data for Reporting• Controlling Integrity of Reporting• Medical Record Review• Other / General Information

Quality Assurance Reporting Requirements (QARR)

Annual Dental Visit

Frequency of Ongoing Prenatal Care

Children's Access to Primary Care Practitioners

Adults' Access to Preventive/Ambulatory Health Services

Ambulatory Care

Inpatient Utilization

Births and Average Length of Stay, Newborns

Discharges and Average Length of Stay-Maternity Care

Mental Health Utilization-Inpatient

Chemical Dependency Utilization-Inpatient

Frequency of Selected Procedures

Practitioner Turnover

Enrollment by County

Board Certification/Residency Completion

Childhood Immunization Status

Lead Testing

Use of Appropriate Medications for People with Asthma

Antidepressant Medication Management

Breast Cancer Screening

Cervical Cancer Screening

Comprehensive Diabetes Care

Controlling High Blood Pressure

Follow-up After Hospitalization for Mental Illness

Well Child and Preventive Care Visits in the 1st 15 Months of Life

Well Child and Preventive Care Visits - Children Ages 3-6 Years

Adolescent Well-Care and Preventive Visits

Timeliness of Prenatal Care

Postpartum Care

Generating QARR Measures• Extract random samples of

member populations relevant to each measure

• Includes visiting primary care sites, physically reviewing medical records, recording compliant instances

• Calculate rates of those showing compliance with requirements for each measure

What is this Metadata all About?

• Metadata Categories• Basic Metadata Purposes• What Quality Management Does• Overview of Relevant Processes• Quality Approach to Information

Metadata Categories• Production Process Factors

– People– Process– Tools– Materials

• Quality Measures– Timeliness– Completeness– Accuracy

• Barriers to Quality (Root Causes)• Improvement Processes

– Quality Assessment– Cleanup and Transfer– Process Improvement

Basic Metadata Purposes:• Create common understanding• Foster process improvement• Enable information stewardship

Metadata CreatesCommon Understanding

Between:• Information Producers:

– Business Process: data acquisition, entry and updating

– Technical Process: developers and analysts• Information Stakeholders:

– Anybody who uses the information for business

Metadata FostersInformation Process Improvement

• Establishes common understanding, from an enterprise standpoint, of all stakeholder requirements for information

• Fosters collaboration, empowerment, teamwork, professionalism, pride of workmanship

• Makes environment of information stewardship possible

Information Stewardship

Willingness to be accountable for a set of business information for the

well-being of the larger organization, by operating in service of (rather than

under control of) those around us.

What Quality Management Does:• Brings producers and stakeholders together• Makes sure stakeholder requirements are fully

represented• Assures completeness and objectivity of measures so

they are reliable basis for understanding and managing functionality and performance

• Establishes factual basis• Makes process improvement possible• Makes cross-functional teamwork possible• Makes performance management possible• Makes resource management possible

The Definition of Quality

Meeting and exceeding stakeholder (customer) expectations and

requirements

Process Improvement:PDCA (Shewhart Cycle)

• Plan an improvement based on factual basis

• Do it for awhile• Check it for improvement• Act to standardize the improvement

(Repeat)

Process Improvement:Root Cause Analysis

Tools Materials

ProcessPeople

Defect

Overview of Production Processes

HEDIS/QARR

PerformanceManagement

CRMSClaims/

Encounters

Providers

Members

QM

DataEntry

DataAcquisition

SystemsDevelopment

OperationalInformation

KnowledgeWorkers

Finance UM

InformationProducers

Data Mart

Etc.InformationStakeholders

Encounters Information Producers

CLAIMS/ENCOUNTERS

Data EntryClaims

Data AcquisitionCare Providers

Vendors

SystemsDevelopmentDevelopers/

Analysts

Members Information Producers

MEMBERS

Data EntryEnrollment

Data AcquisitionMarketing

Member ServicesRecertification


Analysts

Providers Information Producers

PROVIDERS

Data EntryNetwork Information

Data AcquisitionNetwork Development

Provider Relations


Analysts

Encounters Stakeholders

Claims/Encounters

Providers

Members

QM


Finance UM Etc.

HEDIS/QARR

QualityManagement

CRMS

Decision SupportSystem

CommunityHealth

Institute

Data Mining


UtilizationManagement

InformationStakeholders

HEDIS/QARR Quality Assessment and Medical Record Review

• Build CRMS• Create population samples• Extract compliant instances

based on administrative data, along with member and provider site information, to a special QARR database

• Load laptops from this database

CRMS

Data Mart

SpecialQARRReview

Database

Lack of Compliant Instancesby Administrative Data

• Compliance with measure conditions requires the following data elements to be present and accurate:– Recipient information– Service and diagnosis codes– Service location information– Provider information

• Missing or inaccurate information necessitates the conduct of onsite medical record review


• Review claims histories for all members in samples to identify most likely primary care provider

• Sort samples and fax lists to each site, requesting charts for onsite review


• Visit sites and review charts


Track down charts not found at sites:

• Verify Member name, CIN, SSN• Review claims history for other

providers and spans of care• Check Healthy Beginnings /

Mammogram Incentives databases• Check online Immunizations Registry• Check Phone logs• Check NY State Roster• Check http://www.whitepages.com• Call Providers• Call Members

http://www.whitepages.com/


• Visit new locations and review again


• Integrate and upload laptop data to CRMS

• Use CRMS to generate rates

• Send results to NY State Department of Health

CRMS

Data Mart


• Massive costs due to nonquality data• Nurses to sites throughout network, three

times• Hunting down members, what provider

they're getting care from, missing or miscoded services and diagnoses

• 4300+ members in samples

Quality Approachto Information

• Foster common understanding between producers and stakeholders by use of shared metadata

• Accuracy can’t be automated; only people can assess and correct for accuracy

• Information is an enterprise resource• Measure information quality in voice of downstream

customers/stakeholders• To improve a production process, measure its

product• Improve information quality at the point of capture

Foster Common Understanding Between Producers and

Stakeholders

Accuracy Can’t be Automated• Only people can create, assess, or correct for accurate

information• Automatic business rules can only prevent gross errors

Information(Encounters,

Members,Providers)

Data Entry

Data Acquisition

• Checking against a surrogate source: comparison to external datasets, forms

• Checking against authoritative source: comparison to real world entity

Measure Information Qualityin Voice of Stakeholders

HEDIS/QARR


CRMSClaims/

Encounters

Providers

Members

QM

DataEntry

DataAcquisition

SystemsDevelopment


KnowledgeWorkers

Finance UM


Data Mart

Etc.InformationStakeholders

Information Quality Measures• Definitions/Specifications• Timeliness

– Time from first capture to record of reference• Completeness

– Represent each instance in the real world– Completeness of values in existing fields

• Accuracy– To surrogate source– To authoritative source

• Usability– Presence of fields to address functional requirements

Measure the Productto Improve its Process

• Purpose is not to improve a product by measuring it, but to improve its production process

• Product is of concern only from a limited perspective

• Quality measures show how well processes for producing information are functioning; i.e., how well they address all stakeholder requirements

Applications are notthe Product to Measure

• Assessing applications addresses narrow requirements, not downstream requirements for the information

• Address information as the product, understood as an enterprise resource

• Measure the information that applications work on to assess development

• Applications are machines on the assembly line, not the product

Applications are Machineson the Assembly Line

Information

Applications

Improving Information (the Product) is not the Chief Concern

• Right product to measure, but wrong purpose• Improving the product doesn't prevent errors

or assure ongoing quality• Improve production processes• Information product improvement is all cost

basis/scrap and rework• Correction of issues preventable by process

Three Ways to Correct Information Quality

• As a one-time process• As a regular conversion process• At the point of capture

As a One-time Process for a Special Table

• Not preventative; same problems recur because information production process has not changed

• Quality of data decays; not addressing changes in the information

• Creates a redundant table for the entity (encounters, services, providers, members)

As a Regular Conversion for a Data Mart or Data Warehouse

• Performed under constraint of a regular load for up-to-the-moment, daily analysis

• Too late: requires automatic, estimated corrections

• Corrections need to feed back to operational tables

As a Regular Conversion for a Data Mart or Data Warehouse

CRMSClaims/

Encounters

Providers

Members

OperationalSystemsDecision Support

System

Cleanup &Transfer

Regular Estimated,Automated Corrections

At the Point of Capture

• Actual process improvement• Actually preventative• Based on understanding of information as

enterprise resource• Information producers assess and improve

business and technical/development processes based on understanding of stakeholder requirements

At the Point of CaptureMeasure: - Definitions - Timeliness - Completeness - Accuracy - Usability

PlanDoCheckAct (Repeat)

Claims/Encounters

Providers

Members

DataEntry

DataAcquisition

SystemsDevelopment


Do Both:• Improve at Point of Capture: Measure and

improve information production process quality at the point of capture, from the standpoint of the requirements of all downstream stakeholders

• Regular Cleanup and Transfer: Establish and document business rules for integration (cleanup and transfer), through stakeholder involvement (i.e., for CRMS load)

Process Improvement for Information• Plan:

– Identify stakeholders/customers – Survey stakeholders for requirements – Measure quality of product against requirements– Determine root cause for identified issues

• Do:– Implement improvement for awhile

• Check:– Measure again to determine success of improvement

• Act:– Act to standardize the improvement

(Repeat)

Root Cause Analysis for Information

Application /System Source Data

Business ProcessesMethods / ProceduresPersonnel

SpecificDefect

Business Process Quality

DataIntermediaries


ProcessSteps

DocumentedProcedures

SourceForms

External Data

Application

Database Hardware

Root Cause Analysis for Information

DevelopmentTools Source Materials

DevelopmentMethodsPersonnel

SpecificDefect

Development Process Quality

ApplicationDevelopment

Developers

Forms, Reports,Procedures, etc.

Analysts Data Analysis

MetadataRepository

Definitions/Specifications,Business Rules

DataDictionary

Process Improvement Details Useful for the NCQA Baseline

Assessment Tool

Process Improvement Steps• Plan:

– Identify stakeholders/customers – Identify requirements – Measure quality of product against requirements– Determine root cause for identified issues

• Do:– Implement improvement for awhile

• Check:– Measure again to determine success of improvement

• Act:– Act to standardize the improvement

(Repeat)

Metadata Categories• Production Process Factors

– People– Process– Tools– Materials

• Quality Measures– Timeliness– Completeness– Accuracy

• Barriers to Quality (Root Causes)• Improvement Processes

– Quality Assessment– Cleanup and Transfer– Process Improvement

Production Process Factors• People

– Stakeholders, Information Producers, Knowledge Workers– Developers, Analysts

• Process– Business Information Production Processes– Application/Data Development Methods

• Tools– Systems, Applications, Databases, Hardware– Metadata Repositories, Data Dictionaries, CASE Tools

• Materials– Requirements, Specifications, Business Rules, Documented

Procedures, Reports, Forms, External Data Sources

Simplified PictureFor each entity (Encounters, Members, Providers)

– Who are its stakeholders?– What processes use it?– What are their requirements?

– Then standards, measures, rules, definitions, specifications, etc.

Slightly More Detailed Picture• Definition Standards• Goals / Motivations• Product Specifications• Roles – Owners, Stakeholders• Business Requirements• Production Processes• Improvement Processes• Performance Measures• Performance Rates

Very Complex Structure

Purposes of Metadata• Foster process improvement for information• Enable information producers to understand

requirements from an enterprise standpoint, for both business and technical/development processes

• See all stakeholders, their processes that depend on the information, and their requirements

• Provide clear, accurate consensus definitions to enable problem resolution

• Enable developers to standardize and reuse specifications and business rules

• Manage information as a resource, including identifying needless cost, how much value you're deriving, how and where

• Work on information production processes by surveying stakeholders for their requirements

• Measure business and development processes according to requirements for the information as voiced by stakeholders

• Improve information production processes at the point of capture

• Devise rules for cleanup and transfer to decision support systems (CRMS) in collaboration with stakeholders

• Begin to store metadata: Document and share stakeholders, processes, requirements, performance measures, business rules, etc.

Useful Steps to Take

Foster Process Improvement through Common Understanding of

Information Production Processes

Information Quality and Metadata in Healthcare Management

Business

Transcript of Information Quality and Metadata in Healthcare Management