What, Why and How:What, Why and How:Modeling to Address Health Policy Modeling to Address Health Policy

QuestionsQuestions

Deborah CholletDeborah CholletSenior Fellow, Mathematica Policy ResearchSenior Fellow, Mathematica Policy Research

The Robert Wood Johnson Foundation’sThe Robert Wood Johnson Foundation’sState Coverage Initiatives ProgramState Coverage Initiatives Program

Washington, DCWashington, DCNovember 10, 2004November 10, 2004

What is a Model?

A structured way to think about a problemA structured way to think about a problem

A way to measure responses and outcomesA way to measure responses and outcomes

A way to compare alternative optionsA way to compare alternative options

Why Model Policy?

Understand the impacts of policy change and Understand the impacts of policy change and sensitivity to program optionssensitivity to program options– Program enrollment and costProgram enrollment and cost

– Uninsured populationUninsured population

Understand the sensitivity of estimates to Understand the sensitivity of estimates to environmental factorsenvironmental factors– Private insurance premium growthPrivate insurance premium growth

– Changes in employmentChanges in employment

– Demographic changeDemographic change

Types of Models

Determinate modelsDeterminate modelse.g., Number enrollede.g., Number enrolled

= f (X= f (X1 1 . . . X. . . Xnn)+u)+u

Calculate population Calculate population response to a specific response to a specific change, all else held equalchange, all else held equal

“ “Cell-based” Cell-based” (spreadsheet) models(spreadsheet) models

Calculate change by Calculate change by population subgroup, using population subgroup, using average relationshipsaverage relationships

Microsimulation modelsMicrosimulation models Calculate change by Calculate change by individual using determinate-individual using determinate-model relationships and model relationships and other parametersother parameters

Determinate Models

Estimate aggregate response to a program changeEstimate aggregate response to a program change

Support sensitivity analysis only by subgroups Support sensitivity analysis only by subgroups identified in the model identified in the model

““Broad brush” approach, not suited to “fine-grained” Broad brush” approach, not suited to “fine-grained” analysis of complex systems and interactionsanalysis of complex systems and interactions

Offer a measure of precision of estimates, or a Offer a measure of precision of estimates, or a confidence intervalconfidence interval

Spreadsheet Model

Mimics the operation of a program or systemMimics the operation of a program or system

Incorporates average behavior by population subgroupIncorporates average behavior by population subgroup

Relies on assumptions borrowed from populations that Relies on assumptions borrowed from populations that may differ in unmeasured waysmay differ in unmeasured ways

Outputs subgroup estimates only as defined in the Outputs subgroup estimates only as defined in the modelmodel

Relatively inexpensive and fast to assembleRelatively inexpensive and fast to assemble

Example: Simple Spreadsheet Model of the Dirigo Program

Number ofpersons by subgroup

Calculate subgroupoffer and take up from

available evidence

Calculate financingspecific to subgroups:

• State funds • Federal match • Employer contribution

• Individual premiums

Sum acrosssubgroups

Sensitivityanalyses

Microsimulation Model

Mimics the operation of a program or systemMimics the operation of a program or system

Operates on a large database (e.g., CPS) and outputs the Operates on a large database (e.g., CPS) and outputs the same database with variables of interest calculated same database with variables of interest calculated

Incorporates partial responses to many variables at the level Incorporates partial responses to many variables at the level of the individual to calculate final responseof the individual to calculate final response

Supports relatively flexible analysis of subgroups and Supports relatively flexible analysis of subgroups and sensitivity to assumptionssensitivity to assumptions

Often reveals results of complex logical relationships you Often reveals results of complex logical relationships you might otherwise have overlookedmight otherwise have overlooked

Populationinput data file

Createinsurance families

Apply parameters/relationshipestimates to calculate

individual take up probability

Calculate stochasticestimate of take up

Populationoutput data file

Subgroup andsensitivityanalyses;

refine actuarialestimates as needed

Example: Simple MicrosimulationModel of the Dirigo Program

Constrain take upby eligibility rules

Developinitial

actuarial costestimates

Choosing the Right Model

How precisely can you identify what you How precisely can you identify what you need to know? need to know?

How much detail do you need to know?How much detail do you need to know?

How much time and budget do you have?How much time and budget do you have?

What data are available to inform the model?What data are available to inform the model?

Designing the Model

What are the key policy questions?What are the key policy questions?

How many people will enroll? What woodwork effects? How many people will enroll? What woodwork effects? Change in program cost?Change in program cost?

What measures address the policy questions?What measures address the policy questions?

Insurance family membership and income, actuarial factors Insurance family membership and income, actuarial factors (family type, family size, age, gender, location)(family type, family size, age, gender, location)

What subgroups or other responses are of special What subgroups or other responses are of special interest? What program features might be changed?interest? What program features might be changed?

Eligibility groups (parents, childless adults by income), Eligibility groups (parents, childless adults by income), impacts on linked programsimpacts on linked programs

Structural Issues

What program features and system relationships are What program features and system relationships are important?important?– Eligibility rules, outreach activities, income disregardsEligibility rules, outreach activities, income disregards

– Funding caps or links to available fundsFunding caps or links to available funds

What environmental variables should the model What environmental variables should the model include? What sensitivity analyses do you want?include? What sensitivity analyses do you want?

– Aggregate cost growthAggregate cost growth

– Changes in the industry, firm size, or wage structure of Changes in the industry, firm size, or wage structure of employmentemployment

Data Issues

Do available data include the population of Do available data include the population of interest to you?interest to you?

Are adequate estimates of behavioral Are adequate estimates of behavioral response already available from the response already available from the literature? literature?

Can available data be “enhanced” to improve Can available data be “enhanced” to improve sample size and precision?sample size and precision?

Enhancing State Data for Modeling

Merge population samples (e.g., 3 CPS years)Merge population samples (e.g., 3 CPS years)

– Sample overlapSample overlap

– Data are observations of real residents, but trends may Data are observations of real residents, but trends may be lostbe lost

““Balance” a national or regional sampleBalance” a national or regional sample

– National/regional data are “raked” (re-weighted) to state National/regional data are “raked” (re-weighted) to state totalstotals

– Data are synthetic, but reflect the most recent time Data are synthetic, but reflect the most recent time period on key (control) variablesperiod on key (control) variables

When Do You Need an Actuary?

You do not need an actuary to estimate You do not need an actuary to estimate enrollmentenrollment

You do need an actuary to estimate per capita You do need an actuary to estimate per capita cost when:cost when:

– Enrollee demographics are likely to change from Enrollee demographics are likely to change from past experiencepast experience

– The benefit design is newThe benefit design is new

– The benefit design must be targeted to a cost capThe benefit design must be targeted to a cost cap

What Does an Actuary Need?

Estimated eligibles and enrollment, byEstimated eligibles and enrollment, by

– Individual age and genderIndividual age and gender

– Family type and size Family type and size

– Geographic locationGeographic location

– Benefit design optionBenefit design option

Key Lessons for Modeling Policy

Be clear and selective about what the model Be clear and selective about what the model absolutely has to doabsolutely has to do

Be pragmatic about structureBe pragmatic about structure

If you choose microsimulation:If you choose microsimulation:

– Modules are easier to build, debug, and updateModules are easier to build, debug, and update

– Full integration can be costly and unnecessaryFull integration can be costly and unnecessary

– When possible, estimate model performance and When possible, estimate model performance and benchmarkbenchmark

More Lessons

Build into the model what you need out of itBuild into the model what you need out of it

– Variables for subgroup analysisVariables for subgroup analysis

– Parameters for sensitivity analysesParameters for sensitivity analyses

Be aware of tautologies and powerBe aware of tautologies and power

– Are the results showing you only the input Are the results showing you only the input assumptions?assumptions?

– Do just a few observations drive the result?Do just a few observations drive the result?