Ohare 1/28/2011 BSC
Transcript of Ohare 1/28/2011 BSC
![Page 1: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/1.jpg)
esuR lt d I i ht f CSurvey Costs:
T k FR lts and Insights from Census Task Forces
NCHS Board MeetingJanuary 28, 2011
![Page 2: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/2.jpg)
Overview• Task Forces BackgroundTask Forces Background• Survey Management Opportunities• Data Collection OpportunitiesData Collection Opportunities
– Common– Survey Specific
• Adaptive Design Opportunities• Next Steps
2
![Page 3: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/3.jpg)
The ChallengeThe Challenge
• D li ing survey coopera i lDeclin ti tion among general public
• Rising operational survey costs
• Can’t sustain data collection under the oldCan t sustain data collection under the old model
• Need to contain• Need to contain costs maintain datacosts, maintain data quality, increase operational efficiency
3
![Page 4: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/4.jpg)
NCHS: The Demand for SurveyNCHS: The Demand for Survey Information
• High demand for health care information in current climate of health care reform.
• Increasing complexity and sensitivity of medical information.
• Need for timely data. Target release is six months after collection.
• Budget pressures are ongoing and severe.
4
![Page 5: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/5.jpg)
Rising CostsRising Costs
• From 1998 to 2008 NCHS payments to• From 1998 to 2008, NCHS payments to Census for NHIS grew by 42%, while NCHS total program funds grew by only 20%total program funds grew by only 20%
• NAMCS and NHAMCS increases in payments to Census: 2006 vs. 2009– NAMCS: 16% higher
– NHAMCS: 21% higher
![Page 6: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/6.jpg)
NHISNHIS FFieldield Costs:Costs: FY98FY98-FY09FY09$10,000
FR Data Collection Cost RO Office Staff Cost
$7 000
$8,000
$9,000
$5,000
$6,000
$7,000
Thou
sand
s
$2,000
$3,000
$4,000
$0
$1,000
NOTE: FR is Field Representative (interviewer);RO is Regional Office. 6
![Page 7: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/7.jpg)
Current Survey Designy gNational Ambulatory Medical Care Survey:
– Sample: ~ 3,200 office-based physicians and ~ 312 Community Health Center phyp ysicians/pproviders
– Sample unit: patient visit– Cases assigned equally across 52 weeks, quarterly closeout schedule– Attempt to screen for eligibility by phone; in-person induction interview;
sample of patient records for each physiciansample of patient records for each physician– Respondent: sample physician / clinician; nurse or assistant can
provide information as appropriateNational Hospital Ambulatory Medical Care Survey:
– Sample: ~480 hospitals and ~200 freestanding ambulatory surgery centers
– Sample unit: patient visit– Cases divided into 16 subsamples each with a specific 4-week– Cases divided into 16 subsamples, each with a specific 4-week
reporting period; quarterly closeout schedule– Attempt to screen for eligibility by phone; in-person induction interview;
sample of patient records for each facility– Respondent: hospital administrators, medical record personnel
7
![Page 8: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/8.jpg)
NAMCSNAMCS FieldField Costs:Costs: FY98FY98-FY09FY09$1,800
FR Data Collection Cost RO Office Staff Cost
Number of Physician/Provider Offices:3 309 3 824 3 645 3 342
$1 200
$1,400
$1,600 3,309 3,824 3,645 3,342
$800
$1,000
$1,200
Thou
sand
s
$200
$400
$600
$0
SSource: Field DivisionNOTE: FR is Field Representative (interviewer);RO is Regional Office.
8
![Page 9: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/9.jpg)
NHAMCSNHAMCS FieldField Costs:Costs: FY98FY98-FY09FY09
$1,600
$1,800 FR Data Collection Cost RO Staff Cost
Number of Hospital/Care Units:435 512 474 446
$1,200
$1,400
nds
$600
$800
$1,000
Thou
san
$200
$400
$600
$0
Source:Source: FieldField DivisionDivisionNOTE: FR is Field Representative (interviewer);RO is Regional Office.
9
![Page 10: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/10.jpg)
10
PurposePurpose ofof CostCost TaskTask ForcesForces InitiativeInitiative
•• ToTo identifyidentify mostmost promisingpromising opportunitiesopportunities
to improve cost efficiency of survey data
collection procedures in Census
reimbursable surveys.
![Page 11: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/11.jpg)
The Cost Savings Task ForcesThe Cost Savings Task Forces• Designed to specifically focus on cost savings
opportunities.
• Short-lived, small teams, including surveyShort lived small teams including survey methodologist and end-data user external to agenciesagencies.
• Dialogue between Census as data collection agency and surve sponsor agencagenc and s r ey sponsor agency.
11
![Page 12: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/12.jpg)
Task Force SurveysTask Force Surveys
• National Health Interview SurveyNational Health Interview Survey• National Ambulatory Medical Care Survey /
National Hospital Ambulatory MedicalNational Hospital Ambulatory Medical Care SurveyCare Survey
• Consumer Expenditure Surveys
• National Crime Victimization Survey
• Current Population Survey• Current Population Survey
• American Housing Survey12
![Page 13: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/13.jpg)
Key OpportunitiesKey Opportunities
13
![Page 14: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/14.jpg)
Broad Opportunity Themes
• Need for better information to understand and managge cost drivers.
• Less complex management structure and less fragmentation of responsibilitiesless fragmentation of responsibilities.
• Continuous and cooperative cost managemen t throughout th d t ll tit th h t the data collection period.
14
![Page 15: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/15.jpg)
Census Survey ManagementCensus Survey Management
15
![Page 16: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/16.jpg)
Survey ManagementSurvey Management• Streamline survey management responsibilities
to reduce overhead costs.– Fragmentation of respg ponsibilities contributes to
overhead costs.
• Coordinate and consolidate management ofCoordinate and consolidate management of data collection across Regional Offices to reduce redundancy within surveysreduce redundancy within surveys.– Distinguish geo-specific functions from the more
general management functionsgeneral management functions.
16
![Page 17: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/17.jpg)
Survey Management (2)Survey Management (2)• Enhance the program management process at
Census through a project-based organization. – Reinforce “single voice” for Census communications to
our sponsors through improved internal coordination.
• Establish a more detailed survey cost accounting system, including all survey functions from all divisions.– Understand cost drivers, close to real time.– Provide greater transparency on costs.
17
![Page 18: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/18.jpg)
Data CollectionData Collection
18
![Page 19: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/19.jpg)
Data CollectionData Collection• Provide electronic communications options to
field representatives.– Improve efficiency of data transmissions, survey
materials access, and field communications.
• Reduce field address listing to upg pdate the sample pframe. – Master Address File (MAF) for Title 13; Delivery– Master Address File (MAF) for Title 13; Delivery
Sequence File (DSF) for “Title 15” surveys.
19
![Page 20: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/20.jpg)
Data Collection (2)( )
• Conduct more interviews by phone to reduce travel and mileage costs.– Establish cost benefits of centralized CATI phone
interviews vs. field rep CAPI phone vs. CAPI in-person.– Set expected proportion of phone completes by survey
und i d h h ld li ibilit id lider revised household eligibility guidelines.
• Improve CAPI questionnaire flow, response lilinkkage, andd probbes tto redduce postt-inti terviiew editing and respondent burden.
20
![Page 21: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/21.jpg)
Data Collection (3)
• Establish more detailed activity codes for field reps t t k fi ld k ti t l dto track field work preparation, travel and interviewing time. – Develop algorithms to estimate cost per specific case.
• Revise field staff performance evaluation standards to incorporate cost saving and data quality metrics, in addition to response rate. – Distribute ownership of cost containment across all
field staff.
21
![Page 22: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/22.jpg)
NHIS Survey Specific
MMove ffrom WWeeklkly to MMonthlhly fifieldld data collection assignments• Simplifies Regional Office management of
field data collection.– Manage 1 set of field assignments rather than 3 sets on any
given day.More efficient travel schedules of FRs– More efficient travel schedules of FRs.
• No loss in data detail to NCHS.
• Implementation effective January, 2011.
![Page 23: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/23.jpg)
NAMCS Survey Specific
Improve the quality of sample list of physicians provided to field representatives.
• Misinformation in AMA list results in substantial field effort to “clean” case assiggnments.
• Describe and quantify types of file errors.
• Evaluate options to clean list before FR assignmentsEvaluate options to clean list before FR assignments.– Use external business files to verify and update information.
• Determine theDetermine the value of pre-cleaning file by comparing costvalue of pre cleaning file by comparing cost to current cost of having FR resolve discrepancies.
23
![Page 24: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/24.jpg)
NAMCS/NHAMCS Survey Specific
StandarS d didize thhe process ffor contact and d
guididelidata collection attempts through specific
lines.
• Set clear expectations on the schedule to open, contact, and complete cases.
,p p ,
• Monitor case progress more closely and address concerns promptly. Contact History Instrument (CHI) in 2012.
• El t d ithi 7 k f l
stablish rolling closeout schedules with cases to be completed within 7 weeks of release.
24
![Page 25: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/25.jpg)
NAMCS/NHAMCS Survey Specific
Reduce time to gain physician / provider cooperation.cooperation.
• Consider initial phone contact through centralized calling center or RO phone bank to– assess case eligibility, – identify office contact person, y p ,– identify best times for FR visit.
• Analyze case history data from past surveys to understand nonresponse patternsnonresponse patterns.
• Establish field guidelines on steps to be taken to avoid nonresponse, based on the analysis.
25
![Page 26: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/26.jpg)
NAMCS/NHAMCS Survey Specific
Develop electronic data entry option for patient record forms for providers’ office staff use. • One half of patient data collection is done by the physician/provider’s
office staff.
• Computer-based (Blaise) instruments will be available only to Census field repsfield reps.
• Design consultant hired in FY10 Q4 to identify electronic data entry options for office staff to complete patient forms.
• Future: Common platform for both Census and office staff use? Electronic records extraction without rekeying?
26
![Page 27: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/27.jpg)
NAMCS/NHAMCS Survey Specific
I ffi i f datIncrease efficiency of d ta codi dding and
fprocessing with fewer transfers of paper forms.• Current paper process is time consuming with a number
of handoffs.
• In questionnaire and ti t f B t ti t f
2012, Census field reps will have computer-based ti i d patient forms. But, patient forms
completed by provider will continue to be on paper.
• Revise process and reduce the number of handoffs• Revise process and reduce the number of handoffs.
27
![Page 28: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/28.jpg)
NAMCS/NHAMCS Survey Specific
IdIdentifify chharacteriistiics off successffull FiFieldld
killRepresentatives (FRs) and develop these skills among ththe survey FRFRs.
• FR skills to contact and gain cooperation in physician and establishment surveys vary from those for household demographic surveys.
• Review and revise FR selection criteria. May suggest fewer FRs withReview and revise FR selection criteria May suggest fewer FRs with appropriate skills and larger caseloads.
• Consider assigning case tasks to more than one FR, based on strengths. One assignment to gain cooperation and conduct inductionstrengths One assignment to gain cooperation and conduct induction interview; Second assignment to complete patient information forms.
28
![Page 29: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/29.jpg)
NAMCS/NHAMCS Survey Specific
Consider collecting multiple weeks of patient records per physician or healthcare provider.records per physician or healthcare provider.
• Unit of analysis is the py patient record,, with each week representing an independent sample.
• Research needed to determine cost/benefit trade-off in respondent cooperation and the effects on the estimates.
• asP t lP t evaluati f thi id i d b ttion of this idea raised concerns about respondent burden and reduced cooperation and data quality.
29
![Page 30: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/30.jpg)
Adaptive Survey DesignAdaptive Survey Design
30
![Page 31: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/31.jpg)
Adaptive Survey DesignAdaptive Survey Design• Use the Contact History Instrument (CHI) in all
surveys. – Tracking of contact attempts indicates level of effort
by case.
y• S stematicallyy y collect survey py process data (in (addition to CHI) and develop monitoring tools and intervention gguidelines for case management. – Paradata can lead to responsive design.Paradata can lead to responsive design.
31
![Page 32: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/32.jpg)
Adaptive Survey Design (2)Adaptive Survey Design (2)
• Sample nonrespondents late in the field period• Sample nonrespondents late in the field period and target effort on completing the subset of casescases.– Manage expenditures in final weeks in the field.
– Can improve weighted response rate and protect against bias.
32
![Page 33: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/33.jpg)
Adaptive Survey Design (3)• Evaluate alternative sample designs based on
estimated differences in field costsestimated differences in field costs.– Set cluster sizes to achieve more control over
interviewer workloadsinterviewer workloads.– Evaluate panel sample designs to determine
cost/benefit trade-offs (precision,cost/benefit trade offs (precision, cost, quality) ofcost, quality) of alternatives.
• Develop web-based survey instruments andDevelop web based survey instruments and target use of an Internet response option.
33
![Page 34: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/34.jpg)
Adaptive Survey DesignAdaptive Survey Design• Actively pursue use of administrative records to
reduce data collection effort and to improve survey estimates.
• Conduct a responsive design pilot study to demonstrate data-driven interventions to controldemonstrate data driven interventions to control cost and data quality during data collection.
Census is in early stages of defining a pilot study– Census is in early stages of defining a pilot study.
34
![Page 35: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/35.jpg)
Opportunities: CensusOpportunities: Census• Corporate staff restructuring• Contact History Instrument in all surveys• Reduced address listing• Field rep activity coding / guidelines /
exppectations• Paradata pilot study (e.g., nonrespondent
sampling)sampling)• Costs of alternative sample designs• Phone and web modes as cost savers• Phone and web-modes as cost savers
35
![Page 36: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/36.jpg)
Opportunities: SponsorsOpportunities: Sponsors• Survey-specific opportunities
– Process modifications
– Research agendasResearch agendas
– Survey design considerations
• Int re -agency collaboration on costs
36
![Page 37: Ohare 1/28/2011 BSC](https://reader033.fdocuments.in/reader033/viewer/2022041911/6254e7ec8c6edb60691e81b3/html5/thumbnails/37.jpg)
ProgressProgress• Synthesis report of all task forces completed.
• Synthesis presented to Census Operating CommitteeCommittee.
• Draft of action plans in progress.
• Upcoming presentation to all agency heads (ICSP).
• Survey-specific changes in progress and others needeed cocollaboaboratativee atteattenttioon.
37