CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted...
Transcript of CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted...
![Page 1: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/1.jpg)
© 2016. Materials may not be reproduced without permission of the author.
Planning and Performing Quality Control of Computer-Assisted Personal Interviewing
Carol WintheiserUniversity of Wisconsin Survey Center
University of Wisconsin-Madison
International Field Directors & Technologies ConferenceMay 23, 2017
![Page 2: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/2.jpg)
Acknowledgements
• Chris Schlapper• Kate Krueger• Karen Zoladz• Nick Schultz• Rob Schultz• Kyle Krueger• Thomas Francis• Dakota Zarak
2
![Page 3: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/3.jpg)
Overview
• Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI)
• How we perform quality control• Notes review• Verification calls• Monitoring• Feedback
• Forecasting and planning quality control
3
![Page 4: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/4.jpg)
Overview
• Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI)
• How we perform quality control• Notes review• Verification calls• Monitoring• Feedback
• Forecasting and planning quality control
4
![Page 5: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/5.jpg)
Why we perform quality control
• Ensures high quality data for our clients• Guards against data falsification• Helps identify and correct data errors before data delivery
• Ensures high quality interviewers
5
![Page 6: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/6.jpg)
Overview
• Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI)
• How we perform quality control• Notes review• Verification calls• Monitoring• Feedback
• Forecasting and planning quality control
6
![Page 7: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/7.jpg)
Overview
• Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI)
• How we perform quality control• Notes review• Verification calls• Monitoring• Feedback
• Forecasting and planning quality control
7
![Page 8: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/8.jpg)
How we perform quality control
8
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 9: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/9.jpg)
How we perform quality control: Notes Review
9
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 10: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/10.jpg)
How we perform quality control: Notes review
• Closely examine all notes left in a case• Review 100% of all case notes in Field• Notes review identifies potential:• We use a Notes Review Database for notes within CASES instrument
10
![Page 11: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/11.jpg)
How we perform quality control: Notes Review
11
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 12: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/12.jpg)
How we perform quality control
12
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 13: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/13.jpg)
How we perform quality control: Verification Call
13
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 14: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/14.jpg)
How we perform quality control: Verification Call
• Were there any problems or concerns you had with the visit or the interviewer?• Was the interviewer courteous and respectful?• Did the interviewer speak clearly?• Did the interviewer talk about themselves personally in a way that was
unprofessional or a poor use of your time?• The interviewer should have given you ____ cash to thank you for your
participation. Did you receive that amount of cash?• Is there anything else you would like to tell me about the visit?
14
![Page 15: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/15.jpg)
How we perform quality control: Verification Call
15
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 16: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/16.jpg)
How we perform quality control
16
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 17: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/17.jpg)
How we perform quality control: Monitoring
17
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 18: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/18.jpg)
How we perform quality control: Monitoring
18
![Page 19: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/19.jpg)
How we perform quality control: Monitoring
• Interviewer protocols
• Standardized Interviewing
• Interviewer administration
• Study specific protocols
19
![Page 20: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/20.jpg)
How we perform quality control: Monitoring
20
![Page 21: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/21.jpg)
How we perform quality control: Monitoring
21
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 22: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/22.jpg)
How we perform quality control
22
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 23: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/23.jpg)
How we perform quality control: Feedback
23
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 24: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/24.jpg)
How we perform quality control: Feedback
• One-on-one Calls• 1 Interviewer + 1 Supervisor• Weekly at a set time• 30 minutes to over an hour
• Conference Calls• All Interviewers for a project + Available Supervisors• Weekly at a set time• 30 minutes to over an hour
24
![Page 25: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/25.jpg)
How we perform quality control: Feedback
25
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 26: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/26.jpg)
How we perform quality control: Feedback
26
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 27: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/27.jpg)
How we perform quality control: Feedback
27
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 28: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/28.jpg)
How we perform quality control
28
Case Worked
Interview Complete
Notes Review
Verification Call
Monitoring
1 on 1 Calls
Conference Calls
![Page 29: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/29.jpg)
Overview
• Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI)
• How we perform quality control• Notes review• Verification calls• Monitoring• Feedback
• Forecasting and planning quality control
29
![Page 30: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/30.jpg)
Overview
• Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI)
• How we perform quality control• Notes review• Verification calls• Monitoring• Feedback
• Forecasting and planning quality control
30
![Page 31: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/31.jpg)
Forecasting and planning quality control
• Initial meetings with CAPI Supervisors and Project Director• Discuss scope of project and develop plan: Which QC methods to use?• Typical
• Atypical
31
MonitoringInterview Complete Audio Recording
Interview Complete MonitoringAudio Recording?
![Page 32: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/32.jpg)
Forecasting and planning quality control
• Frontload monitoring effort • 18-25% of completed interviews monitored in initial weeks of data collection• Ensure every interviewer delivered feedback in first three weeks
• Average monitoring interviewers 1x per month• Varies based on field period & sample size• Prioritize those with lower scores
• 10% completed interviews monitored overall by end of field period
32
![Page 33: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/33.jpg)
Forecasting and planning quality control
• Midlife in the United States – Milwaukee Subsample• Interviews average over 2.5 hours, the longest being just under 7 hours• After full monitoring of first several interviews, shorten the QC Monitoring by
sections:• Certain sections are mandatory to QC in every monitoring• Other sections can be chosen by the person performing the QC
33
![Page 34: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/34.jpg)
Forecasting and planning quality control
• Consolidate reporting into one central location: • Project Management Database (PMDB)
• Streamline QC & feedback process• Better track trends we notice during QC and 1 on 1 calls• More effectively track when interviewers need re-monitoring
34
![Page 35: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/35.jpg)
Looking Ahead
• Review systems used for quality control to make most effective & efficient
35
![Page 36: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/36.jpg)
© 2016. Materials may not be reproduced without permission of the author
Questions?
![Page 37: CAPI Quality Control · Overview • Why we perform quality control (QC) in Computer-Assisted Personal Interviews (CAPI) • How we perform quality control • Notes review • Verification](https://reader035.fdocuments.in/reader035/viewer/2022071607/61457e1a07bb162e665fba25/html5/thumbnails/37.jpg)
Please visit us at:www.uwsc.wisc.edu
Thank You!
For copies of this presentation or more information, contact:
Carol [email protected]