Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Citizen Report Card Methodology THE...
-
Upload
abraham-gordon -
Category
Documents
-
view
216 -
download
1
Transcript of Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Citizen Report Card Methodology THE...
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Citizen Report Card Citizen Report Card MethodologyMethodology
THE BUILDING BLOCKS
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Overview of CRC Overview of CRC MethodologyMethodology
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling Design
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The CRC JourneyThe CRC Journey
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling Design
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Examples of Research Examples of Research ObjectivesObjectives
General-Getting feedback from people on quality of
public servicesSpecific - Finding out satisfaction with quality of
service, satisfaction with behaviour of staff, extent of corruption, problem resolving capabilities of the agency
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The CRC JourneyThe CRC Journey
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling Design
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Designing the InstrumentDesigning the Instrument
Basic structure of a schedule/ questionnaire
Investigator informationIntroductionFilter questionsDemographicsThe body of the schedule - sections if
needed - sub- schedules if required
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Thumb Rules for a Schedule Thumb Rules for a Schedule
All research objectives to be coveredShortest length possibleLanguage - simple and convenient
based on respondents’ and investigator’s level of understanding
Clear instructions for the investigator
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Basic Tools in Instrument Basic Tools in Instrument DesignDesign
Different types of questions Open-ended and closed questions /
combinations yes/ no questions
Coding - pre and post codingUse of show cardsScaling - 5 pt or 2 ptSkipsGrids
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
CRC Process ChartCRC Process Chart
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Sampling DesignSampling Design
Why sampling?Importance of representativenessHow to assure representativeness
Fixing quotas by area or category or both Choosing appropriate sample size
• time constraints and cost constraints
• standard error
• level of sub classification required for analysis and conclusions
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
SAMPLING DESIGNSAMPLING DESIGN
BMP: 606 general households 812 slum households
General Households: 10 wards from each of 3 zones selected Number of households allocated according to population of ward Mains and crosses selected from each ward Proportionate number of households selected from each street
MAIN SAMPLE:2000 households from city and suburban areas: 800 general households 1200 slum households
Slum Households: Slums selected from each region using PPS Proportionate allocation according to slum type & size within regions Households selected using right-hand rule in each slum through systematic sampling
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Research Journey...The Research Journey...
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Field Issues...Field Issues...
Method of data collection Right field teamTraining/Briefing of investigatorsRegular reportingSpot checks and back checksTime scheduleAdherence to budget
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Right TeamThe Right TeamHow do we decide?How do we decide?
Familiarity with locality, languageAble to relate to target audienceIntelligentAvailable for the entire duration of
fieldwork And, INTEGRITY
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Right TeamThe Right TeamHow many?How many?
f {Sample size, Timeline, Productivity }
Sample size : 2000Timeline : 20 daysProductivity per person/ day : 5Team size = [(2000/20) / 5]
= 20 Investigators
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Final Team...The Final Team...
Field Supervisor
Field Supervisor
5 Interviewers5 Interviewers
Field ManagerField Manager
5 Interviewers5 Interviewers5 Interviewers5 Interviewers5 Interviewers5 Interviewers
Field Coordinator
Field Coordinator
Field Supervisor
Field Supervisor
Field Supervisor
Field Supervisor
Field Supervisor
Field Supervisor
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Data Collection Data Collection
Random sampling
Starting points selection Sample Size : 1000
100 starting points
10 listings around each starting point
Right hand rule
Boosters-- phone call, exit interviews
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
At all stages
Before launching fieldwork
During fieldwork
After fieldwork
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
Before fieldwork Pilot exercise10-15 interviewsCheck for
LengthComprehensionFlowSensitiveness
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
During fieldwork
Accompanied interviews
Sample of each interviewer’s work spot checked by field supervisor
10% back checks by the field coordinator
100% scrutiny of schedules
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Quality ChecksQuality Checks
After fieldwork
Sample sizes
Quotas
Representation
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Cost Heads in FieldworkCost Heads in Fieldwork
Cost is a function of Nature of projectType of respondentSelection criteriaSample size, sampling
Two broad types
OverheadExpenses
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Cost Heads in FieldworkCost Heads in Fieldwork
OverheadsTime cost, personnel
Communication: telephone, computers
Courier / mailings
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Cost Heads in FieldworkCost Heads in FieldworkExpenses
Printing of questionnaires Briefing and piloting Interviewer fees Supervisor fees Local conveyance Outstation travel : Field officer, interviewers Others: Venue, equipment, moderation,
gifts, transcriptions Translation
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Research Journey...The Research Journey...
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Data AnalysisData Analysis
Preliminary analysis by research agencyFurther analysis by PAC
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Framework of AnalysisFramework of Analysis
Select computer package - Foxpro, SPSS
Run basic demographics
Check if this is in line with population
If yes, proceed with further analysis
If no, weight data to represent population
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Framework of AnalysisFramework of AnalysisWhat is the depth of analysis required?What are the relevant analysis heads?
By demographics By agency By responses to other questions :
• satisfied vs. dissatisfied
• had problem vs. did not have problem
• paid bribe vs. did not pay bribeSimple measures used- percentages and mean
scores. Recently regression models
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
The Research Journey...The Research Journey...
Defining Research Objectives
Development of Instrument
Data collection
Analysis
Interpretation
Sampling
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Compare across agencies - rank them on satisfaction, responsiveness and corruption- using percentages and means
Within agencies - specific information - timings of water supply, quality of water - again using percentages and means
Two ways of interpreting same information - eg - number of visits to agency
Interpretation of DataInterpretation of Data
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
Interpretation of DataInterpretation of Data
Assimilation of scattered findings into cohesive whole
Translation of findings into diagnostic statements
Conversion of diagnosis into conclusions
Formulation of recommendations based on conclusions
Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore
From Findings to From Findings to RecommendationsRecommendations
Satisfaction with behaviour of staffObjective statement of finding -
percentage satisfied, dissatisfied, ambivalent.
Conclusion - general / high/low satisfaction/dissatisfaction
Recommendation - training for staff to improve their behaviour