Paper-Based Questionnaire Surveysin U.S. ranch-management literature

Kristin GangwerFebruary 22, 2010GEOG 5161-002

• Method of generating primary data for quantitative or mixed-methods studies

• “Provides a quantitative or numeric description of trends, attitudes, or opinions of a population by studying a sample of that population.”

• Data then used to calculate statistical information

Overview of Questionnaire Surveys

• Questionnaire design– Open, closed

• Sample– Population, size, demographic characteristics, selection of individuals,

stratification• Delivery medium

- Face-to-face, postal, telephone, email• Cross-sectional or longitudinal• Response bias• For the rest of the presentation I will be speaking specifically

about: Paper-based, postal questionnaires that are self administered and cross-sectional.

Considerations

Strengths• Increases potential sample size• Reaches geographically-disperse respondentsWeaknesses• Lower response rates (especially without return

postage)• Incomplete responses• Increased costs• Can’t clear up misunderstandings

General Strengths & Weaknesses

• Aging population• Geographic dispersion• Access to contact information• Sample size • Popular for studying natural resource issues

(Didier and Brunson 2004, 331)1. Population parameters can be quantified2. Analytical tools are well defined and can be used to provide generalizations about the

population under study3. Variance of key parameters and the degree of confidence associated with hypotheses

can be estimated4. Replication and validation are relatively easy when standardized procedures are used5. Survey method generally accepted by the scientific community

Advantages for Ranch-Management Studies

1. “Socioeconomic behavior of cattle ranchers, with implications for rural community development in the West” (Smith and Martin 1972)

2. “Motivation of Colorado ranchers with federal grazing allotments” (Bartlett et al. 1989)

3. “To ranch or not to ranch: Home on the urban range?” (Liffmann, Huntsinger and Forero 2000)

4. “Classifying federal public land grazing permittees” (Gentner and Tanaka 2002)

Examples

• Sought to understand cattle ranching and ranchers by viewing the ranch resource as generating both production and consumption outputs.

• 89-person random sample of Arizona ranch owners• 33-question questionnaire • Economic and attitudinal variables

Socioeconomic Behavior of Cattle Ranchers, with Implications for Rural Community Development in the West

Arthur H. Smith and William E. MartinAmerican Journal of Agricultural Economics, 1972

• Factor analysis– 11 factors (explaining 69.2 percent of the variance of the original variables)– 4 basic groups (not mutually exclusive)

• Discriminant analysis– Used to determine if a significant difference in goals and attitudes existed between ranchers willing to sell their ranches and those who

weren’t• Final discriminant function

– Used 3 of the original factors, which were found to significantly explain differences between groups (i.e., keep or sell)

• Predict keep or sell with 80% accuracy

Socioeconomic Behavior of Cattle Ranchers, with Implications for Rural Community Development in the West

Arthur H. Smith and William E. MartinAmerican Journal of Agricultural Economics, 1972

• Determined ranchers’ willingness to sell their ranches and determined which factors were important in their decision to ranch.

• Postal survey• Questions developed from literature• Random sample of the 1,530 ranchers who had summer federal grazing permits in

CO in 1983, and who had cattle• Excluded other seasons and grazing for sheep from sample• Pre-survey of 238 to see if population understood and responded well to the

format and questions. Also to estimate response variance and response rate to determine the required sample size. Result: shortened survey and clarified some questions.

• Improved survey sent to 1,000 ranchers, with one additional mailing to nonrespondents. Pooled pre-survey and survey responses.

• 313 useable surveys returned.

Motivation of Colorado Ranchers with Federal Grazing Allotments

E.T. Bartlett, R.G. Taylor, J.R. McKean and J.G. HofJournal of Range Management, 1989

• Simple correlation analysis didn’t appear promising• Cluster analysis -- 4 groups identified

– 7 variables relating to continuing ranching examined– K-mean clustering– Nonhierarchical

Motivation of Colorado Ranchers with Federal Grazing Allotments

E.T. Bartlett, R.G. Taylor, J.R. McKean and J.G. HofJournal of Range Management, 1989

• Ranchers in 3 CA counties (2 areas) surveyed about effects of increasing development, land use change, and attrition of the ranching community on their commitment to ranching -- and also to assess land conservation program acceptability.

• Questions: practices, reasons for ranching, and what influences ranching’s future.• Sample randomly selected from compilation of U.C. Extension’s Farm Advisor list, NRCS

list, and an emergency feed program list.• Respondents had to 1) graze livestock in the study areas and 2) be the main decision

maker.• Postal questionnaire, 4-wave mailing technique, winter of 1993/94• Eligible and usable questionnaires:132 of 178 in Tehama County, and 113 of 204 in

Alameda/Contra Costa County.• Booklet form, 19 pages long, 48 questions and groups of questions (topically based)• Likert-scale, 1-5 ranking of importance (1=not at all important, 4=very important,

5=does not apply

To Ranch or Not to Ranch: Home On the Urban Range?

Robin H. Liffmann, Lynn Huntsinger and Larry C. ForeroJournal of Range Management, 2000

• Statistical tests performed:– Chi-square: significant differences in

categorical responses between ranchers in the 2 study areas

– Student’s t-test: compare grouped continuous variables (ex: respondent age, how long ranch owned).

– Alpha level = .1 (p<.1 discussed as significant)

To Ranch or Not to Ranch: Home On the Urban Range?

Robin H. Liffmann, Lynn Huntsinger and Larry C. ForeroJournal of Range Management, 2000

• Identified the characteristics and attitudes of public land ranchers (U.S. Forest Service and BLM).

• Duplicate addresses and institutional permits removed from permittee lists, leaving total individual ranch-operator population of 21,018.

• Random sample of 2,000 operators• 4 mailings spread over 8 weeks• 53.5% useable response rate• <3 items left blank, responses kept and missing values imputed from rest of data• Follow-up telephone survey of 100 randomly selected non-respondents: the

means of key demographic characteristics not significantly different between respondents and non-respondents. Data can be interpreted to accurately represent the population of all public land permittees.

Classifying Federal Public Land Grazing Permittees

Bradley J. Gentner and John A. TanakaJournal of Range Management, 2002

• Cluster analysis -- 8 groups identified– Observational units– Rancher attributes

• Previous literature (Bartlett et al. 1989; Smith and Martin 1972)

• Focus group interviews in Oregon and New Mexico helped hone list

– K-means algorithm– Validation process

Classifying Federal Public Land Grazing Permittees

Bradley J. Gentner and John A. TanakaJournal of Range Management, 2002

• Inhibits discovery– Pre-determined questions and appropriate range of responses

• Inflexible– Can’t adapt throughout research process

• Focus on aggregates rather than individuals– Heterogeneity of ranchers (difficult to make generalizations, etc.)

• Can’t account for unanticipated outcomes• Misses detailed, ranch-level analysis and complexity (Eakin

and Conley 2002, 272)• Unable to explore rancher decision-making frameworks

(Sayre 2004, 668)

Limitations

• Other surveys– Face-to-face, structured interviews– Web, telephone questionnaires

• Qualitative methods– In-depth interviews (Eakin and Conley 2002)– Semi-structured interviews (Knapp and Fernandez-Gimenez 2009)– Grounded theory: inductive approach where researcher begins with observations and

then identifies patterns (Didier and Brunson 2004, 331)– “The authors are currently planning a study in Arizona to examine these “impact”

relationships in detail and to explain the viability or lack of viability of small rural communities and the ranches nearby. In doing so, the authors do not feel they can use aggregate methods such as input-output analysis or economic base analysis, traditionally used by economists in impact studies. Such methods are too mechanical and gloss over the complex social interrelationships so important in the business of small rural communities. The authors feel they must take an almost anthropological view of the communities’ inhabitants in order to examine the detailed interactions involved” (Smith and Martin 1972, 8).

• Mixture of interviews and questionnaire (Rowe, Bartlett and Swanson, Jr. 2001)

Other Methods

• Bartlett, E.T., R.G. Taylor, J.R. McKean, and J.G. Hof. 1989. Motivation of Colorado ranchers with federal grazing allotments. Journal of Range Management 42: 454-457.

• Creswell, J.W. 2009. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.

• Didier, E.A. and M.W. Brunson. 2004. Adoption of range management innovations by Utah ranchers. Journal of Range Management 57: 330-336.

• Eakin, H., and J. Conley. 2002. Climate variability and the vulnerability of ranching in southeastern Arizona: a pilot study. Climate Research 21: 271-281.

• Kitchin, R. and N.J. Tate. 2000. Conducting research in human geography: theory, methodology and practice. London: Prentice Hall.

• Liffmann, R.H., L. Huntsinger, and L.C. Forero. 2000. To ranch or not to ranch: Home on the urban range? Journal of Range Management 53: 362-370.

• Rowe, H.I., E.T. Bartlett, and L.E. Swanson, Jr. 2001. Ranching motivations in 2 Colorado counties. Journal of Range Management 54: 314-321.

• Smith, A.H. and W.E. Martin. 1972. Socioeconomic behavior of cattle ranchers, with implications for rural communities. Journal of Agricultural Economics: 1-9.

Sources

Questions?