Jaki S. McCarthy National Agricultural Statistics Service,
-
Upload
yeo-harper -
Category
Documents
-
view
72 -
download
0
description
Transcript of Jaki S. McCarthy National Agricultural Statistics Service,
![Page 1: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/1.jpg)
Like – But Oh, How Different: The effect of different questionnaire formats in the 2005 Census of Agriculture Content Test.
Jaki S. McCarthyNational Agricultural Statistics Service,
United States Department of Agriculture
![Page 2: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/2.jpg)
Census of Agriculture conducted every 5 years
• Includes all known operations with agriculture potential of any type (~3mil)
• Information collected on acreage, production, sales and expenses, production practices, and demographics
• Primarily mail out/mail back form, with follow up mailings, limited CATI and interview follow up
• Reporting is mandatory
![Page 3: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/3.jpg)
Prior to Census, Census Content Test is Conducted• Test used to finalize forms, test data
collection and processing procedures• Small scale mail out/mail back with follow up reminder
• Follow up cognitive interviews conducted with subset of respondents
• Reporting was not
mandatory
![Page 4: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/4.jpg)
2005 Census Content Test (CCT)
• Two forms tested (long and short)
• In 2007 Census, short form will be targeted to specific types of operations
• For CCT, split sample of all farms used for analysis
![Page 5: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/5.jpg)
Both CCT Forms collected the same information
• Differences in:– Format– Layout– Number of pages
![Page 6: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/6.jpg)
Why use a short form?
• May increase response rates
• May be faster and/or easier to complete
• Costs less to print and mail
![Page 7: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/7.jpg)
Why use a long form?• More specific examples and instructions
can be included
• Explicit screening questions and routing instructions can be incorporated
• Common items are pre-printed
• Flexibility with layout can
allow related items to be
placed together
![Page 8: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/8.jpg)
Differences Between Forms
LONG FORM• 24 pages• Acres operated by land type
collected immediately after acres operated by ownership
• Separate sections for different commodities; Most common items pre-printed, others listed
• Sales collected separately from production in list
• Government programs listed
SHORT FORM• 12 pages• Acres operated by land type
collected several pages after acres operated by ownership
• All crops and livestock collected in open write-in tables; Only a few items included as examples
• Value of sales collected with production in open tables
• Government program info collected in abbrev. tables
• Qs on production contracts and irrigated pasture omitted
![Page 9: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/9.jpg)
Short Form, Field Crops and Hay Table
![Page 10: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/10.jpg)
Long Form, Field Crop Section
![Page 11: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/11.jpg)
Short and Long Forms
• Split sample used to send short and long forms to all types of operations
• Response rate for each form tracked • Content was the same, so comparisons of length were
not confounded with content• Data compared across forms• Information from cognitive
interviews compared across forms
![Page 12: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/12.jpg)
Less pages ≠ Higher response
Form Total Mailed Receipts Response Rate % (Weighted*)
Short 12808 6870 53.24 (52.73)
Long 14860 8200 54.68
*short form response weighted for comparability to long form subgroup sizes
![Page 13: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/13.jpg)
Long Form: page 2
Short Form: page 2
Long Form: Facing on Page 3
Short Form: 3 pages later on page 5
![Page 14: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/14.jpg)
If you want people to report the same number twice,
don’t ask them 3 pages apart
Form % Total Land 1 missing (n)
% Total Land 2 missing (n)
% Total land 1 <> Total land 2 (n)
Short 6.3%(364)
14.6%(841)
39.61%(2180)
Long 5.1%(358)
11.4%(790)
26.54%(1779)
Problems reporting Total Acres
![Page 15: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/15.jpg)
Quantity Unit preprintedCalculated yields = 1
Form Corn Soy
Short 11% 13%
Long 2% 2%
If you want bushels, ask for bushels
![Page 16: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/16.jpg)
Calculated yields = 1
Form Corn Soy
Short 11% 13%
Long 2% 2%Quantity Unit must be written by respondent
![Page 17: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/17.jpg)
Long Form
Short Form
![Page 18: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/18.jpg)
If you want something reported, ask for it explicitly
Total reporting horses in prior census
Horses reported in prior census and in CCT
% (n)
Horses reported in prior census and 0 in CCT
% (n)
Short 1108 32.1%
(356)
67.9%
(752)
Long 1409 75.0%
(1056)
25.0%
(353)
![Page 19: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/19.jpg)
Other problems
• Short Form: Fewer gov. payments reported• Short Form: More manual processing required
(extra sheets, uncodable responses, more than one number entered in a cell)
• Short Form: Slightly more omitted agriculture reported in cognitive interviews (9.9% vs 7.4%)
• Long Form: Self reported completion times slightly longer (44.46 vs 47.87 minutes)
![Page 20: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/20.jpg)
Changes based on these findings
• Added additional types of operations to exclude from short form (e.g. horses, specialties, etc.)
• Use of more long form question formats on short form (e.g. government payments)
• Use of additional edit checks on short form
![Page 21: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/21.jpg)
Conclusions• Decreasing # of pages does not increase
response rate
• More data problems with the short form, particularly underreporting
• Forms may collect the same
data, but their performance
is NOT equivalent—
Like, but different
![Page 22: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/22.jpg)
Plans for use of Short Form
• Short Form will be used ONLY with limited set of operations (e.g. operations known to have 4 or less commodities; operations without specialty commodities; operations not previously reporting horses, goats, sheep, bees, etc)
• With over 3 million mailing
addresses, cost savings for
lighter mailing can be substantial
![Page 23: Jaki S. McCarthy National Agricultural Statistics Service,](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813518550346895d9c6eed/html5/thumbnails/23.jpg)
YES, it was the mountain Echo, Solitary, clear, profound, Answering to the shouting Cuckoo, Giving to her sound for sound!
Unsolicited reply To a babbling wanderer sent; Like her ordinary cry,
Like--but oh, how different! --W. Wordsworth