Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry”...

56
Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department of Agriculture Regional Workshop on Sampling for Agricultural Surveys 23 June 2011 Montevideo, Uruguay

Transcript of Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry”...

Page 1: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Overview of Sample Surveys forForecasting & Estimating U.S. Crops

presented byTheresa “Terry” Holland

National Agricultural Statistics ServiceU.S. Department of Agriculture

Regional Workshop onSampling for Agricultural Surveys

23 June 2011Montevideo, Uruguay

Page 2: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

NASS Data Series for U.S. Crops

Crop Progress & Condition

Areaplanted & harvested

Production

Stocks

Prices Received by Farmers

Monthly Marketings

Values

Page 3: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

U.S. Crop Standards

standard weightstandard moisture

standard price

marketing yearstarts:

Wheat 60 lb (27.2 kg) per bushel 13.5% $/bu June 1

Barley 48 lb (21.8 kg) per bushel 14.5% $/bu June 1

Oats 32 lb (14.5 kg) per bushel 14.0% $/bu June 1

Flaxseed 56 lb (25.4 kg) per bushel 8.0% $/bu June 1

Cotton 480 lb (217.4 kg) per bale $/lb August 1

Rice 100 lb (45.3 kg) per hundredweight $/cwt August 1

Corn 56 lb (25.4 kg) per bushel 15.5% $/bu September 1

Sorghum 56 lb (25.4 kg) per bushel 14.0% $/cwt September 1

Soybeans 60 lb (27.2 kg) per bushel 14.0% $/bu September 1

Sunflowers 100 lb (45.3 kg) per hundredweight 8.0% $/cwt September 1

Page 4: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

System of Crop Forecasts & Estimates

early seasonarea/trees planted

area/trees expected for harvest

during seasonyield forecasts

end of seasonarea/trees harvested

total production

end of marketing yearrevised total production

Page 5: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Possible Methods for Crop Forecasting & Estimation

growers’ actual dataarea/trees planted/harvested

quantities harvested/sold/stored

growers’ expectationsareas expected to be planted/harvested

expected yields

objective counts & measurementsplant/fruit counts & measurements

expert opinioncrop progress

growing conditions

remote sensing

Page 6: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

NASS Survey Program for Major Crops

jan feb mar apr may jun jul aug sep oct nov dec

Crop Yield Surveys small grains

summer crops

Quarterly Agricultural Surveys

summer crops: planting intentions planted areaharvest intentions

– planted areaharvested area

production

small grains: planted areaharvest intentions

planted areaharvest intentions

planted areaharvested area

production

planted area(for harvest next year)

on-farm grain stocks: storage capacityquantity stored

storage capacityquantity stored

small grains - final old crop

storage capacityquantity stored

summer crops - final old crop

storage capacityquantity stored

Off-farm Grain Stocks Surveys

Prices Received by Farmers

Page 7: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Quarterly Agricultural Surveys

sample:based on –

total croplandareas of specific cropsgrain storage capacity

data obtained:areas

intended vs. actualplanted vs. harvested

productiongrain stored on farm

data collection modes:mail questionnaire

Internet questionnairetelephone/CATI interview

personal interview

Page 8: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Acreage, Yield & Production

production

area x yield

QuarterlyAgricultural

Surveys

Agricultural Yield Surveys

Objective Yield Surveys

QuarterlyAgricultural

Surveys

Page 9: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Yield Surveys

Agricultural Yield Objective Yield

samples:

March & June Ag Surveys March & June Ag Surveyscrops of interestrotated out reps

exclude high strataexclude NOL

crops of interest

supplemental

Page 10: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Yield Surveys

Agricultural Yield Objective Yield

commodities surveyed:

small grains small grainsbarleyoats

winter wheatdurum wheat

other spring wheat

winter wheat

summer crops summer cropscorn

cottondry beans

haypeanuts

rice

tobaccosorghumsoybeanssugarcanesunflowers

canola

corncotton

soybeanspotatoes

Page 11: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Yield Surveys

Agricultural Yield Objective Yield

data obtained:

farmer reported yieldsyields based on

field counts & measurements

area updates area updates

data collection modes:

mail questionnaireInternet questionnaire

telephone/CATI interviewpersonal interview

field visitpersonal interview

Page 12: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Yield Surveys

Agricultural Yield Objective Yield

advantage:

inexpensive objective

disadvantage:

subjective expensive

Page 13: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Yield Surveys

small grains: May thru Septembersummer crops: August thru November

yields to reflect conditions as of 1st of month

25

1

data collection starts25th of previous month

Page 14: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sample Sizes

number ofStates

% ofU.S. crop

samplesize

Wheat 10 53% 1410

Corn 10 84% 1920

Soybeans 11 84% 1835

Cotton 6 75% 1227

Potatoes 7 75% 1290

updated 2010

Page 15: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

WHEAT n=141010 states average 67% of U.S. winter wheat production

winter wheat in the 10 states averages 50% of U.S. production of all wheat

110

160

80

80

180

80

280

210

130

100

Objective Yield Sample Sizes

Page 16: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sample Sizes

CORN n=192010 states average 84% of U.S. corn production

180

260 290

150

280 170140

150

150 150

Page 17: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sample Sizes

SOYBEANS n=183511 states average 84% of U.S. soybean production

120

120 210

150

230 180150

120

130

140

285

Page 18: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sample Sizes

COTTON n=12276 states average 75% of U.S. upland cotton production

upland cotton in the 6 states averages 65% of U.S. production of all cotton

525 110

112150

150

180

Page 19: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sample Sizes

POTATOES n=12907 states average 83% of U.S. fall potato productionfall potatoes in the 7 states average 75% of U.S. production of all potatoes

155300

160

150

150200175

Page 20: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sampling

Fields are selected based on area

expected for harvest.

Page 21: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sampling

segment tract fieldcrop acres

expansion factor

expanded crop acres

cumulativeexpanded crop

acres selected acres

4087 6 1 130.0 53.000 6890.00 6890.00 1306.11

5030 1 2 69.0 53.000 3657.00 10547.00 10269.71

5030 1 5 59.0 53.000 3127.00 139674.00

6049 3 2 17.0 53.000 901.00 14575.00

6049 3 4 62.0 53.000 3286.00 17861.00

5031 2 2 33.0 53.000 1749.00 19610.00 19233.31

6145 2 3 34.0 53.000 1802.00 21412.00

6145 2 5 48.5 53.000 2570.50 23982.50

4381 1 2 65.0 194.000 12610.00 36592.50 21896.91

5391 1 3 78.0 194.000 15132.00 51724.50 37160.51 46124.11

4476 4 1 105.0 106.143 11145.01 62869.51 55087.71

4485 1 2 80.0 106.143 8491.44 71360.95 64051.31

4485 1 4 100.0 106.143 10614.30 81975.25 73014.91

4494 5 1 87.0 106.143 9234.44 91209.69 81978.51 90942.11

4494 5 2 48.0 106.143 5094.86 96304.55

4494 5 3 85.0 106.143 9022.15 105326.70 99905.71

4494 5 4 100.0 106.143 10614.30 115941.00 108869.31

6467 1 4 32.0 106.143 3396.58 119337.58 117832.91⋮ ⋮ ⋮ ⋮ ⋮ ⋮

4153 1 6 9.0 53.000 477.00 895617.99

4153 1 7 14.0 53.000 742.00 896359.99

interval = total expanded acres/sample size = 896359.99/100 = 8963.60

random start ∊ (0. interval] = 1306.11

Page 22: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sampling

Samples consist of 2 plotsrandomly located within each selected field.

starting corner

Plot 1

Plot 2

Page 23: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Sampling

Each plot consists of a 5-foot buffer & a count area.

5-ft buffer zone

count area

count area

Wheat 3 rows x 21.6 inchesCorn 2 rows x 15 feet

Cotton 2 rows x 10 feetSoybeans 2 rows x 3.5 feetPotatoes 1 row x 20 feet

Page 24: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Yield Components

number of fruitper acre x weight

per fruit – harvest lossper acre

gross yield

Page 25: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Gross Yield Components

WHEAT

heads per acre grain weight per head

at harvest: heads grain weightmoisture content

forecast: stalksheads in boot

emerged heads

historical averagefertile spikeletsgrains per head

head weight

Page 26: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Gross Yield Components

CORN

ears per acre grain weight per ear

at harvest: ears grain weightmoisture content

forecast: stalksears & ear shootsears with kernels

historical averagekernel row length

ear diameterear weight

Page 27: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Gross Yield Components

SOYBEANS

plants per acre x pods per plant bean weight per pod

at harvest: plantspods

bean weightmoisture content

forecast: plantsmain stem nodeslateral branches

blooms, dried flowers & podspods with beans

historical averagelateral branchespods with beans

pod weight

Page 28: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Gross Yield Components

COTTON

bolls per acre lint weight per boll

at harvest: open bollslarge bolls

lint weightmoisture content

forecast: squaresblooms

bollsburrs

historical averageblooms

bollslarge boll weight

Page 29: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Gross Yield Components

POTATOES

hills per acre potato weight per hill

at harvest: hills potato weight

forecast: — —

Page 30: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Harvest Loss Component

gross yield – harvest loss

after harvest: gleanings

forecast: historicalaverage

Page 31: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Forecasting the futureis dependent on

patterns of the past...

Page 32: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Current Counts& Measurements

Historical Data ModelsCounts & Measurements Final Yield⇒

Forecast

Objective Yield Forecasting

Page 33: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Model #1

0

200

400

600

800

1000

1200

0 200 400 600 800 1000 1200

final

frui

t cou

nt

early season variable

0.0

0.2

0.4

0.6

0.8

1.0

1.2

0 5 10 15 20 25 30 35 40 45 50

final

wei

ght

early season variable

Number of Fruit

Weight per Fruit

Page 34: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Objective Yield Model #2

Early Season Yield Forecast

06

0497

0196

0895 02

98 03

09

05

07

9900

y = 0.0013x - 76.025

80

90

100

110

120

130

140

150

120000 130000 140000 150000 160000 170000

independent variable

NA

SS y

ield

Page 35: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Yield Survey Indications

Objective Yield 1:➀ regress sample level counts &/or measurements to final crop-cutting data➁ forecast yields for each sample➂ average sample level yields to state & regional forecasts

overestimates

Objective Yield 2:➀ aggregate sample counts &/or measurements to state & region – independent

variable➁ regress independent variable to final NASS yield

unbiased estimates

Agricultural Yield:➀ summarize grower reported yields using sampling weights

underestimates

Page 36: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Yield Forecasting Process

➊ regress Objective Yield1 & Agricultural Yield indications to final NASS yield➋ determine prediction intervals for each regression (± 1 SE)➌ determine the overlap of the three prediction intervals

➍ use expert knowledge to decide where within the overlap intervalto set forecast

110

120

130

140

150

160

170

OY 1 OY 2 AY overlap

Page 37: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Analyzing Yield Data

survey data• impact of unusual reports• maturity stages• yield components• other data relationships

crop progress reports• planting dates• pollination dates

weather data• rainfall• heat degree days

industry contacts• insect problems• disease problems

Page 38: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Reliability of U.S. Crop Production Forecasts

Page 39: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Reliability of U.S. Crop Production Forecasts

Page 40: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Grain & Oilseed Stocks Surveys

Off-farm On-farm

sample:

commercialgrain storage operations

elevatorswarehouses

terminalsgrain & oilseed processing plants

other storage facilities not classified as farms

farms in Agricultural Surveys

Page 41: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Grain & Oilseed Stocks Surveys

Off-farm On-farm

data obtained:

all whole (unprocessed) grains & oilseedson hand or stored in permanent or temporary facilities

regardless of ownership or intended use

data collection modes:

mail questionnaireInternet questionnairetelephone interviewpersonal interview

mail questionnaireInternet questionnaire

telephone/CATI interviewpersonal interview

Page 42: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Grain & Oilseed Prices Surveys

sample:

firms that purchase whole grains & oilseeds from farmers30 States in monthly program account for 90% of all U.S. sales

data obtained:

quantity purchased from farmers during previous month

total dollars paid for this quantityincluding quality discounts & premiums,

excluding deductions for drying, handling, cleaning, storage, grading, checkoff fees, etc.

data collection modes:

mail questionnaireInternet questionnairetelephone interviewpersonal interview

Page 43: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Surveys

monthly Crop Production reportsissued on or about 10th day of each month

reflect conditions as of 1st day of month

weekly Crop Progress & Condition reportsissued 1st business day of each week, April-November

reflect status & conditions as of previous Sunday

farmer activities, such as planting and harvestingprogress of crops through various phenological stages of development

crop, pasture & range conditionssoil moisture ratings

Page 44: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Coverage

Crop progress & condition:barley, corn, cotton, peanuts, oats, rice,sorghum, soybeans, sugarbeets, sunflowers,winter wheat, spring wheat

States that account forat least 75% of areaplanted to crop of interest

Pasture & range condition: all States except Alaska & Hawaii

Soil moisture &days suitable for field work:

all States except Alaska, Arizona, California, Hawaii & Nevada

Narrative: all States

At least one reporter for each agricultural county in each State

Page 45: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Corn Progress & Condition

18 states account for 92% of acres planted to corn in 2009

planted

emerged

silk

dough

dent

mature

harvested

condition

Page 46: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Soybean Progress & Condition

18 states account for 95% of acres planted to soybeans in 2009

planted

emerged

bloom

set pods

drop leaves

harvested

condition

Page 47: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Winter Wheat Progress & Condition

18 states account for 89% of acres planted to winter wheat for 2010

planted

emerged

headed

harvested

condition

Page 48: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Spring Wheat Progress & Condition

6 states account for 99% of acres planted to spring wheat in 2009

planted

emerged

headed

harvested

condition

Page 49: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Cotton Progress & Condition

15 states account for 99% of acres planted to cotton in 2009

planted

squaring

set bolls

bolls open

harvested

condition

Page 50: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Other Crops Progress & Condition

numberof

States

% of2009

plantedarea progress stages

Barley 5 81% planted, emerged, headed, harvested, condition

Oats 9 64% planted, emerged, headed, harvested, condition

Peanuts 8 98% planted, pegging, harvested

Rice 6 100% planted, emerged, headed, harvested, condition

Sorghum 11 98% planted, headed, coloring, mature, harvested, condition

Sugarbeets

4 84% planted, harvested

Sunflowers 4 85% planted, harvested

Page 51: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Reporters

5,000 reporterswhose occupations provide frequent opportunities

to observe crops and/or contact farmers in their counties

Extension Service agentsUSDA Farm Service Agency directors

NASDA enumeratorsAgribusiness representatives

Crop advisorsFarmers

Page 52: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Data Collection

mail

phone

fax

personal interview

e-mail

Internet~ 80-85% ~

Page 53: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Questionnaire

Required items specified weekly by HQhistorical trends at national levelcurrent trends at national level

“rules”

States may deviate from requirementsstart earlier or end later for particular items

additional items of interest to State

All questions must use standardized wording!

Page 54: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Summary & Analysis

• County averages weighted by planted areasto State-level estimates

• State estimates weighted by planted areasto U.S. estimates

• State & U.S. progress estimates compared withlast week, last year and 5-year average

• within-season progress cannot go backwards• unusually large or small changes must be explained

Page 55: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Reports

issued at 16:00 Eastern1st business day of week

issued at 12:00 Eastern2nd business day of week

issued by World Agriculture Outlook Boardafternoon 2st business day of week

Page 56: Overview of Sample Surveys for Forecasting & Estimating U.S. Crops presented by Theresa “Terry” Holland National Agricultural Statistics Service U.S. Department.

Crop Progress & Condition Uses & Users

producersagribusinesses

commodities tradersgovernment agencies

researchersassess current growing conditions

in order to reduce inherent risks of doing businessplanning, decision-making, research

NASSinterpretation of survey results for crop yield forecasts