5th Sen2-Agri Webinar, 19 July 2018
The webinar will last around 1h
The slides will be available on the Sen2-Agri website in the coming 24hrs(http://www.esa-sen2agri.org/)
Moderator:
Pierre Defourny from UCLouvain
Presenters:
Sophie Bontemps and Nicolas Bellemans from UCLouvain
Sen2Agri users
Members of the consortium available to answer your questions:
Cosmin Cara, Cosmin Udroiu from Cs-Romania
Welcome to the
5th webinar
5th Sen2-Agri Webinar, 19 July 2018
1. Welcome
2. Information about the new version 1.8.2 of the system
3. In situ data collection: experience sharing within Sen2-Agri community
4. Summary of the different data collection techniques and important challenges
5. Next events
Webinar outline
5th Sen2-Agri Webinar, 19 July 2018
Sen2-Agri: open source system to deliver
4 Sen2-Agri products along the season
EARLY AREA INDICATOR
EARLY AREA INDICATOR
Binary map identifying annually cultivated land at 10m updated every month
Crop type map at 10 m for the main regional
crops including irrigated/rainfed
discrimination
Vegetation status map at 10 m delivered every
week (NDVI, LAI, pheno index)
Monthly cloud free surface reflectance composite at 10-20 m
in line with the GEOGLAM core products
5th Sen2-Agri Webinar, 19 July 2018
Sen2-Agri system
A system designed to run in an automated near real time (and off line) mode to deliver agric. products as satellite images are ingested
=> Orchestrator concept
5th Sen2-Agri Webinar, 19 July 2018
2 operating modes
Automated mode
a) based on the Orchestrator with by-default parmeterization, automatic data download and processing until the end of the season, on-time delivery => operational scenarios
b) Processor execution on user request, with by-default parameterization
Manual mode: manual processing, independently for any processor, without installing and configuring the whole system
System can be operated through GUI, SNAP, terminal window / command line
5th Sen2-Agri Webinar, 19 July 2018
• 716 system downloads on www.esa-sen2agri.org/
Sen2-Agri users
Ongoing supports
30 downloads
1 download
Map of Sen2-Agri system download
5th Sen2-Agri Webinar, 19 July 2018
Sen2-Agri support
http://www.esa-sen2agri.org/resources/faq/
http://forum.esa-sen2agri.org/
> 70 active users (without staff)
5th Sen2-Agri Webinar, 19 July 2018
1. Welcome
2. Information about the new version 1.8.2 of the system
3. In situ data collection: experience sharing within Sen2-Agri community
4. Next events
Webinar outline
5th Sen2-Agri Webinar, 19 July 2018
Easy and transparent update through an update script
Sen2-Agri: a continuously improving system !
Version 1.8.2
released on
16/07/2018
Version Release date Purpose
1.0 July 2016 First beta-version release to project demo users
1.6 June 2017 Pre-release of the system to project champion-users
->1.6.1 July 2017 First public release
1.7 November 2017 New User interface functionalities Handling of S2B together with S2A and L8Able to use local repository of L1C data (IPT, EODC)
1.8 April 2018 Pre-release of the v1.8- New User interface functionalities - New downloader (sen2agri-services application)- Pre-release of a new Vegetation indices processor (LAI-FAPAR-FCOVER-NDVI)
-> 1.8.1 May 2018 Stable release of the v1.8
-> 1.8.2 July 2018 Evolutions of the downloader and of the parameters of L2A processor
5th Sen2-Agri Webinar, 19 July 2018
Version 1.8.2 new features
Version 1.8.2 includes two important evolutions:
1. New features introduced in the downloader (sen2agri-services application)– handling of multi-polygons site extent
– handling of long decimal precision in the geometry coordinates of the site extent
– handling of the time lag between the S2 images availability on the ESA-Scientific Data Hub and on the AWS store
2. Update of the Ground Image Processing Parameters (GIPP) files used by the atmospheric correctionprocessor (L2A)
5th Sen2-Agri Webinar, 19 July 2018
Different scenarios
to update your system
Standard installation
Standard update+ GIPP update
Standard update! GIPP update only before new site definitionCheck http://www.esa-sen2agri.org/release-of-the-version-1-8-2/ for details
5th Sen2-Agri Webinar, 19 July 2018
1. Welcome
2. Information about the new version 1.8.2 of the system
3. In situ data collection: experience sharing within Sen2-Agri community
4. Next events
Webinar outline
5th Sen2-Agri Webinar, 19 July 2018
Sen2-Agri system : parameters settings
Area of Interest Shapefile to be uploaded
Monitoring period Start and end dates to be defined
S2 or S2 + L8 To be selected
Other parameters …
Field data as a key but challenging activity
SoS
Monitoring periodBefore the monitoring period
System initialization
EoS
Sen2-Agri system : field campaigns
Sampling design Stratification and sampling
Field visit In situ data collection – early survey
In situ data collection – mid-season survey
Data upload Field data quality control
5th Sen2-Agri Webinar, 19 July 2018
• World Food Programme – South Sudan
• ESA PECS Project – Bulgaria
• CIMMYT – Mexico
• ICRISAT / IER – Mali
Users Experience
Sentinel-2:
WFP Approaches for In Situ Data Collection
WFP Existing System for Crop Type Mapping
Crop Type Ground Data Collection
Sentinel-2 Data Assimilation
Cloud Storage and Processing Centre
Crop Type Map
ONA Platform
Assimilation, Quality Control, Conversion
on
Crop type data collection:Uses JECAM (Joint Experiment in Crop Area Measurement) compatible guidelines complemented by UCL Geomatics (Belgium).
Basic minimum information set required:
IDCrop / No CropCrop TypeCrop/LC ClassIrrigation
Stratification: Farming Systems / Sectors, Agro-climatic.
30-50 samples per strata and crop type.
Non cropland data also collected, mostly by photo-interpretation.
Additional data collected to ease interpretation and quality control:
Development stageCrop planted previous seasonPlanting and Harvest dateFree comments (crop status, flood, weeds, trees)
Crop Type Data
Data Collection Tool: GeoODK
Widespread use of ODK in household survey in WFP, guaranteeing a shallow learning curve and instant familiarity
Simple to set up field forms, very simple to use and teach
Very few fancy features / very robust
Data Collection Tool
GeoODK-Collect selected for in-situ data collection
Used in 2 small pilots and one large pilot.
Now in use on two large scale operational deployments
Starting off…
Tap the form that you want to use.
Survey date: there is no need to edit the date. The default date is today’s.
Field Type Sequence
Field Type: Choose one of Cultivated or Fallow.
If Cultivated, you progress to the choice of crop types, both mono or mixed cropping.
For mixed cropping, the first crop should be the taller and denser crop.
Next: GPS Work
GeoLocation Save Coordinate
GeoTrace Capture
Single Point Capture Watch out for Accuracy!
Click Play
Select ModeWalk around, check your path
Save your work Add a Comment
Types of Samples
Field Work Requirements
Given the areas we operate in, the preferred option is to collect geo-referenced perimeters.
Crop Type Data
Field Work Requirements
Where large fields are present and well defined, the preferred option may change to collect transect lines or single points. This requires back-office work to derive polygons from the lines or points
Crop Type Data
Captured Office Completion
Field Work Requirements
Where large fields are present and well defined, the preferred option may change to collect transect lines or single points. This requires back-office work to derive polygons from the lines or points
Crop Type Data
Field Work Requirements
Points and lines make for very rapid data capture, particularly in areas with contiguous large fields
Crop Type Data
Drive along track
Collect a Single Point inside each of the 4 fieldsMove to the nextIntersection and repeat
Stop at corner of 4 fields
1 2
34
5
Start
Corner
End
ONA Platform
Captured data uploaded to on-line platform (real time if feasible). Quality control and basic display and summaries)
ONA Platform
Capacity Development
Training Enumerators in Crop Type Data Capture
Capacity Development
What is a sample?
Capacity Development
Good Work: Footpaths do not matter, several fields included in a single sample.
Sample <> Fields
Mixed Cropping!
Capacity Development
Good Work: Footpaths do not matter, several fields included in a single sample.
Field Campaigns
First pilots carried out with dedicated campaigns
Good control of the sampling, free to go everywhere, guaranteed coverage of all strata (irrigated, commercial, subsistence)
Expensive, though reach and implementation easy for WFP
Field Campaigns
Recommended for WFP: Piggyback on Household Surveys
No control of the sampling, focused on settlements, requires special arrangements for coverage of non-subsistence strata (irrigated, commercial)
Cost is diluted in wider data collection exercise
No need for dedicated logistical apparatus
Large number of samples can be captured
Some numbers
South Sudan FSNMS: 2 rounds / year, now 700 clusters visited
Karamoja FSMS: 2 rounds / year, this year 300 cluster visited
We recommend 2 samples per crop type present with a minimum of 6 samples
Ideally 4200 samples for South Sudan, realistically 3000
Ideally 1800 samples for Karamoja, realistically 1300
Field Campaigns
Ask us in early September!
The Future
Field Campaigns
DRONES!
Testing Sentinel-2 vegetation indices for the assessment of the state of winter crops in Bulgaria (TS2AgroBG) - ESA PECS Project
Vlaamse Instelling voor Technologisch Onderzoek NV
(VITO)
Space Research and Technology Institute – Bulgarian Academy of
Sciences (SRTI-BAS)
Institute of Soil Science, Agrotechnologiesand Plant Protection "Nikola Poushkarov“
(ISSAPP )
Project start date: 1 September 2016; Duration: 24 months
Project Officer: Assoc. Prof. Dr. Petar Dimitrov, Technical Officer: Assoc. Prof. Georgi Jelev
The main objectives are:
1. To conduct series of field campaigns in a selected test site and to provide geo-database containing ground measurements of winter wheat biophysical variables;
2. To develop regression models for retrieval of different biophysical variables of winter wheat using Sentinel-2 vegetation indices applicable to winter wheat grown in Bulgaria;
3. To propose a methodology for generating assessment maps of crop condition for winter wheat grown in Bulgaria;
4. To produce crop type map at national level using classification of PROBA-V 100 m time series.
19.07.2018 Sen2Agri webinar
Field Campaigns07 – 11.11.2016 Tillering (Z20)
20 – 24.03.2017 Tillering (Z21-Z26)
24 – 28.04.2017Stem elongation (Z31-Z34)
15 – 19.05.2017 Anthesis (Z65 – Z69)
06 – 10.11.2017 Tillering (Z20-Z22)
02 – 04.04.2018 Tillering (Z29-Z30)
Date of field campaign and corresponding development stage of winter wheat
•Above Ground Biomass [g m-2]
•Leaf Area Index
•Nitrogen content [%]
•Nitrogen uptake [g m-2]
•Canopy Chlorophyll Content [g m-2]
•Fraction Vegetation Cover
•Fraction of Absorbed Photosynthetically
Active Radiation
Field check of north part of Zlatiya -2018
• 4 working days• 28 maps with crop
classification• Application for the
field check –OruxMaps 7.0.2 (Android, Google Play)
• Data post-processing• 678 fields checked in
2018. Defining the winter wheat phenophase – Knezha
Address
SRTI-BASScientific complex 1,Acad. Georgi Bonchev Str., bl. 11113 Sofia, P.O.Box 799, Bulgaria
E-mail: [email protected]: [email protected]
Bulgarian Academy of Sciences (BAS)
Space Research and Technology Institute (SRTI-BAS)
Remote Sensing and GIS Department
Field data collection campaign for
Sen2Agri validation in Mexico
Urs Schulthess
Francelino Rodrigues
Ivan Ortiz-Monasterio
Nicolas Bellemans (UCL)
July 19, 2018
Sen2Agri Webinar
Mexico: wind shield survey
Data Sets: Crop types reported by
farmers
Mexico: Major crop types
Crop (Spanish) # Ave Area (ha) Crop (English) Start End
TRIGO 2666 24.5 Wheat nov-dic 10-abr-20may
MAIZ (Otono) 340 19.7 Maize Fall 15-nov-30nov mayo-15jun
GARBANZO 261 18.1 Chickpea dic abril-may Anytime (90 day crop)
ALFALFA 128 8.9 Alfalfa Perennial
SOYA 119 18.3 Soybean mayo-10jun 15 sep-15oct
FRIJOL 113 18.4 Beans Regular 20-sep-15oct enero/febrero Anytime (90 day crop)
CARTAMO 106 11.5 Safflower dic jun
MAIZ (Prim.) 89 17.6 Maize Spring 15feb-30marzo agosto-sep
MAIZ (Ver.) 49 20.9 Maize Summer junio octubre
NARANJA 38 24.0 Citrus Perennial
TOMATILLO 37 8.5 Tomato nov/dic may Anytime (90 day crop)
PAPA 36 40.5 Potato oct enero-feb
CHILE 35 20.3 Chili nov/dec junio Anytime (90 day crop)
FRIJOL (Prim.) 32 23.2 Bean Spring enero/febrero mayo Anytime (90 day crop)
ESPARRAGO 24 34.2 Asparagus Perennial
NOGAL 19 39.3 Walnut Perennial
TOMATE 18 17.2 Tomato nov-enero junio Anytime (90 day crop)
ZACATE 15 18.0 Forage Perennial
CALABAZA 14 15.3 Pumpkin octubre -enero 30-Apr Anytime (90 day crop)
CEBOLLA 13 9.2 Onion 15 ago- 15 dec 30-Apr Anytime (90 day crop)
BROCOLI 11 22.4 Broccoli octubre-enero 30-Apr Anytime (90 day crop)
LECHUGA 9 31.7 Salad octubre-enero 15-May Anytime (90 day crop)
Perennial
Perennial
Perennial
Perennial
Perennial
Farmer reported crop type by
block and parcelMY_ID Block Lote Area(ha) Crop
5963 1006 1 20.0 ALFALFA
5952 1006 3 12.0 MAIZ (Otoño)
5933 1006 4 13.9 TRIGO
5967 1006 6 20.0 TRIGO
5965 1006 8 7.0 MAIZ (Otoño)
5938 1006 9 7.0 TRIGO
5956 1006 10 7.0 TRIGO
5956 1006 10 1.0 ZACATE
5955 1006 11 1.6 TRIGO
5955 1006 11 2.9 TRIGO
5955 1006 11 21.1 TRIGO
5955 1006 11 4.5 TRIGO
5953 1006 14 1.5 TRIGO
5953 1006 14 21.5 TRIGO
5941 1006 16 7.0 TRIGO
5942 1006 17 10.0 TRIGO
5942 1006 17 10.0 TRIGO
5942 1006 17 7.5 TRIGO
5972 1006 20 2.5 TRIGO
Research questions
1) What is the optimal size of the calibration data set? I.e. how sample size affects the mapping
accuracy? How the relative proportion of classes inside of the calibration dataset affects the map
accuracy?
2) How does the the accuracy change over the season? i.e., how early can we detect wheat (or any
other major crop within wheat season)?
3) Does road side data collection cause an over-fitting? What is its impact on the mapping accuracy?
4) Can we predict 2018 crop types (at least, wheat/no wheat) using a model trained with 2017
dataset?
5) How sensitive is the random forest classifier to errors in the in situ data set (specifically when
grouping in non wheat for all the other crops than wheat)? How does the robustness of RF
compare to other algorithms (incl., deep learning)?
Thank you
for your
interest!
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
Sen2AgriField data collection campaign, 2017
Mali
Souleymane S. TRAORE
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
Mali – Sen2Agri 2016• Country size: 1 241 238 km2
•Population: 17 990 000 (WB, 2016)
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
• The region covers 12% of
Malian geographical territory
and around 51% of country
population (RGPH, 2009)
• Major agricultural production
region in Mali - ~50% National
crop production (CMDT-OHVN, 2013)
• Main crops: Cotton, Sorghum,
Millet, Maize, Rice
Sen2Agri 2017 – area covered
Malian cotton belt150 000km²
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
Survey using JECAM protocol
• Windshield survey using motor
bike - 3 teams
• Survey on 28 villages randomly
selected in the area
• Smart phone app “GeoODK”
for data capturing
• GeoTrace along field boundary
and GeoPoint in the field
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
• 6155 GeoTraces and 2897 GeoPoints collected in 2 weeks field work
• Rice and groundnut represent respectively 32% 15% of GeoPoint
• From the 9052 samples, 6005 field were extracted for the analysis
Samples collected
GeoTraces GeoPoints
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
Preliminary results
Non cropCrop
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
Aw ni ce…..
Thank you….
3rd Sen2Agri User Workshop - Rome 28-29 June 2017
In situ data collection for Sen2Agri system
1. Farmers declaration for their fields
2. Existing surveys for ag. statistics or insurance purpose
3. Piggyback on existing household surveys
4. Specific field campaign for Sen2Agri
5th Sen2-Agri Webinar, 19 July 2018
In situ data collection for Sen2Agri system
In situ data collection tool : - ground survey apps : GeoODK (WFP, Mali), ESRI app, OruxMaps 7.0.2., … from motorbike, car or in the field
- drones (STARS, WFP)- very light aircrafts (South Africa)
IT platform to compile and quality control the in situ data : - ONA Platform (WFP)- Google Earth or Bing combined with in season Sentinel-2 imageryfor quality control (Mali)
5th Sen2-Agri Webinar, 19 July 2018
To capture the entire diversity of the cropland and all the non cropland classes
To cover the full diversity of the crop types of interest and all the minor crop types (it isimportant to define the targeted crop types to emphasize the field data collection on themand to capture all the gradients in crop growing conditions of the crop types of interest)
To integrate the control the quality of the in situ data (labeling error, location error, editingerror, etc. ) as the in data quality is of paramount importance
To ensure the spatial distribution of the in situ data across the whole study area to avoid a local overfitting of the classification model to specific growing conditions
To use spatially independent in situ data for robust accuracy assessment (closer the calibration and validation data set are in space, more the accuracy metrics represent the quality of the classification model rather than the quality of the entire map); Sen2Agri system delivers accuracy metrics for the entire map when the in situ covers the wholeregion of interest
Challenges for in situ data collection
5th Sen2-Agri Webinar, 19 July 2018
In situ data collection
2 different objectives
In situ data for calibration (training): sampling to cover the diversity of situationsexisting in the study site (possibly the national territory) in order to represent the rangeof possible signatures for the different elements of interest (i.e. croplands vs noncroplands on one hand and, the five main crop types and the other frequent crops onthe other hand).
In situ data for validation to estimate the products accuracy (with a confidence interval)using a statistically-sound sampling to be objective and independent; for logisticreasons, sampling not strictly random but 2-stage sampling (with PSU and ESU) toassess the crop types (one field campaign)
Calibration and validation field campaigns for cropland and crop type can be allcombined but the sampling design should be explicitly different to beindependent.
Sometimes, 2 field campaigns are needed for early and end of season maps when all
crop types can not be identified during the mid-season campaign (before some crop emergence).
5th Sen2-Agri Webinar, 19 July 2018
JECAM Guidelines - http://www.jecam.org/
5th Sen2-Agri Webinar, 19 July 2018
1. Stratification according to existing agro-ecological zoning to sample the range of diversity
2. On screen visual interpretation to select samples (min. 1 ha) of land cover types different than cropland
3. Ground survey to delineate crop type samples (min. 1 ha but larger is better)
Calibration – 3 steps
5th Sen2-Agri Webinar, 19 July 2018
Calibration
➢ Stratification according to existing agro-ecological zoning to sample the range of diversity ex. Ukraine: 4 zones
5th Sen2-Agri Webinar, 19 July 2018
1. Stratification according to existing agro-ecological zoning to sample the range of diversity
2. On screen visual interpretation to select samples (min. 1 ha) of land cover types different than cropland on recent aerial photographs, Google Earth or Bing imagery to capture the diversity of the non cropland land cover types. The sample distribution between strata could also consider the stratum size and their respective diversity (~ 15 samples by land cover type by stratum)
ex. Ukraine : 720 samples for non-cropland (15 samples x 12 land cover types x 4 strata)=> 5 days in office
3. Ground survey to delineate crop type samples (min. 1 ha but larger is better)
Calibration – 3 steps
5th Sen2-Agri Webinar, 19 July 2018
Selection of calibration polygons for each of
non cropland class on Google earth
Check the image date (recent!)
5th Sen2-Agri Webinar, 19 July 2018
1. Stratification according to existing agro-ecological zoning to sample the range of diversity
2. On screen visual interpretation to select samples (min. 1 ha) of land cover types different than cropland on recent aerial photographs, Google Earth or Bing imagery to capture the diversity of the non cropland land cover types. The sample distribution between strata could also consider the stratum size and their respective diversity (~ 15 samples by land cover type by stratum)
ex. Ukraine : 720 samples for non-cropland (15 samples x 12 land cover types x 4 strata)=> 5 days in office
3. Ground survey to delineate crop type samples (min. 1 ha but larger is better): for each stratum, 75-100 samples for each main crop and 20-30 samples for each minor crop. No strict sampling design but need to capture each crop diversity. Visual delineation could use the most recent color composite. A this time of the year, summer crops are not yet visible.
ex. Ukraine: 2000 samples for major crops (5 major crop types x 100 samples x 4 strata)840 samples for minor crops (7 minor crop types x 30 samples x 4 strata)
=> 10 days before the mid-season
Calibration – 3 steps
5th Sen2-Agri Webinar, 19 July 2018
National demo in Ukraine
Windshield surveyIn-situ data set – 7689 parcels75% for algorithm calibration25% for products validation
4+3 days, 1 car, 2 persons/car2 weeks/campaings, including days in the office
5th Sen2-Agri Webinar, 19 July 2018
1. Stratification
2. On screen interpretation of non cropland samples on GE imagery
3. Ground survey of 75-100 / 20-30 samples per crop and stratum
CROPLAND MAP
CROP TYPE MAPCalibration set made of non
cropland (on screen) and cropland (ground survey) samplesto train each stratum separatelyand produce the first cropland
mask at the mid-season
Calibration set made of croptypes (ground survey) to
train each stratum separatelyand produce the first crop
type map at the mid-season
=> 1st field campaign data delivery before the mid-season (calibration only)
Calibration – 3 steps for 2 products
5th Sen2-Agri Webinar, 19 July 2018
➢ Stratification according to existing agro-ecological zoning to sample the range of diversity
➢ Two-stage sampling strategy for crop type validation
❖ Delineation of large Primary Sampling Units (PSU) based on ancillary data set (typically admin. regions) ex. : Ukraine = around 30 oblasts (districts); local site = 5 to 15 districts
❖ Random selection of few (2 or 3) PSU distributed in the different strata according to their cropland area (cropland area-weighted sampling probability).
❖ “Windshield survey” for each selected PSU to identify the crop type for each Elementary Sampling Unit (ESU) along the roads (e.g. tablet onboard of vehicle). ESU corresponds to parcels covered by the same crop/crop association. Delineation should use the best contrasted color composite for parcel identification. The road selection and the ESU selection should be as systematic as possible, unbiased with a min. parcel size larger than 0,25 ha (25 S2 pixels). Typical density of 1 ESU / 100 sq km.
National case. Ukraine : 2 oblasts/stratum * 4 strata => 8 oblasts => 1600 crop samples
1 day to reach the selected oblast (PSU), 200 ESUs/day by windshield survey => 16 days campaign
Local case. 1 district/stratum * 3 strata => 3 districts => 600 crop samples200 ESUs/day by windshield survey => 3-4 days campaign
➢ Sampling strategy for non-cropland validation (2 options)
1) Collection of non cropland ESU during the windshield survey2) If up-to-date very high resolution imagery (GE) is available for large parts of the area of interest, on screen
identification of cropland/non-cropland samples of randomly selected in each stratum. Typically 100 – 150 ESUs of 0,25 ha per stratum. => 3 days in the office
Validation
JECAM Guidelines
5th Sen2-Agri Webinar, 19 July 2018
1. Welcome
2. Information about the new version 1.8.2 of the system
3. In situ data collection: experience sharing within Sen2-Agri community
4. Next webinar: 13 September 2018
Webinar outline
Top Related