Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in...
-
Upload
silas-chapman -
Category
Documents
-
view
215 -
download
3
Transcript of Annual Meeting, June 15-16 2015, Istanbul, Turkey WP2: Data aggregation and products generation in...
Annual Meeting, June 15-16 2015, Istanbul, Turkey
WP2: Data aggregation and products generation
in the Atlantic Sea: results and criticalities
EMODnet Chemistry
With IFREMER, IEO, IHPT and RIHMI‐WDC contributions
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Outline
• Data inventory of the second harvesting
• Data aggregation / validation phase : issues and solutions
• Regional data buffer : space and time distribution by parameter
• DIVA maps production
3/15
Data collected by MARIS robot (28/10/2014) :
Data inventory (1)
CDI type Vertical profiles
Time Series
Total
New CDIS 13613 99 13712
Updated CDIs 2601 27 2628
Old CDIs 7040 231 7271
Total 23254 357 23611
Annual Meeting, June 15-16 2015, Istanbul, Turkey
4/15
Data collected by MARIS robot (28/10/2014) :
Data inventory (1)
CDI type Vertical profiles
Time Series
Total
New CDIS 13613 99 13712
Updated CDIs 2601 27 2628
Old CDIs 7040 231 7271
Total 23254 357 23611
More than twice the first harvesting !
Annual Meeting, June 15-16 2015, Istanbul, Turkey
5/15
First harvesting
Data inventory (2)
Second harvesting
Old Updated
New
Enlargeddata coverage
Annual Meeting, June 15-16 2015, Istanbul, Turkey
6/15
Distribution by partners
Annual Meeting, June 15-16 2015, Istanbul, Turkey
CDI type
IEO Ifremer IHPTRIHMI-NODC
IMR UniHB
New CDIs
2396 40 0 1670 3 9
Updated CDIs
5242 0 3677 146 0 0
Old CDIs
7764 6465 876 2676 33 0
Total 15402 6505 4553 4492 36 9
Data inventory (3)
7/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
CDI type IEO Ifremer IHPTRIHMI-NODC
IMR UniHB
New CDIs 2396 40 0 1670 3 9
Updated CDIs 5242 0 3677 146 0 0
Old CDIs 7764 6465 876 2676 33 0
Total 15402 6505 4553 4492 36 9
Data inventory (3)
Distribution by partners
8/15
2nd harvestingCDI type
Phos Amon Silicate Nox
New CDIs 2230 250 762 835
Updated CDIs 2551 1835 2241 2416
Old CDIs 6538 2916 4107 4221
Total 11319 5001 7110 7472
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Distribution by parameters
Data inventory (4)
9/15
• Use only data with Quality Flag = 0, 1, 2, 6 as initial collection to use only good data
• Unit conversion and aggregation (P35 vocabs): mol/l
• Deletion of empty lines in a station : some levels with no measurements for the parameter
• Deletion of lines with negative depth (incorrect missing value)
• Perform QC with ODV
Search for out of “broad range” values with Flag = 1
Set a QC for all measurements with Flag = 0
Set default values with Flag = 0, 1 to Flag = 9
Deal with Flag 6 and zero values
• Write a report on QC and send results to MARIS and data providers
Quality Checks - common rules
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
10/15
Quality checks – problems faced
• Problem of unit conversion : ug/l to umol/l e.g. reload russian data and some spanish data
• Problem in CDI updating e.g.high spanish Phosphate to delete
• Some of the reported errors to data centres have not been corrected : we cannot just import the new collection in the
collection built up after the first harvesting. We have to re-make the whole QC work.
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
11/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
5 Aggregated Data Collection Qced produced and sent to MARIS :- Phosphate : Global_PHOS_collection.txt- Ammonium : Global_AMON_collection.txt- Nox : Global_NOX_collection.txt- Silicate : Global_SLCA_collection.txt- Timeseries of these 4 parameters : Global_nutrients_TS_CDIs.txt
12/15
Perform QC with ODV Search for out of range values with Flag = 1 Set a QC for all measurements with Flag = 0 Set default values with Flag = 0, 1 to Flag = 9 Deal with Flag 6 and zero values
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
Annual Meeting, June 15-16 2015, Istanbul, Turkey
13/15
Flag value
PHOS Before
QC
PHOS after QC
0 43 0
1 29994 29417
2 19 19
3 40 114
4 4 5
9 0 2
total 30100 29557
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data aggregation and QC
Perform QC on New Phosphate with ODV Search for out of range values with Flag = 1 Set a QC for all measurements with Flag = 0 Set default values with Flag = 0, 1 to Flag = 9 Deal with Flag 6 and zero values
14/15
Regional data buffer
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Phosphate data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
15/15
Nox data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Regional data buffer
16/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Silicate data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
Regional data buffer
17/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Regional data buffer
Ammonium data
a) Station map
d) Data bins e) X/Y distribution f) Season histogram
b) Vertical plot c) Time histogram
18/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
DIVA maps settings :
settings Atlantic area
DIVA version 4.6.9 + weighting patch
Time period 10-years running periods with yearly step for the 4 seasons
Time range 1962-2014
Seasons winter (0103), spring (0406), summer (0709), autumn (1012)
Vertical layers IODE standards (33 levels)
parameters PO4, SiO4, Nox
19/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
Problems faced :•Some difficulties in installing DIVA on a Window PC via cygwin : inconsistencies between netcdf and gfortran librairies
•Handle the different steps to produce an analyse
•Optimizing the parameter estimation: Do not run on the 46 periods – 4 seasons
chose a decade with high coverage Some SN profiles very noisy
test binning and weighting options
•Very long time-consuming runs reduce the space and time resolution
•Pb of generation of the big 4D netcdf file from the 3D files
DIVA maps production
20/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Phosphate at surface (0m) – summer (0709)
DIVA – 2005/2014 WOA – 1955/2012
21/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Phosphate at surface (0m) – summer (0709)
DIVA 2005/2014 Data bins
22/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Silicate at surface (0m) – summer (0709)
DIVA 2002/2011 WOA – 1955/2012
23/15Annual Meeting, June 15-16 2015, Istanbul, Turkey
DIVA maps production
Silicate at surface (0m) – summer (0709)
DIVA 2002/2011 Data bins
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
Data extraction (and weighting)
Parameters optimization
Run analysis Internal expert validation of products
Phosphate
Silicate
Nox
DIVA maps production
4D Netcdf files ready
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
Data extraction (and weighting)
Parameters optimization
Run analysis Internal expert validation of products
Phosphate
Silicate
Nox
If needed
DIVA maps production
4D Netcdf files ready
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
Data extraction (and weighting)
Parameters optimization
Run analysis Internal expert validation of products
Phosphate
Silicate
Nox
If needed
DIVA maps production
Work with other parameters : oxygen, chlorophylle, …
4D Netcdf files ready
Thanks!
Annual Meeting, June 15-16 2015, Istanbul, Turkey