Regional production Quarterly report on the daily analyses ......Regional production Quarterly...
Transcript of Regional production Quarterly report on the daily analyses ......Regional production Quarterly...
Regional productionQuarterly report on the daily analyses and forecasts activities, and verification of the MOCAGE performances
December 2015 – January 2016 – February
2016
Issued by: METEO-FRANCE
Date: 04/05/2016
REF.:
CAMS50_2015SC1_D50.3.2.MOCAGE-2016Q1_201605
CAMS50_2015SC1_D50.3.4.MOCAGE-2016Q1_201605
CAMS50_2015SC1_D50.5.1.MOCAGE-2016Q1_201605
Copernicus Atmosphere Monitoring Service
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances |
This document has been produced in the context of the Copernicus Atmosphere Monitoring
Service (CAMS). The activities leading to these results have been contracted by the
European Centre for Medium-Range Weather Forecasts, operator of CAMS on behalf of the
European Union (Delegation Agreement signed on 11/11/2014). All information in this
document is provided "as is" and no guarantee or warranty is given that the information is
fit for any particular purpose. The user thereof uses the information at its sole risk and
liability. For the avoidance of all doubts, the European Commission and the European Centre
for Medium-Range Weather Forecasts has no liability in respect of this document, which is
merely representing the authors view.
Copernicus Atmosphere Monitoring Service
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances |
Quarterly report on the dailyanalyses and forecasts activities, and verification ofthe MOCAGE performances
December 2015 – January 2016 – February 2016
METEO-FRANCE (M. Pithon, M. Plu, J. Parmentier,
J. Arteta, S. Guidotti, N. Asencio, N. Assar)
Date: 04/05/2016
REF.:
CAMS50_2015SC1_D50.3.2.MOCAGE.02_201605
CAMS50_2015SC1_D50.3.4.MOCAGE.02_201605
CAMS50_2015SC1_D50.5.1.MOCAGE.02_201605
Copernicus Atmosphere Monitoring Service
Contents:
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances |
1. Executive Summary.........................................................................................42. The MOCAGE model (METEO-FRANCE)................................................................5Product portfolio..................................................................................................5Availability statistics.............................................................................................5
Use for observations for data assimilation...............................................................73. Verification report..........................................................................................10Verification of NRT forecasts................................................................................10Verification of NRT analyses................................................................................15
Copernicus Atmosphere Monitoring Service
1. Executive Summary
The Copernicus Atmosphere Monitoring Service (CAMS, www.copernicus-atmosphere.eu) is establishing the core global and regional atmospheric environmental
service delivered as a component of Europe's Copernicus programme. The regionalforecasting service provides daily 4-days forecasts of the main air quality species andanalyses of the day before, from 7 state-of-the-art atmospheric chemistry models andfrom the median ensemble calculated from the 7 model forecasts. The regional service
also provides posteriori reanalyses using the latest validated observation datasetavailable for assimilation.
This report covers the deliverables related to Near Real Time Production (NRT) for
MOCAGE: D50.3.2.MOCAGE-2016Q1, D50.3.4.MOCAGE-2016Q1, D50.5.1.MOCAGE-2016Q1, for the quarter December 2015 – January 2016 – February 2016. Verificationis done against in-situ surface observations; they are described in the report D50.1.2-2016Q1, that will be delivered shortly. The verification of analyses is done against non
assimilated observations.
During this quarter, the reliability of the MOCAGE production was good, with a 99%availability for the analyses and for the first-day forecasts, and decreasing down to
89% for the fourth-day forecast. The failures were due to the acquisition of surfacemeteorological forcings and to the provision of pollen input data, that have beensecured since then.
During this quarter, the performance of MOCAGE forecasts and analyses was similar asduring past winter, except for ozone, for which a bug had been inducing an increasingbias. This bug is under correction. NO2 scores have improved since past winter. PM10
performances remain poor as long as the formation of secondary aerosols is not taken
into account. Development work is on-going for MOCAGE on this aspect. Theassimilation of ozone and NO2 surface observations improves significantly thecorresponding surface concentration fields. The assimilation of PM10 surface observationis under development.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 4
Copernicus Atmosphere Monitoring Service
2. The MOCAGE model (METEO-FRANCE)
Product portfolio
Name Forecast Analysis
Description Forecast at surface, 50m,
250m, 500m, 1000m, 2000m,
3000m, 5000m above ground
Analysis at the surface
Available for users at 0:30 UTC 10:30 UTC for the day before
Species O3, NO2, CO, SO2, PM2.5, PM10,
NO, NH3, NMVOC, PANs,
Birch pollen at surface during
season
O3, NO2
Time span 0-96h, hourly 0-24h for the day before,
hourly
Availability statistics
The statistics below describe the ratio of days for which the MOCAGE model outputs
were available on time to be included in the ENSEMBLE fields (analyses and forecasts)that are computed at METEO-FRANCE. They are based on the following schedule forthe provision at METEO-FRANCE of:
- forecasts data before: 05:30 UTC for D0-D1 (up to 48h), 07:30 UTC for D2-D3 (from49h to 96h) - analyses data: before 11:00 UTC
These schedules have been set to meet the IT requirements for ENSEMBLE products(no later than 8 UTC for 0-48h, 10 UTC for 49-96h and 12 UTC for analyses).
Indicators:
Availability_MOCAGE_Forecast
Quarterly basis
D0: 99%
D1: 96%
D2: 90%
D3: 89%
Availability _MOCAGE_Analysis
Quarterly basis
D: 99%
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 5
Copernicus Atmosphere Monitoring Service
During this quarter, the following issues have been encountered by the MOCAGE
production system:
Date Problem description
(origin, effects)
Impact
on production
02/12/2015 Analysis not provided
17/12/2015 Step 95 missing (timeout) D3 too late for ENSEMBLE
From 09/01/2016 to
11/01/2016
Problem of acquisition of met.
surface forcing due to HPC
switch
D2-D3 forecasts not provided
on Jan 9th
D1-D2-D3 forecasts not
provided on Jan 11th
From 14/01/2016 to
15/01/2016
Issue regarding met. forcing
acquisition
D2-D3 forecasts not provided
30/01/2016 Issue regarding met. forcing
acquisition
D2-D3 forecasts not provided
From 27/02/2016 to
29/02/2016
Issue related with treatment
of pollen input data
D2-D3 forecasts not provided
on Feb 27th
D1-D2-D3 forecasts not
provided on Feb 28th
D0-D1-D2-D3 not provided on
Feb 29th
Comments:
The issue related to pollen input data non available has been definitively corrected and
production of other species has been secured against potential pollen forecast failure.
Acquisition of surface meteorological data, which was the cause of 3 main failures, willgain a full operational status at the end of Phase 0.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 6
Copernicus Atmosphere Monitoring Service
Use for observations for data assimilation
Day
Use of observation for MOCAGE
December
O3 NO2 NO SO2 CO PM10 PM2.5
1 8,687 9,281
2 8,658 9,509
3 8,990 9,644
4 9168 9,792
5 8,858 9,119
6 8,864 5,161
7 8,779 9,212
8 8,766 9,173
9 9,254 9,694
10 9,222 9,477
11 9,170 3,660
12 9,248 9,882
13 8,593 8,815
14 3,436 9,551
15 9,366 9,559
16 8,601 8,933
17 3,557 3,174
18 8,737 4,429
19 3,893 3,917
20 8,384 9,332
21 3,758 9,621
22 9,054 9,621
23 9,142 9,122
24 8,674 8,922
25 8,487 8,775
26 689 508
27 0 0
28 701 611
29 929 798
30 1,033 870
31 1,006 809
average 6,823 7005
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 7
Copernicus Atmosphere Monitoring Service
Day
Use of observation for MOCAGE
January
O3 NO2 NO SO2 CO PM10 PM2.5
1 1,527 1,292
2 1,843 1,530
3 286 247
4 3,871 4,120
5 6,223 6,149
6 8,658 9,056
7 1,347 996
8 8,019 8,315
9 1,328 1,004
10 0 0
11 2,645 2,187
12 6,519 6,215
13 9,373 10,028
14 9,145 9,887
15 3,581 3,371
16 2,570 2,214
17 2,485 2,148
18 9,687 10,575
19 9,404 10,027
20 9,624 10,357
21 9,599 10,505
22 9,538 10,434
23 9,430 9,877
24 9,204 9,814
25 9,556 10,339
26 4,258 4,342
27 9,263 10,381
28 9,304 10,232
29 9,464 10,467
30 8,943 9,670
31 8,799 8,986
average 6,056 6,343
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 8
Copernicus Atmosphere Monitoring Service
Day
Use of observation for MOCAGE
February
O3 NO2 NO SO2 CO PM10 PM2.5
1 9,321 10,074
2 2,556 2,005
3 9,464 10,377
4 9,224 10,147
5 9,280 10,223
6 9,265 9,968
7 9,136 9,743
8 9,149 9,795
9 9,004 9,726
10 8,594 9,592
11 8,966 9,881
12 8,123 9,052
13 7,934 8,804
14 8,182 9,227
15 8,585 9,582
16 8,645 9,849
17 8,079 9,260
18 4,540 4,778
19 0 0
20 8,289 9,242
21 8,322 9,176
22 8,359 9,460
23 7,182 8,083
24 8,368 9,403
25 8,349 9,493
26 5,989 6,909
27 221 48
28 220 48
29 0 0
average 7,012 7,722
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 9
Copernicus Atmosphere Monitoring Service
3. Verification report
This verification report covers the period December 2015 – January 2016 – February2016. The MOCAGE skill scores are successively presented for four pollutants: ozone,NO2, PM10 and PM2.5. The skill is shown for the entire forecast horizon from 0 to 96h(hourly values), allowing to evaluate the entire diurnal cycle and the evolution of
performance from day 0 to day 3. The forecasts and the analyses cover a largeEuropean domain (25°W-45°E, 30°N-70°N). The statistical scores that are reported arethe root-mean-square error, the modified mean bias and the correlation. The surfaceobservations that are acquired by METEO-FRANCE and used for verification are
described in D50.1.2-2016Q1, that will be delivered shortly.
Verification of NRT forecasts
The following figures present, for each pollutant (ozone, NO2, PM10, PM2.5):
- in the upper-left panel, the root-mean square error of daily maximum (for ozone andNO2) or of daily mean (PM10) for the first-day forecasts with regards to surface
observations, for every quarter since DJF2014/2015, a target reference value isindicated as an orange line,
- in the upper-right panel, the root-mean square error of pollutant concentrationforecasts with regards to surface observations as a function of forecast term,
- in the lower-left panel, the modified mean bias of pollutant concentration forecastswith regards to surface observations as a function of forecast term,
- in the lower-right panel, the correlation of pollutant concentration forecasts withregards to surface observations as a function of forecast term.
The graphics show the performance of MOCAGE (black curves) and of the ENSEMBLE(blue curves).
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 10
Copernicus Atmosphere Monitoring Service
MOCAGE forecasts: ozone skill scores against data from representative sites,period December 2015 - January 2016 - February 2016
The MOCAGE ozone forecasts showed a very high RMSE (around 30 µg.m-3) and a veryhigh positive bias during this winter. Complementary diagnoses showed that the biashas been increasing steadily since summer 2015, and that it is due to a bug with the
implementation of gas deposition with the new surface IFS fields after the IFS changeof cycle in April 2015. This bug does not affect significantly correlation, which remainssimilar as during winter 2014. Correlation decreases with forecast time.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 11
Copernicus Atmosphere Monitoring Service
MOCAGE forecasts: NO2 skill scores against data from representative sites,period December 2015 - January 2016 - February 2016
The MOCAGE NO2 forecast bias oscillates around 0, being slightly positive during night-time and slightly negative during daytime. This is different from the ENSEMBLE bias,which remains negative at every hours of the day. The MOCAGE RMSE oscillates twice
daily and follows rather well the ENSEMBLE, except in late night, when the MOCAGERMSE is much higher than the ENSEMBLE. The correlation of MOCAGE is about 0.1point below the ENSEMBLE and both scores decrease with forecast time. The RMSE ofthe NO2 daily maximum forecasts has improved significantly for MOCAGE since winter
2014 and is now very close to the ENSEMBLE performance. On other scores, theperformance of MOCAGE for NO2 forecasts remains similar compared to winter 2014.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 12
Copernicus Atmosphere Monitoring Service
MOCAGE forecasts: PM10 skill scores against data from representative sites,period December 2015 - January 2016 - February 2016
The MOCAGE PM10 forecast bias is strongly negative. Values are between -0.7 and-0.85, which is similar as during past winter. MOCAGE NMMB is far from the ENSEMBLENMMB (~ -0.1). MOCAGE RMSE oscillates around-17 µg.m-3, which is slightly lower
than for past winter. There is no significant tendency to increase with time. MOCAGERMSE is higher than the ENSEMBLE. MOCAGE correlation for PM10 is still poor (~0.23)and significantly lower than the ENSEMBLE. The absence of secondary aerosolformation process in MOCAGE is the main reason that explains the poor performance of
PM10 MOCAGE forecasts.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 13
Copernicus Atmosphere Monitoring Service
MOCAGE forecasts: PM2.5 skill scores against data from representative sites,period December 2015 - January 2016 - February 2016
The scores for PM2.5 MOCAGE forecasts are consistent with the PM10 forecasts: strongnegative bias, high RMSE compared to the ensemble and poor correlation. The absenceof secondary aerosol formation process in MOCAGE is the main reason that explains
the poor performance of PM10 MOCAGE forecasts.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 14
Copernicus Atmosphere Monitoring Service
Verification of NRT analyses
The following figures present, for each pollutant (ozone, NO2, PM10):
- in the upper-left panel, the root-mean square error of daily maximum (for ozone andNO2) or of daily mean (PM10) for the analyses (solid line) and for the first-day forecasts(dashed line) with regards to surface observations, for every quarter since
DJF2014/2015, a target reference value is indicated as an orange line,
- in the upper-right panel, the root-mean square error of pollutant concentration of theanalyses (solid line) and of the first-day forecasts (dashed line), with regards tosurface observations as a function of forecast term,
- in the lower-left panel, the modified mean bias of pollutant concentration forecasts ofthe analyses (solid line) and of the first-day forecasts (dashed line), with regards tosurface observations as a function of forecast term,
- in the lower-right panel, the correlation of pollutant concentration of the analyses
(solid line) and of the first-day forecasts (dashed line), with regards to surface observations as a function of forecast term.
The graphics show the performances of MOCAGE (black curves) and of the ENSEMBLE(blue curves). The superposition of the analysis scores (solid lines) and of the forecast
scores (dashed lines) computed over the same observation dataset is helpful to assessthe added value of data assimilation.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 15
Copernicus Atmosphere Monitoring Service
MOCAGE analyses: ozone skill scores against data from representative sites,period December 2015 - January 2016 - February 2016
The MOCAGE ozone analyses scores show a positive bias and a RMSE similar to theENSEMBLE forecasts. The correlation of MOCAGE ozone analyses are lower than theENSEMBLE. The assimilation of surface observations improves very significantly the
performance compared to the forecasts, on all scores. The bug impacting gasdeposition that was reported for MOCAGE ozone forecasts may also explain therelatively degraded scores of MOCAGE ozone analyses, in terms of bias and RMSE. Still,the assimilation of surface observations seems to limit the impact of this bug.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 16
Copernicus Atmosphere Monitoring Service
MOCAGE analyses: NO2 skill scores against data from representative sites,period December 2015 - January 2016 - February 2016
The performance of MOCAGE NO2 analyses daily maximum analyses is very close to theENSEMBLE and is significantly better than the MOCAGE forecasts. The daily variation ofMOCAGE RMSE is close to the ENSEMBLE, except during the late night-time. The
MOCAGE correlation is improved after assimilation of surface observations, but itremains about 0.06 below the ENSEMBLE correlation.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 17
Copernicus Atmosphere Monitoring Service
MOCAGE analyses: PM10 skill scores against data from representative sites,period December 2015 - January 2016 - February 2016
Neither MOCAGE nor ENSEMBLE production of PM10 analyses have been implementedyet.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 18
Copernicus Atmosphere Monitoring Service
Analysis of MOCAGE performances for the quarter
The scores of MOCAGE ozone forecasts have shown a strong positive bias and high
RMSE this quarter, due to a bug that in gas deposition. The ozone correlation remains
similar as during past winter, but overall MOCAGE ozone forecasts were degraded. This
bug is related to the implementation of surface deposition after a IFS change in April
2015 and it will be corrected soon. The performance of MOCAGE NO2 daily maximum is
better than past winter and very close to the ENSEMBLE performance. On other scores,
the MOCAGE NO2 forecast performance is similar as past winter. The implementation of
the TNO/MACC-III anthropogenic emission inventory will help to further improve the
MOCAGE NO2 scores.
The scores of MOCAGE PM (10 and 2.5) forecasts show, as for past quarters, a strong
negative bias and poor correlation, with regards to the ENSEMBLE performance. This
poor performance of MOCAGE for PM is due to the lack of representation of secondary
aerosols, which is expected to change very soon with the introduction of secondary
inorganic aerosols into MOCAGE.
The MOCAGE analyses shows overall a clear improvement for ozone and NO2 with
regards to the MOCAGE forecasts. MOCAGE assimilates ozone and NO2 surface
observations. The assimilation of ozone surface observations limits the impact of the
bug that was detected on ozone forecasts. The differences of scores between the
MOCAGE and ENSEMBLE analyses is lower than the differences between the MOCAGE
and ENSEMBLE forecasts. The assimilation of surface observations by MOCAGE leads to
a moderate but constant improvements of the NO2 scores during the day.
Qr. report on daily analyses and forecasts activities, verification of the MOCAGE performances | 19