Quarterly report on MOCAGE NRT productions (daily analyses ...€¦ · Copernicus Atmosphere...

Copernicus Atmosphere Monitoring Service

ECMWF COPERNICUS REPORT

Quarterly report on MOCAGE NRT productions (daily analyses and forecasts) and their verification

September – October – November 2019

Issued by: METEO-FRANCE / G. Collin

Date: 31/03/2020

Ref: CAMS50_2018SC2_D5.2-3.1.MOCAGE-2019SON_202003_NRTProduction_Report_v1

This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of CAMS on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view.


CAMS50_2018SC2 – MOCAGE NRT Production report – SON2019 of 30

Contributors

METEO-FRANCE M. Pithon J. Arteta V. Petiot M. Joly G. Collin N. Asencio N. Assar

INERIS A. Colette



Table of Contents 4

1. The MOCAGE model 6

1.1 Product portfolio 61.1 Availability statistics 61.1.1 Indicators 61.1.2 Problems encountered 71.2 Use of observations for data assimilation 71.2.1 Use of observations – September 2019 81.2.2 Use of observations – October 2019 91.2.3 Use of observations – November 2019 10

2. Verification report 11

1.2 Verification of NRT forecasts 112.1.1 MOCAGE forecasts: ozone 122.1.1.1Skillscoresagainstdatafromrepresentativesites 122.1.1.2Detectionofmodeloutlierresultsoverthequarter 132.1.2 MOCAGE forecasts: NO2 142.1.2.1Skillscoresagainstdatafromrepresentativesites 142.1.2.2Detectionofmodeloutlierresultsoverthequarter 152.1.3 MOCAGE forecasts: PM10 162.1.3.1Skillscoresagainstdatafromrepresentativesites 162.1.3.2Detectionofmodeloutlierresultsoverthequarter 172.1.4 MOCAGE forecasts: PM2.5 182.1.4.1Skillscoresagainstdatafromrepresentativesites 182.1.4.2Detectionofmodeloutlierresultsoverthequarter 192.2 Verification of NRT analyses 202.2.1 MOCAGE analyses: ozone 212.2.1.1Skillscoresagainstdatafromrepresentativesites 212.2.1.2Detectionofmodeloutlierresultsoverthequarter 222.2.2 MOCAGE analyses: NO2 232.2.2.1Skillscoresagainstdatafromrepresentativesites 232.2.2.2Detectionofmodeloutlierresultsoverthequarter 242.2.3 MOCAGE analyses: PM10 252.2.3.1Skillscoresagainstdatafromrepresentativesites 252.2.3.2Detectionofmodeloutlierresultsoverthequarter 262.2.4 MOCAGE analyses: PM2.5 272.2.4.1Skillscoresagainstdatafromrepresentativesites 272.2.4.2Detectionofmodeloutlierresultsoverthequarter 282.3 Analysis of the MOCAGE performances over the quarter 29



Executive summary The Copernicus Atmosphere Monitoring Service (CAMS, atmosphere.copernicus.eu/) is establishing the core global and regional atmospheric environmental service delivered as a component of Europe's Copernicus programme. The Regional forecasting service currently provides daily 4-day forecasts of the main air quality species and analyses of the day before, from 9 state-of-the-art atmospheric chemistry models and from the median ENSEMBLE calculated from the 9 model forecasts1. The Regional service also provides posteriori re-analyses using the latest validated observation dataset available for assimilation. This report covers the MOCAGE Near Real Time Production (NRT) for the quarterly period ending August 31st, 2019. Verification is done against in-situ surface observations. Verification of analyses is done against non-assimilated observations. Besides average performances over Europe and the whole period, the diagnostics include an indication of potential outlying days and locations. During this quarter, production reliability was excellent with a score of 100% of dataset provision on time, both for analyses and forecasts. No incident was reported, which confirms the production stability. The validation conducted in the present document shows that: 1. MOCAGE reaches the target for every pollutant (forecasts and analyses). 2. Compared to the ENSEMBLE, the performance of MOCAGE is comparable to the previous

quarters. There is no anomaly in the validation plots. 3. MOCAGE analyses are better than the forecasts for all pollutants, even PM2.5 (whereas only PM10

observations are assimilated). 4. The performance of MOCAGE is significantly better than during the same quarter last year.

1 The original pool of 7 models contributing to the Near Real Time ENSEMBLE was extended to 2 new models on 16/10/2019. Their first Near Real Time production reports will be issued in spring 2020 and will cover the December 2019 to February 2020 production quarter.



1. The MOCAGE model

1.1 Product portfolio Item Forecast Analysis Description Forecast at surface, 50m, 250m,

500m, 1000m, 2000m, 3000m, 5000m above ground

Analysis at the surface, 50m, 250m, 500m, 1000m, 2000m, 3000m, 5000m above ground

Available for users at 6:00 UTC 09:45 UTC for the day before Species O3, NO2, CO, SO2, PM2.5, PM10,

NO, NH3, NMVOC, PANs, dust in the PM10 fraction and SIA in the PM2.5 fraction Birch, olive, grass and ragweed pollen at surface during season

O3, NO2, CO*, SO2*, PM2.5*, PM10, NO*, NH3*, NMVOC*, PANs*, dust** in the PM10 fraction and SIA** in the PM2.5 fraction

Time span 0-96h, hourly 0-24h for the day before, hourly * Non-assimilated species, ** species indirectly assimilated through PM10

1.1 Availability statistics The statistics below describe the ratio of days for which the MOCAGE model outputs were available on time to be included in the ENSEMBLE fields (analyses and forecasts) that are computed at Meteo-France. They are based on the following schedules for the provision at Meteo-France of: • Forecasts data before: 05:30 UTC for D0-D1 (up to 48h), 07:30 UTC for D2-D3 (from 49h to 96h). • Analyses data: before 11:00 UTC. These schedules were set to meet the IT requirements for ENSEMBLE products (no later than 8 UTC for 0-48h, 10 UTC for 49-96h and 12 UTC for analyses).

1.1.1 Indicators Availability_model_Forecast Quarterly basis

D0: 100% D1: 100% D2: 100% D3: 100%

Availability_model_Analysis Quarterly basis

D: 100%



1.1.2 Problems encountered No issue was encountered by the MOCAGE production system this quarter.

1.2 Use of observations for data assimilation Please see the next 3 pages.



1.2.1 Use of observations – September 2019

Day O3 NO2 NO SO2 CO PM10 PM2.5 1 11919 7598 6196 2 11048 7590 6138 3 10525 7777 6064 4 12209 7453 6159 5 11747 7580 6207 6 12494 7688 6385 7 12105 7549 6225 8 11984 7683 6229 9 12023 7711 6216

10 12191 7744 6265 11 12631 7543 6307 12 12879 7661 6239 13 12569 7749 6374 14 12386 7702 6345 15 350 222 173 16 1638 1467 831 17 12509 7753 6344 18 12500 7710 6340 19 12107 7738 6215 20 12404 7861 6408 21 12190 7798 6286 22 11459 7362 5879 23 11946 7712 6272 24 12141 7877 6217 25 12267 7867 6342 26 12216 7785 6217 27 12081 7746 6243 28 11953 7621 6102 29 6085 5588 3225 30 12121 7917 6223



1.2.2 Use of observations – October 2019


10 12436 7549 6160 11 12463 7506 6187 12 11644 7528 6127 13 10275 7709 5971 14 10296 7667 6084 15 10331 7769 6141 16 10675 7630 6157 17 12302 7630 6157 18 12788 7849 6310 19 12578 7877 6222 20 11820 7842 6057 21 11623 7739 5814 22 11398 7646 5903 23 12087 7857 6083 24 12268 7812 6018 25 12342 7639 5912 26 12094 7568 5762 27 12042 7519 5746 28 12169 7747 6094 29 12049 7763 6002 30 12439 7813 6066 31 12302 7746 6011



1.2.3 Use of observations – November 2019


10 11632 8295 5475 11 11963 7823 5948 2525 12 12282 7774 5900 2479 13 499 277 257 109 14 12429 7665 5944 2529 15 12294 7610 5834 2492 16 12346 7566 5831 2444 17 12246 7472 5766 2394 18 12291 7724 5939 2614 19 12077 7759 6057 2621 20 12056 7629 6026 2684 21 12082 7540 5992 2686 22 12352 7587 6033 2637 23 11998 7400 5898 2739 24 12034 7541 5859 2764 25 12004 7499 6084 2712 26 12195 7628 5975 2607 27 12236 7646 5807 2511 28 12457 7510 5863 2500 29 12344 7650 6059 2592 30 12299 7648 6096 2717



2. Verification report The MOCAGE skill scores are successively presented for 4 pollutants: ozone, NO2, PM10 and PM2.5. The skill is shown for the entire forecast horizon from 0 to 96h (hourly values), allowing to evaluate the entire diurnal cycle and the evolution of performance from day 0 to day 3. The forecasts and the analyses cover a large European domain (25°W-45°E, 30°N-72°N). The statistical scores that are reported are the root-mean-square error, the modified mean bias and the correlation. In all cases, we present the median score over all monitoring stations. Using the median is extremely robust and ensures consistency for all scores presented. It however hides any potential outlying day and location. That is why an additional diagnostic is presented: the mean daily RMSE over all stations of the domain. The surface observations that are acquired by Meteo-France and used for verification are described in D50.1.1.2 covering the same period.

1.2 Verification of NRT forecasts The following figures present, for each pollutant (ozone, NO2, PM10 and PM2.5): • In the upper-left panel, the root-mean square error of daily maximum (for ozone and NO2) or of

daily mean (PM10 and PM2.5) for the first-day forecasts with regard to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line.

• In the upper-right panel, the root-mean square error of pollutant concentration forecasts with regard to surface observations as a function of forecast term.

• In the lower-left panel, the modified mean bias of pollutant concentration forecasts with regard to surface observations as a function of forecast term.

• In the lower-right panel, the correlation of pollutant concentration forecasts with regard to surface observations as a function of forecast term.

Another diagnostic allows the detection of potentially outlying days and location, by showing the mean daily RMSE over all stations. The graphics show the performance of MOCAGE (black curves) and of the ENSEMBLE (blue curves).



2.1.1 MOCAGE forecasts: ozone

2.1.1.1 Skill scores against data from representative sites

• The RMSE of daily max is just below the target. • The bias and the RMSE are very high during the night. • The trend of the scores (as a function of forecast day) is similar to that of the ENSEMBLE.



2.1.1.2 Detection of model outlier results over the quarter

MOCAGE ozone RMSE are higher than the ENSEMBLE all along the quarter.



2.1.2 MOCAGE forecasts: NO2


• The RMSE of daily max is below the target. • The diurnal cycle of the scores is quite coherent with the ENSEMBLE. • The trend of the scores (as a function of forecast day) is similar to the ENSEMBLE.




At the beginning of the period, RMSE of daily max are very close to the ENSEMBLE. At the end, RMSE get stronger.



2.1.3 MOCAGE forecasts: PM10


• The RMSE of daily max is below the target. • PM10 are characterized by a strong negative bias in MOCAGE. • The diurnal cycle of the scores is coherent with the ENSEMBLE, but strongly shifted, with much

higher RMSE, and much weaker correlations. • The trend of the scores as a function of forecast day is similar to the ENSEMBLE.




RMSE of daily means are stronger than the ENSEMBLE, but the daily evolution is quite similar.



2.1.4 MOCAGE forecasts: PM2.5


• The RMSE of daily max is below the target. • The diurnal cycle of the RMSE is coherent with the ENSEMBLE, but with higher values.

Correlations are significantly lower. • The trend of the scores as a function of forecast day is similar to the ENSEMBLE.




The daily evolution of RMSE of daily means is very close to the ENSEMBLE



2.2 Verification of NRT analyses The following figures present, for each pollutant (ozone, NO2, PM10 and PM2.5): • In the upper-left panel, the root-mean square error of daily maximum (for ozone and NO2) or of

daily mean (PM10) for the analyses (solid line) and for the first-day forecasts (dashed line) with regard to surface observations, for every quarter since DJF2014/2015, a target reference value is indicated as an orange line.

• In the upper-right panel, the root-mean square error of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term.

• In the lower-left panel, the modified mean bias of pollutant concentration forecasts of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term.

• In the lower-right panel, the correlation of pollutant concentration of the analyses (solid line) and of the first-day forecasts (dashed line), with regard to surface observations as a function of forecast term.

Another diagnostic allows the detection of potentially outlying days and location, by showing the mean daily RMSE over all stations. The graphics show the performances of MOCAGE (black curves) and of the ENSEMBLE (blue curves). The superposition of the analysis scores (solid lines) and of the forecast scores (dashed lines) computed over the same observation dataset is helpful to assess the added value of data assimilation.



2.2.1 MOCAGE analyses: ozone


• Ozone in situ observations are assimilated in MOCAGE. • The RMSE of daily max is below the target, close to the ENSEMBLE. • The diurnal cycle of the scores is coherent with the ENSEMBLE. MMB values are in the same

range. For RMSE and correlations, the difference between MOCAGE and the ENSEMBLE is rather constant over the day.

• MOCAGE ozone analyses are much better than the forecasts.




The daily evolution of RMSE is very close to the ENSEMBLE.



2.2.2 MOCAGE analyses: NO2


• NO2 in situ observations are assimilated in MOCAGE. • The RMSE of daily max is below the target. • The diurnal cycle of the scores is coherent with the ENSEMBLE. • MOCAGE NO2 analyses are significantly better than the forecasts.




At the end of the period, the daily evolution of RMSE presents 3 peaks, despite the assimilation of in situ observations.



2.2.3 MOCAGE analyses: PM10


• PM10 in-situ observations are assimilated in MOCAGE. • The RMSE of daily means is below the target. • The diurnal cycle of the scores is similar to the ENSEMBLE, but slightly shifted. • MOCAGE PM10 analyses are significantly better than the forecasts.




After assimilation of in situ observations, MOCAGE RMSE get close to the ENSEMBLE.



2.2.4 MOCAGE analyses: PM2.5


• PM2.5 in-situ observations are not assimilated in MOCAGE. • The RMSE of daily means is below the target. • PM2.5 analyses are better than the forecasts, thanks to the assimilation of PM10 observations. • The main drawback is a positive PM2.5 bias in MOCAGE analyses, which may be corrected

when PM10 and PM2.5 will be jointly assimilated in the system.




The daily evolution of RMSE is consistent with the ENSEMBLE, but values are significantly greater at the beginning of the period.



2.3 Analysis of the MOCAGE performances over the quarter There is no relevant anomaly to mention for the SON2019 quarter. MOCAGE reaches the target for every pollutant (forecasts and analyses). Performances are significantly better than during SON2018. During the year 2020, the operational forecasts based on MOCAGE will undergo an important update. Performances are expected to increase for all monitored pollutants, as shown by the validation simulations currently conducted at Meteo-France CNRM.


atmosphere.copernicus.eu copernicus.eu ecmwf.int

ECMWF - Shinfield Park, Reading RG2 9AX, UK Contact: [email protected]

Quarterly report on MOCAGE NRT productions (daily analyses ...€¦ · Copernicus Atmosphere...

Documents

Transcript of Quarterly report on MOCAGE NRT productions (daily analyses ...€¦ · Copernicus Atmosphere...