Regional Deterministic Prediction System (RDPS) Technical … · Regional Deterministic Prediction...
Transcript of Regional Deterministic Prediction System (RDPS) Technical … · Regional Deterministic Prediction...
© Environment Canada, 2014
1
Regional Deterministic Prediction System (RDPS)
Technical Note
Last update:
November 18, 2014
from version 3.2.1 to version 4.0.0
© Environment Canada, 2014
2
Version Date Author Variation or revocation
1.0 01/10/2014 J.-F. Caron Creation of document
First version of content for Sections 1 to 6.1
2.0 30/10/2014 D. Anselmo Addition of text covering UMOS, SCRIBE, the
subjective evaluation, and dependent sub-systems.
3.0 17/11/2014 J.-F. Caron Some corrections
4.0 18/11/2014 D. Anselmo Final modifications before publishing.
© Environment Canada, 2014
3
Table of Contents Summary .......................................................................................................................................... 4
Nomenclature ................................................................................................................................... 5
1. Introduction .................................................................................................................................. 6
2. Forecast model configurations and cycling strategies .................................................................. 7
3. Changes to the data assimilation system ...................................................................................... 8
4. Changes to the forecast models .................................................................................................. 11
5. Objective evaluation of the final R&D test series ...................................................................... 11
6. Evaluation of the parallel run ..................................................................................................... 27
7. Performance of dependent systems ............................................................................................ 34
8. Availability of products .............................................................................................................. 36
9. Summary of the results ............................................................................................................... 36
10. Acknowledgements .................................................................................................................. 37
11. References ................................................................................................................................ 37
© Environment Canada, 2014
4
Changes to the Regional Deterministic Prediction System (RDPS)
from version 3.2.1 to version 4.0.0
Research, Development, and Operations Divisions at the Canadian Meteorological Centre,
Environment Canada
Summary
The modifications to the data assimilation component of the regional deterministic prediction
system (RDPS) implemented at Environment Canada operations during the fall of 2014 are
described. The main change is the replacement of the limited-area 4DVar data assimilation
algorithm for the limited-area analysis and the associated 3DVar scheme for the synchronous global
driver analysis by the 4D ensemble-variational (4DEnVar) scheme presented in the companion
TechNote (GDPS-TN) for the global deterministic prediction system (GDPS). Further forecast
improvements were also made possible by upgrades in the assimilated observational data and by
introducing the improved global analysis presented in GDPS-TN in the RDPS intermittent cycling
strategy. The computational savings brought about by the 4DEnVar approach are also discussed.
© Environment Canada, 2014
5
Nomenclature
3DVar: Three-dimensional variational data assimilation
4DVar: Four-dimensional variational data assimilation
3DEnVar: Three-dimensional ensemble-variational data assimilation
4DEnVar: Four-dimensional ensemble-variational data assimilation
AD: Adjoint
CMC: Canadian Meteorological Centre
EC: Environment Canada
ECMWF European Centre for Medium-Range Weather Forecasts
EnKF: Ensemble Kalman Filter
GB-GPS ZTD: Ground-based global positioning system zenith tropospheric delay
GDPS Global Deterministic Prediction System
GEM: Global Environmental Multi-scale model
ETS: Equitable threat score
LAM: Limited area model
LBC: Lateral Boundary Condition
RDPS: Regional Deterministic Prediction System
SCRIBE: EC’s software to assist in the preparation of forecast bulletins
TL: Tangent-linear
UMOS: Updateable Model Output Statistics
© Environment Canada, 2014
6
1. Introduction
In the companion technical note (Buehner et al. 2014; hereafter referred to as GDPS-TN), the latest
modifications to the global deterministic prediction system (GDPS) implemented operationally at
Environment Canada (EC) in the fall of 2014 were described. The most notable change is the
replacement of the four-dimensional variational data assimilation (4DVar) scheme by a four-
dimensional ensemble-variational (4DEnVar) scheme in which the background-error covariances
are represented by a blend of climatological covariances and 4D flow-dependent covariances
derived from a global ensemble Kalman filter (EnKF). In this document we report on the
implementation of the same 4DEnVar scheme in the regional deterministic prediction system
(RDPS). Further forecast improvements were obtained from changes to the assimilated
observational data and the introduction of the improved global analysis presented in GDPS-TN in
the RDPS intermittent cycling strategy.
Similarly to global data assimilation applications, considerable effort has been devoted in recent
years to both comparing and combining ensemble-based and variational-based limited-area data
assimilation approaches (Wang et al. 2008a,b; Zhang and Zhang 2012; Zhang et al. 2013; Liu and
Xiao 2013; Schwartz and Liu 2014; Gustafsson and Bojarova 2014, Pan et al. 2014). As reported by
Zhang and Zhang (2012), using 4DVar in combination with an ensemble-derived covariance matrix
(hybrid 4DVar) can outperform a stand-alone EnKF or a 4DVar in a limited-area forecasting system,
in agreement with the results from Buehner et al. (2010) in a global configuration. However, the
latter approach still requires the use of tangent-linear (TL) and adjoint (AD) versions of the forecast
model, whose time integrations dominate the cost of the analysis step and which requires, moreover,
significant development and maintenance effort. On the other hand, as explained in Buehner et al.
(2013; hereafter B13), 4DEnVar uses 4D ensemble covariances in a way that essentially replaces the
use of TL and AD versions of the forecast model and has been shown to produce forecasts with
similar or improved accuracy at short lead times in a global context by B13 and in GDPS-TN of this
paper. Recently Gustafsson and Bojarova (2014) reported that 4DEnVar can outperform both 4DVar
and hybrid 4DVar in the context of the limited-area HIRLAM forecasting system. Therefore, the
4DEnVar approach described in GDPS-TN seemed appropriate for the RDPS in order to 1) improve,
or at least maintain, the RDPS forecast accuracy obtained using the operational limited-area 4DVar
scheme and 2) make more efficient use of resources at EC by moving towards a more unified data
assimilation approach for deterministic and ensemble forecasting.
Since there is currently no operational equivalent to the global EnKF (Houtekamer et al. 2014) at the
regional scale at EC, we simply based our 4DEnVar scheme for the RDPS limited-area analysis on
the use of 4D ensemble covariances derived from the global EnKF as in the GDPS configuration
described in GDPS-TN. Our approach is thus similar to the National Centers for Environmental
Prediction (NCEP), which recently replaced the 3DVar scheme in the North American Mesoscale
(NAM) and the Rapid Refresh (RAP) regional forecasting systems by a 3DEnVar scheme based on
their global EnKF system (see NWS 2014a,b).
© Environment Canada, 2014
7
2. Forecast model configurations and cycling strategies
The forecast model configurations and the cycling strategy of the RDPS remain the same (except for
one bug correction in the models explained in section 4) in v.4.0.0. Nevertheless they are
summarized again here to facilitate the comprehension of this document.
The RDPS represents the main NWP guidance for forecasters of the Meteorological Service of
Canada for day one and two. It produces forecasts over North America (see again Fig. 1) up to T +
48 h (T + 54 h) at 00 and 12 UTC (06 and 18 UTC) using a limited-area (LAM) version of the GEM
model (Côté et al. 1998) on a cylindrical equidistant grid with a horizontal grid spacing of 0.09o
(approximately 10 km) and 80 vertical levels (model lid at 0.1 hPa, as in the GDPS). Unlike the
GDPS, the RDPS still uses a hydrostatic-pressure coordinate (defined in Charron et al. 2012) and a
discretization on a regular (unstaggered) vertical grid, due to compatibility issues between some
physical process parameterizations used in the RDPS and the log-hydrostatic pressure vertical
coordinate defined on a staggered grid proposed by Girard et al. (2014).
Figure 1. Map showing the domain of the limited-area model in the
RDPS using an approximate 10 km horizontal grid spacing.
This forecasting system employs an intermittent upper air cycling strategy (Fig. 2) where the
(approximately 25 km) analysis from the GDPS serves to initialize the 6 h LAM forecast before the
analysis time T (named LB in Fig. 2). This forecast serves as the background state for the analysis
step at time T (named LF in Fig. 2). In parallel, the same procedure is applied to a global GEM
driving model with a horizontal grid spacing of approximately 33 km. The resulting synchronous
global driving analysis (named DF in Fig. 2) and forecast, as proposed and tested by Fillion et al.
(2010; hereafter F10), allows the observations outside the LAM analysis domain to influence the
LAM forecasts through the lateral boundary conditions (LBCs).
© Environment Canada, 2014
8
Figure 2. Schematic of the RDPS intermittent upper air cycling
strategy showing its dependency on the GDPS. The distances refer to
the approximate horizontal grid spacing of the nonlinear models. See
text in section 2 for further details.
The driving model uses the same vertical levels as the LAM in order to minimize interpolations in
the prescription of the LBCs, whose formulation in the LAM version of GEM is based on Thomas et
al. (1998). In each forecast a digital filter procedure (Fillion et al. 1995) is applied to eliminate the
spurious gravity waves triggered by imbalances in the initial conditions. Interested readers are
referred to CMC (2012) and references therein for further details on the model configuration used in
the RDPS, especially regarding the parameterization of the physical processes.
Both the LAM and the driving component use the Interactions between Soil–Biosphere–Atmosphere
(ISBA) land surface scheme, which has its own data assimilation and cycling strategy (see Bélair et
al. 2003a,b for complete details). In the driver, the surface variables from the GDPS at T – 6 h are
used to initialize the DB forecast (see Fig. 2) and no update of these variables is made in DF at time
T (i.e., DF starts from 6 h forecast surface variables). The LAM component uses a continuous data
assimilation cycling strategy where surface temperatures and soil moisture contents are updated
every 24 h at 00 UTC. LAM forecasts (LB and LF) initialized at 06, 12, and 18 UTC simply rely on
the forecast surface variables from the most recent 00 UTC surface analysis.
3. Changes to the data assimilation system
3.1. From 4D/3DVar to 4DEnVar
The previously operational 4DVar scheme used to perform the LAM analysis is based on the
formulation described in T12 and is a temporal extension of the 3DVar scheme proposed by F10. It
operates with an incremental formulation (Courtier et al. 1994) to produce analysis increments on a
© Environment Canada, 2014
9
global 400 200 Gaussian grid (i.e. with a horizontal grid spacing of approximately 100 km). The
homogeneous and isotropic background-error covariances used at the start of the assimilation
window (T – 3 h) are defined with a global spherical-harmonics spectral representation, but the TL
and AD models are defined over a limited-area domain exceeding (by 30% in each direction) the
horizontal grid of the high-resolution (~10 km) nonlinear model (see Fig. 5 in T12). Because the TL
and AD model grid chosen is an exact sub-domain of the global Gaussian analysis grid, the
communication of information (i.e., analysis increments and adjoint sensitivities) is direct, with no
spatial interpolations required.
This configuration mimics the solution of a global 4DVar (but at a reduced computational cost) in
order to prevent distortions in the analysis increments near the lateral boundaries of the nonlinear
model and to optimize the analysis of the large scales in this continental-scale LAM. The TL and
AD models used here include the same dynamical processes as the high resolution forecast model,
but with only two physical processes: simplified vertical diffusion and simplified grid-scale
condensation. Only one outer loop with 45 iterations is performed, and the resulting global analysis
increments at low resolution (~100 km) are then interpolated to the high resolution LAM grid (~10
km) and added to the background state at T – 3 h. A 3 h nonlinear forecast is then needed to carry
this analysis to time T. See T12 and F10 for further details.
The global driver analysis, on the other hand, is obtained from a 3DVar system since a global 4DVar
system would require too much time to meet EC's operational constraints for RDPS products. The
analysis increments are also produced on a global 400 200 (~100 km) Gaussian grid, but using the
same background-error covariances employed by the global 4DVar in the previously operational
version of the GDPS (see Charron et al. 2012) and with all the observational data available over the
entire globe.
The 4DEnVar scheme tested and adopted in the RDPS is identical to the system implemented in the
GDPS and presented in section 2a of GDPS-TN and in B13. Using an analysis grid that matches the
horizontal grid spacing of the 4D background-error covariances obtained from the global EnKF (~66
km in section 3 and 4; ~50 km in the final configuration presented in section 5), it is possible to
obtain analysis increments at a higher resolution than the limited-area 4DVar scheme presented
above. Because no TL or AD model integrations are necessary in 4DEnVar, the computational cost
of performing a LAM analysis at a higher resolution using the ensemble covariances from the global
EnKF is still lower than the limited-area 4DVar scheme. Moreover, the computational efficiency of
the 4DEnVar makes it possible to use this approach for the global driver analysis. A comparison of
timings and computational resources between the different data assimilation approaches is presented
in section 5d.
As in GDPS-TN, the ensemble-derived background-error covariances are blended with the same
homogeneous and isotropic background-error covariances used in the GDPS for each analysis
© Environment Canada, 2014
10
(LAM and global driver) using the same vertical profile of weighting factors and the same spatial
localization design. Unlike the GDPS, the RDPS based on 4DEnVar still uses a cold start strategy
and a non-incremental digital filter initialization procedure in forecasts initialized at time T, simply
because the incremental analysis update and the physics recycling (warm start) capability now used
in the GDPS (see Section 2g in GDPS-TN) are not available in the older version of the GEM model
used in the LAM and the driver components. Therefore, it was decided to select the 4DEnVar
analysis increment valid at time T obtained after 50 (70) iterations in the LAM (driver) analysis,
which further reduces the computational cost compared to 4DVar, since no forecast step is needed to
carry the analysis from T – 3 h to T.
3.2. Changes to the observational data assimilated
The various improvements to the processing and the volume of data assimilated in the GDPS
described in section 2 of GDPS-TN were also implemented in the RDPS. In short, the most
important changes are: 1) coefficients from a revised satellite radiance bias correction scheme were
implemented1, 2) the processing and the assimilation of radiosonde and aircraft data were upgraded
(this includes introducing a bias correction strategy for aircraft temperature data and taking into
account the horizontal drift of radiosonde balloons as in Laroche and Sarazin, 2013), 3) the number
of assimilated channels for AIRS and IASI radiances was increased and, finally, 4) zenith
tropospheric delay (ZTD) data from ground-based GPS (GB-GPS) receivers (sensitive to
precipitable water) over North America are now assimilated. See Sections 2b to 2f and Tables 2 and
3 in GDPS-TN for complete details.
3.3. Computational resources
The forecast improvements in v.4.0.0 are also associated with a major reduction in the
computational cost of the analysis steps in the RDPS. Table 2 shows the average computational
resources of the analysis steps (LAM and global driver) for v.4.0.0 and v.3.2.1, where the
computational cost is defined in terms of the product between the number of CPUs used and the
elapsed wall-clock time. In the LAM component, the replacement of the 4DVar reduces the
computational cost by an order of magnitude due to significant reductions in both the required
number of CPUs (320 vs 2048) and in the wall clock time (7 min vs. 17 min). This is largely the
result of replacing the integration of the TL and AD models in 4DVar by the use of an ensemble of
nonlinear model states to estimate 4D background-error covariances over the assimilation time
window. However, it is important to stress that 4DEnVar analyses are generated using an updated
version of the variational analysis program, and benefit from improvements in the optimization of
the FORTRAN code, especially from a better usage of the Message Passage Interface (MPI)
protocol. Note that the computational cost of 4DVar does not include the 3 h nonlinear model
forecast step (3 min using 1024 CPUs) to carry the analysis increments from T – 3 h to the analysis
1 No bias correction coefficients are computed in the RDPS. The coefficients are simply taken from the GDPS.
© Environment Canada, 2014
11
time T, which is not needed in 4DEnVar. In the global driver, in order to maintain a wall clock time
similar to that in the 3DVar analysis in v.3.2.1 (i.e., 9 min), the 4DEnVar component in V.4.0.0
demands more CPUs (320 vs. 64) and therefore shows an increased cost (see again Table 2).
Nevertheless, the total cost of the two analyses in V.4.0.0 remains well below the total cost in
v.3.2.1.
TABLE 2. Comparison of averaged computational resources used for the
analysis steps of the RDPS in experiments v.4.0.0 and v.3.2.1 on an IBM Power
7 system. The wall clock time represent the elapsed time between the start of
the variational analysis program and the end of the writing of the analysis
increments. The cost is simply defined as the product of the wall clock time and
the number of cores used. The cost of the nonlinear forecast from T – 3 h to T in
4DVar is not included.
cores Wall clock (min) Cost (core min)
LAM
v.3.2.1 (4DVar) 2048 17 34816
v.4.0.0 (4DEnVar) 320 7 2240
Global driver
v.3.2.1 (3DVar) 64 9 576
v.4.0.0 (4DEnVar) 320 9 2880
4. Changes to the forecast models
As in the GDPS v.4.0.0, a unit conversion error in the formulation of the prognostic equations for
the snow canopy density was corrected in the model version used for the LAM and for the driving
forecasts in v.4.0.0, leading to an approximate doubling of the snow density seen by the models.
This correction in the model formulation had a significant impact on the near-surface temperature in
winter, as will be shown later in Section 5.3.
5. Objective evaluation of the final R&D test series
For each winter (February and March) and summer (July and August) period of 2011, a total of 118
48 h forecasts were initialized at 00 and 12 UTC. In order to measure the full impact of the changes,
© Environment Canada, 2014
12
the surface variable cycling strategy in the LAM (see Section 2.1) was activated in both the new
configuration (v.4.0.0) and in the control experiment (v.3.2.1). The only difference between the final
series and the parallel run performed during the Summer and Fall 2014,with the exception of the
time period, is that the assimilated observations were obtained using a data cutoff time of T + 9h
instead of the data cutoff time used at EC operations: T + 2h.
The results reported below show the impact on forecasts from all the updates in the RDPS. Readers
interested in the impact on forecasts from partial updates (e.g. replacing only 4DVar by 4DEnVar)
are referred to Caron et al. (2014).
5.1. Upper air verification
To be consistent with the assimilation of radiosonde data at their reported or estimated geographical
position during the balloon ascent in v.4.0.0, forecasts from both v.4.0.0 and v.3.2.1 where verified
against radiosonde data by taking into account the horizontal drift. However, as in Laroche and
Sarazin (2013), all the observations used for the verification were assumed to be valid at the
synoptic time (00 or 12 UTC) despite that reported and estimated observation times were taken into
account in the data assimilation step of v.4.0.0. This verification strategy is also used in Section 6.1.
Figure 3 and Figure 4 show the verification for the winter period while Figure 5 and Figure 6
show the results during the summer period. Forecasts from v.4.0.0 better match radiosonde
observations at most vertical levels compared to v.3.0.0 for both winter and summer periods.
Figure 7 and Figure 8 show the fit of the analyses to the observations. It can be seen that the
4DEnVar-based analyses are closer to radiosonde observations than the 4DVar-based analyses, but
this DOES NOT necessarily mean that 4DEnVar-based analysis are closer to the true state of the
atmosphere. Independent observations (observations that were not assimilated in either 4DEnVar or
4DVar) would be needed to estimate the accuracy of the analyses. However, based on the forecast
improvements observed above and below, we can infer that 4DEnVar-based analyses represent an
improved estimate of the true state of the atmosphere compared to 4DVar-based analyses.
© Environment Canada, 2014
13
Figure 3. Final series verification for 24 h forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against
North American radiosondes. The standard deviation (solid lines) and bias (observation minus
forecast; dashed lines) are shown for zonal wind (UU; upper left panels), wind module (UV; upper
right panel), geopotential height (GZ; middle left panel), temperature (TT; middle right panel), and
dew point depression (ES; lower left panel). Scores are averaged over 118 winter cases in 2011.
Boxes on the left (right) of the figures indicate statistical significance levels for biases (standard
deviation). Red (blue) boxes indicate that the v.4.0.0 (v.3.0.0) experiment is better. No box indicates
that the null hypothesis (that statistics of the two samples are the same) cannot be rejected. The
radiosondes’ horizontal drift during balloon ascent was taken into account in order to be consistent
with the changes introduced in the data assimilated in v.4.0.0.
© Environment Canada, 2014
16
Figure 6. As in Fig. 3, but for 118 summer cases in 2011 and for 48 h forecasts.
© Environment Canada, 2014
18
Figure 8. As in Fig. 3, but for 118 summer cases in 2011 and for 00 h forecasts (analyses).
© Environment Canada, 2014
19
5.2. Ground-based GPS
Figure 9 shows the verification of precipitable water forecasts against the values derived from GB-
GPS ZTD observations. An improved fit can be seen at every lead time during the summer period.
These improvements are largely due to the addition of GB-GPS ZTD observations in the data
assimilated in both the RDPS and the GDPS in v.4.0.0 and are in agreement with the findings of
Macpherson et al. (2008). In the winter period, when precipitable water is very low, no change was
observed in the verification scores (not shown).
Figure 9. Final series verification of forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against
precipitable water (mm) derived from GB-GPS ZTD observations over North America as a function
of lead times (h) for the summer period. The standard deviations (biases; forecast minus
observation) are shown as solid (dashed) lines. Scores are averaged over 118 cases and the error bars
represent a 90% confidence interval.
© Environment Canada, 2014
20
5.3. Surface verification
The forecasts were also verified against METAR (aviation routine weather report) and synoptic
observation data available operationally at EC as in Wilson and Vallée (2003) using the USTAT
verification package. Only reports from surface-based stations with an elevation difference with the
LAM topography of less than 100 meters were used, which led to a total of 1282 stations being
considered: 984 in Canada and 210 in the US. Therefore the verification scores presented here are
more representative of the forecast performance over Canada. Statistics were partitioned between
forecasts initialized at 00 UTC and 12 UTC in order to capture the different behavior during daytime
and nighttime regimes.
Figure 10 and Figure 11 show the verification for screen level temperature and dew point,
respectively, for the summer period while Figure 12 and 13 show the scores for the winter period.
While some improvements can be seen in summer, the largest improvements can be seen in winter
due to the bug correction in the prognostic equations of the snow canopy density in the model
formulation.
Precipitation accumulations were also verified using the equitable threat score (ETS; see e.g., Mason
2003) for 5 thresholds (≥0.5, ≥2, ≥5, ≥10, and ≥15 mm) of 12 h accumulations from synoptic
reports. The scores are reported in Figure 14 (for summer) and Figure 15 (for winter). Again, some
improvements in v.4.0.0 compared to v.3.2.1 can be seen at most of the forecast lead times in each
season, although many are barely statistically significant.
© Environment Canada, 2014
21
1.5m T – Summer
00 UTC 12 UTC K
K
Lead Time (hours) Lead Time (hours)
Figure 10. Final series verification for forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against
temperature from surface-based stations over North America as a function of lead times (h) for
forecasts initialized at 00 UTC (12 UTC) in left (right) panel for the summer period. The standard
deviations (biases; forecast minus observation) are shown in the top (bottom) panel and were
obtained from an average over 59 cases. For each comparison, the difference between the two
experiments (red minus blue) is shown in the bottom part of the figure where the grey shaded area
represents a 90% confidence interval.
a) STDE b) STDE
a) Std Dev
c) Bias
b) Std Dev
d) Bias
© Environment Canada, 2014
22
1.5m Td – Summer
00 UTC 12 UTC K
K
Lead Time (hours) Lead Time (hours)
Figure 11. As in Fig. 10, but for dew point temperature.
a) Std Dev
c) Bias
b) Std Dev
d) Bias
© Environment Canada, 2014
23
1.5m T – Winter
00 UTC 12 UTC K
K
Lead Time (hours) Lead Time (hours)
Figure 12. As in Fig. 10, but for the winter period.
a) Std Dev
c) Bias
b) Std Dev
d) Bias
© Environment Canada, 2014
24
1.5m Td – Winter
00 UTC 12 UTC K
K
Lead Time (hours) Lead Time (hours)
Figure 13. As in Fig. 10, but for dew point temperature and the winter period.
a) Std Dev
c) Bias
b) Std Dev
d) Bias
© Environment Canada, 2014
25
00 UTC 12 UTC
ET
S
ET
S
ET
S
ET
S
Figure 14. Final series verification for forecasts from v.4.0.0 (red) and v.3.2.1 (blue) against 12-h
precipitation accumulation measured in terms of the equitable threat score (ETS) as a function of
accumulation thresholds (greater or equal to; mm) for forecasts initialized at 00 UTC (12 UTC) in
left (right) panel for the summer period. Verifications for accumulation from 0h to +12h are shown
in top row; +12h to +24h in second row; +24h to +36h in third row, and +36h to +48 in bottom row.
The scores were obtained from an average over 59 cases. For each comparison, the difference
between the two experiments (red minus blue) is shown in the bottom part of the figure where the
grey shade represents a 90% confidence interval.
a) +00 to +12h
c) +12 to +24h
e) +24 to +36h
g) +36 to +48h
b) +00 to +12h
d) +12 to +24h
f) +24 to +36h
h) +36 to +48h
© Environment Canada, 2014
26
00 UTC 12 UTC
ET
S
ET
S
ET
S
ET
S
Figure 15. As in Fig. 14, but for the winter period.
a) +00 to +12h
c) +12 to +24h
e) +24 to +36h
g) +36 to +48h
b) +00 to +12h
d) +12 to +24h
f) +24 to +36h
h) +36 to +48h
© Environment Canada, 2014
27
6. Evaluation of the parallel run
The parallel run of the RDPS began on 15 July 2014 (although the complete set of sub-systems,
including UMOS and SCRIBE, only started on 24 July 2014) and lasted until November 2014 (more
than 3 months).
6.1. Objective evaluation
The objective verification was computed using 118 cases from the parallel run (as in the final series)
starting on 15 July 2014 00 UTC and ending on 11 September 2014 12 UTC.
Figure 16 and Figure 17 show the comparison with radiosonde observations for 24h and 48h
forecasts, respectively. The improvements are similar to the results from the final series (see Figures
5 and 6), except for the degradation of the geopotential height bias above 500 hPa, especially in the
24h forecasts (note that the same apparent degradation was also observed in the GDPS v.4.0.0; see
GDPS-TN). We remark that temperature measurements from radiosondes (and thus geopotential
height) can be significantly biased (see e.g. Sun et al. 2013) and that no bias correction is currently
applied in our systems. Therefore, it is difficult to draw any conclusion based on radiosonde
observations alone. On the other hand, as shown in the GDPS-TN, a comparison of the GDPS
forecasts with ECMWF analyses (in which radiosonde temperatures are bias corrected) did not
revealed any degradation in geopotential or temperature biases.
Figure 18 show the verification of precipitable water forecasts against the values derived from GB-
GPS ZTD observations. The improvements are similar to the results seen in the final series (see
Figure 7).
Figure 19 and Figure 20 show the verification for screen-level temperature and dew point,
respectively. Improvements can be seen, but not as much as in the final series (see Figures 8 and 9).
Figure 21 shows the precipitation accumulation verification in terms of ETS. Some improvements
can be seen for some thresholds at most lead times.
© Environment Canada, 2014
28
Figure 16. As in Fig. 3, but for 118 cases of the parallel run in summer 2014.
© Environment Canada, 2014
29
Figure 17. As in Fig. 3, but for 118 cases of the parallel run in summer 2014 and for 48 h forecasts.
© Environment Canada, 2014
30
Figure 18. As in Fig. 9, but for 118 cases of the parallel run in summer 2014.
© Environment Canada, 2014
31
1.5m T – Summer
00 UTC 12 UTC
K
K
Lead Time (hours) Lead Time (hours)
Figure 19. As in Fig. 10. but for 118 cases of the parallel run in summer 2014.
a) STDE b) STDE
a) Std Dev
c) Bias
b) Std Dev
d) Bias
© Environment Canada, 2014
32
1.5m Td – Summer
00 UTC 12 UTC
K
K
Lead Time (hours) Lead Time (hours)
Figure 20. As in Fig. 11. but for 118 cases of the parallel run in summer 2014.
a) Std Dev
c) Bias
b) Std Dev
d) Bias
© Environment Canada, 2014
33
00 UTC 12 UTC
ET
S
ET
S
ET
S
ET
S
Figure 21. As in Fig. 14. but for 118 cases of the parallel run in summer 2014.
a) +00 to +12h
c) +12 to +24h
e) +24 to +36h
g) +36 to +48h
b) +00 to +12h
d) +12 to +24h
f) +24 to +36h
h) +36 to +48h
© Environment Canada, 2014
34
6.2. Subjective evaluation
The output of the RDPS parallel run was evaluated by CMC operational meteorologists for most of
the summer and part of the autumn of 2014. Daily evaluations were performed by verifying
forecasts from RDPS v.4.0.0 and RDPS v.3.2.1 against the operational analysis at 00 and 12 UTC.
Forecasts with lead times of 12, 24, 36, and 48-hours were verified for mass fields (geopotential
height at 500 hPa and sea-level pressure) and for quantitative precipitation forecasts (QPF).
In the shorter lead-times, results showed small differences when it came to the verification of the
geopotential height at 500 hPa and sea-level pressure. The differences became more frequent at 36
hours and more significant at 48 hours in favour of RDPS v.4.0.0. At 48-hour lead time, the two
systems were judged to be equivalent nearly two-thirds of the time, either having the correct
solution or not. The remaining third of the time, the new RDPS was preferred over the operational
model by a small margin.
When considering the verification by region, a slight gain was noted for v.4.0.0 over the Pacific and
slightly more over central North America. It is over the Atlantic regions where we saw a more
significant gain with RDPS v.4.0.0. Where the operational RDPS and new RDPS were different, two
thirds of the cases were in favour of the v.4.0.0 solution. The timing and intensity of systems were
better handled over the Atlantic regions, which had a positive impact for QPF as well. The
subjective evaluation also showed slight gains for v.4.0.0 over the Arctic, which were more
pronounced at 500 hPa.
Results for the QPF showed a more pronounced signal in favour of RDPS v.4.0.0 over the Atlantic
region, whereas they were judged to be equivalent over central regions, and somewhat better over
the Pacific region.
In summary, the subjective assessment concluded that the two systems are essentially equivalent in
the first 24 hours, but RDPS v.4.0.0 performs better on day 2, especially in terms of QPF and mass
fields over regions of eastern Canada.
7. Performance of dependent systems
7.1. UMOS
Training of the UMOS system for RDPS v.4.0.0 was carried out using the hindcasts (final series of
forecasts) for the periods from February 1st to March 31
st, 2011 for the winter season and from July
1st to August 30
th, 2011 for the summer season. In addition, training for the summer season was
advanced using data from the parallel run.
Verifications performed in both the summer (July 24th
to September 30th
, 2014) and winter (March
3rd
to March 31st, 2011) seasons show that the performances of the UMOS-RDPS system are
globally maintained although some individual stations show greater or weaker performances. Such
variations are normal, and moreover, individual performances should generally improve with the
addition of more cases in the UMOS statistical database.
© Environment Canada, 2014
35
7.2. SCRIBE
Since the forecasts for days 1 and 2 in the SCRIBE system originate from the RDPS, the results
from the RDPS v.4.0.0 parallel run were verified using the SCRIBE verification system for the
period from August 1st to September 30, 2014. A statistical post-processing (UMOS) is performed
for cloudiness, temperature at 2 m, probability of precipitation, and the direction and speed of the
wind. Other weather elements are generated directly from model output.
Observing stations used by the SCRIBE verification system.
The following weather elements were verified for the 00, 06, 12, and 18Z model runs, for the 136
stations shown in the figure above, and for the 0-48 hour forecast period:
• Precipitation accumulation
• Cloudiness
• Probability of precipitation or PoP
• Maximum (daytime) and minimum (nighttime) temperatures
• Precipitation type
• Wind speed
• Wind direction
The verification results generally indicated a neutral outcome. However, there was a slight
improvement for the RDPS v.4.0.0 forecast accumulation for the 0 mm category (no precipitation),
as well as for the probability of precipitation, especially for the category ≤ 50%. Improvements were
© Environment Canada, 2014
36
also noticed in the prediction of wind speeds from 0 to 19 km/h and, to a lesser extent, from 20 to 39
km/h.
For the predicted precipitation types and wind direction, the results showed a slight degradation
compared to RDPS v.3.2.1. However, these differences are not considered to be significant.
7.3. Other
The impact of changes to the RDPS (from v.3.2.1 to v.4.0.0) on the various sub-systems which
depend on RDPS output to drive their forecasts and/or analyses was in most cases largely neutral.
This was true for the High-Resolution Deterministic Prediction System (HRDPS) – West window,
the Regional Deterministic Wave Prediction System (RDWPS), the Integrated Nowcasting System
(INCS), the Regional Air Quality Deterministic Prediction System (RAQDPS), and the Regional
Deterministic Air Quality Analysis (RDAQA).
For three other systems, in particular, 1) the Regional Deterministic Prediction Analysis (RDPA),
otherwise known as the Canadian Precipitation Analysis (CaPA), 2) the Regional Deterministic
Prediction System – Coupled to the Gulf of St. Lawrence (RDPS-CGSL), and 3) the Regional
Marine Prediction System for the Gulf of St. Lawrence (RMPS-GSL), major upgrades were tested in
parallel runs over the summer of 2014. Based on the results of these runs, new versions of these
systems were approved for implementation into operations at the same time as the implementation
of RDPS v.4.0.0. For further information concerning the changes made to each of these systems, as
well as the performance of the parallel runs, the reader is referred to the appropriate technical note.
The list of changes to operational systems at the CMC is available on the following web page:
http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/changes_e.html
8. Availability of products
With this update, there are no changes to the timing or offering of products.
9. Summary of the results
The updated version 4.0.0 of the RDPS provides improved forecast accuracy relative to the previous
system, version 3.2.1, based on results from two sets of two-month R&D data assimilation and
forecast experiments during 2011 and also the parallel run during the summer and fall of 2014. The
improvements were shown to result from the combined impact of numerous changes, notably: the
replacement of 4DVar by 4DEnVar, an improved treatment of radiosonde and aircraft observations,
an improved radiance bias correction procedure, and the assimilation of new radiance and GB-GPS
observations. Due to the replacement of 4DVar with 4DEnVar, the new system is more
computationally efficient and easier to parallelize, facilitating a doubling of the analysis increment
horizontal resolution, which would have been prohibitively expensive with the 4DVar-based system.
© Environment Canada, 2014
37
The RDPS adopted the same 4DEnVar configuration as the GDPS for the LAM analysis only
because there is currently no operational equivalent to the global EnKF at the regional scale at EC.
However, significant efforts have recently been devoted at EC to developing limited-area EnKF
systems, and a North American continental scale configuration is currently being tested for
operational implementation in the near future. There are also plans to increase the horizontal
resolution of the forecast model in the RDPS at the convection-permitting scale, i.e., with a grid
spacing of a few km. Our future work will thus be devoted to developing a limited-area 4DEnVar
scheme to improve the analyses and forecasts at small scales, where the importance of moist
processes and nonlinearities should give further advantage to the 4DEnVar approach relative to
4DVar.
10. Acknowledgements
The authors thank the large number of people in the research, development, and operations divisions
at the Canadian Meteorological Centre who made this project possible.
11. References
Bélair, S., L.-P. Crevier, J. Mailhot, B. Bilodeau, and Y. Delage, 2003a: Operational implementation
of the ISBA land surface scheme in the Canadian regional weather forecast model. Part I:
Warm season results. J. Hydromet., 4, 352-370.
Bélair, S., R. Brown, J. Mailhot, B. Bilodeau, and L.-P. Crevier, 2003b: Operational implementation
of the ISBA land surface scheme in the Canadian regional weather forecast model. Part
II: Cold season results. J. Hydromet., 4, 371-386.
Buehner, M., P. L. Houtekamer, C. Charette, H. L. Mitchell, and B. He, 2010b: Intercomparison of
variational data assimilation and the ensemble Kalman filter for global deterministic
NWP. Part II: One-month experiments with real observations. Mon. Wea. Rev., 138,
1567–1586.
Buehner, M., J. Morneau, and C Charette, 2013: Four-dimensional ensemble-variational data
assimilation for global deterministic weather prediction, Nonlin. Processes Geophys., 20,
669-682
Canadian Meteorological Center (CMC), 2013: Improvements to the Global Deterministic
Prediction System (from version 2.2.2 to 3.0.0), and related changes to the Regional
Prediction System (from version 3.0.0 to 3.1.0), Technical note of the Canadian
Meteorological Center. Available at:
© Environment Canada, 2014
38
http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opc
hanges/technote_gdps300_20130213_e.pdf
Canadian Meteorological Center (CMC), 2012: Improvements to the Regional Deterministic
Prediction System (RDPS) from version 2.0.0 to 3.0.0. Technical note of the Canadian
Meteorological Center. Available at:
http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/lib/op_systems/doc_opc
hanges/technote_rdps300_20121003_e.pdf
Caron, J.-F., T. Milewski, M. Buehner, L. Fillion, M. Reszka, S. Macpherson and J. St-James, 2014:
Implementation of deterministic weather forecasting systems based on ensemble-
variational data assimilation at Environment Canada. Part II: The regional system.
Submitted to Mon. Wea. Rev.
Charron, M., and Coauthors, 2012: The Stratospheric Extension of the Canadian Global
Deterministic Medium-Range Weather Forecasting System and Its Impact on
Tropospheric Forecasts. Mon. Wea. Rev., 140, 1924–1944.
Côté, J., S. Gravel, A. Méthot, A. Patoine, M. Roch, and A. Staniforth, 1998: The operational CMC-
MRB Global Environmental Multiscale (GEM) model. Part I: Design considerations and
formulation. Mon. Wea. Rev., 126, 1373–1395.
Courtier, P., J.-N. Thépaut, and A. Hollingsworth, 1994: A strategy for operational implementation
of 4D-Var using an incremental approach. Quart. J. Roy. Meteor. Soc., 120, 1367–1387.
Fillion, L., and Coauthors, 2010: The Canadian Regional Data Assimilation and Forecasting system.
Wea. Forecasting, 25, 1645–1669.
Fillion, L., H. L. Mitchell, H. Ritchie, and A. Staniforth, 1995: The impact of a digital filter
finalization technique in a global data assimilation system. Tellus,47A, 304–323.
Girard, C., and Coauthors, 2014: Staggered Vertical Discretization of the Canadian Environmental
Multiscale (GEM) Model Using a Coordinate of the Log-Hydrostatic-Pressure Type.
Mon. Wea. Rev., 142, 1183–1196.
Gustafsson, N. and Bojarova, J., 2014: Four-dimensional ensemble variational (4D-En-Var) data
assimilation for the HIgh Resolution Limited Area Model (HIRLAM), Nonlin. Processes
Geophys., 21, 745-762.
© Environment Canada, 2014
39
Houtekamer, P. L., Xingxiu Deng, Herschel L. Mitchell, Seung-Jong Baek, Normand Gagnon, 2014:
Higher Resolution in an Operational Ensemble Kalman Filter. Mon. Wea. Rev., 142,
1143–1162.
Laroche, S., and R. Sarrazin, 2013: Impact of Radiosonde Balloon Drift on Numerical Weather
Prediction and Verification. Wea. Forecasting, 28, 772–782.
Liu, C., and Q. Xiao, 2013: An ensemble-based four-dimensional variational data assimilation
scheme. Part III: Antarctic applications with the advanced WRF using real data. Mon.
Wea. Rev., 141, 2721–2739.
Macpherson, S. R., G. Deblonde, J. M. Aparicio, and B. Casati, 2008: Impact of NOAA Ground-
Based GPS Observations on the Canadian Regional Analysis and Forecast System. Mon.
Wea. Rev., 136, 2727–2746.
Mason, I. B., 2003: Binary events. Forecast Verification—A Practitioner’s Guide in Atmospheric
Science, I. T. Jolliffe and D. B. Stephenson, Eds., Wiley, 37–76.
National Weather Service (NWS), 2014a: Updates to the Rapid Refresh (RAP) Analysis and
Forecast System, NWS National Technical Implementation Notice 13-38. Available at:
http://www.nws.noaa.gov/os/notification/tin13-38rap_aab.htm
National Weather Service (NWS), 2014b: Changes to the North American Mesoscale (NAM)
Analysis and Forecast System, NWS National Technical Implementation Notice 14-29.
Available at: http://www.nws.noaa.gov/os/notification/tin14-29namcca.htm
Pan, Y., K. Zhu, M. Xue, X. Wang, M. Hu, S. G. Benjamin, S. S. Weygandt, and J. S. Whitaker,
2014: A GSI-Based Coupled EnSRF–En3DVar Hybrid Data Assimilation System for
the Operational Rapid Refresh Model: Tests at a Reduced Resolution. Mon. Wea. Rev.,
142, 3756–3780.
Schwartz, C. S., Z. Liu, 2014: Convection-Permitting Forecasts Initialized with Continuously
Cycling Limited-Area 3DVAR, Ensemble Kalman Filter, and “Hybrid” Variational–
Ensemble Data Assimilation Systems. Mon. Wea. Rev., 142, 716–738.
Sun, B., A. Reale, S. Schroeder, D. J. Seidel, and B. Ballish, 2013: Toward improved corrections for
radiation-induced biases in radiosonde temperature observations, J. Geophys. Res.
Atmos., 118, 4231–4243
© Environment Canada, 2014
40
Tanguay, M., L. Fillion, E. Lapalme, M. Lajoie, 2012: Four-Dimensional Variational Data
Assimilation for the Canadian Regional Deterministic Prediction System. Mon. Wea.
Rev., 140, 1517–1538.
Thomas, S. J., C. Girard, R. Benoit, M. Desgagné, and P. Pellerin, 1998: A new adiabatic kernel for
the MC2 model. Atmos.–Ocean, 36, 241–270.
Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008a: A hybrid ETKF–3DVAR data
assimilation scheme for the WRF model. Part I: Observation system simulation
experiment. Mon. Wea. Rev., 136, 5116–5131.
Wang, X., D. M. Barker, C. Snyder, and T. M. Hamill, 2008b: A hybrid ETKF–3DVAR data
assimilation scheme for the WRF model. Part II: Real observation experiments. Mon.
Wea. Rev., 136, 5132–5147.
Wilson, L. J., and M. Vallée, 2003: The Canadian Updateable Model Output Statistics (UMOS)
System: Validation against Perfect Prog. Wea. Forecasting, 18, 288–302.
Zhang, F., M. Zhang, and J. Poterjoy, 2013: E3DVar: Coupling an Ensemble Kalman Filter with
Three-Dimensional Variational Data Assimilation in a Limited-Area Weather Prediction
Model and Comparison to E4DVar. Mon. Wea. Rev., 141, 900–917.
Zhang, M., and F. Zhang, 2012: E4DVar: Coupling an ensemble Kalman filter with four-
dimensional variational data assimilation in a limited-area weather prediction model.
Mon. Wea. Rev., 140, 587–600.