CALPUFF status update - Cleanairinfo.Com · 2016-11-02 · CALPUFF Regulatory Update • Began...

CALPUFF: Status and Update

Dennis Atkinson*Meteorologist (NOAA)

*in partnership with US EPA, OAQPS

Tyler Fox and Roger BrodeUSEPA/OAQPS

Regional/State/Local Modelers WorkshopMay 16, 2007

Outline

• CALPUFF Modeling System Background– Scorecard of CALPUFF/CALMET versions

• First update of EPA regulatory version of CALPUFF/CALMET

• Initial efforts to update to VISTAS version– Engaging TRC through VISTAS

• Results of update process • Findings and next steps

Scorecard of CALPUFF Versions

- 5.7 April, 2003 (promulgated version)- 5.711a July, 2004 (current regulatory)- 5.711b December, 2005- 5.754 February, 2006 (original VISTAS)- 5.756 August, 2006 (latest VISTAS)- 6.112 April, 2006 (MMS)

April, 2003 July, 2004 Dec., 2005

Feb., 2006

April, 2006 August, 2006

5.7 5.711a 5.711b

5.754

5.7566.112

CALPUFF/CALMET Version Linkages

- 5.7………. 5.53- 5.711a…. 5.53a - 5.711b…. 5.53b- 5.754…… 5.724- 5.756…… 5.726- 6.112…… 6.211

Regulatory Approval ofCALPUFF Modeling System

• Question about regulatory status of CALMET? (by community, model developer)

• EPA approves each component of the CALPUFF modeling system

• Regulatory approval is for both, e.g. current regulatory version is 5.711a/5.53a

CALPUFF Update Process• Incumbent upon EPA to perform an independent

assessment of CALPUFF when updating to new versions

• CALPUFF requires extensive assessment and understanding of changes made; approvals made by EPA, not developer

• CALPUFF Update Tool – Introduced at 8th

Modeling Conference in Sept 2005

CALPUFF Update Tool

• Developed by MACTEC• Compares 2 CALPUFF versions;

proposed (beta) vs. current version (base)• Determines code changes and differences

in results across 10 scenarios• Provides a standardized methodology for

evaluating changes in model codes

AnalysisTool

SourcesMeteorology

Terrain

Differences

?

Resolve Document

BETA

BASE

CALPUFF Update Tool

Description of Scenarios1. Large-scale domain in Pacific NW-NWS data only2. Same as 1 with MM5 NOOBS option3. Medium-scale domain in Pacific NW with

MM5/NWS4. Medium-scale domain near Shenandoah NP5. Small-scale complex flow with deep valley6. Idealized hill with steady-state met – similarity7. Idealized hill with steady-state met – PG8. Flat terrain with steady-state met – similarity9. Flat terrain with steady-state met – PG10. Idealized hill with simulated wind shear

Description of Sources

• Four core sources included in all scenarios:(1) Ground-level area source (20m x 200m)(2) 10m volume source(3) 30m non-buoyant point source(4) 65m buoyant point source

CALPUFF Regulatory Update:Version 5.7 to 5.711a

• Model Change Bulletin-A provided description of changes– Only minor changes in coding

• Application of Update Tool showed minimal differences/impacts on concentrations– Clearly attributed differences to bug fixes

• Approved 5.711a - June 15, 2006

CALPUFF Regulatory Update:Version 5.7 to 5.711a

FEDs Discussions on CALPUFF:Spring 2006

• Reason: EarthTech sells CALPUFF rights to TRC (April 2006) and EPA experiences difficulties in updating to VISTAS version 5.754

• Representatives from Federal Agencies discussed current situation with CALPUFF (status and regulatory updates)– EPA– FLM– MMS

• Discussed the following issues:– lack of adequate documentation– response time from developer slow– developer requires funding to address any issues from Federal

community, despite regulatory classification of the model

EPA contacts TRC:June 30, 2006

• CALPUFF must meet Appendix W requirements for regulatory models, Section 3.1.1(b)-(d):– *.exe to run on a common platform– current model documentation – input/output test data w/model– users can operate model from available

documentation– documentation must include comparison

w/tracer data– model/code cannot be proprietary

TRC to EPA response:July 11, 2006

• TRC provides copy of Use Agreement

• States that … “Permission to modify the Fortran computer codes is granted to users with the provision that modified versions of the model codes must be provided promptly to TRC, along with appropriate written documentation and instructions for use”

TRC to EPA response:July 11, 2006

• Use Agreement also states…

“It is the user's responsibility to determine if the CALPUFF modeling system is appropriate for any particular application, and the user assumes all responsibility and liability for its use.”

CALPUFF Regulatory Update• Began update process with v5.754 in June,

2006

• Engaged with TRC in discussions to understand 5.711a vs. 5.754 differences

• August 15, 2006—Received e-mail from TRC indicating a new VISTAS v5.756 would be released

• OAQPS decided to focus on update for new VISTAS v5.756– newest features, latest corrections, etc. for user

community (including MMS updates for over water)

CALPUFF Regulatory Update: 5.711a -> 5.756

• Found large ‘unresolved’ differences between 5.711a/5.756– CALMET versions (5.53a vs. 5.726)—up to

83% for scenarios 2, 3, (Pacific NW), 5 (deep valley)

– CALPUFF versions (5.711a vs. 5.756)— up to 7% for scenario 5 (deep valley)

• PROBLEM = incomplete documentation– Missing Model Change Bulletins (MCBs)– No in-code comparisons– Inadequate model documentation

While we wait for response . . .

• BART usage– “Good faith” effort to use “best” available

model version– Attempt to be “consistent” with App W and

‘wall off’ any potential bad precedents• OAQPS becomes increasingly concerned

about NSR/PSD permit modeling– Appendix W provides “rules of the game”

Clearing the Air on CALPUFF

• OAQPS makes clear position statement at monthly call with RO modelers (January 2007)

• Region 4 communicates with VISTAS so Pat Brewer contacts OAQPS and facilitates discussion with TRC (Feb 2007)

• Request for documentation from TRC via VISTAS (Feb. 2007)

• Received documentation from TRC on March 15, 2007

CALPUFF Update Process:Initial Assessment of v5.756

• Significant differences in high ranked values between v5.711a and v5.756– Largest differences (up to 83%) attributable to

CALMET (v5.53a vs. v5.726) for Scenarios 2 & 3 (Pacific NW), and 5 (near field/complex flow)

– Smaller differences attributable to CALPUFF (v5.711a vs. v5.756); up to 7% for Scenario 5 (near field/complex flow)

• Much larger differences found across full modeling domain – LRT distances

CALPUFF Update Process:Initial Assessments of v5.756

• Results from initial assessment of VISTAS version -range of percent differences of high ranked values across all sources and averaging periods by scenario:– Scenario 1: -22.0 to +23.3 – Scenario 2: -0.1 to +83.4– Scenario 3: -17.7 to +60.8– Scenario 4: -13.6 to +28.1– Scenario 5: -46.0 to +21.1– Scenario 6: -10.3 to +6.3– Scenario 7: -1.7 to +1.0– Scenario 8: -10.0 to +5.4– Scenario 9: -1.2 to +1.0– Scenario 10: No differences

Maximum Percent Differences by Averaging Period Due to CALMET:Short Term Averages:

Max % Diffs by Aver Period for Scenario 1:ARE1 VOL1 P301 P651 ARE2 VOL2 P302 P652----- ----- ----- ----- ----- ----- ----- -----

1 -0.019 0.107 0.675 15.212 1.641 1.879 -15.804 19.7503 0.011 0.081 -1.132 -10.811 0.218 -1.139 -22.012 -1.44024 -1.757 -0.272 2.609 12.242 0.676 0.800 12.437 11.6598 -0.041 0.075 1.985 9.376 -0.062 0.908 21.914 23.260

Max % Diffs by Aver Period for Scenario 2:ARE1 VOL1 P301 P651 ARE2 VOL2 P302 P652----- ----- ----- ----- ----- ----- ----- -----

1 22.278 54.430 0.153 42.447 -0.084 -0.091 -0.133 7.2953 60.666 51.456 0.137 34.825 -0.047 0.393 -0.070 2.70424 59.770 83.390 3.771 46.302 0.837 0.803 0.185 10.6348 68.987 39.749 1.448 63.646 -0.202 0.732 0.472 15.409

Max % Diffs by Aver Period for Scenario 3:ARE VOL P30 P65 P99----- ----- ----- ----- -----

1 0.008 0.023 2.704 60.830 5.1983 0.177 -0.141 0.175 25.405 17.06324 0.715 0.440 5.879 21.892 -7.8938 -0.025 2.368 4.788 -17.722 11.605

Maximum Percent Differences for High Ranked Values byScenario and Source Due to CALMET v5.53a vs. v5.726

Maximum Percent Differences by Averaging Period Due to CALMET:Short Term Averages:

Max % Diffs by Aver Period for Scenario 4:

ARE VOL P30 P65 BAR----- ----- ----- ----- -----

1G 4.244 6.878 -3.408 7.026 -6.9913G 16.207 9.008 17.049 4.782 -11.00224G 15.403 16.381 16.672 13.379 3.3468G 18.695 16.323 18.239 5.697 -13.6271D -6.718 -8.323 6.937 6.176 3.5853D 17.231 20.079 22.897 8.938 -3.99524D 22.332 23.710 16.899 21.473 -9.3538D 21.152 12.227 19.777 28.060 -3.349

Max % Diffs by Aver Period for Scenario 5:

ARE1 VOL1 P301 P651 ARE2 VOL2 P302 P652----- ----- ----- ----- ----- ----- ----- -----

1 -0.020 -0.188 2.522 -33.268 -19.460 -0.015 16.664 -40.9943 -0.003 -0.069 3.798 -21.227 -5.749 -0.058 11.722 -33.89324 -0.558 2.006 19.485 -27.681 11.601 1.498 11.318 -28.3638 0.005 -0.300 17.110 -31.322 0.133 0.304 21.106 -46.021

Maximum Percent Differences for High Ranked Values byScenario and Source Due to CALMET v5.53a vs. v5.726

Scenario 2 - 65m Point Source, Salem LocationPercent Difference - 1st Highest, 1-Hour Averages

-400 -300 -200 -100 0 100 200 300 400

-400

-300

-200

-100

0

100

200

300

400

710 712 714 716 718 720 722 7245242

5244

5246

5248

5250

5252

Scenario 5, Area SourceWest Side of River Near Rock Island Dam

Percent Differences - 1st Highest, 1-Hour Average

CALPUFF Update Process:Additional Assessments of v5.756

• Under auspices of VISTAS, TRC provided additional documentation regarding model changes– New MCB-C and MCB-D (not released yet)– Tables of in-code documentation annotated to indicate

category of change and whether change may affect results (previous MCB’s only included bug fixes)

– Summary of TRC model tests, claiming that “the VISTAS code is equivalent to the EPA-approved code once the coding errors in the EPA code are corrected.”


• Annotated in-code documentation identified several categories of changes, including:– Bug fixes– Non-optional technical enhancements– Optional technical enhancements– Non-technical enhancements– Enhancement adjustments– Coordinate conversion fixes


• TRC provided the following interim versions, to facilitate isolating impacts due to (1) bug fixes and (2) non-optional technical enhancements:– CALMET v5.53c = CALMET v5.53a + bug fixes– CALMET v5.53c2 = CALMET v5.53c + non-optional

technical enhancements– CALPUFF v5.711c = CALPUFF v5.711a + bug fixes


• Additional potential source of differences identified (indirectly) by TRC, namely “new default parameters for optional technical enhancements”

• User must manually override these new default parameters to maintain consistency with EPA-approved version

• CALMET and CALPUFF output list files do notdocument these new parameters unless the user has specified them in the input file


• New default parameters for optional technical enhancements in CALMET include:– IMIXH = -1 for convective mixing height option based on Maul-

Carson for land cells only and original OCD mixing height overwater(default value of 1 will select Maul-Carson for land and water cells if IMIXH is omitted);

– THRESHL = 0.0 for threshold buoyancy energy flux per meter of boundary layer required for Mixing Growth over land (default value of 0.05 W/m2/m will be selected if THRESHL is omitted);

– THRESHW = 0.0 for threshold buoyancy energy flux per meter of boundary layer required for Mixing Growth over water (default value of 0.05 W/m2/m will be selected if THRESHW is omitted); and

– ICOARE = 0 for option to use delta-T method for the Coupled Ocean Atmosphere Response Experiment (COARE) bulk flux model for computing Zo, u*, L and QH overwater


• TRC later identified the following new default parameter for an optional technical enhancement in CALPUFF:– SVMIN(7:12) = 0.50 for minimum sigma-v value (m/s) for over water

cells for stability classes 1 through 6 (default value of 0.37 m/s will be used for each stability class for over water grid cells if these parameters are omitted)


• EPA conducted several tests in order to compare and isolate affects of the following categories of changes:– Bug fixes– Non-optional technical enhancements– New default parameters for optional technical

enhancements• Test No. 1 designed to test TRC’s assertion that

“the VISTAS code is equivalent to the EPA-approved code once the coding errors in the EPA code are corrected”– CALPUFF v5.711c/CALMET v5.53c vs. current VISTAS

version (CALPUFF v5.756/CALMET v5.726)


• Results from Test No. 1 - range of percent differences of high ranked values across all sources and averaging periods by scenario:– Scenario 1: -22.0 to +24.5 – Scenario 2: -21.7 to +81.8– Scenario 3: -17.9 to +60.3– Scenario 4: -3.3 to +30.5– Scenario 5: -41.8 to +19.5– Scenario 6: -9.7 to +6.3– Scenario 7: -1.7 to +1.0– Scenario 8: -9.3 to +5.6– Scenario 9: -1.2 to +1.0– Scenario 10: No differences


• Conclusion? Range of differences similar to initial assessment so at odds with the assertion that it’s all about the bug fixes.

• Moving on, Test No. 4 was designed to eliminate all documented causes of differences relative to the VISTAS version– 1) bug fixes; 2) non-optional technical enhancements; 3)

new default parameters for optional technical enhancements

– CALPUFF v5.711c/CALMET v5.53c2 vs. current VISTAS version (CALPUFF v5.756/CALMET v5.726), with modified inputs to override new default parameters


• Results from Test No. 4 - range of percent differences of high ranked values across all sources and averaging periods by scenario:– Scenario 1: -17.7 to +0.6– Scenario 2: -20.9 to +10.3– Scenario 3: -0.4 to +5.1– Scenario 4: No differences– Scenario 5: No differences– Scenario 6: No differences– Scenario 7: No differences– Scenario 8: No differences– Scenario 9: No differences– Scenario 10: No differences


• Conclusion? Some progress made (no differences for Scenarios 4 – 10), but significant unresolved differences remain for Scenarios 1 – 3

• While range of differences is smaller than previous tests, much larger differences are found across domain for LRT distances

• The following example shows one source from Scenario 2

Figure 1: Contour Plot of Percent Differences for Scenario 2Jordan Valley 65m Point Source; 4th-Highest 24-Hour Averages

Test 4 - CALPUFF (5.711c) and CALMET (5.53c2) vs. VISTAS (5.756/5.726) (with modified inputs to override new default parameter)

0 100 200 300-200

-100

0

100

200


• Given results from Tests 1 through 4, additional tests were performed to further isolate differences due to each of the three factors contributing to differences

• Results from Tests 5 through 7 are shown in the following table as the range of high ranked values by scenario


Table 1. Range of Maximum Absolute Percent Differences by Scenario

Test 5 – Bug Fixes Test 6 – Non-Optional

Technical Enhancements Test 7 – New Default

Parameters Scenario 1 -1.0 to +22.6 -18.6 to +1.9 -22.0 to +24.4 Scenario 2 -15.4 to +27.8 -30.5 to +0.7 -21.8 to +118.3 Scenario 3 -5.1 to +6.0 -0.8 to +1.7 -19.3 to +60.8 Scenario 4 -18.4 to +8.4 -0.7 to +4.2 -3.2 to +30.5 Scenario 5 -32.4 to +9.5 -9.2 to +0.6 -41.8 to +20.9 Scenario 6 -1.4 to +0.6 -2.0 to +6.3 -12.6 to +5.5Scenario 7 No differences > |0.01| -1.7 to +1.0 0.0 to +0.4 Scenario 8 -1.4 to +0.6 -2.0 to +3.1 -12.1 to +5.5Scenario 9 No differences > |0.01| -1.2 to +1.0 No differences > |0.01|Scenario 10 No differences No differences No differences


• Conclusions? – All three factors cause significant differences– New default parameters appear to cause the largest

differences overall– Differences vary significantly across different scenarios

and source types, with no significant overall bias evident• Reminder: These summaries of differences for

high ranked values are only the tip of the iceberg, and may significantly understate magnitude of differences for LRT portions of domain


• Given the unresolved differences found in Test 4, EPA sent a copy of Test 4 to TRC on April 18, 2007

• TRC identified four (4) additional bug fixes and two (2) additional non-optional technical enhancements in the VISTAS version that were not included in the previous interim versions “c” and “c2”

• EPA received a DVD from TRC on May 11, 2007 containing test results showing no differences across all scenarios once these additional changes were incorporated


• TRC also sent a new version of CALMET, v5.53c2 – Level 070501, containing the six (6) additional changes

• EPA has requested that TRC provide a new version 5.53c of CALMET containing only the four new bug fixes in order to isolate differences due to each category of change

• Once new version 5.53c – Level 07???? of CALMET is received, at a minimum EPA will redo Tests 5 through 7 to document impact of each category


• Beyond all of the tests and multiple versions of CALMET and CALPUFF, EPA has serious technical concerns regarding how the optional technical enhancements for mixing height have been implemented in CALMET

• These concerns are related to the new default parameters THRESHL and THRESHW for the threshold buoyancy energy flux over land and water, respectively

• According to TRC, these new thresholds were implemented as part of the MMS-sponsored enhancements, to prevent unlimited growth of the convective boundary layer over water


• These new thresholds are defined (in the limited documentation available) as the “threshold buoyancy flux required to sustain convective mixing height growth”

• However, the code indicates that when the sensible heat flux falls below the threshold, the convective mixing height is immediately assigned to 0m for that grid cell, eliminating convective turbulence

• A new convective boundary layer may form on subsequent hours


• This behavior is masked somewhat by other defaults within CALMET, including the default minimum mixing height of 50m, the use of the maximum of convective and mechanical mixing heights as the overall mixing height (similar to AERMOD), and the default option for upwind averaging of mixing heights

• The default threshold is 0.05 W/m2/m. For example, if Zic = 3000m at 2pm, and H = 149 W/m2

at 3pm, then Zic = 0m at 3pm. A new CBL will likely form at 4pm.

• The following figures from Scenario 4 illustrate this behavior

Convective Mixing Heights for Scenario 4 – 10/18/1992 Hour 13

Current Options for Regulatory Approval

• Approval of latest VISTAS version “as is”is simply NOT an option at present so . . .

• Step 1: Approve “de-bugged” version 5.711c– Addressed main concern by all which is use

of version with bugs & have understanding of implications

• Step 2: Continue process to approve latest VISTAS version with pre-set regulatory defaults by resolving issues and problems w/ implementation of tech enhancements

How we expect to proceed• Step 1: Approve “debugged” version 5.711c –

Level 07???? – Replicate and document review and apply update tool

to revised versions from TRC• Step 2: Engage with model developer and user

community on appropriateness of technical enhancements– Approval process will include revised MOU restating

responsibilities and expectations of model developer wrt regulatory version and interaction with user community

Lessons Learned

CALPUFF status update - Cleanairinfo.Com · 2016-11-02 · CALPUFF Regulatory Update • Began...

Documents

Transcript of CALPUFF status update - Cleanairinfo.Com · 2016-11-02 · CALPUFF Regulatory Update • Began...