WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.
-
Upload
beatrix-pearson -
Category
Documents
-
view
213 -
download
0
Transcript of WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.
![Page 1: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/1.jpg)
WQX Web and FIFRA Data
Larry Theller, ABE, Purdue UniversityLeighanne Hahn, OISC
March 22, 2012
![Page 2: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/2.jpg)
WQX and FIFRA Data
• What is WQX• How is it used• Original concept for data upload• Current “state of the art” data loader• Process flow• Data loader upgrades created for this project• Future Tool (2 year NEIEN project)
![Page 3: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/3.jpg)
WQX as Framework
• The Water Quality Exchange (WQX) is a framework that makes it easier for States, Tribes, and others to submit and share water quality monitoring data over the Internet.
• States, Tribes and other organizations can now submit data directly to the publicly-accessible STORET Data Warehouse using the WQX framework.
• WQX will eventually replace the distributed STORET, the STORET Data Entry Module, Reports Module, and STORET Import Module or SIM as the primary means of submitting water quality monitoring data to EPA.
![Page 4: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/4.jpg)
http://www.epa.gov/storet/wqx/products/WQX_factsheet.pdf
CDX
STORET
NODEWEB
![Page 5: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/5.jpg)
Win 2003
Server
Purdue-OISC WQX NODE Network Design
Red Hat Entp
MySQL
ECN 8-Core Red Hat Server
Ubuntu
MySQL
MySQL
Oracle
Authenticated QUERY to CDX
Development (Test) computer
Main Development Server
Interim Production ServerRollover to:
Permanent Production Server
abeip867
abe2k3s2
Templeton.ecn
laptops
Data Loader Test Query Test
Backup of Prod. database
Proposed, Tested, but Not Implemented
![Page 6: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/6.jpg)
Current WQX Web Design
Staging
CDX
Authenticated upload to staging database
Interim Production Server
Spreadsheet of results
Excel
QC done and
Reformat to XML
OISC
Authenticated XML upload to CDX
1
2
![Page 7: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/7.jpg)
WQX_Web_User_Guide_v2.0.pdf
![Page 8: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/8.jpg)
![Page 9: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/9.jpg)
WQX_Web_User_Guide_v2.0.pdf
![Page 10: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/10.jpg)
http://www.epa.gov/storet/wqx/products/WQX_factsheet.pdf
Project Name Monitoring Locations
2011 Activity - Results
2010Activity - Results
2009 Activity - Results
![Page 11: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/11.jpg)
Sample XML for monitoring location upload.
![Page 12: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/12.jpg)
In this typical spreadsheet layout, the measurements are in columns, and each set of samples from a location is a row.
![Page 13: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/13.jpg)
The columns need to pivot so each measurement becomes one row, and the data collection notes need to be replicated for each measurement.
![Page 14: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/14.jpg)
![Page 15: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/15.jpg)
The pink colored tab is used to export data for upload; The yellow tabs to set-up the input data;
The green tabs provide a linked reference for documented elements in WQX.
![Page 16: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/16.jpg)
![Page 17: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/17.jpg)
![Page 18: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/18.jpg)
![Page 19: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/19.jpg)
![Page 20: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/20.jpg)
![Page 21: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/21.jpg)
New Tool under development
• Load and Flow Duration Curve (LDC)– Upload of flow data from USGS– Upload of water quality data from USGS– Upload of water quality data from WQX
Current Grant: WQX data in Action
![Page 22: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/22.jpg)
![Page 23: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/23.jpg)
23
![Page 24: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/24.jpg)
Load and Flow Duration Curves
![Page 25: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/25.jpg)
![Page 26: WQX Web and FIFRA Data Larry Theller, ABE, Purdue University Leighanne Hahn, OISC March 22, 2012.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e575503460f94b4f768/html5/thumbnails/26.jpg)
The End