3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review
description
Transcript of 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review
![Page 1: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/1.jpg)
3SAQS Technical WorkshopMay 29-30, 2013
Data WarehouseUpdate, Assessment, and Work Plan Review
Zac Adelman (UNC-IE)Shawn McClure (CSU-CIRA)Tom Moore (WGA-WRAP)
![Page 2: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/2.jpg)
Summary Status of 3SAQS Technical Work Completed To Date Via
Cooperative Agreements
![Page 3: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/3.jpg)
Identification, Acquisition, Pre- and Post-processing, Extraction
Storage, Backup, Restore, Security, Summarizing, Statistics
Searching, Querying, Filtering, Aggregating, Formatting, Packaging
Charting, Graphing, Mapping, Analyzing
Architecture, Design, Implementation, Management, and Operation
WGANPS CIRA
Acquisition Integration Management Distribution Presentation
Verification, Validation, QA/QC, Mapping, Flagging, Tranformation
Tools
3SDW
Results
Documents
Raw Data
Modelers
Planners
Managers
Guidance, Requirements, Feedback, Funding
Monitored
Modeled
Aerosol
Deposition
Gaseous
Emissions
Met
Air Quality
AQS, VIEWS
WestJump, future modeling, etc
Review of the 3SDW Overall System Ecosystem and Architecture
![Page 4: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/4.jpg)
• Continued to refine the database, software, and website infrastructure supporting the data warehouse
• Refined various pre-processing components• XML Generator for metadata
• SQL scripts for importing modeled emissions data
• Extended portions of the data import system• SQL scripts for importing modeled emissions data
• Integration layer for the Emissions Management Framework (EMF)
• Refined the logical and physical file system design
• Refined the data verification and validation system
• Refined the system for acquiring and managing emissions data
Summary of Past Quarter Activities
![Page 5: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/5.jpg)
• Interviewed several applicants for the TSDW software developer position
• Offered the position to the best candidate and received a verbal acceptance
• Candidate reneged on their acceptance the week before starting the appointment
• Designed an alternate plan for continuing the work plan with minimal impacts via an increased involvement by UNC
• Populated the data warehouse with the full complement of WestJump Base08b CAMx Input/Output data for June/July ‘08
• Refined the modeled emissions review tool
• Conducted the First Use Case Beta Test
Summary of Past Quarter Activities (cont’d)
![Page 6: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/6.jpg)
Operational Website Components
• User Login Form
• User Registration/Modification Form
• User Profile/Account Form
• User Feedback Form
• Dataset Request Form
• Database Query Wizard
• Raw Data Download
• Interactive Charts
• Dynamic Contour Maps
• Site Metadata Reports
• Monitoring Site Metadata Browser
• File Explorer
Authentication and Authorization System
![Page 7: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/7.jpg)
Future Website Components
• Modeled Emissions Summary Tool
• Modeled-to-Observed Data Comparison Tool
• Air Quality Summary Reports
o Visibility
o Deposition
o Ozone
o Other
• Model Data Mapping Tool
• Source Apportionment Tool
• Various Unpublished Monitoring Data Tools
• Backend Web Services and Processing Components
![Page 8: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/8.jpg)
User Login Form
![Page 9: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/9.jpg)
User Registration/Modification Form
![Page 10: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/10.jpg)
User Profile/Account Form
![Page 11: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/11.jpg)
User Feedback Form
![Page 12: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/12.jpg)
Dataset Request Form
![Page 13: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/13.jpg)
![Page 14: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/14.jpg)
Raw Data Download (Query Wizard)
![Page 15: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/15.jpg)
Time Series Charts (Query Wizard)
![Page 16: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/16.jpg)
Dynamic Contour Maps (Query Wizard)
![Page 17: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/17.jpg)
Site Metadata Report (Query Wizard)
![Page 18: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/18.jpg)
Monitoring Site Browser
![Page 19: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/19.jpg)
![Page 20: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/20.jpg)
Modeled Emissions Summary Tool
![Page 21: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/21.jpg)
Modeled-to-Obs Comparison Tool
![Page 22: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/22.jpg)
Air Quality Summary Reports
![Page 23: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/23.jpg)
Model Data Mapping Tool
![Page 24: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/24.jpg)
Future Online Visualization Tools
![Page 25: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/25.jpg)
First TSDW Modeling Use Case Report and Results
![Page 26: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/26.jpg)
• Testers visited the TSDW website and registered with the system to create an account
• Testers visited the Data Request web page and entered their requests for the WestJump Base08b dataset
• Each request was stored in the database
• The system determined whether or not each request could be automatically filled or had to be manually assembled
• The system sent emails to the appropriate TSDW team members to notify them of the data requests
• TSDW team members assembled the dataset requests (copied the relevant data files onto hard drives)
• The datasets (hard drives) were delivered to the beta testers
• The system updated the dataset requests to reflect their “filled” status
First Use Case - Beta Test Steps
![Page 27: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/27.jpg)
• Using the delivered datasets, testers ran the models and generated results
• Testers returned the model output results to the 3SDW
• The test results were assessed by TSDW team members
• Testing outcomes were summarized for the May 3-State AQ Study Technical Workshop
• The TSDW team refines the dataset ordering, download, packaging, and delivery system according to lessons learned
• The TSDW team develops the next Use Case testing scenario(s)
First Use Case - Beta Test Steps (cont’d)
![Page 28: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/28.jpg)
Summer (June – October) 2013 3SAQS Technical Work Review
Data Warehouse Activities
![Page 29: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/29.jpg)
• Implement the collaborative components of the warehouse
• Implement the ongoing news and updates section
• 3SDW on-line for NEPA air quality analysis projects by end of October
• Out-bound data delivery and in-bound data ingestion for NEPA and other air quality studies
• Data warehouse operations (hosting, data analysis and processing, maintenance, et cetera)
• Plans for storage/access/visualization for modeling results and evaluation tools
• Store UT BLM ARMS and other studies’ data in 3SDW after evaluation using protocols
Summary of Coming Activities
![Page 30: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/30.jpg)
Testing and Refinement Help
• All users, collaborators, and partners can help with testing
• Please report bugs – don’t endure them
• Use the website Feedback form
• Send direct email to team members
• Provide as much information as possible up-front
• Stay abreast of ongoing additions and updates
• Be an active part of the design process - make suggestions for features and refinements
• Don’t assume it can’t be done
• Don’t assume it can be done
![Page 31: 3SAQS Technical Workshop May 29-30, 2013 Data Warehouse Update, Assessment, and Work Plan Review](https://reader035.fdocuments.in/reader035/viewer/2022062815/5681690d550346895de02362/html5/thumbnails/31.jpg)
Thanks.