1

Build EPA’s Bay Barometer in the Cloud

Brand NiemannUS EPA

July 15, 2010http://semanticommunity.net

Disclaimer: These slides do not reflect the views of the U.S. Environmental Protection Agencyand does not constitute endorsement by the EPA of the standards or products mentioned.

2

Overview

• The Challenge• The Data.gov Program• The Expert and His Advice• The Cloud Tools• The Inspiration• The Data Sources• Other Sources of Data• The Process• The Results• Comments• Acknowledgements• References

3

The Challenge

• The Chesapeake Bay Program and its partners have developed the Chesapeake Action Plan (CAP) to strengthen and expand partnerships in the watershed, enhance coordination of restoration activities, and increase the collective accountability for protecting the Chesapeake Bay. The CAP will become a valuable asset in efforts to protect the Chesapeake Bay and its watershed because it is a collaborative product of many Bay Program partners, contains tools to advance restoration work and is a dynamic system that will be adapted and expanded in response to new information and analysis.

http://chesapeakeactionplan.wik.is/#The_Challenge

4

The Data.gov Program

• Data.gov: Pretty Advanced for a One-Year-Old, Vivek Kundra, White House Blog, May 21, 2010: As we look to the next year, we recognize that the Web itself is evolving into a data platform and how important it is to link data from one agency to another or one country to another.  True value lies at the intersection of multiple datasets and what we are witnessing is a continued movement across the world to democratize data, but more importantly the explosion of applications created by the emergence of a community of innovators. So all you innovators out there – what data sets can we try to get out there to help you go further? Tweet your ideas for data we should try to put out with hashtag #datagov, and we’ll see what we can do in year 2.

http://chesapeakeactionplan.wik.is/#The_Challenge

5

The Data.gov Program• XML: Better suited for consumption by automated programs capable of handling raw

XML files.• CSV/Text: Use this format for easy access to the data. CSV/Text files can be opened

by most desktop spreadsheet applications (e.g. MS Excel).• Excel: The spreadsheet format used by MS Excel which holds data in charts,

worksheets, and macros.• KML/KMZ: Used to display geospatial data in Google Earth, Google Maps, and

similar applications.• Shapefile: Used for consumption by shapefile-compatible mapping applications. Most

datasets in shapefile format are updated on a monthly or quarterly basis as they are not “operational” in nature.

• Maps: Not defined.• RDF: Resource Description Framework (RDF) is the standard model for data

interchange on the Web.• PDF: Portable Document Format (PDF) is a file format used to display documents in

a manner that is independent of the original application software, hardware, and operating system used to create those documents. A PDF file can describe documents containing any combination of text, graphics and images in a device independent and resolution independent format. PDF is an open standard and anyone may write applications that can read and write PDFs royalty-free.

• Source: http://www.data.gov/catalog/raw

http://chesapeakeactionplan.wik.is/#The_Data.gov_Program

6

The Expert and His Advice

• Edward Tufte Presidential appointment announced by White House, March 5, 2010.

• Tufte Comment on iPhone interface design: Better to have users looking over material adjacent in space within our eyespan rather than stacked in time. This is especially the case for statistical data, where the fundamental analytical task is to make comparisons. Also see page 159 in the book reference below.– Edward R. Tufte, Beautiful Evidence (2006), 

Graphics Press LLC.

http://chesapeakeactionplan.wik.is/#The_Expert_and_His_Advice

7

The Cloud Tools

http://cloud.mindtouch.com/

8

The Cloud Tools

http://chesapeakeactionplan.wik.is/

9

The Cloud Tools

http://spotfire.tibco.com/

10

The Cloud Tools

http://ondemand.spotfire.com/public/Help/index.htm

11

The Inspiration

H1N1 Spread Courtesy of TIBCO Spotfire. See Web Player.http://chesapeakeactionplan.wik.is/#The_Inspiration

12

The Data Sources

http://chesapeakeactionplan.wik.is/Bay_Barometer

13

The Data Sources

http://chesapeakeactionplan.wik.is/Special:Sitemap

14

The Data Sources

http://chesapeakeactionplan.wik.is/Bay_Barometer/Bay_Health

Metadata: http://chesapeakeactionplan.wik.is/@api/deki/files/108/=bayhealthoverarchingindex2009.doc

15

The Data Sources

Data: http://chesapeakeactionplan.wik.is/@api/deki/files/109/=bayhealthoverarchingindex2009.xls

16

Other Sources of Data

http://chesapeakeactionplan.wik.is/@api/deki/files/167/=BayWatershedClimate.xls

17

The Process

• The Basic Steps:– Inventory Data Sources and Plan Application– Prepare and Import Data and Metadata– Implement Layout and Analytics– Add Bookmarks and Create Data Stories– Publish and Test in Web Player– Get Feedback and Improve

• First create visualizations, faceted search (filters), and analytics for each individual data source and then look for relationships between the data sources.

http://chesapeakeactionplan.wik.is/#The_Process

18

The Results• This is the thirteenth part of Put Your Desktop in the Cloud to

Support the Open Government Directive and Data.gov/semantic, April 19, 2010.

• The author followed Edward Tufte's advice on interface design: "Better to have users looking over material adjacent in space within our eyespan rather than stacked in time. This is especially the case for statistical data, where the fundamental analytical task is to make comparisons“.

• The author followed a six step process and example results are shown in the slides and many more are possible with the open, interactive, and creative environment offered by Mindtouch and Spotfire.

• Bay Barometer databases were organized in 75 tabs, each with 2-5 adjacent panels, in Spotfire Analytics: Navigation and Metadata, Data Dictionary, Database, and Interactive Graphics.

http://chesapeakeactionplan.wik.is/#The_Results

19

The Results

http://chesapeakeactionplan.wik.is/#The_Results

20

The Results

http://chesapeakeactionplan.wik.is/#The_Results

21

The Results

http://chesapeakeactionplan.wik.is/#The_Results

22

The Results

http://chesapeakeactionplan.wik.is/#The_Results

23

Comments

• More analytics for Integrated Analyses and Mapping are in process.

• Please use the Add Comment feature at the bottom of this wiki page to provide feedback and suggest additional analyses you would like to see. To use the Add Comment feature you first need to register by providing your email address. Your privacy will be respected and your email addressed will not be available to others or used for any other purpose. You can also download the Spotfire File from this Wiki and a 30-day free evaluation copy from http://spotfire.tibco.com/ and reuse these analyses, add your own data to this file or new Spotfire files that you create. Have fun and give us your feedback!

http://chesapeakeactionplan.wik.is/#Comments

24

Acknowledgements

• The author acknowledges gratefully Dean Allemang, Cory Casanave, Sean Connors, Mills Davis, Li Ding, David Eng, Lee Feigenbaum, Aaron Fulkerson, Jim Hendler, Ralph Hodgson, Kevin Kirby, Kevin Jackson, Bob Marcus, John McMahon, Richard Murphy, Brand Niemann, Jr., Barry Nussbaum, Matthew Phoenix, Tony Shaw, Jeff Stein, George Strawn, George Thomas, Pete Tseronis, and Edward Tufte.

http://chesapeakeactionplan.wik.is/#Acknowledgements

25

References• A Gov 2.0 Platform for Open Government with Data Science (see

Data Science Library in the Cloud, JumpStart, and Public): – First there was Put Your Desktop in the Cloud to Support the Open Government

Directive and Data.gov/semantic, Semantic Universe, April 19, 2010. – Second was Build Your Own Data.gov (Spotfire) and EPA Microsite (Spotfire)

with Semantics and Statistics in the Cloud, Slides, May 15, 2010. – Third was Build Your Community Health Information "Design for America" Using

Mindtouch and Spotfire, Slides, May 17, 2010. See Spread of H1N1 in Spotfire. – Fourth was Build EPA's CASTNET in less that two hours, see Mindtouch and

Spotfire, Slides, May 21, 2010. Completed in 8 more hours by May 30, 2010. – Fifth was Build Your Own Data.gov/semantic with Mindtouch and Spotfire in the

Cloud: The White House Visitor Database, Slides, May 22, 2010. See Data.gov takes the 'Mumsy' test, FCW, May 26, 2010. See Mindtouch and Spotfire, Updated July 7, 2010 (see below). Compare to Findthebest's WH Visitor Logs Comparison Tool.

– Sixth was Build EPA's EPA's Facility Registry System (FRS) and Locational Reference Database with Mindtouch and Spotfire in the Cloud: Virginia, No Slides, June 1, 2010.

– Seventh was Build the UK’s COINS in the Data Science Library Cloud (Mindtouch and Spotfire), Slides, June 9, 2010.

– Eight was Build EPA's Envirofacts in the Cloud: Virginia FRS, NPL, and TRI (Mindtouch and Spotfire), Slides. June 14, 2010.

http://chesapeakeactionplan.wik.is/#References

26

References• A Gov 2.0 Platform for Open Government with Data Science (see

Data Science Library in the Cloud, JumpStart, and Public) (continued): – Ninth was Build the SemTech 2010 in the Cloud (Mindtouch and Spotfire), No Slides, July 2,

2010. – Tenth was Build the White House Staff Salaries in the Cloud (Mindtouch and Spotfire).

Slides, July 3, 2010. – Eleventh was Build EPA’s Synaptica in the Cloud: An Enterprise Vocabulary Catalog for

Data.gov/semantic (Mindtouch and Spotfire). Slides, July 8, 2010. – Twelfth was Build EPA’s BP Oil Spill Data Tools in the Cloud (Mindtouch and Spotfire).

Slides, July 10, 2010. – Thirteenth was Build EPA’s Bay Barometer in the Cloud (Mindtouch and Spotfire). Slides,

July 15, 2010 (in process). – Fourteenth was Build the Federal IT Dashboard in the Cloud (Mindtouch and Spotfire). No

Slides, July 15, 2010. Completed in 3 hours. Added agency functionality in three more hours, July 18, 2010.

– Fifteenth was Build the TRI Explorer in the Cloud (Mindtouch and Spotfire), July 20, 2010. – Sixteen was Build the Top Secret America in the Cloud (Mindtouch and Spotfire), July 21,

2010. – Seventeen was Build the EPA Enterprise Architecture Information Management Planning

Analysis & Reuse Tool in the Cloud (Mindtouch and Spotfire). Slides, July 21, 2010 (in process).

– Eighteenth was Build the EPA 2010 Environmental Information Symposium in the Cloud (Mindtouch and Spotfire) in Support of One EPA and Strategic Planning, July 26, 2010 (in process).

– More in process of Spotfire: Linked Data Between Panels and Tabs and of GAIN: Linked Data Between Data Catalogs and Apps.

http://chesapeakeactionplan.wik.is/#References