The Evolution of Open Data@JeanneHolm Jeanne Holm

Evangelist, Data.gov11 June 2014

2

The Times They Are A’Changing

PeoplePeople Knowledge

Problems

June 11, 2014

3

Data, Data Everywhere Smart phones Smart cars

Smart people Sensors

RFID Cameras…everywhere

June 11, 2014

Americanis.net

4

Move from Structured to Unstructured Heterogeneous sources of data

Structured (tables, transactions) = schema Semi-structured (human-readable, XML, JSON) Unstructured (images, audio, videos) = no

relationship

June 11, 2014

5

Recent Data Growth Web 2.0

Social media Facebook Twitter Skype

The Internet of Things Many sources Varied formats Relatively timely

Web content

Many authors Unstructured Highly variable

trust and provenance

Gaming Highly specific Huge transactional

data Real-time, high

bandwidth usageJune 11, 2014

6

Recent Data Growth Web 2.0 Open data

Government and industry

Structured and unstructured

Accessible

Private data Apps Health data Credit card and financial

data

The Web

Browsers Search engines Web site metrics

June 11, 2014

7

Creating Order from the Chaos

Open vs. closed

Multiple formats

Unstructured

Trusted vs. unvalidated

June 11, 2014

Releasing and using open data is about empowering people

to make better decisions

June 11, 2014

9

Presidential Executive Order

June 11, 2014

10

Project Open Data: Open Source Policy

Open source government policy, technical guidance, and software

Citizen contributions to policy, code, and content

http://project-open-data.github.io/

June 11, 2014

105,000+ collections

349 citizen apps500,000 data

resources175 agencies

450 APIs

June 11, 2014

12

Creating the Open Data Community

Open Data is

an Ecosyste

m

June 11, 2014

13

Open Exchanges with Citizens

Questions and answers at the new Open Data Stack Exchangehttp://opendata.stackexchange.com/

Data jams and data paloozas at the White House

June 11, 2014

14

Connecting Over Social Media

June 11, 2014

15

Developer Feedback

June 11, 2014

https://github.com/GSA/data.gov

16

Open Exchange with Developers

Created a new Open Data Stack Exchange to field questions to the global community: http://opendata.stackexchange.com/

June 11, 2014

17

Citizen Participation: Redesigning Data.gov

In looking at the redesign, conducted multiple places for citizens to say what they wanted Formal usability testing (3 rounds) Blogs Next.Data.gov Quora Twitter @usdatagov Open Data Stack Exchange Multiple social media platforms

All the comments in one place Github Issues tracked at

https://github.com/GSA/data.gov/issues?labels=&milestone=&page=1&state=open

June 11, 2014

18

Usability Testing

Created and vetted usability test that focused on what actions people completed on the site and expectations they had for what they would find Face-to-face testing in Washington D.C. Virtual testing via Skype and phone Online testing using Loop 11

June 11, 2014

Reached out to key users Data journalists Researchers Developers Entrepreneurs Data scientists

Businesses Students and teachers Advocacy groups People who had

complained about Data.gov

19

Evaluate the Feedback

All issues were copied, connected, or added to Github from any public communication channel

Issues were assigned to a person and a build

Discussion was encouraged on each and people were invited to the conversation

June 11, 2014

20

Designs Exposed Openly

June 11, 2014

21

U.S. Open Data for Cities, Counties, and States

June 11, 2014

22

Linked Data and the Semantic Web

June 11, 2014

Join the W3C eGovernment Interest Groupwww.w3.org/egov

23

Open CommunitiesCommunity

Developers ✓

Safety ✓

Energy ✓

Health ✓

Law ✓

Education ✓

Ocean ✓

Manufacturing ✓

Business ✓

Ethics ✓

States ✓

Counties ✓

Cities ✓

Agriculture ✓

+ many more…June 11, 2014

24

Agriculture Drives Innovation and Saves Lives

Farmers’ Markets

iCow

June 11, 2014

Open Government Platform (OGPL)

Email, Github, Facebook, and Twitter for discussionhttps://github.com/opengovtplatform http

://www.opengovplatform.org

June 11, 2014

A Global Movement Has Begun to Provide Transparency and Democratization of Data

June 11, 2014

Is open data useful?

June 11, 2014

Creating a multi-billion dollar

industry

June 11, 2014

29

NOAA weather data + Air Force GPS services = $100B industry

June 11, 2014

30

Climate Corporation

June 11, 2014

Helping others through a disaster

June 11, 2014

32

Hurricane Sandy

700,000 downloads of a transportation app

using data from Data.gov

Early warnings save lives

June 11, 2014

Saving Lives

June 11, 2014

June 11, 2014

It’s Not Rocket Science…or Maybe

It Is

June 11, 2014

International Space Apps

Open annual event, around the world and in space

95 physical locations in 46 countries participated;, 8,195 participants, 671 projects Where on Earth Exomars Rover is My Robot Asteroid Prospector Space Wearables Growing Food for A Martian Table

June 11, 2014 36

37June 11, 2014

Predicting the Future

June 11, 2014

39June 11, 2014

40June 11, 2014

Empowering Others to Use Open Data

41

Organizing and Understanding the Data

Web searching, mining, and crawling

Algorithms

Visualizations

Text mining

Clustering

Semantic analysis

Linked data

Machine learning

June 11, 2014

42

New Role: Data Scientist

Combines technical and business skills

Looks at complex data problems with subject matter expertise

Applies technologies to mine, analyze, and visualize the data

Understands statistics and math, coding and algorithms

Can explain the significance of the data to others

Leader of the data scientists: The Chief Data Officer

June 11, 2014

43June 11, 2014

44

Open Data Matters

Connect citizens to open data to transform their world and empower them through education

Connect developers to open data to create new ways of using the data to inform others

Connect businesses to open data to provide new services and products for everyone to use

Connect data scientists to open data to analyze the past and predict the future

Encourage governments to release more open data

June 11, 2014

Helping to improve the lives of people in

our community

June 11, 2014