Observational Data Standard, v 1.0

Lynn Kutner, Donna Reynolds, Jennifer Nichols, Kristin Barker, Kristin Snow

April 2007

A Provisional Standard to Facilitate Data Sharing and Aggregation

Working GroupNatureServe Lynn Kutner Carol Fogelsong Donna Reynolds Jennifer Nichols Bruce Stein Rickie White Keith Carr Kelly Gravuer Larry Master Kristin Barker

Member Programs Karen Cieminski (MN) Karen Walker (MT) Pete Sorrill (ON) John Fleckenstein (WA) Nicole Firlotte (MB) Jim Morefield (NV) Tim Howard (NY) Melanie Arnett (WY)

Partners Parks Canada Canadian Wildlife Service Cornell Lab of Ornithology Larry Morse

Reviewers Member Programs: Alberta, British Columbia, Connecticut,

Idaho, Maine, Manitoba, Maryland, Massachusetts, Minnesota, Montana, Nevada, New Hampshire, New York, Ontario, Pennsylvania, Saskatchewan, Texas, Vermont, Virginia, Washington, West Virginia, Wyoming

NatureServe Central NatureServe Canada Canadian Wildlife Service Cornell Lab of Ornithology Information Center for the Environment (ICE), University of

California, Davis Parks Canada U.S. National Park Service VegBank Washington Department of Fish and Wildlife

Development of the Standard

Process Funded by the Gordon and Betty Moore Foundation NatureServe is a co-sponsor of the Taxonomic

Databases Working Group (TDWG) Observational Data Subgroup: this is a provisional standard as an input for that group’s work towards an international observational data standard

Multi-institutional Observation Working Group Compared 16 observation databases to identify

breadth and commonalities among attributes: Salvias, ABCD, VegBank, Darwin Core2, Member Program custom databases

Project began mid-2005. Version 1.0 published September 2006

Initial implementation in NatureServe’s Kestrel Revisions expected

What is an Observation?

An observation characterizes evidence for the presence or absence of an organism or set of organisms through a data collection event at a location.

Goals & Requirements Useable—minimal requirements, simple Extensible, e.g., discipline-specific data Universal

core set of concepts, broadly applicable, for data sharing and aggregation

compatible with (and incorporating) existing standards Software-independent

Accommodate: species AND ecological communities (inc. plots) historical data different protocols negative (absence) data data quality and validation attributes monitoring

Overview of the Standard

http://www.natureserve.org/prodServices/obsStandard.jsp

Major Entities

Observation

Observation Grouping

Survey

Species List

Project (group of surveys)

Protocol (methods)

Search Area

External Documentation

Ranking of Attributes

Required – cannot be null, but in some cases can be “unknown”

Priority Core – should always be recorded if possible

Core – important but not required (not always available or applicable)

Additional – supplementary data

Attribute Table

Observation: Required Attributes

1. What: Scientific nameCalypte anna or

Quercus velutina / Ilex opaca Forest or

“a community”

2. Where: GIS shape, coordinates, or text New Mexico: Eddy County, 3 mi S of Artesia along Rt. 285

3. When: Date (to year, at least)

30 November 1978

4. Who: Observer Name(s) or “Unknown”D.J. Davis

Observation: “What” Scientific Name Author – of scientific name

Concept reference & name used in concept ref

Name Type – “taxon” or “ecological community” Identification Confidence – high, medium, low

Concept Fit – extent to which observed community fits with published concept

Secondary Designation – other communities / taxa that this observation may relate to

Verbatim Scientific Name

GUID – global universal identifier of a taxonomic concept

Observation: “Where” Link to GIS feature (point, line, polygon)

Latitude, Longitude, Datum

Location Description (text)

OR

Country

Mapping Accuracy – distance within which the location of the observation has believed to have been captured

Location confidence – high, medium, low

Location fuzzed – indicator and/or description of process if data have been randomized / generalized

Date of the observation – includes either full date, partial date, or range of dates

Date Accuracy – proposed values: accurate within 1 day, 1 week, 1 month, 3 months, 1 year, 3 years, 10 years, >10 years

Observation Start Time and End Time

Observation: “When”

Observer Name – full name(s) of the person(s) who collected data

Observer Affiliation

Observer Postal Address, Phone Number, E-mail Address

Observer Role – primary observer, observer, collector, submitter, verifier

Observation: “Who”

Protocol – protocol used to collect data

Found Indicator – used to indicate negative (absence) data

Confidence – confidence in search result (High, Medium, Low)

Level of Effort – effort expended on a given visit

Additional Inventory Needed – Is additional inventory of the area needed for this element? (Y / N)

Evidence Type – sample values: Specimen, Sighting, Literature, Photograph, Tracks

Evidence Comments

Observation: “How”

Observation: Evidence

Observation: Biology Number of Individuals (species)

Estimated / Observed – whether number is estimate or actual count

Reproductive Evidence – element reproducing at the location? (Y / N)

Condition of Element – description of the quality of element (e.g., health, alive or dead)

Density / Distribution – distribution of the element on the landscape (e.g., solitary individual, patchy, scattered, solid cover)

Strata – vegetation strata in the community

Origin – sample values: Native, Nonindigenous, Unknown/Undetermined

Invasiveness Comments – notes about the degree of invasiveness

Observation: Environment Habitat Description – text description of the local or surrounding

habitat

IUCN Habitat Category – habitat type, selected from a picklist

Condition of Site – condition of the surroundings (e.g., flooded, burned, etc.)

Weather – description of the weather conditions at the time of the visit

IUCN Threat Category – primary threat, selected from a picklist

Threat Comments – notes about primary or other threats

Management Activities – activities conducted during a particular visit (e.g., pulling or pesticides applied to invasives)

Management Needs – most important management needs (for enhancement or control) at this observation site

Data Sensitive – Is the observation information sensitive? (Y / N)

Reason Data Sensitive – primary reason why data are sensitive

General Comments – notes about observation not addressed elsewhere

Internal Notes – comments or issues about the observation that are internal to the organization that created the record

Associated Element Scientific Name

Associated Element Origin sample values: Native, Nonindigenous, Unknown/Undetermined

Associated Element Relationship Comments

Other Information

Associated Elements

Major Entities

Observation

Observation Grouping

Survey

Species List

Project (group of surveys)

Search Area (inc. plot) - GIS Shape

Protocol (methods)

Documentation

Observation Grouping

Name of Grouping

Criteria – the common characteristic or other criteria used to group the observations (same element, same location, or any other criteria)

Owner – the person who created the observation grouping

Monitoring Comments – changes over time and trends within the grouping

Species List For community plot data

observation ID search area ID (plot) scientific name / concept stratum cover class percent cover number (per species)

May make an observation record for each species, but not required

Next Steps Current and future software development

projects at NatureServe:

Kestrel

Handheld field data collection tool - funding received from NSF

XML schema, mapping to existing TDWG standards

TDWG Observational Data Subgroup

Kestrel NatureServe’s Web Application for

Observations Data Management

XML-based Web interface Extensible: Data model is dynamically generated

Libraries and Templates Open Source

Kestrel Architecture

KestrelXML Web Service

KestrelXML Web Service

KestrelWeb User Interface

KestrelWeb User Interface

OracleDatabase

Internet

BrowserBrowser

AggregationPortal

AggregationPortal

DesktopApplication

DesktopApplication

Field DeviceField Device

XMLHTML

Application Framework

KestrelXML Web Service

KestrelXML Web Service

KestrelWeb User Interface

KestrelWeb User Interface

OracleDatabase

IDDSpecies Web Service

IDDSpecies Web Service

BioticsDatabase

SecurityXML Web Service

SecurityXML Web Service

Users & Permissions

Database

ArcGIS ServerXML Web Service

ArcGIS ServerXML Web Service

SDEDatabase

Observation Standard Support

Core Attributes Who, what, where, when Managed as relational fields

Extended Attributes Managed as XML

Attribute Extensibility

Individual Attributes defined in XML Help to describe observation or survey Non-trivial effort Signed by author,

optionally by organization

Exam

ple

Att

ribu

te:

Obse

rvati

onTyp

e

Exam

ple

Att

ribu

te:

Aci

dit

y

Attribute Collection

Group of Extended Attributes Strongly associated with observation or survey

Signed by author/organization “Work together” Ad-hoc OR Protocol support, e.g.

Heritage Methodology CI Rapid Assessment Citizen Bird Survey

Timeline

Parks Canada: June 2007 – very basic functionality

NSF Handheld Data Collection Tool: June 2009 – accommodate plots, multivalued data, etc.

Questions / Discussion