Fun with Geospatial Metadata, CUGIR, CORC, MARC, and OAI: The CSDGM to MARC

Grant Project

Adam Chandler, Olin LibraryElaine Westbrooks, Mann Library

Vivek Uppal, Dept. of Computer Science

Metadata Working GroupOctober 12, 2001

2

Project Workflow

Converter

Authority

Control

3

Problem

How do we create a persistent URL pointing to “deep web” digital objects across very different systems?

What is GIS?What is CUGIR?What are Geospatial Metadata? What is CORC?

4

Overview (1)

Overview (2)

What is the FGDC Clearinghouse?Bucket ArchitectureWhat is OAI?DemonstrationOutcomes

5

“A system for capturing, storing, checking, manipulating, analyzing and displaying data which are spatially referenced to the Earth.” (DoE, 1987)

6

What is GIS?

GISGIS

ProceduresProcedures

DataData

HardwareHardware

SoftwareSoftwarePeoplePeople

7

8

Examples of GIS

CUGIR

Brief HistorySubject areasUnrestricted access to data &

metadataNational Geospatial Data

Clearinghouse & NSDI

9

Geospatial Metadata

Brief HistoryDefinitionPurpose / Uses / BenefitsCUGIR Examples: TAZ

Content Standard for Digital Geospatial Metadata

10

What is CORC?PathfindersHarvests MetadataAuthority ControlClassification

Metadata: MARC / DC-RDF

11

CORC

FGDC Clearinghouse

12

13

14 14

CUGIR

15

16

Show Metadata

17

18

http://130.11.52.184/servlet/FGDCServlet/retrieve?on=2&el=F&db=current&rp=3&mr=1&ac=current&cid=181

Non-Persistent URL

Clearinghouse Problems

0 hits for users 50% of time (unpublished research)

> 10% or more or nodes are offline (Z39.50)

FGDC Clearinghouse is not well known

Record maintenance is a undefined

Harvesting is not a part of model

Data/Metadata are coupled with the server software (i.e., “smart archive, dumb object” DL model)

19

Open Archives Initiative

Data Provider

Service Provider

http (get request)

xml response

low-barrier interoperability data-provider & service-provider model metadata harvesting model shared metadata format & parallel,

community-specific metadata formats authentication : on purpose outside protocolSource: Herbert Van De Sompel, CS 502, Spring 2001

20

21

OAI example

22

23

“Buckets are part of the larger ‘Smart Object, Dumb Archive’ DL Model (Maly, Nelson, & Zubair, 1999). SODA is a reaction to the vertically integrated (and non-interoperable) DLs that tended to grow from the ad-hoc origins of many popular DLs (Esler & Nelson, 1998). Separating the functionality of the archive from that of the DL allows for greater interoperability and federation of DLs. The archive's purpose is to provide DLs the location of buckets (the DLs can poll the buckets themselves for their metadata), and the DLs build their own indexes.”

Buckets

Source: M. Nelson: “Smart Objects and Open Archives.” D-Lib Magazine, February 2001, 7(2) http://www.dlib.org/dlib/february01/nelson/02nelson.html

24

On Buckets and OAI

“Just as buckets break the dependency of the information objects on archives, the Open Archives Initiative (OAI) breaks the dependency of archives on DLs …”

- Michael L. Nelson

25

On Buckets and OAI

“The OAI does not address the issue of smart objects, but the archives in the OAI are very similar to the archives described in the SODA model in that they have minimal functionality. OAI archives aim for greater interoperability through performing less sophisticated functions (no keyword search functions defined, T&C is not handled at the protocol level) -- a sort of Reduced Instruction Set Computer philosophy for archives.”

- Michael L. Nelson

26

Problem Revisited

How do we create a persistent URL pointing to “deep web” digital objects across very different systems?

27

Project Workflow

Converter

Authority

Control

28

Step 1: Convert SGML -> MARC

29

Step 2: MARC & Bucket Location

30

Step 3a: Import to CORC (MARC)

30

MARCView:

856 “Bucket”

31

Step 3b: Import to CORC (DC RDF)

31

DC RDFView:

856 “Bucket”

32

Step 4a: Import DC-RDF

33

Step 4b: Import DC RDF to Local Mysql Database

34

MARC Record in Voyager

34

Click here to see bucket

35

Table: Bucket

36

Table: DC

37

Project Outcomes

1. Access for patrons-1,300 MARC Records in OPAC-1,300 MARC Records in CORC/WorldCat-2,600 DC-RDF Records in OAI

2. OAI data provider (DLF agreement)3. CORC evaluation4. Foundation for CUGIR 2.05. Data for cross-collection access

research