The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local...
Transcript of The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local...
![Page 1: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/1.jpg)
Interoperability and Collection of Preservation Metadata for Digital
Repository Content
Matt Cordial, Tom Habing, Bill Ingram, Robert ManasterUniversity of Illinois Urbana-Champaign
cordial;thabing;wingram2;[email protected]
The Hub and Spoke Framework
![Page 2: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/2.jpg)
• Conceptual Overview– Conceptual diagrams– Framework architecture
• Components– METS Profiles (The Hub)– Content processing, transformation, and metadata
generation (Spokes)– Facilitating repository ingestion and dissemination
(The Handoff)
• Conclusion– Finding more information
Outline
![Page 3: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/3.jpg)
A Birds-eye View
![Page 4: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/4.jpg)
![Page 5: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/5.jpg)
The Essentials
• Extensible METS profile• Repository specific processing and
transformation utilities• Java API for local integration and
extensibility– Apache XML Beans
• Dissemination/Submission Web-service– RESTful
![Page 6: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/6.jpg)
Apache XMLBeans
Hub and Spoke METS Profile API
Web Profile API ... ...
Dspace Packager ...EPrints Packager
Packager
LRCRUD Client
LRCRUD ServiceDspace LRCRUD
Service EPrints LRCRUD Service ...
DSpace
Dspace Workflow
Workflow Manager
EPrints Workflow
EPrints ...
...
JHOVE
![Page 7: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/7.jpg)
Processing and Transformation
METSConstruction
Descriptive MetadataAugmentation
Bitstream Verification
Profile Validation
Functional Overview
to hub
from hub
METS Profiles Repository
Technical Metadata
Augmentation
XSLT
TechMDAugmenter
JHOVE
Handoff
Web Service Client
Web Service
![Page 8: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/8.jpg)
• Non-prescriptive in regards to structure or file formats
• Intended to overlay other profiles which specify case-specific needs (i.e. web captures)
• PREMIS • MODS
– Must conform to the DLF Aquifer profile
• File-format specific technical metadata – MIX, VIDEOMD, AUDIOMD, others as
appropriate
METS Profiles
![Page 9: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/9.jpg)
Processing and Transformation
METSConstruction
Descriptive MetadataAugmentation
Bitstream Verification
Profile Validation
Functional Overview
to hub
from hub
METS Profiles Repository
Technical Metadata
Augmentation
XSLT
TechMDAugmenter
JHOVE
Handoff
Web Service Client
Web Service
![Page 10: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/10.jpg)
Technical Metadata Generation/Augmentation
• JHOVE Output + Custom XSLT• Java “Applicators” for specific technical
metadata schemas– MIX– TEXTMD– AUDIOMD– PREMIS – Class hierarchy to support new Applicators
![Page 11: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/11.jpg)
Processing and Transformation
METSConstruction
Descriptive MetadataAugmentation
Bitstream Verification
Profile Validation
Functional Overview
to hub
from hub
METS Profiles Repository
Technical Metadata
Augmentation
XSLT
TechMDAugmenter
JHOVE
Handoff
Web Service Client
Web Service
![Page 12: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/12.jpg)
Hub
Data Store / DIPs
metadata.xml
image.jpg
Generate/collect provenance metadata
Extract format-specific technical
metadata
Transform/enrich native metadata
Embed native metadata
Generate/collect digital provenance metadata
To-Hub Processing
Embed links to digital items
Model structure of the item
![Page 13: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/13.jpg)
Hub
SIPs
hubMets.xml
Generate provenance metadata
Add the METS file as an item in the
submission package
Transform hub metadata to repository-compatible
metadata
Assemble into packages for repository ingest
From-Hub Processing
metadata.xml
![Page 14: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/14.jpg)
Processing and Transformation
METSConstruction
Descriptive MetadataAugmentation
Bitstream Verification
Profile Validation
Functional Overview
to hub
from hub
METS Profiles Repository
Technical Metadata
Augmentation
XSLT
TechMDAugmenter
JHOVE
Handoff
Web Service Client
Web Service
![Page 15: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/15.jpg)
• Packages usable by a repository’s native ingestion routines
• REST Web-Service – Client integrated into processing workflow– DSpace, EPrints, and others in the next year– Specification and API to create service for other
repository systems
LRCRUD
![Page 16: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/16.jpg)
Data
DSpace native requestfor handle 2135.89342
GET /dspace-lrcrud/2135.89342 HTTP/1.1
Item + metadata
Zip File
4
2
1
3
LRCRUD Client
LRCRUD Service
Repository Server
• Client submits a GET request to LRCRUD service for a specific item
• Service calls the native DSpace dissemination routine
• Service receives the output from the dissemination, creates a header file, and adds both the header file and the disseminated content to a zip-file
• Service returns a zip-file containing the package to the client
![Page 17: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/17.jpg)
PUT Fedora IDHTTP/1.1
Zip File
Item + metadata
HTTP Status 204
Confirmation
POST /fedora-lrcrud/
HTTP/1.1
HTTP Status 201 Location header w/
Fedora ID
Request creation of stub recordFedora ID
1
2
3
4
8
7
6
5
Data
LRCRUD Service
LRCRUD Client
Repository Server
Create stub record
2) Client issues a POST request to LRCRUD specifying “where” to create the record (e.g. communities or collections) if needed
3) LRCRUD calls the native Fedora creation routine
4) Fedora supplies LRCRUD with the ID for the newly created record
5) LRCRUD responds to the client with an HTTP 201 “Created” message and returns the ID in the Location: header
Upload and ingest the item
8) Client issues a PUT request to LRCRUD to replace the package identified by the URI. The entity body of the request must contain the zip-file containing the package to be ingested.
9) LRCRUD unpacks the files and calls the native Fedora ingestion routine.
10) Fedora tells LRCRUD that ingestion was successful
11) LRCRUD responds to the client with an HTTP 204 “No Content” message indicating that the request was successful.
![Page 18: The Hub and Spoke Framework - COnnecting REpositories · 2015-05-29 · •Java API for local integration and extensibility –Apache XML Beans ... Descriptive Metadata Augmentation](https://reader034.fdocuments.in/reader034/viewer/2022042206/5ea8377a75dbc66d3b6f485c/html5/thumbnails/18.jpg)
Open Source Code:http://sourceforge.net/projects/echodep
LRCRUD Service Specification:
http://dli.grainger.uiuc.edu/echodep/hns/LRCRUDS.htm
METS Profiles:Generic - http://www.loc.gov/standards/mets/profiles/00000015.xml
Web Capture - http://www.loc.gov/standards/mets/profiles/00000016.xml
Java API Documentation (Javadoc):http://echodep.sourceforge.net/javadoc/index.html
Project Web Sitehttp://ndiipp.uiuc.edu/
More Information