Networked Digital Libraryof Theses and Dissertations (NDLTD, www.ndltd.org)

Oregon State UniversityCorvallis, OR20 July 2005

Edward A. FoxVirginia Tech, Blacksburg, VA 24061 USA

fox@vt.edu http://fox.cs.vt.eduSlides later at: http://fox.cs.vt.edu/talks/2005

Acknowledgements(Selected)

• VT Faculty/Staff: Tony Atkins, Debra Dudley, John Eaton, Gary

Hooper (provided funds for work in 1987), Carolyn Kletnieks, Gail McMillan, Len Peters, …

• VT Students: Shahrooz Feizabadi, Marcos Goncalves, Bing

Liu, Ming Luo, Paul Mather, Rao Shen, Ohm Sornil, Hussein Suleman, …

ACKNOWLEDGEMENTS (NDLTD)• NDLTD Board of Directors, previous Steering Committee + other

NDLTD committees; those running Electronic Thesis & Dissertation (ETD) initiatives in universities, regions, countries

• Helpful sponsorship by many organizations, especially Adobe, CONACyT, DFG, FIPSE (US Dept. Education), IBM, Microsoft, NSF (IIS-9986089, 0086227, 0080748, 0325579; DUE-0121679, 0136690, 0121741, 0333601), OCLC, SOLINET, SUN, SURA, UNESCO, VTLS, many governments (Australia, Germany, India, …), …

• Colleagues (faculty, staff, students) / collaborators at many universities

• Slides included from: Vinod Chachra, Thom Hickey, Joan Lippincott, Gail McMillan, Axel Plathe, Hussein Suleman, …

Part 1

Introduction

Digital Libraries --- Objectives

• World Lit.: 24hr / 7day / from desktop• Integrated “super” information systems: 5S:

streams, structures, spaces, scenarios, societies • Ubiquitous, Higher Quality, Lower Cost • Education, Knowledge Sharing, Discovery• Disintermediation -> Collaboration • Universities Reclaim Property• Interactive Courseware, Student Works• Scalable, Sustainable, Usable, Useful

SynchronousScholarly Communication

Same time, Same or different place

Asynchronous, Digital Library Mediated Scholarly Communication

Different time and/or place

Digital LibrariesShorten the Chain from

Editor

Publisher

A&I

Consolidator

Library

Reviewer

DLs Shorten the Chain to

Author

Reader

Digital

LibraryEditor

Reviewer

Teacher

Learner

Librarian

Information Life Cycle

AuthoringModifying

OrganizingIndexing

StoringRetrieving

DistributingNetworking

Retention/ Mining

AccessingFiltering

UsingCreating

D ig ita l L ib ra r y C o n te n t

A rtic le s ,R e p o rts,

B o o ks

T e xtD o cum e n ts

S p ee ch ,M u s ic

V id eoA u d io

(A e ria l)P h o tos

G e og rap h icIn fo rm ation

M o d e lsS im u la tio ns

S o ftw a re ,P ro g ra m s

G e no m eH u m a n,a n im a l,

p la n t

B ioIn fo rm ation

2 D , 3 D ,V R ,C A T

Im ag es a ndG ra p h ics

C o nte n tT yp e s

Definitions

• Library ++ (library+archive+museum+…)• Distributed information system + organization

+ effective interface• User community + collection + services• Digital objects, repositories, IPR management,

handles, indexes, federated search, hyperbase, annotation

• Or, we can use the “5S” framework:

5S Model: Examples, Objectives

Models Examples ObjectivesStream Text; video; audio; image Describes properties of the DL

content such as encoding and language for textual material or particular forms of multimedia data

Structures Collection; catalog; hypertext; document; metadata; organization tools

Specifies organizational aspects of the DL content

Spatial Measure; measurable, topological, vector, probabilistic

Defines logical and presentational views of several DL components

Scenarios Searching, browsing, recommending,

Details the behavior of DL services

Societies Service managers, learners, Teachers, etc.

Defines managers, responsible for running DL services; actors, that use those services; and relationships among them

5S and DL formal definitions and compositions (April 2004 TOIS)

5S

structures (d.10)streams (d.9) spaces (d.18) scenarios (d.21) societies (d. 24)

structural metadataspecification(d.25)

descriptive metadataspecification(d.26)

repository(d. 33)

collection (d. 31)

(d.34)indexingservice

structured stream (d.29)

digitalobject (d.30)

metadata catalog (d.32)

browsingservice

(d.37)

searchingservice (d.35)

digital library(minimal) (d. 38)

services (d.22)

sequence (d. 3)

graph (d. 6)function (d. 2)

measurable(d.12), measure(d.13), probability (d.14), vector (d.15), topological (d.16) spaces

event (d.10)state (d. 18)

hypertext(d.36)

sequence (d. 3)

transmission(d.23)

relation (d. 1) language (d.5)

grammar (d. 7)

tuple (d. 4)*

DL Services/Activities Taxonomy (Gonçalves)

BrowsingCollaboratingCustomizingFilteringProviding accessRecommendingRequestingSearchingVisualizing

AnnotatingClassifyingClusteringEvaluatingExtractingIndexing

MeasuringPublicizing

RatingReviewing (peer)

SurveyingTranslating (language)

ConservingConverting

Copying/ReplicatingEmulatingRenewing

Translating (format)

AcquiringCataloging

Crawling (focused)DescribingDigitizingFederatingHarvestingPurchasingSubmitting

PreservationalCreational

AddValue

Repository-Building

Information SatisfactionServices

Infrastructure Services

AuthoringModifying

OrganizingIndexing

Storing

Archiving

NetworkingAccessing

Filtering

Creation

DistributionUtilization

Reputation

Similarity

Desirability

AccuracyCompletenessConformance

Discovery

SearchingBrowsingRecommending

Relevance

Timeliness

Accessibility

Usage

Inactive

Active

Discard

RetentionMining

Semi-Active

Preservability

Timeliness

Institutional Repositories

• “Institutional repositories are digital collections that capture and preserve the intellectual output of a single university or a multiple institution community of colleges and universities.”

• Crow, R. “Institutional repository checklist and resource guide”, SPARC, Washington, D.C., USA

• www.arl.org/sparc/IR/IR_Guide_v1.pdf

Fedora™Repository

E x ter n a lC o n ten tS o u r c e

E x ter n a lC o n ten tS o u r c e

HT

TP

E x ter n a l C o n ten tR etr iev er

X M L F ile s

Re la t io n a l D B

S e s s io n M a n a g e me n tU s e r A u th e n t ic a t io n

P o l icies

U s ers /G ro u p s

H T T P

F T P

D atas tr eam s

D ig ita l O b jec tsS to rag e S u b s ys te m

S e c u rityS u b s ys te m

W e b Se r vi c eE xpo s ur eL aye r

SO

AP

R em o teS er v ic e

L o c alS er v ic e

M an ag e A c c e s s S e arc h O A I P ro v id e r

M an ag e m e n tS u b s ys te m

A c c e s sS u b s ys te m

HT

TP

FT

P

H T T PH T T P S O A P H T T P S O A P H T T P S O A P

C lie n tA pplica t io n

B a tchPro g ra m

S e rv e rA pplica t io n

W e bB ro ws e r

Co mp o n e n t M g mt

O b je c t M g mt

O b je c t Va lid a t io n

P ID Ge n e ra t io n

O b je c t D is s e min a t io n

O b je c t Re fle c t io n

P o lic y En fo rc e me n t

P o lic y M g mt

Co n te n t

Web Service Web Service Exposure Exposure LayerLayer

Adapted from Slide by V. Chachra, VTLS

VITAL / Fedora Relationship

Open Archives Initiative (OAI)

• Advocacy for interoperability• Standard for transferring metadata among

digital libraries• Protocol for Metadata Harvesting (PMH)

• Simplicity• Generality• Extensibility

• Support for PMH => Open Archive (OA)

OAI = Technical Umbrella forPractical Interoperability…

ReferenceLibraries

PublishersE-Print

Archives

…that can be exploited by different communities

Museums

OAI – Repository Perspective

Required: Protocol

DODO DO DO

MDO

MDO MDOMDOMDO

MDOMDOMDO

OAI – Black Box Perspective

OA 1

OA 2

OA 4

OA 3

OA 5OA 6

OA 7

DiscoveryCurrent

AwarenessPreservation

Service Providers

Data Providers

Meta

data

harv

estin

g

The World According to OAI

Case Study: Education

• Refactoring Scholarly Communication:• Creating, Sharing, Reviewing, Teaching, Learning, …

• Start with focused efforts (ETDs, courseware, technical reports), but eventually cover everything

• Physics: PhysNet• OCKHAM• NCSTRL, CSTC, CITIDEL, NSDL• NDLTD

Part 2

NDLTD

A Digital Library Case Study

• Domain: graduate education, research

• Genre:ETDs=electronic theses & dissertations

• Submission: http://etd.vt.edu

• Collection: http://www.theses.org

Project: Networked Digital

Library of Theses & Dissertations

(NDLTD) http://www.ndltd.org

The Networked Digital Library of Theses and Dissertations

www.NDLTD.org

Leader of the Worldwide ETD(Electronic Thesis and Dissertation) Initiative

Training AuthorsExpanding Access

Preserving KnowledgeImproving Graduate Education

Enhancing Scholarly CommunicationEmpowering Students & Universities

Main Message

• Digital libraries can help advance education.

• Oregon State University and its neighbors are invited to engage in NDLTD, as well as CITIDEL, NSDL, and other DL ventures.

• UNESCO Analytical Survey on Digital Libraries in Education is recommending DLE in each nation.

• Local and national support can• stimulate activities, including collaboration• promote a sharing culture, especially in research and teaching• leverage others’ investments (networking, computing, …)• encourage / facilitate learning

• Please join NDLTD!

What led to today’s meeting?• 1987 mtg in Ann Arbor: UMI, VT, …• 1992 mtg in Washington: CNI, CGS, UMI, VT and 10 universities

with 3 reps each• 1993 mtg in Atlanta to start Monticello Electronic Library (regional,

US Southeast): SURA, SOLINET• 1994 mtg at VT: std: PDF + SGML + multimedia objects• 1996 funding by SURA, US Dept. of Education (FIPSE)• 1997 meetings in UK, Germany, ...• 1998 – 1st symposium – Memphis (20)• 1999 – 2nd symposium – Blacksburg (70)• 2000 – 3rd symposium – St. Petersburg (225)• 2001 – 4th symposium – Caltech (200)• 2002 – 5th syposium – BYU, Provo, Utah• 2003 – 6th syposium – Berlin (215) • 2004 – 7th syposium – U. Kentucky• 2005 – 8th syposium – Sydney, Australia

What are the long term goals?

• 400K US students / year getting grad degrees are exposed / involved

• 200K/yr rich hypermedia ETDs that may turn into electronic portfolios (images, video, audio, …)

• Dramatic increase in knowledge sharing: literature reviews, bibliographies, …

• Services providing lifelong access for students: browse, search, prior searches, citation links

• Hundreds/thousands of downloads / year / work

ETDs: Library Goals • Improve library services

• Better turn-around time • Always available

• Reduce work • catalog from e-text • eliminate handling: mailing to ProQuest, bindery

prep, check-out, check-in, reshelving, etc.• Save space

NDLTD Incorporation

• Networked Digital Library of Theses and Dissertations incorporated May 20, 2003 in Virginia, USA

• Charitable and educational purposes (501 c 3)• Can accept donations, collect dues, receive funds

• LeClair Ryan provides legal counsel

• Officers• Executive Director (Ed Fox)• Secretary (Gail McMillan)• Treasurer (Scott Eldredge)

Board of Directors• Suzie Allard (ETD 2004, U. Kentucky)• Denise A. D. Bedford (World Bank)• Julia C. Blixrud (ARL, SPARC)• José Luis Borbinha (National Lib Portugal)• Alex Byrne (ETD 2005, ADT: Australia)• Tony Cargnelutti (ETD 2005, Australia)• Vinod Chachra (VTLS)• Susan Copeland (RGU, UK)• Jude Edminster (Bowling Green St. Univ.)• Scott Eldredge (Treasurer, ETD 2002, BYU)• Edward A. Fox (Exec Director,Virginia Tech)• John H. Hagen (West Virginia U.)• Thomas B. Hickey (OCLC)• Christine Jewell (U. Waterloo, Canada)

• Delphine Lewis (ProQuest)• Joan K. Lippincott (CNI)• Mike Looney (Adobe)• Gail McMillan (Secretary, Virginia Tech)• Joseph Moxley (ETD 2000, USF)• Eva Müller (U. Uppsala, Sweden)• Ana Pavani (PUC Rio, Brazil)• Axel Plathe (UNESCO, Paris)• Sharon Reeves (National Library Canada)• Peter Schirmbacher (ETD 2003, Humboldt)• Hussein Suleman (U.Cape Town, S. Africa)• Shalini R. Urs (U. Mysore, India)• Eric F. Van de Velde (ETD 2001, Caltech)

NDLTD Committees (Chairs)• Awards (John Hagen)• Conferences (Tony Cargnelutti)• Development (Peter Schirmbacher)• Executive (Edward Fox)• Finance (Scott Eldredge)• Implementation (Ana Pavani)• Membership (Shalini Urs)• Nominating (Sharon Reeves)• Standards (Thomas Hickey)• Union Catalog (Vinod Chachra)

Selected Projects / Sponsors• Australia (ADT)• Brazil (BDT, IBICT)• Canada• Catalunya• Chile (Cybertesis)• Germany• India (Vidyanidhi)• Korea• OhioLINK: 79

colleges/univs

• Portugal (National Library)

• South Africa• UK (British Library,

JISC, Edinburgh, …)• UNESCO (especially

Latin America, Eastern Europe, Africa)

• Venezuela

Some Countries

• Australia• Belgium• Brazil• Canada• Chile• China, Hong Kong• Columbia• Finland• France• Germany• Greece• India• Italy• Jamaica• Korea• Lithuania• Malaysia• Mexico

• Namibia• Netherlands• Norway• Poland• Russia• Singapore• S. Africa• S. Korea• Spain• Sudan• Sweden• Switzerland• Taiwan• Thailand• Turkey• UK• USA• Venezuela• Yugoslavia

Some Institutional Members• British Library• Cinemedia• Coalition for Networked Information (CNI)• Committee on Institutional Cooperation (CIC)• Consorci de Biblioteques Universitàries de Catalunya• Diplomica.com• Dissertation.com• Dissertationen Online (Germany)• ETDweb, a Division of Answer4.com• Ibero-American Sci. & Tech. Education Consortium (ISTEC)• National Documentation Centre (NDC), Greece• National Library of Portugal (for all universities)• OCLC Online Computer Library Center• OhioLINK• Organization of American States (SEDI/OAS)• Southeastern Library Network (SOLINET)• UNESCO (www.unesco.org/webworld/etd)

UNESCO and ETDs(by Axel Plathe at ETD2003)

• Promoting the use of the Internet as a tool for disseminating scientific knowledge

• Facilitating the transfer of ETD expertise from developed to developing countries

 

• 1998: Member of the NDLTD Steering Committee• 1999: First UNESCO ETD meeting on ETD internationalisation  

• 2002: “UNESCO Guide to Electronic Theses and Dissertations” 

• 2003: Model training programmes and training courses• 2003: Sponsor pilot projects• 2003: Pilot projects (Africa, Europe, Latin-America)

Convene Local Planning Group

ETD

ETD project participants

• Academic administrators

• Faculty

• Students

• Staff

• Graduate school / provost / registrar

• Information technologists

• Librarians

Build Local ETD Site

Digital Library

Policies

Inspection/Approval

Workshop/Training

ETD

ETD

NDLTD

Computer Resources

Research

Literature

Student Prepares Thesis/Dissertation

Student Defends & Finalizes ETD

My Thesis

ETD

Student Gets CommitteeSignatures and Submits ETD

Signed

Grad School

Graduate School Approves ETD, Student is Graduated

Ph.D.

Library Catalogs ETD, Access isOpened to the New Research

WWW

NDLTD

QuickTime™ and aCinepak decompressor

are needed to see this picture.

http://scholar.lib.vt.edu/theses/available/etd-2227102539751141/

http://www.ngo.org/

Status of the VT Project

• Approved by university governance Spring 1996; required starting 1/1/97

• Submission & access software in place

• Submission workshops for students (and faculty) occur often: beginner/adv.

• Faculty training as part of Faculty Development Initiative

• Over 5000 ETDs in collection – some have audio, video, large images, software, …

Archiving ETDs

• Every 15 minutes back-ups made of not-yet-approved submissions

• Hourly back-ups of newly approved ETDs

• Weekly back-ups of entire ETD collection

• Copies stored on-site and off-site

VT ETD Cataloging

• same as current cataloging policies, except:• author-assigned keywords (not LCSH)• generic (not LC) call no.• fields/subfields as required for computer files• full abstracts

• time savings• cataloger familiar with computer files• equipment, software for word processing• 5 minutes avg. (10-15 minutes for paper TDs)

Library Resources• Hardware: with Apache web server

• Maintenance and security• Started small; now: Sun 2-processor Enterprise 250--Solaris 2.7

• Software• Submission scripts written by DLA

• Includes e-mail notifications to authors, advisors, UMI• Use it too: http://scholar.lib.vt.edu/ETD-db/

• Log files analyzed with Analog • Survey scripts written by DLA

• Data from authors and readers• Use it too: http://lumiere.lib.vt.edu/surveys/

• Search Engine• Started small; now: InfoSeek’s ULTRASEEK

Digital Library Benefits:Low margin, high use

• Incorporate ETDs with other digital library activities• Ejournals, online class materials, digital images, etc.• Additional equipment, staff may not be necessary

• http://scholar.lib.vt.edu/theses/data/setup.html• Use VT programs, scripts, etc.

• http://scholar.lib.vt.edu/ETD-db/• Or other options: DSpace, eprints, bePress, VITAL

• Online accesses vs. circulation of copies• 1990-1994, average circulation per copy per year:• 2.2 for theses, 3.2 for dissertations

Access to VT’s ETDshttp://scholar.lib.vt.edu/theses/

-

500,000

1,000,000

1,500,000

2,000,000

2,500,000

3,000,000

3,500,000

4,000,000

4,500,000

5,000,000

ETD files requested 231,709 483,030 578,152 2,173,420 4,497,199

Abstracts requested 165,710 215,493 260,699 573,149 471,917

1997/98 1997/98 1999/00 2000/01 2001/02

Server log records of accesses to Virginia Tech ETDs

  1997/98 1998/99 1999/00 2000/01 2001/02 2002/03

Requests for PDF files 221.6 K 481 K 578 K 2,173 K 4,497K 7,320 K

Requests for HTML files 165.7 K 215.5 K 260.7 K 400K 472K 368 K

Requests for Multimedia 1.7 K 4.5 K 12.6 K 44 K 169.1 K 121 K

Distinct files requested 6.4 K 21 K 16.4 K N/A 51 K 31.9 K

Distinct hosts served 29.8 K 57.9 K 87.8 K N/A 425 K 681 K

Average transfer/day 156 Kb 219 Mb 382 Mb 945 Mb 2.15 Gb 3.49 Gb

Data transferred 55.6 Gb 78.1 Gb 137 Gb 332 Gb 780 Gb 1.2 Tb

Info Available at VT

• Informationhttp://scholar.lib.vt.edu/theses

• Automated submission system ready for customization

http://scholar.lib.vt.edu/ETD-db/

• Student guidelines, training materials, FAQ's, multimedia educational materials

http://etd.vt.edu

ETD Union Collection (OAI)

VIRTUA

Merged Metadata Collection

ODL (VT)

Virginia Tech ETD Archive

Brazil ETD

Archive

OCLC ETD

Archive

Future: recommender, …

… OAI Data Provider

OAI Service Provider

OAI Harvesting

LEGEND

Union catalog: OCLC

• OCLC will expand OAI data provider on TDs.

• Is getting data from WorldCat (so, from many sites!).

• Will harvest from all others who contact them.

• Need DC and either ETD-MS or MARC.

• Has a set for ETDs.

OCLC SRU Interface

Union catalog: VTLS, VT

• VTLS will enhance search/browse service for ETDs

• Will harvest from OCLC’s set of ETD records

• Will receive through other mechanisms

• Will work with MARC-21 and ETD-MS

• VT will continue to offer experimental services

ETD Union Search Mirror Site in China (CALIS)(http://ndltd.calis.edu.cn – popular site!)

VTLS Union CatalogContent Languages

The VTLS NDLTD Union Catalog has data in 6 different languages. These are: English German Greek Korean Portuguese Spanish

Examples follow

Language = German; hits = 137

Full record display

Relationship with publishers

• Concern of faculty and students that still wish to publish books or journal articles, voiced: campus, Chronicle, NPR, Times

• Solution: Approval Form gives students, faculty choices on access, when to change access condition; use IPR controls in DL

• Solution: by case, work with publishers and publisher associations to increase access• AAP, AAUP• AAAS, ACM, ACS, Elsevier, ...

Summary: ETDs and Publishing

• Early controversies waning• Faculty: prior publication?

• Protective of future academics• Surveys of publishers

• No specific policies largely• Consider submissions individually

• VT ETD Alumni• None had problems getting published

• Authors• Retain some rights, e.g., link to curriculum vitae,

online course materials

ETDs and Copyright

• Author’s rights• Retain copyright!• Rights include reproduction, modification,

distribution, public performance, public display

• Share non-exclusive rights:• Permit library to store / provide access

• Author’s obligations: fair use• Balance factors or get permission

ETDs and Long-term Preservation

• Concerns: Access without paper• Long term preservation• Standard multimedia formats• Adobe support (www.adobe.com/education/etd)

• Free (and compatible) PDF Reader, work on PDF/A• http://scholar.lib.vt.edu/theses/archive.html

• Addressed Concerns

• Cooperatives, e.g., OhioLink, plus LOCKSS • Commercial options

• ProQuest: traditional microfilming, CDs, …• Frequent, regular back-ups available on-, off-site

ETD-MS

• ETD Metadata Standard• XML-encoded metadata standard

(content and encoding) for Electronic Theses and Dissertations (ETDs)

• in part conforming to Dublin Core (DC)

• using RDF

• using UNICODE

• Will specify relationship with MARC

Complex to Simple

MARC ($50) Dublin Core (DC)

+thesis

Added Support by NDLTD

• Links from NDLTD site• ETD individuals support – submit ETD

• ETD discussion (e-prints) – community activities

• Conference papers and presentations – community activities

• Automated support to “join NDLTD”• Members have discount rates for int’l conference• Marcel Dekker book

• Edward A. Fox, Shahrooz Feizbadi, Joseph M. Moxley, and Christian R. Weisser, eds., The ETD Sourcebook: Theses and Dissertations in the Electronic Age, New York: Marcel Dekker, 2004

Spirit of NDLTD• Help make a better (smaller) world• Win-win-win (everyone can benefit)• Have fun helping others• Helpers/teachers learn more than those they work with• Build on standards• ETDs are preservable, popular, expressive, “better”

• Doable, feasible, learnable, affordable, sharable

• Please join and participate in NDLTD!

Part 4

Conclusion

Future Educational DLs

• All are invited to join NDLTD!

• All are invited to broaden and integrate activities for digital libraries for education, toward:

• Institutional repositories at every site.

• Regional, national, and international initiatives integrating research, education, and libraries.

For More Information

• http://fox.cs.vt.edu

• www.dlib.vt.edu

• scholar.lib.vt.edu

• www.ndltd.org

Questions/Discussion?