WDL Technical Architecture Working Group (TAWG)

June 2010

Achievements and Recommendations

Co-chaired by Noha Adly, Bibliotheca Alexandrina Babak Hamidzadeh, Library of Congress

TAWG• TAWG is responsible for establishing policies

and guidelines needed to govern the technical aspects of the WDL.

• TAWG will advise the WDL Project Manager and the future Executive Committee on matters relating to – Architecture– Workflow– Standards– Site functionality– Technology transfer

Meetings- April 2007

8 Members• UNESCO• National Library of China• National Library of Russia• Mexico• Bibliotheca Alexandrina• Library of Congress• National Library of the Netherlands• Germany

 

Meetings- April 2007• Defining the objectives and the target audience of WDL• Discuss the merits of a centralized vs a portal model • Discuss workflow models• Define the main aspects of WDL website (Connectivity,

access points, content, search, etc.)• Types of content and ingestion models• Presentation and usage of the website• Intellectual added value• Access (multilingual search and display)• Context (narrative and interpretive content)• Participation (inclusion of social networking)

Members in December 200918 Members:• Bibliotheca Alexandrina• Brown University Library• King Abdullah University of Science and Technology• Central Library, Qatar Foundation• Italian Ministry for Cultural Heritage and Activities• Library of Congress • National Diet Library• National Library of Bulgaria• National Library of China• National Library of Finland• National Library of France• National Library of Israel• National Library of the Netherlands• National Library of Sweden• UNESCO• University of Pretoria• Yale University• National Library of Serbia

The WDL Architecture Overview

The WDL Content Pipeline

The WDL Catalog Application

The WDL Web Application

Akamai NetStorageAkamai NetStorage

Akamai Global Edge NetworkAkamai Global Edge Network

http://www.wdl.org

LC ServerLC Server

Solaris 10Solaris 10

MySQLMySQL JavaJava PythonPython File SysFile Sys

LuceneLucene DJangoDJango

nginxnginxSolrSolr ApacheApache

memcachedmemcached

SquidSquid

Uncached Searches

All Item ContentStatically Rendered HTML

Recommendations - Technical• Ensure the WDL website has the following features

– Reliability– Scalability, especially with the increase amount of content– Has multiple access points– Powerful searching tools

• Multilingual• Metadata and content

– Deliver services to users with low bandwidth• Provide full-text search on metadata and contents• Enabling bulk transfer of metadata• Automation of fetching objects and data• Bandwidth and the transfer of large files• Improve documentation on digitization

Recommendations - Technical• Content transfer, metadata compliance, translation– Document current practice and desired improvements– Review existing tools and specifications.– Adopt tools and workflow that are scalable

• Improving the cataloguing tools to allow partners editing data

• Develop tools for quality control to ensure consistent use of terminology

• Discussions with Wikipedia for linking individual WDL documents with relevant Wikipedia pages

Recommendations - Content

• Current methods and tools used for transferring content and standardizing metadata needs to be more scalable

• Consistent use of terminology for translation: Cooperation with the Translation Working Group.

• Content selection guidance: Cooperation with the WDL Content Selection Working Group

• Improve documentation on metadata standards

• Asking for richer metadata to enrich functionality in the future

Recommendations- Policy

• Developing requirements and policies for data distribution

• Collaboration between WDL and Europeana

• Extending the languages of the WDL beyond the existing seven

Future Considerations• Preservation– Role of WDL wrt providing preservation services– WDL could undertake a survey of existing preservation

practices among partners.– WDL partners with preservation experience train other

partners to build capacity for preservation and digital conversion.

• Sharing the WDL software through licensing, open source, APIs

• Website features– Mobile devices– Building a user interactive community– Allowing users to use and annotate the content– Display options for comparing items

Future Considerations

• TAWG’s role towards new partners and partners with less experience:– Technology transfer– Training– Adopting a country/partner

• Establishing sub-groups to deal with particular issues

• Sharing relevant experiences – Social networking– Preservation – Metadata– Access

Thank You