Bratislava WS - Schlarb - ONB - technical tools_pdf
-
Upload
impact-centre-of-competence -
Category
Education
-
view
889 -
download
0
description
Transcript of Bratislava WS - Schlarb - ONB - technical tools_pdf
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May, Bratislava
The challenges of historical materials and an overview on the technical solutions in IMPACT
Sven Schlarb, Austrian National Library
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Overview Challenges Techical solutions Integration
– Interoperability– Modularisation
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Challenges of historical materials Warped book pages (caused by thick spines) Skewed and distorted scans Curved text lines (caused by creased or due to humidity warped
paper) Annoying colour blots, different print intensities Shine through and bleed through Gothic font Handwritten annotations Complex layouts (e.g. newspaper pages and the article reading order) Historical languages and time-specific words in the documents
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tables – Curved cell bordersChallenges
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Extreme warping Gothic font Annotations Chapter numbers
Challenges
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Warpage due to humidity Distortion Crinkles Dots and blots Page/Chapter numbers
Challenges
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Complex layout Reading order
Challenges
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Skewed image Gothic font Bleed through Page number
Challenges
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Gothic font Warping Page borders Curved text lines Page/Chapter numbers
Challenges
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Consortium (including new partners)• 11 Libraries
– Koninklijke Bibliotheek (Netherlands)– British Library– Bibliothèque national de France– Deutsche Nationalbibliothek– Bayrische Staatsbibliothek– Niedersächsische Staats- und
Universitätsbibliothek Göttingen– Österreichische Nationalbibliothek– Universitätsbibliothek der Universität Innsbruck– “St. Cyril and Methodius” National Library, Sofia– National Library of the Czech Republic, Prague– National Library, Madrid (Spain)
• 2 Industry partners– IBM (Research Centre Haifa, Israel)– ABBYY (Moscow)
• 13 Universities and Research Centres– Instituut voor Nederlandse Lexicologie, Leiden
(Netherlands)– National Research Centre Demokritos, Athens– University of Salford, Great Britain– University Munich, Centrum für Informations-
und Sprachverarbeitung (CIS), Germany– University Innsbruck (InfMath), Austria– University Bath, Great Britain– Institute for Parallel Processing, Bulgarian
Academy of Sciences– Jožef Stefan Institute, Ljubljana (Slovenia)– Institute of the Czech National Corpus,
Charles University Prague (Czech Republic)– Analyse et Traitement Informatique de la
Langue Française (ATILF), Nancy (France)– Foundation Virtual Library Miguel de
Cervantes, Alicante (Spain)– Poznan Supercomputing and Networking
Center, Poznan (Poland)– University of Warsaw, Department of Formal
Linguistics, Warsaw (Poland)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Border detection/removal
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Geometric Dewarping
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Geometric Deskewing
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Binarisation
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Historical Lexicons Lexicons for German, Dutch, English, French, Spanish, Polish,
Bulgarian and Czech available. Tools for building historical lexicons Interface to ABBYY FRE to integrate external lexicons
Basically ABBYY provides the information on how the weighing parameters of word lists with word frequencies have to be created.
Procedural information disclosed but results can be evaluated against each other, e.g. by evaluating the results with or without or with different dictionaries against each other.
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Named Entities Registry Named entities (=
persons names, geographic locations, organizations) and general
Collaborative Named Entities Registry
Named Entities to be integrated into ABBYY FR as word lists
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Linguistic Post-Correction
OCR (ABBYY) and OCR analysis (CIS group, LMU) The colors indicate different types of analysis results, like a
word being found in the historical or hypothetical dictionary, or a supposed OCR error, etc.
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Collaborative Correction Integrated web-
based system for collaborative post-correction of OCR results
Character/Word/Page modi
Main purpose: Collaboratively correct OCR errors and use results for improving OCR
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Functional Extension Parser Recognition of the structure
of book pages– Print space– Standard font of the main
text– Page numbers
Enrichment of OCR results with structural information
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Tools: Word Spotting Alternative technique for indexing
historical documents After word segmentation relevant
words are detected and highlighted Key words can be person and
location names (e.g. taken from the Named Entities Registry)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Interoperability ABBYY XML METS/ALTO AltoEx (IBM) PAGE XML (TEI)
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
7 May 2010, Bratislava
Modularisation
IMPACT is supported by the European Community under the FP7 ICT Work Programme. The project is coordinated by the National Library of the Netherlands.
http://www.impact-project.eu