Many Hands Make Light Work, the American Version

Experiences with User-Text-Correction at California Digital Newspaper Collection (CDNC):

How crowd-sourcing OCR text correction impacts a historic newspaper collection

The California Digital Newspaper Collection contains over 490,000 pages of significant California newspapers published from 1846 to 1922.

The newspapers were digitized to both page and article level METS/ALTO data as part of the National Digital Newspaper Program.

The collection is displayed using Veridian digital library software.

visits per month

minutes per visit

pages per visit

About the Collection

site statistics between Nov. 2010 and Aug. 2011

poor OCR reduces search recall to low levels OCR quality ranges between 50%-90% of word level accuracy

post OCR text correction is expensive

≈ $0.50 per 1000 characters or $5.00 to $10.00 per newspaper page $$

Daily Alta California, 2 January 1850

Like the users of many digital newspaper collections, patrons of the CDNC visit the site for personal reasons, consider themselves genealogists or family historians, and return to the site frequently.

The Average CDNC User users above 40 years old

users who consider themselves genealogists

users who visit the site at least weekly

Wikipedia on Crowdsourcing:

“distributed problem-solving and production model”

“sourcing tasks traditionally performed by specific individuals to an undefined large group of people or community (crowd)

through an open call”

Crowd-Sourcing Projects Project Gutenberg

Family Search The National Library of Australia The National Library of Finland

FreeBMD.org

Site Statistics Since User Text Correction

visits per month

minutes per visit

pages per visit

lines per month corrected by the top corrector

total lines corrected since 2008

total number of text correctors

lines corrected per month in 2011

30,000

49 Million 30,000

‘Engaging with users and building virtual communities is just as important to the users as providing the data itself. They want to be part of a community.’

Rose Holley, The National Library of Australia

2,000,000 +

User Text Correction added to CDNC

Results August 22 - October 22

Users who have corrected text

Lines corrected by top corrector

Total number of lines corrected

Lines Corrected Per Month

Goals

•  Improve OCR text at low cost

•  Improve search precision / recall

•  Build user community

Risks?

•  User text correction of newspapers is (relatively) new

•  Users won’t know what to do, interface is confusing

•  Users don’t understand errors in OCR text

•  Vandalism of text

Benefits

•  Text quality improved

•  Cost effective

•  Community involvement

•  Users empowered

$

User Reaction

“Great feature (I tested it during the beta) for a great site, which I have used extensively.  I plan to use the edit feature when I get back to research in the Los Angeles Herald and the Daily Alta California.”

~Lawrence B.

“STUNNINGLY  FANTASTIC!!!! is what I think!” ~A fifth generation Californian of multiple Forty-niner families

“I have used the new system and like it. The user correction is great idea.”

~Pat

“Exactly what the system needed!!! Pulled up a couple articles in the beta system and made some text corrections. Went back and tried the old system using the words I corrected and it worked!! Outstanding enhancement!”

~Mary B.

“The addition of user text correction (UTC) to the California Digital Newspaper Collection has dramatically improved the quality of the computer-generated text and enlivened our relationship with our

users.  Within a couple of weeks of implementing UTC, and with little publicity, a handful of users had already corrected thousands of lines

of text.  Many of those users emailed us directly with questions about or praise for the UTC, building direct, personal connections between

our staff and users that hadn’t existed before.”

~Brian Geiger, Center for Bibliographic Research, UC Riverside

? Brian Geiger, Director Center for Bibliographic Studies and Research

University of California Riverside bgeiger@ucr.edu

Frederick Zarndt, Chair IFLA Newspapers Section frederick@frederickzarndt.com