Digital Library Quality and

User Expectation: Investigating

Assumptions About What Is "Good Enough"

Aaron Brenner Digital Projects Librarian, University of Pittsburgh

abrenner@pitt.eduComputers in Libraries – April 17, 2007

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

An Old Cliché

pick two:

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Plenty of Guidance on Standards

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Implementation Quality Issues

† Incompleteness of selection at collection level

† Incompleteness of digital capture† Inappropriate or insufficient descriptive

standard† Mistakes in transcription or uncorrected

OCR † Hastily researched or inappropriate

metadata† Insufficient user support

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

A Gentle Example:

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

An Uglier Example:

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Even the Resource-Rich Make Compromises:

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Dissatisfaction with lapses in quality

>Appreciation for free and unique digital library content

Current User Equation

The Good News:

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Users get most upset when

1) it’s an incorrect fact2) it's personal 3) no one takes responsibility“There is a listing for seven individuals living in a household in the 2nd Ward of Allegheny City all with the surname "GREEN" [...] I know conclusively that the spelling of the name should be "GREER" and that the transcriber read the last letter as "N" rather than "R".  SARAH GREER is my great-grandmother and she is living with her siblings (both parents were deceased). [...] You would do a service to the GRIER descendants to make the necessary change to your database.”

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Users also don’t like a lack of completeness:

“The atlases are also extremely useful to me. I would like to see one for the South Side around 1890 or 1900 (if one is available), as you now only cover 1872 and then there is a gap to 1916.”

“I have found the census information available on your site to be invaluable. Will census information past 1880 be available any time soon?”

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Principle:Avoid presenting users one-

off, incomplete, or unsupported

digital collections

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Principle:Make it easy for users to

give feedback on quality issues

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Principle:Give extra care to quality

when you data is not visible

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Principle:Quality problems linger in inverse proportion to an

item’s popularity

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Courtesy of Mark Phillips, University of North Texas Libraries

Digital Library Quality and User ExpectationAaron Brenner, University of Pittsburgh – Computers In Libraries, April 17, 2007

Other Quality Strategies Involving Users

Run logged user queries against a known sub-set of your dataagainst your complete data, review

those returning no resultsLook for patterns in user complaintsIn collaborations, encourage

“ownership” of quality with partners/users