EVERY

CONNECTIONhas a

starting point.

Andrew K. Pace

Executive Director, Networked Library Services

Web-scale Management:The Promise and Peril of Creating a Next-

generation Library Management System

5th NOVEMBER 2010

Disclaimers (―about me, before OCLC‖)

I am a Librarian with a Humanities background

• BA (Rhetoric), The University of Virginia

• M.S.L.S., The Catholic University of America

I have a long and jaded history with library automation

• Innovative Interfaces, Inc. (1996-1999)

• product manager WebPac, Z39.50 client/server, Electronic Course Reserves, Web Management Reports

• North Carolina State University, Head of IT (1999-2008)

• DRA-to-Sirsi-to-SirsiDynix

• Home grown ERM (―E-Matrix‖)

• Project Director, NCSU Endeca discovery engine

• Library Systems author, speaker, consultant, and provocateur (1994 – present)

• Blogger, ―Hectic Pace,‖ (2006-present)

• Columnist, ―Technically Speaking,‖ American Libraries (2004-2008)

• Contributing Editor, Smart Libraries Newsletter (2003-2004)

• Columnist, ―Coming Full Circle,‖ Computers in Libraries (2000-2005)

Agenda

Web scale, cloud computing, and OCLC

• Libraries before web scale

Web-scale Management Services

• Why?

• What?

“Web-scale management services”

• An opportunity for truly next-generation library management

services

Breaking new ground in Library Automation

Taking things apart and putting things together

Web Scale and Cloud Computing

Cloud Computing

Infrastructure Platform Applications Services

KP

MG

A style of computing in which

scalable and elastic IT-enabled

capabilities are delivered as a

service to external customers

using Internet technologies.-Gartner Group

Simple: Web-based applications with shared data and services.

Why cloud computing is different

“Unlike most innovations that started in the enterprise

and went to the consumer, this innovation started with

consumers and is now coming to the enterprise.”-Geoffrey Moore

“Core Content and the Cloud”

http://www.youtube.com/watch?v=0swJCYLH2C

The Web is all about

scale, finding ways to

attract the most users

for centralized resources,

spreading those costs

over larger and larger

audiences as the

technology gets more

and more capable.

— Chris Anderson,

Editor-in-Chief, WIRED

and author of The Long Tail

The cloud provides scalability

Libraries Before Web Scale

The Timeline

1936

1960

1970

1980

1990

2000

2010

Inte

gra

tion &

sophis

ticati

on

1936

First library automation – punchcard

circulation at University of Texas

1960

Stand-alone modules, separated by

department or function

1970

Mainframe and dumb terminals – the birth

of integration

1980

Coming of age era – the best of the text-

based library systems

1990

GUI, WWW, RDBMS – the great distraction,

or “the beginning of legacy status”

2000

The “dance with the one that brung ya”

era. Making the best of a bad situation.2010

Peril or Promise?: XML, Java, Open Source,

SaaS, cloud computing, web scale. The

“new” ILS.

In the 1970s…

ILS

OPAC

Circulation

CatalogingUsersPrint

Vendors

Library

In the 1980s…

ILS

OPAC

Circulation

CatalogingUsersPrint

Vendors

Library

Self

Service

National/

Global

System

Consortial

System

Cataloging

Utility

Acquisitions

In the 1990s…

ILS

OPAC

Circulation

CatalogingUsersPrint

Vendors

Library

Self

Service

National/

Global

System

Consortial

System

Cataloging

Utility

Acquisitions

A to Z

List

Electronic

Vendor

Resolver

ERM

In the 2000s…

ILS

OPAC

Circulation

CatalogingUsersPrint

Vendors

Library

Self

Service

National/

Global

System

Consortial

System

Cataloging

Utility

Acquisitions

A to Z

List

Electronic

Vendor

Resolver

ERM

Institutional

Repository

Meta-

search

―[We signed on to be a pilot

partner] because this is

pretty much what our library

systems look like.‖

Michael Dula

Pepperdine University Libraries

Why web-scale management

services?

Changing demand

Changing collections

Web scale value proposition

70% 30%INFRASTRUCTURE INITIATIVE

Amazon.com: http://www.slideshare.net/goodfriday/amazon-web-services-building-a-webscale-computing-architecture

Before cloud computing…

Web scale value proposition

70%30%INITIATIVE

Amazon.com: http://www.slideshare.net/goodfriday/amazon-web-services-building-a-webscale-computing-architecture

After cloud computing

INFRASTRUCTURE

Why?

• Libraries are doing more than ever

• Silos, costs, time requirements, inefficiency are hindering

libraries

• Cloud computing has helped other industries

• OCLC members have been building ―cloud‖ services for 40

years (cataloging, resource sharing, online reference…)

• OCLC Web-scale Management brings the power of the

cooperative to core services

• Member libraries will be freed to focus on innovation

A Sea Change

―For 25 years of library automation,

we‘ve had a choice of brand, and now

we have a choice of kind.‖

-Marshall Breeding

Libraries and Web Scale

―Library Web scale‖

Libraries worldwide 1,212,383

Books: physical processing 15,517,196,010

Back-office transactions 61,879,349

OPAC searches 105,607,800,600

Database searches 36,555,852,000

Circulation / ILL 4,983,393,968 + Adds/deletes; patron record maintenance, etc.

____________________________________________________________________

Annual transactions 166,041,975,140

18,954,563 transactions / day 5,265 transactions / second

Worldwide libraries and worldwide library transactions (PER YEAR)

Possible with a handful of commodity servers

Side note (mixing metaphors):

The Cloud is Greener

If all the world‘s library transactions could be

managed with a handful of commodity

servers, what is the current carbon footprint

of the world‘s libraries?

What if…

UsersPrint

Vendors

LibraryOPAC

ILS

Circulation

Cataloging

Self

Service

Acquisitions

Cataloging

Utility

National/

Global

System

Consortial

System

Electronic

Vendor

A to Z

List

Resolver

ERM

Institutional

Repository

Meta-

search

What if…

Data

Library

Users Suppliers

Partners

Library

Users Suppliers

Partners

Efficient storage of data in the cloud:

Common use data

Bib

Holdings

User

Contributed

Common Use Data

Users

Partners

Suppliers

Library

Users Suppliers

Partners

Data by Agreement

Patron

Auth

Circ

Counts

Ordering

LicensesCommon Use Data

Library

Efficient storage of data in the cloud:

Data by agreement

Library

Users Suppliers

Partners

Users

Partners

Suppliers

Efficient storage of data in the cloud:

Private data

Private Data

Data by Agreement

Common Use Data

Patron

C‘Out

Order

Details

Licensing

Library

Data

Library

Users Suppliers

Partners

Roles

Aggregators

Book

Sellers

Jobbers

Local

Users

Local

Visitors

Worldwide

Users

Partners

Libraries

Search

Engines

Utilities

Consortia

Social

Sites

Scaling privacy and security

Web-scale Management Services

for Libraries

Data

Workflow

Accession &

Description

Access,

Circulation, &

Delivery

Discovery

Identification

& Selection

Acquisition

Deaccession &

Preservation

Library

Users Suppliers

Partners

Data

Web-scale Management Components

Cooperative

Intelligence

Unified

Selection &

AcquisitionsCirculation

WorldCat Local

Cataloging

Library

Users Suppliers

Partners

Subscriptions

& License

Management

Workflow

Engine

The Software

1: Discover items in WorldCat and add titles to an order

2: Place your order: email, print, batch

3: Patron discovers titles in WorldCat Local

4: Staff checks out the title to the patron

5: Patron self-service account

Advisory Council, Pilots,

& Early Adopters

Next steps: Strategy

Helene Blowers

Digital Strategy Director,

Columbus Metropolitan Library

John HelmerExecutive Director,

Orbis Cascade Alliance

Jan Ison

Executive Director,

Lincoln Trail Libraries System

David Lankes

Associate Professor,

Syracuse University

Web scale Management Services Library Advisory CouncilStrategic direction, reality checks and moral support

Sarah McHughStatewide Projects

Librarian, Montana State

Library

Mary PiorunAssociate Director,

University of Mass.

Medical Center Library

Tim RogersExecutive Director,

NCLIVE

John TeskeyDirector of Libraries,

University of New

Brunswick

Andrew PaceOCLC, ex officio

Tested by pilot libraries

Orbis Cascade Alliance

• Linfield College

• Washington State University

Pepperdine University Libraries

Idaho Commission for Libraries

• Boundary County District Library

• Payette Public Library

• Cooperative Information Network (CIN)

CPC (Craven, Pamlico, Carteret) Regional Libraries (NC)

Early adopters: Starting the user community

• Simpson University

• St. Thomas University Law Library

• Samuel Roberts Noble Foundation

• University of Tennessee at Chattanooga

• Vanguard University

• Texas A&M San Antonio

Early adopters: Norway

The BIBSYS organization is a government agency under the

Norwegian Ministry of Education and Research and is organized as

a part of The Norwegian University of Science and Technology

(NTNU. In addition to the HE-institutions BIBSYS also provides ILS

services to The National Library of Norway and other

governmental and private institutions.

The BIBSYS Consortium has been active for 37 years. The main

service is a highly Integrated Library System (ILS) built around a

single joint catalogue: More than 100 institutions share the same

bibliographic records with addition of their own holdings and

classification.

Observations so far…

From Early Adopters‘ Perspective—

Foremost concern is not disrupting patrons

Discovery cannot be separated from Management System

A NextGen Catalog as the sole OPAC is a new and sometimes

uncomfortable concept

WorldCat as local catalog is a new and sometimes uncomfortable

concept

• Accustomed to ‗hiding places‘ for some records.

• WorldCat still perceived as union catalog.

• Local display and searching of local metadata in WorldCat

Observations so far…

From WMS Team‘s Perspective—

Already seeing early epiphanies of the power of cloud-based

management.

• Allowing for cataloging further upstream, benefiting patrons.

• Allowing for sharing of traditionally siloed data (e.g. course reserve

lists; vendor data).

• ‗Hidden‘ local collections are shared globally.

Changing library workflows & resourcing.

• Cataloging staff are now Acquisitions staff and vice versa.

• Reference staff role as OPAC searching experts is eroding.

WMS—Scaling Globally

Circulation

Workflow

IDM/UDS

User Interface Cookbook

Acquisitions

Vendor Information Centre

License

Management

KnowledgeBase

WorldCat

Infrastructure

Being Global: WMS Product & Development team

3 continents, 5 countries, 12 cities

Management Platform

WMS—Scaling Savings

More than an application—

Web-scale Management Services

represent a platform

Today‘s problem

• The difficulties of establishing an effective infrastructure for

community collaboration and innovation

• Very little economy of scale

• A real need to move from library solutions and software

development as an individual ―hobby‖ to a core industry

business activity

The Black Box System

And

Library innovation is a

―Cottage Industry‖

Library collections are largely inaccessible and unaddressable.

Why cloud computing is different

“If you think that you know better

than ‘the cloud’, you don’t.”-Geoffrey Moore

“Core Content and the Cloud”

http://www.youtube.com/watch?v=0swJCYLH2C

Towards a developers‘ platform for library

management services

• The cloud should not stifle library innovation—it should

better enable it

• Discovery layer APIs are only the tip of the iceberg

• Next-generation solutions must enable levels of access for

local development and participation unparalleled in the

history of library automation

Cloud Computing

Infrastructure Platform Applications Services

Required OCLC Cloud Infrastructure Web-scale Management Services for libraries

Value-add Definition:

The exclusive difference is the

“network effect” that bringing all

our members together provides.

Our cooperative efforts will

create scale savings and

efficiencies, bring wider

recognition for libraries, provide

cooperative intelligence for

better decision-making, and

provide the platform on which

libraries can innovate.

How the platform works

Your application here

Example: Putting a Bestseller Widget

into AcquisitionsLive Demo anyone?

Installing the Bestseller Widget from the

Application Gallery

Example: Putting a Bestseller Widget

into Acquisitions

Example: Working with external services:

The Amazon Budget Widget

Price: $8.86

Current Budget: 4,573.21 Remaining Budget: 4,357.29Current Budget: 4,573.21 Remaining Budget: 4,348.43

―The application platform will allow us to

develop power tools that could really, really

streamline operations for staff.‖

Kyle Banerjee, Digital Services Program Manager

Orbis Cascade Alliance (pilot participant)

Putting things together:

The value of a library cooperative

Top reasons that OCLC Web-scale Management

Services define a different future

• Concentration of sharable data

• Built by the community: WorldCat, holdings, vendors, etc.

• Added for the community: knowledge base, evaluative content, etc.

• Innovative and enabling technology

• At best, other services are “hosted” in the cloud

• System is Service Oriented Architecture (SOA) by design

• Provides a Platform on which to build extended services

• Built on a real workflow engine

• Global collaborative community

• OCLC is uniquely positioned to add the OCLC network to the data,

services, and infrastructure that is being offered

• Allows cooperative intelligence among libraries

• Allows cooperative workflows between libraries

Top reasons that OCLC Web-scale Management

Services define a different future

• Concentration of sharable data

• Built by the community: WorldCat, holdings, vendors, etc.

• Added for the community: knowledge base, evaluative content, etc.

• Innovative and enabling technology

• At best, other services are “hosted” in the cloud

• System is Service Oriented Architecture (SOA) by design

• Provides a Platform on which to build extended services

• Built on a real workflow engine

• Global collaborative community

• OCLC is uniquely positioned to add the OCLC network to the data,

services, and infrastructure that is being offered

• Allows cooperative intelligence among libraries

• Allows cooperative workflows between libraries

―OCLC is the organization we should be

working with. If we had to start from scratch,

we‘d want to create a nonprofit, member-

driven international organization to oversee

this activity. The concept is right and the

organization is right.‖

John F. Helmer

Executive Director

Orbis Cascade Alliance

(Web-scale Library Advisory Council Member)

OCLC Web-scale as a ―revolutionary approach‖

“...companies distinguish

themselves through

defining different

futures for their library

customers.”

— Marshall Breeding

SCELC Colloquium

11 May 2010

OCLC and Libraries: Inventing the Future

http://www.oclc.org/webscale

pacea@oclc.org

THANK YOU!