Engaging the public in tagging and researching the UK’s paintings: two case studies

Andrew Greg, Director, National Inventory Research Project, University of Glasgow

andrew.greg@glasgow.ac.uk

Willem van Haecht (1593-1637), The Gallery of Cornelis van der Geest, 1628, oil on panel, 100 x 130 cm, Rubenshuis, Antwerp

The projects• National Inventory Research Project: NICE

Paintings http://www.vads.ac.uk/collections/NIRP/index.php

• The Public Catalogue Foundation: Your Paintings http://www.bbc.co.uk/arts/yourpaintings/

• Your Paintings Tagger http://tagger.thepcf.org.uk/

• Art Detective http://www.thepcf.org.uk/artdetective/

National Inventory Research Project NICE Paintings

• National Inventory Research Project (NIRP) set up 2001 to aid research in regional museums and improve access to collection information

• Initiated by the National Gallery, London, and a steering committee – now the Advisory Committee for Research on European Paintings

• Now based in University of Glasgow, NIRP’s first project 2004-7: 8,000 paintings in 200 UK collections

• 2009 work with the V&A, the Royal Academy and Apsley House

• Over 9,000 paintings and newly researched records now online as NICE Paintings: the National Inventory of Continental European Paintings

• New project 2014-15 NIRP in the North, working with four major collections: Liverpool, Manchester, Leeds and York.

32 project researchers and Neil MacGregor Scholars (below, with Neil MacGregor and Dr Susan Foister of NG), 12 now in curatorial posts, 7 in academic art history posts and 3 freelance art historians.

Core record:David Teniers II, Interior of a Tavern, oil on panel, Shipley Art Gallery, Gateshead. http://www.vads.ac.uk/large.php?uid=84081&sos=1

Full record: David Teniers II, Interior of a Tavern, oil on panel, Shipley Art Gallery, Gateshead. http://www.vads.ac.uk/large.php?uid=84081&sos=1

The Public Catalogue FoundationYour Paintings

Your Paintings comprises images and basic information on all 211,000 oil paintings in 3,000 UK collections ranging from hospitals to national museums. Originally in book form.

Data collected by PCF:• Artist name and

attribution qualifier (‘after’, ‘attributed to’, etc.)

• Title• Production date, if known• Medium• Size• Accession number • Acquisition method

Above: Public Catalogue Foundation, Paintings in Public Ownership. West Yorkshire: Leeds, 2004, p.31

Your Paintings now on line hosted by BBC.

• Limited information, but

• Links to other works by the artist

• To other works in the collection

• To more detailed NICE Paintings data and

• To Art Detective.

Right: painting page from Art Detective

Additional data requirements for effective searching:• Structured content

keywords• Names of people,

places and events• Types, or genres of

paintings• Dates and periods• Styles and artistic

movements• Social tags

The tasks and constraints:• 200,000 paintings• respond to user

research • reliable and consistent • minimise cost • short timescale

http://www.brooklynmuseum.org/opencollection/objects/1452/Kiss_Me_and_Youll_Kiss_the_Lasses

Two examples of other museums collecting public tags:

1. Detail of object page from Brooklyn Museum with ‘Posse’ tags on the right.

2. Detail of object page from Indianapolis Museum of Art using the ‘Steve Tagger’ with tag cloud.[N.B. Accessed 10 Jan 2012. New page accessed 8 Oct 2012 has a different layout with the tags at the bottom.]

Working towards more structured tagging:

1. Match data requirement to potential tagger groups

• No expert knowledge required– Social tagging– Types– Subjects

• Local knowledge useful– Names (people, places

and events)• Art history knowledge

required– Dates– Styles and movements

The solution: crowd sourcing

© Glasgow Life

Your Paintings Tagger

Your Paintings TaggerSearch for a crowdsourcing model

The model, Galaxy Zoo:• Launched July 2007• 1,000,000 images of

galaxies• 150,000 worldwide

volunteers in first year• 50,000,000 classifications in

first year.

Key principles:• No expertise required• Images delivered at random• Multiple classifications.

Results as good as professionals.

www.galaxyzoo.org

Your Paintings TaggerWorkflowsPublic workflows• Social tagging (‘things and ideas’)• Types• Subjects• Names (people, places and events)Expert workflows• Dates• Styles and movements

Quality assuranceThresholds• 15 taggers per painting• Tags accepted after: – Things: 2 or more taggers use a term– Names: 2 or more taggers use a name– Types: 4 or more taggers select a type– Subjects: 2 or more taggers select a

subjectReferrals to ‘Supervisors Interface’– Specific problem subjects– Two or more names selected in one

painting– Abstracts, portraits and still lifes

Launched in June 2011.

As of March 2013:

• 9,090 registered taggers of whom 671 (7.3%) are ‘expert’ taggers

• 40% have tagged fewer than five paintings

• Only 15.5% have tagged more than 50 paintings

• 49 of these (0.5% of total) have tagged over 1,000 paintings each

• Over 3.5 million tags have been created

• Over 23,000 paintings have been completed.

25.7%

17.3%41.5%

13.0%

1.4% 0.6% 0.5%

No pictures tagged1-4 pictures tagged5-49 pictures tagged50-249 pictures tagged250-499 pictures tagged500-999 pictures tagged1,000 or more pictures tagged

Productivity of taggers March 2013

Your Paintings Tagger

Screenshots from Your Paintings Tagger, 27 August 2014

Your Paintings TaggerSupertaggers:

• Tags for 23,000 paintings delivered to BBC• Now about 50-100 taggers

active every day• The first 2,278 completed

paintings received an average of:– 61 different free text tags, of

which 22 were accepted– 15 different subject terms

(excluding ‘Other’), of which 7 were accepted– 87 different tags and

keywords of which 30 were accepted.Right: detail of painting page on Your Paintings

showing 36 public tags for: Emily Mary Osborn (1828-1925), Study for 'Nameless and Friendless', 1857, oil on panel, York Museums Trust

Your Paintings Tagger

Your Paintings TaggerWork with Oxford Visual Geometry Group, which is developing image recognition software.

Abstract:“The objective of this work is to find objects in paintings by learning object-category classifiers from available sources of natural images. Finding such objects is of much benefit to the art history community as well as being a challenging problem in large-scale retrieval anddomain adaptation…”

Image from: Elliot J. Crowley and Andrew Zisserman, Visual Geometry Group, Department of Engineering Science, University of Oxford, ‘In Search of Art’, Workshop on Computer Vision for Art Analysis, ECCV, 2014

http://www.robots.ox.ac.uk/~vgg/publications/2014/Crowley14a/crowley14a.pdf

Your Paintings Tagger

Benefit of automated tag creation for Your Paintings Tagger, through manual check:

1. Taggers shown computer-selected images and confirm identification

2. In two months 250 taggers confirmed nearly 1 million tags for over 200 subjects in 99,600 paintings

Art DetectiveAims:• Improve the knowledge

collection managers have about the art in their care

• Improve communication between curators and sources of expertise

• Engage the public in discussions around art history, historical research and connoisseurship.

A free-to-use online interface bringing together:• Curators in search of

specialist information regarding their collection

• Specialist knowledge from academics, the art trade and other experts

• Interested members of the general public.

Above: screenshot of early design of home page for Art Detective

Art Detective

• Anyone can browse Art Detective without signing in

• To make a comment they must sign up with name and email address

• To start a discussion they must enter via a specific painting page on Your Paintings (right).

Right: some of the regional and thematic groups on Art Detective

Art Detective• Discussion topics on

Art Detective are linked to special interest groups with expert group leaders

• Users can subscribe to groups and receive updates on new discussions in areas that interest them

• Users can follow particular discussions.

Screenshot of Art Detective Discussions page with most recently commented upon discussions from top

July 2014:

5176 unique users (average time on site 4.5 mins)

47% returning visitors (average time on the site 8 mins)

845 registered users (i.e. contributors)

Art Detective

A discussion on Art Detective

Art DetectiveSome successes: identifications of a doctor through archival and newspaper research; and of some back gardens through local knowledge.

Art DetectiveAnd two outstanding mysteries, several months old: the identity of an obscure subject, possibly connected to the Venetian poetess Veronica Franco; and the identity of a minor artist, possibly the Dutchman Ary de Vois (c 1632-1680) or the obscure English gentleman artist Henry Gibbs (1631-1713).

Engaging the public in tagging and researching the UK’s paintings: two

case studiesThank you

• National Inventory Research Project: NICE Paintings http://www.vads.ac.uk/collections/NIRP/index.php

• The Public Catalogue Foundation: Your Paintings http://www.bbc.co.uk/arts/yourpaintings/

• Your Paintings Tagger http://tagger.thepcf.org.uk/

• Art Detective http://www.thepcf.org.uk/artdetective/

Andrew.greg@glasgow.ac.uk