Nichesourcing: Harnessing the Power of Crowds of Experts

Victor de Boer, Michiel Hildebrand, Lora Aroyo, Pieter De Leenheer, Chris Dijkshoorn, Binyam Tesfa, Guus Schreiber

EKAW 2012, Galway, Ireland. 8-12 Oct. 2012

Rijksmuseum Prints

Img: Flickr/ only_point_five

What is this? Iris

What is this? Castle “Wijk bij Duurstede”

A colossal effort

For Print Room Online 11 professional cataloguers are expected to describe 20% of the collectionduring the 3 years of the project.

Also, in some cases they lack the specific expertise

Crowdsourcing?

Not suited for this knowledge intensive taskNot ‘a castle’, but castle ‘Wijk bij Duurstede’Not ‘a flower’, but ‘Iris’, or even more preciseNot ‘frog’ but the symbolic meaning of a frog in a

Japanese print

High quality annotations

Long-term motivation

In general, we observe that there is

(i) a growing demand for solving also complex knowledge-intensive tasks and

(ii) a natural expectation to focus on the quality of the final result.

Oomen, J., Aroyo, L.: Crowdsourcing in the cultural heritage domain: opportunities and challenges. Oded Nov, O.A., Anderson, D.: Dusting for science: motivation and participation of digital citizen science volunteers.

Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on amazon mechanical turk. Gligorov, R., Hildebrand, M., van Ossenbruggen, J., Schreiber, G., Aroyo, L.:

On the role ofuser-generated metadata in audio visual collections

Human-based computation

Nichesourcing

CrowdsourcingProfessional effort

Niche groups of amateur experts with shared characteristics

ACCURATOR - Nichesourcing Specific Knowledge for Cultural Heritage Institutions

Step 1: identify the main topics for annotation from Rijksmuseum prints

Step 2: allocate the annotation to specific niche groups

Step 3: Gather and rate the quality of annotations provided by niches

Step 4: Select the most complete and accurate annotations

Specific windmill

Adapted from C. Dijkshoorn, M Leyssen, A Nottamkandath, J. Oosterman, M Traub [SEALINCMedia project ]

Specific boat

Crowd vs. Niche

Img: Flickr/creativejenna

The Atomic Task: Simple vs. Knowledge-intensive

Crowdsourcing for simple atomic tasks.

Nichesourcing for atomic tasks that a) require specific knowledgeb) require more effort

Product: Quantity vs. Quality

Success of crowdsourcing lies in quantityQuality can be achieved through either careful application design or post processing.

In nichesourcing the expertise and intrinsic courtesyleads to higher quality individual results

Resource Pool: Crowd vs. Community of Practice

Motivating the crowd requires careful design (gamification, payment,…)

In a niche, we expect intrinsic motivation throughconnection with the task, shared goal or context.

Furthermore, we can exploit the social relations within the community of practice for motivation

altruism,reputation

Second Case Study:Who wants to be a regreening super hero?

Photo credit: Chris Reij

Nichesourcing for pluvial data digitization for the Sahel

[W4RA-related M.Sc. thesis by Binyam Tesfa]

Nichesourcing challenges

• Identification of and access to niche(s)

• Task distributionUse intrinsic social connections cf. expert finding

• Adequate motivation reputation, altruism,…

• Quality assurance within peer networksTrust, reputation

Thank youv.de.boer@vu.nl@victordeboer

Web for regreening in Africa (W4RA)w4ra.few.vu.nl

SEALINCMedia project sealincmedia.wordpress.com