REfine a gamifiedplatform for participatory requirements engineering
-
Upload
engineering-of-social-informatics-esotics -
Category
Technology
-
view
139 -
download
0
Transcript of REfine a gamifiedplatform for participatory requirements engineering
REfine: A Gamified Platform for
Participatory Requirements Engineering
R. Snijders, F. Dalpiaz, S. Brinkkemper (Utrecht University, NL)
Mahmood Hosseini, Raian Ali (Bournemouth University, UK)
Atilla Özüm (KPMG Technology Advisory, NL)
Agenda
1. Stakeholders involvement in RE
2. The potential: crowdsourcing, gamification
3. The REfine platform
4. Results from a case study
5. Conclusions and Future Work
REfine: a gamified platform for participatory RE
1. Stakeholders involvement in RE
Software projects keep failing, or being challenged
Results from the 2014 Chaos Standish Report
16.2
31.1
52.7
Successful Cancelled Challenged
REfine: a gamified platform for participatory RE
1. Stakeholders involvement in RE
What are the key determinants for success/failure?
Taken from the 2014
Standish Chaos Report
REfine: a gamified platform for participatory RE
1. Stakeholders involvement in RE
But what can stakeholder/user involvement deliver?
More accurate requirements
Expensive and unnecessary features avoided
Improved acceptance (Kujala, 2003)
Higher quality of requirements
Higher chances of project success (Kujala et al., 2005)
Improved loyalty
Broadened market (Kabbedijk et al., 2009)
REfine: a gamified platform for participatory RE
2. The potential: crowdsourcing, gamification
Involving the crowd! Relying on a crowd of humans to
get things done!
Feature requests (Win10)
Crowdwork (Mturk)
REfine: a gamified platform for participatory RE
2. The potential: crowdsourcing, gamification
Gamify it! The trend is manifest
4Square, Stackoverflow, LinkedIn, etc.
Gamification: “the use of game design elements in non-
game contexts” (Deterding et al., 2011)
Intrinsic kinds of motivation
Personal gratification
Social status
REfine: a gamified platform for participatory RE
3. The REfine platform
An interactive platform to elicit and refine requirements
Crowd-centric: all stakeholders are welcome to participate
Gamified: incentives through game elements
REfine: a gamified platform for participatory RE
3. The REfine platform
Discussing needs via REfine (perform actions)
Agree/disagree with needs
Add comments
“Liking” comments
Branching needs
Actions have a cost
Money obtained
when others find
one’s contributions
useful
REfine: a gamified platform for participatory RE
3. The REfine platform
Leaderboards to determine the most useful participants
General leaderboard (overall points)
Per type of contribution: ideator, assessor, commenter
Per need
REfine: a gamified platform for participatory RE
3. The REfine platform
The CCRE method
From feasibility to focus group
Fits in agile development (sprints)
Highly iterative, despite figure
Main characteristics
Determines if crowd involvement
has a potential
Uses an interactive platform like REfine
Rewards the most active stakeholders
by inviting them to the focus groups
REfine: a gamified platform for participatory RE
4. Results from a case study
The Qubus software product
A governance, risk, and compliance tool developed by KPMG
Under the hood, it hosts a workflow management system
Object of our study: needs for version 7 beta
REfine: a gamified platform for participatory RE
4. Results from a case study
Crowdsourcing preparation
Three-steps invitation: internal stakeholders, clients & users,
off-stage actors
Small crowd involved (19 people)
Crowd involvement
21 needs, 37 comments, 130 votes
Little moderation
One-month period
REfine: a gamified platform for participatory RE
4. Results from a case study
Stakeholder type N Active Needs Comments Votes Points
Avg. Total Avg. Total Avg. Total Avg. Total
Community
manager
1 - 2 2 8 8 21 21 41 41
Product
management
2 67% 0 0 3 6 6 12 13.5 27
Development team 4 100% 0.8 3 3.8 15 14.8 59 26.8 107
Experts 4 100% 1 4 1.3 5 2.5 10 16 64
Client 1 100% 0 0 0 0 0 0 3 3
Users 1 6% 1 1 0 0 3 3 11 11
Off-stage actors 6 86% 1.8 11 0.5 3 4.2 25 20 120
Total 19 50% 1.1 21 1.9 37 6.8 130 19.6 373
REfine: a gamified platform for participatory RE
4. Results from a case study
Questionnaire: comparing with previous experiences in
communicating requirements
0% 20% 40% 60% 80% 100%
Difficulty
Usefulness
Engagement
1 2 3 4 5
3.88 (more engaging)
4.19 (more useful)
2.69 (as difficult)
REfine: a gamified platform for participatory RE
4. Results from a case study
Questionnaire: perceived experience/usefulness of Refine
3.47
2.82
4.12
1.92
4.71
4.41
2.94
REfine: a gamified platform for participatory RE
4. Results from a case study
Evaluation with experts from the industry (generality)
1 = strong disagreement
7 = strong agreement
REfine: a gamified platform for participatory RE
5. Conclusions and Future Work
User involvement is crucial for project success
Potential: crowdsourcing and gamification
The REfine tool
Gamification to maximize stakeholder involvement
Mixed results from a case study
Gamification improves the experience
It is difficult to attract a large crowd
REfine: a gamified platform for participatory RE
5. Conclusions and Future Work
Large-scale studies with more stakeholders
Long-term incentives (real-life gamification)
How to assess the actual effect of gamification?
Ongoing work using user stories
REfine: a gamified platform for participatory RE
Thanks for listening!
Questions?
REfine: a gamified platform for participatory RE
References
S. Deterding, D. Dixon, R. Khaled, and L. Nacke, “From Game Design Elements to
Gamefulness: Defining Gamification,” in Proc. of MindTrek: Envisioning Future Media
Environments, 2011, pp. 9–15
J. Kabbedijk, S. Brinkkemper, S. Jansen, and B. van der Veldt, “Customer Involvement
in Requirements Management: Lessons from Mass Market Software Development,”
in Proc. of RE, Aug. 2009, pp. 281–286
S. Kujala, “User Involvement: A Review of the Benefits and Challenges,” Behaviour &
Information Technology, 2003
S. Kujala, M. Kauppinen, L. Lehtola, and T. Kojo, “The Role of User Involvement in
Requirements Quality and Project Success,” in Proc. of RE, 2005, pp. 75–84
R. Snijders, F. Dalpiaz, M. Hosseini, A. Shahri, and R. Ali, “Crowd-Centric
Requirements Engineering,” in Proc. of the 2nd International Workshop on
Crowdsourcing and Gamification in the Cloud, 2014
REfine: a gamified platform for participatory RE