WITH CROWD-RE TO BETTER REQUIREMENTS
Transcript of WITH CROWD-RE TO BETTER REQUIREMENTS
© Fraunhofer IESE
WITH CROWD-RE TO BETTER REQUIREMENTS
Dr. Jörg Dörr
© Fraunhofer IESE
2
BA Distance
Number of people
Obstacles (oceans, mountains)
Flexibility
Travel time
Travel comfort
Price
Vehicle(s)
Origin Destination
© Fraunhofer IESE
3
ScalabilityScalability
© Fraunhofer IESE
4
0
50
100
150
200
250
300
350
Max. Bus Passenger Capacity(Standing and Seating)
Articulated
Bi-articulated
Straight-sided design
You
ng
man
JN
P625
0G
Ikar
us
280
Yel
low
Co
ach
Cro
wn
Sup
erco
ach
© Fraunhofer IESE
5
0
100
200
300
400
500
600
700
800
900
1000
Max. Airplane Passenger Capacity(All-Economy Configuration)
Jet engines
Wide-body planes
Bo
ein
g 7
47-X
Air
bu
s A
380-
X
Siko
rsky
IM /
S-40
Do
ug
las
DC
-X
Wri
gh
t B
roth
ers
© Fraunhofer IESE
6
Vehicles Getting People from A to BVehicles Getting People from A to B
© Fraunhofer IESE
7
There Will Always be Exceptions…There Will Always be Exceptions…
© Fraunhofer IESE
8
BA Availability of requirements
Number of stakeholders
Project challenges
Constraints
Time
Quality
Cost
…
Vehicle(s)
No requirements Requirements elicited, validated, and documented
© Fraunhofer IESE
9
0
5
10
15
20
25
1.1.1970 1.1.1980 1.1.1990 1.1.2000 1.1.2010
Recommended Number of Participants
1970 1980 1990 2000 2010
RE Interpersonal Elicitation “Vehicles”RE Interpersonal Elicitation “Vehicles”
Workshops
Focus groups
Interviews
On
line
focu
s g
rou
ps
© Fraunhofer IESE
10
Involving the Crowd in InnovationInvolving the Crowd in Innovation
EvaluateUSAGE
UnderstandPEOPLE
DesignSOLUTIONS
AnalyzeUSER FEEDBACK
.
What RE is Essentially
About
InvolveSTAKEHOLDERS
GatherUSE DATA
.
.
© Fraunhofer IESE
11
Why Ask a Crowd for Their Opinion before Realizing a “Good” Idea?Why Ask a Crowd for Their Opinion before Realizing a “Good” Idea?
Any product available online (potentially) has a “crowd”
Knowing how to approach customers (i.e., manage the crowd) means companies are able to channel and unleash the power of the crowd and possibly gain market advantage
Sometimes requires dealing with hundreds to millions of people!
Can reduce “blind spots” when taking management decisions (from visionary down to the implementation level)
The crowd holds great potential to bring the knowledge and resources a company needs
“Thecrowd isagroupofcurrentorpotentialstakeholders,largeenoughinsizetodisplaygroupbehavior,withacommoninterestinaparticularservice.”
FraunhoferIESE
© Fraunhofer IESE
12
What the Crowd Produces –A new GenerationWhat the Crowd Produces –A new Generation
The internet is a predominantly text-based medium
Emails, social media posts, app store reviews, bug tracker entries, etc.
Social media in 2015
2.1 billion unique users (27% of the world’s population, 68% of the active internet users)
Per minute: 2.5 million Facebook posts, 300,000 tweets, 220,000 Instagram photos, 72 hours of YouTube video content
Reviews
“Only” about 1 in 1,000 users write an app review
But: about 50,000 downloads from the Apple AppStore per minute
Around 10,000 reviews on Amazon per month, just for electronics
Reviews are on average nearly 600 characters long
[Sources: Jeff Bullas, Minimaxir, ACI, BBC, Cisco]
© Fraunhofer IESE
15
What Motivates the Crowd?What Motivates the Crowd?
Social incentives
Recognition: winning a competition, prestige
Altruistic motives: moral obligation, social obligation
Monetary incentives
Bounty, cash reward, contest draw
Entertainment
Gamification elements, exclusive access (e.g., previews)
Personal gains
Improving one‘s own world (e.g., a better working service)
Distorting the public view (e.g., sabotage)
© Fraunhofer IESE
17
Some Common ProblemsSome Common Problems
Selection bias
Different preferences towards providing feedback or being involved
Some are very concerned about their privacy, others contribute (classic crowdsourcing) or even pay money (crowdfunding)
Over- / underrepresentation of stakeholder (sub)groups
100% of the users‘ true needs will never be obtained no “One Truth”
Sabotage
Intentional bias by (unknown) motivations
Loss of nuance
Often forced in a template/wizard, excludes exceptions
Tacit information, e.g.: “This new feature is awful.”
© Fraunhofer IESE
20
Despite the Problems: there is so much potential benefit! Despite the Problems: there is so much potential benefit! We need to achieve systematic user participation
Feedback analysis: observing individuals
Context awareness: observing communities
For this, there is much power and potential in a crowd
Contagious mass behavior, reciprocal relationships
Benefits
Large (representative) sample size statistical analyses
Involve the uninvolved (based on trust and expectation)
Discussions drafting, prioritization, validation, reconciliation
Identify specific members who stand out selection
Identify groups attend to minorities
© Fraunhofer IESE
21
Crowd-based RECrowd-based RE
Analyzes any kind of feedback in a remote setting
Interaction (crowdsourcing, community management)
Text mining verbalized conscious needs
Usage mining unconscious needs
Text & usage mining subconscious needs
Use presently available data (passive crowd) and stimulate data generation (active / activated crowd)
Focus on a specific analysis purposes
“Crowd‐BasedRequirementsEngineeringisasemi‐automatedrequirementsengineeringapproachforobtainingandanalyzinganykindofuserfeedback fromacrowd,withthegoalofderivingvalidateduserrequirements.”
FraunhoferIESE
© Fraunhofer IESE
22
Traditional RE vs. Crowd-based RETraditional RE vs. Crowd-based RETraditional RE Crowd-based RE
Eli
cita
tio
n
Mode Manual, co-present(analyst(s) & stakeholder(s) in same place & time)
Semi-automated, remote(analyst(s) & stakeholder(s) can be in different place & time)
Techniques Co-present: interviews, work-shops, focus groups, etc.
Log file & feedback analysis, prototyping, video conferencing, etc.
Results Verbalized requirements Derived online statements & patterns
Documentation Manual and/or computer-assisted processing
Semi-automated algorithm-baseddistillation, derivation, drafting, and compilation (ranking, prioritization, correlation, clustering)
Validation & Negotiation
Co-present Crowdsourced, remote
Management Iterative within one project Iterative, repeated at leisure
Performer of RE Requirements Engineer System, Requirements Engineer
Ideal sample size
Depends on budget/complexity
Unlimited (the more stakeholders, the higher the validity)
Total duration Typically several months,in various stages
Recurring analysis of static data,continuous crowd management
© Fraunhofer IESE
23
Crowd-based RE is about Processing Feedback!Crowd-based RE is about Processing Feedback!
Traditional RE asks for requirements in a co-present setting
Crowdsourcing in RE asks for requirements in a remote setting
Crowd-based RE analyzes any kind of feedback in a remote setting
Crowdsourcing
Text mining
Data mining (usage mining)
These approaches can be performed in parallel and (iteratively) in series
© Fraunhofer IESE
24
Crowd-based RE Aspect: CrowdsourcingCrowd-based RE Aspect: Crowdsourcing
Crowdsourcing Crowd-Based RE
Description a form of RE outsourcing semi-automated approach to analyze user feedback from a crowd
Role of the crowd problem solvers informants
Stakeholder involvement
active mostly passive
Contribution provided upon request, always directly
provided continuously, often indirectly
Processing of data performed manually performed semi-automatically
Crowd-based RE can integrate crowdsourcing, but not vice versa
© Fraunhofer IESE
25
Integrating different Information for REIntegrating different Information for RE
Track and Observe User Behavior
Identify Associated Behavioral Pattern
Aggregated Results
Quantitative Feedback
Qualitative Feedback
Event Logging Usage Mining
Motivate Users to Provide Feedback
Discover Problems, Needs, and Ideas
Motivational Instruments Text Mining
Derived Requirements
© Fraunhofer IESE
26
Crowd-based RE Aspect: Text MiningCrowd-based RE Aspect: Text Mining
Collect and analyze both actively generated (verbalized) opinions
Manually processing feedback is tedious
Especially when trying to draw parallels, identify trends and topics
NL analysis of text-based user feedback using language queries (specifications of linguistic expressions)
Identify statements from stakeholders
Positive strengths, key selling points, popular features
Negative problems, unfulfilled expectations
Feature requests demands, needs
Structuring the statements
Prioritization, clustering, consistencies, conflicts, outliers
May reveal pressing issues, market trends, innovative ideas, etc.
The outcomes need to be validated before we can speak of requirements
© Fraunhofer IESE
27
Crowd-based RE Aspect: Usage MiningCrowd-based RE Aspect: Usage Mining
Can uncover needs of which users are not (consciously) aware or fail to verbalize
Behavioral patterns (e.g., shortcuts, preferential workflows)
Bottlenecks (frequently encountered problems of users)
Deviant activities (which may spur innovative ideas)
Detect issues that require addressing, new uses for the product (including new markets), and opportunities for optimization
Used in combination with prototyping (cf. A/B testing), it can help:
discover new requirements (even redesigning user interactions)
validate the results from text analyses
reconcile identified conflicts
© Fraunhofer IESE
28
iRequireiRequire
Users are triggered to provide requirements, are guided through the documentation process
Incentive: improving (changing) one’s own world
© Fraunhofer IESE
30
PRO-OPT Project OverviewPRO-OPT Project Overview
Challenges in Collaborative Analytics of Diagnostic Data
Workshops
Integrated Data
Product quality analysis andimprovement
Specific Reports for different use cases
Production / Engineering
Predictive and preventivemaintenance
Improvement of the diagnostics system
3rd Party
Supply‐Chain
Usage Mining Text Mining
Aspects Involving CrowdRE
© Fraunhofer IESE
32
IN ZUSAMMENARBEIT MIT
GEFÖRDERT VOM
KONTAKT
Crowdsourcing Logistik vom Land fürs Land
LOGISTIK
Pakete werden nicht mehr nur über den Paketdienst transportiert, sondern erhalten eine Mitfahrgelegenheit – sogar in privaten Fahrzeugen.
Dr.-Ing. Mario TrappThemenverantwortlicher Smart Rural Areas Hauptabteilungsleiter Embedded SystemsFraunhofer-Institut für Experimentelles Software Engineering IESEmario.trapp(at)iese.fraunhofer.deTelefon +49 (631) 6800-2272www.iese.fraunhofer.de
HANDELDer stationäre Einzelhandel profitiert von gemeinsamen Logistiklösungen und einem komfortablen Einkaufserlebnis seiner Kunden, die gerne regionale Produkte kaufen.
MOBILITÄTSoftware vernetzt Mobilitäts- und Logistiksysteme, dadurch entstehen Dienste aus unterschiedlichen Bereichen und schaffen für alle einen großen Mehrwert.
DIE VISION
© Fraunhofer IESE
34
SummarySummary
Crowd-based Requirements Engineering
helps scale RE to much larger groups of stakeholders by employing a set of usage & text mining tools, as well as techniques to get the crowd to participate systematically
makes optimal use of the large amounts of available knowledge to prevent or uncover blind spots and over- or underrepresentation of stakeholder groups
extends the portfolio of existing RE elicitation techniques
already plays an important role in several projects of Fraunhofer IESE in various branches (emergency response, automotive, public sector, smart ecosystems)
© Fraunhofer IESE
35
References – Crowdsourcing / Crowd-REReferences – Crowdsourcing / Crowd-REOrigins of Crowdsourcing
Howe, J. (2006). The rise of crowdsourcing. Wired, 14(6).
Malone, T. W., Laubacher, R. & Dellarocas, C. (2009). Harnessing crowds: Mapping the genome of collective intelligence. MIT Sloan Research Paper No. 4732-09.
Principles of crowdsourcing
Brabham, D. C. (2008). Crowdsourcing as a model for problem solving: An introduction and cases. The International Journal of Research into New Media Technologies, 14(1), 75–90.
Surowiecki, J. (2004). The wisdom of crowds. New York: Anchor.
Lanier, J. (2010). You are not a gadget: A manifesto. New York: Alfred A. Knopf.
RE and Crowdsourcing
Sutcliffe A. & Sawyer, P. (2013). Requirements elicitation: Towards the unknown unknowns. Rio de Janeiro, Brasil: RE 2013, Research Track.