MSR populations talk v2.key
-
Upload
matthew-chalmers -
Category
Technology
-
view
132 -
download
0
description
Transcript of MSR populations talk v2.key
A Population Approach to System Design
Matthew Chalmers
560
30
Zippy
FanBook
5
A Population Approach to Ubicomp System Design
Matthew Chalmers
560
30
Zippy
FanBook
5
Plus ça change...
“Users’ control of [ubicomp] systems should be maximised and, given the heterogeneity of likely patterns of use, such systems must also be tailorable by the user. It is not possible to predict the social conventions governing the modes of use of ubiquitous computing that will come into being, so empirical studies of new applications [are] essential.”
Perceptions and Expectations of Ubiquitous Computing: Experiments with BirdDog, a Prototype Person Locator, O’Shea et al., Proc. IEE ITaP, 1991.
Seamful Design: Treasure
Limits, variation and dynamics in wifi infrastructure made into resources for designers and users
Seamful Design: Feeding Yoshi
Limits, variation and dynamics in wifi infrastructure made into resources for designers and users
Seamful Design: Castles
Limits, variation and dynamics in software module infrastructure made into resources for designers and users
Analysis Tools: Replayer
Analysis Tools: Replayer
Analysis Tools: Replayer
Analysis Tools: Replayer
Shifting to iPhones
Shifting to iPhones
Shifting to iPhones
Yoshi (2009)
28GB of quantitative log data representing 18,000 hours of use by 182,000 users, and qualitative data
from 8,000 users
Analysis tools: SGVis
SGVis: Categorising users
SGVis: Categorising users
SGVis: Categorising users
Calculate centroids of each identified cluster
4 categories of users: top players, commuters, static players, beginners
Seamful Design: FlexBook
Mobile social networking app
Dynamic integration of software modules at runtime
Shared via our “module store” server, or ad hoc peer-to-peer
Reviewing and user feedback
Implicit logging of use
Explicit comments and ratings shown in app and Facebook
Post Factory
A software engineering revolution
Conventional models of the software development process all claim that “at some point, a product is delivered to a user community, at which point, for that revision of the software, the design process is over.”Paul Dourish
“When we look at an interactive system as an evolving artifact in use, it follows that the process of design does not end with the delivery of the system to some community of users. Instead, it continues as they use and adapt the system.”
A software engineering revolution
2011: 85% of users had extended v4 with at least 1 add-on
(or module or plug-in or add-in or component or...)
No single structural definition—or class—defines it, and no one pattern of use either
Similar situation in IE, Photoshop, Outlook and beyond...
In structural terms, what is Firefox?
Established approaches are based on program structure being the same wherever and whenever it is used
How to deal with the scale of human and software interaction?
How to reason predictively about structure and use?
How to make tools for designers, evaluators and users that support adaptation and appropriation?
Practice ahead of theory and engineering
A population approach to system design
Class as a varied population of software structures, being changed in the real world of use
Class as population:
Dynamic instead of static
Stochastic instead of deterministic and discrete
Coupled with programs in use, instead of standing apart from them
Complementary to (and to be used along with) existing class concepts
A simple example: EdinburghFestivalGuide
Modules FestivalEvents & FestivalVenues in 100% of instances
Module NextBusToMyNextVenue in 50% of instances and the module QuickestWalkingRoute in 40%
All recorded instances have a variable meanLocationOfUse that shows 90% of its values within the city of Edinburgh
The population approach
Multiple representations of a software class (or type)
Tools to use, create or adapt these representations
Social interactions and practices involving those tools
The traditional representation
A class is a structure, and all instances conform to it
Data structure: variables or state
Code structure: methods, procedures or functions
These form a structural description of potential behaviour and use
Assumption: exact match between the structure of the class definition, and the structure of each instance
The traditional representation
Tests and changes need only be done on one thing at one time: the class structure
What’s true for the class is true for the deployed instances
Methodologically convenient but not consistently true
Borrowing from biology: population thinking
A species is a varied and changing population of individuals
Assumed to be undergoing continual change and evolvution
Each individual member may differ from others, and variation among individuals is natural and vital to adaptation
Described by family resemblances among observable attributes
The species can be described by the set of current (or recently recorded) members: the aggregate of each individual’s genes, size, colour, shape, etc.
A new representation: class as population
A class as a varied and changing population of instances
Assumed to be undergoing continual change and evolvution
Each individual instance may differ from others, and variation among individuals is natural and vital to adaptation
Described by family resemblances of structure and use
A class can be described by the set of current (or recently recorded) members: the aggregate of each instance’s structure, context, use, etc.
Populations: variation and dynamism
What’s true for one instance may not be for another
90% of unmodified instances crash, but those with module A added don’t crash. Overall, 75% of instances crash
Analysis results may be different at different times
We advertise A and the percentages; a week later only 50% crash
Results for different contexts may differ
News of module A spreads faster in one country than another, so in Spain 25% crash while in the US 60% crash
New and old representations
Variation/dynamism in the match between class & instances
Probabilistic statements/models, coupled to ongoing use, at the core of designs, analyses and interactions
Population serves as overarching and connecting concept
Populations with family resemblances
Fit with how ‘classes’ are used in everyday language
Moving away from narrow reliance on necessary and sufficient membership properties
Application to software structures demands a holistic approach
Why ubiquitous computing?
It’s about more than just mobile devices...
Systems that fit with context, interaction and activity
...which are varied and dynamic in ways difficult to predict
The system is part of the user’s context
System interactions, and how users are modelled and presented to others in them, are resources for new user activity
Ubiquitous computing
A holistic view spanning technology, use and users
Mark Weiser: The unit of design should be social people, in their environment, plus your device
Robin Milner: Here we look for synergy between the societal vision on the one hand, and the development of scientific models and engineering principles on the other.
New tools and new interactions
Analysis of modules, configurations and contexts
Which configurations and contexts are popular? Which are problematic? How are they used? Are there clusters that should be separately managed and designed for?
How does my setup compare to my friends’? Are there configurations like my own that doesn’t crash so much? How are they used?
Access to modules and usage data
May support free dissemination (P2P) or retain a degree of control
‘Lead users’ keen to try out new or specially instrumented versions
A third representation: ostension
One or more population members are pointed out as examples
Many potential ways to make an ostension
e.g. a category or cluster selected from a population of instances
From observed use patterns to software structure
MapTool is usually run along with DGPSLocationStream
UKTrainSchedule is frequently logged as being used in UK locations tagged as train station
FestivalEvents and FestivalVenues modules are frequently used with an unofficial module AlternativeGuideToTheFringe when the user is in Edinburgh Fringe Festival venues
SGVis: Categorising users
Girolami: statistical analysis of populations
Inferring significant family resemblances among disparate features of large dynamic populations
Reducing a chosen set of instances to representative code and usage data
Girolami: statistical analysis of populations
Extending RJMCMC and TI approaches to online environment via Dirichlet process approach
Heterogeneous data => non-parametric hierarchical mixture model
Stochastic structures abstracted into formal models
e.g. CTMCs, reliant on MCMC posterior inference
Calder: formal modelling of populations
An online framework set within end-user apps
Real-time, critical feedback to users
Longer-term feedback to evaluators and developers
Calder: role-based interface automata
Methods are classified as Master, Servant or Valets, and as Input or Output :
Master (M) actions must be performed
Servant (S) actions are offered
Valet (V) actions are either servant or hidden
Input (?) actions receive data
Output (!) actions send data
45
Example: a simple GPS system
46
0,0,0,0(c1,c4).(M,S)loc
1,0,0,0
2,0,2,0 2,0,1,0
(c1,c8).(M,S)req
(c8,c5).(M,S)log3,0,0,0 (c1,c8).(S,M)map
c1.M!display
Calder: component replacement
When can we replace/substitute one component by another?
C1 ⊗ C2 ⊗ C3 ⊗... ⊗ Cn ⤳ C1’ ⊗ C2 ⊗ C3 ⊗... ⊗ Cn
C1’ has to offer the environment C2 ⊗ C3 ⊗... ⊗ Cn whatever the environment needs to work
Formalise work by services specified in a temporal logic
C1 ⊗ Env ⊨ Σ : the current system works
C1’ ⊗ Env ⊨ Σ : replace C1 by C1’ and the system still works
Not aiming to check C1’ ⊗ Env ⊨ Σ, but derive it...
Deploy app at a large scale, and collect log data
Infer statistical patterns of module structure and use
Make event-based formal models of module configurations
Make probabilistic judgements about failures/exceptions
Feed patterns and judgements into role-specific tools for awareness and analysis
Support people in these roles in adapting the app and its use
Scenario: FanPhoto
FanPhoto: two core components, for text and photo sharing
We give it out to 100 people
and they start using it
We start developing new modules
one for sharing data via Facebook
one for fast compression and sharing of text, photos and video via MANETs
Scenario: FanPhoto
A few participants are interested in new modules
download them and show off that they are trying them out
Analysts keep track via log data streaming back
start to see a new subcluster forming
973
Scenario: FanPhoto
In the IDE, developers can see the FanPhoto class
Modules used by a minority of users are shown in grey
▽FanPhoto▷TextSharing▷PhotoSharing▷FacebookSharing▷MANETmodule
Scenario: FanPhoto
After some time, developers and evaluators meet to play back several weeks’ use
973
Scenario: FanPhoto
After some time, developers and evaluators meet to play back several weeks’ use
919
Scenario: FanPhoto
After some time, developers and evaluators meet to play back several weeks’ use
5126
23
Scenario: FanPhoto
After some time, developers and evaluators meet to play back several weeks’ use
560
30
FanBook
5
Scenario: FanPhoto
After some time, developers and evaluators meet to play back several weeks’ use
560
30
Zippy
FanBook
5
Scenario: FanPhoto
A developer sets his IDE to do automatic updates
Code defining each new subclass appears
labelled with users’ and analysts’ names
linked to tools to analyse the ostensions that made them
FanPhoto
FanBook Zippy
Ontology as structure
“An ontology is a formal specification of a shared conceptualization” Gruber, via SemanticWeb.org
Traditionally, the core of software design is about finding the perfect hierarchical structure
One structure for all known uses, contexts and people
Adaptation is the problem of changing it
Ontology as process
Population approach couples process and structure
Design process in which structure is a resource for use, and in which use creates or adapts structure
Giddens’ duality of structure, Heidegger’s hermeneutic circle
Part of the process is adaptation of a structure to new contexts and uses
A design requirement for ubicomp... and CompSci?
Designing for duality of structure
Moving from structure to use is commonplace
That’s what happens when you compile code and users run it
The trick is how to move back again
Making code for a class from a set of instances and usage histories
A process that allows for gradual adaptation of a class
Moving between complementary forms of class definition
Each move is (or should be) a social interaction
Code as intension
A class’ code, compiled to create an executable shared among users
Population as extension
The deployed instances, that are adapted by users and which create log data shared with evaluators and developers (and other users)
Sub-population as ostension
A subset of instances selected by one of these people as particularly interesting or useful, and shared among them for comment and use
Reduction to a code signature reflecting family resemblances among the configurations and logs of that subset
Ostension revisited
Most forms of ‘analysis’ afford ostensive definition
A way to focus on a subset of structures, contexts, uses and users
e.g. an evaluator’s video of particular users from a system trial, a user’s list of friends on Facebook, a developer choosing a name of a city to customise design for
Log data allow us to link to the software used
Extend previous work that links from video to log data
A population approach
A combination of theory, design, evaluation and use
Advances in all four needed for success
A circular process designed for duality of structure
Change and human agency as essential elements of the process
Multiple ways of representing a software class
Tools to use, create or adapt these representations
Users’, evaluators’ and developers’ social interactions and practices involving those tools... that feed back into low-level representations