A Preliminary Field Study of Game Programming on Mobile Devices
-
Upload
tao-xie -
Category
Technology
-
view
453 -
download
3
description
Transcript of A Preliminary Field Study of Game Programming on Mobile Devices
A Preliminary Field Study of Game Programming on Mobile Devices
Sihan Li Tao Xie
University of Illinois at Urbana-Champaign
Eric Anderson North Carolina State University
1
http://arxiv.org/abs/1310.3308
Mining App Stores: Mobile App Markets
Apple App Store Google Play Microsoft Windows (Phone) Apps
Microsoft Research TouchDevelop
o Previous approaches look at permissions code (runtime behaviors)
o What does the users expect?
Mining App Stores: Program Analysis + Text Analytics
3
App Description
App Code
App Permissions
Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/publications/usenixsec13-whyper.pdf
o Previous approaches look at permissions code (runtime behaviors)
o What does the users expect?
Mining App Stores: Program Analysis + Text Analytics
3
App Description
App Code
App Permissions
Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/publications/usenixsec13-whyper.pdf
o Previous approaches look at permissions code (runtime behaviors)
o What does the users expect?
Mining App Stores: Program Analysis + Text Analytics
3
App Description
App Code
App Permissions
Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/publications/usenixsec13-whyper.pdf
o Previous approaches look at permissions code (runtime behaviors)
o What does the users expect?
Mining App Stores: Program Analysis + Text Analytics
3
App Description
App Code
App Permissions
Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/publications/usenixsec13-whyper.pdf
o Previous approaches look at permissions code (runtime behaviors)
o What does the users expect?
Mining App Stores: Program Analysis + Text Analytics
3
App Description
App Code
App Permissions
Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/publications/usenixsec13-whyper.pdf
Perhaps it's my ignorance, but would anything related to storage, system tools, your location, camera, development tools, phone calls, and network communication be ridiculously more accessible to the smartphone capabilities than what a flashlight app would require?
Mining App Stores: Open Source Mobile App Repo
https://f-droid.org/
http://research.microsoft.com/en-us/projects/touchdevelop/
Rapid Increase of TouchDevelop Popularity
5
vs. 17322 Feb this year
Mining App Stores: Educational/Learning Analytics
6
TouchDevelop Team@MSR
TouchDevelop Users
SE/HCI/Education … Researchers
Mining App Stores: Educational/Learning Analytics
6
TouchDevelop Team@MSR
TouchDevelop Users
SE/HCI/Education … Researchers
Mining App Stores: Educational/Learning Analytics
6
TouchDevelop Team@MSR
TouchDevelop Users
SE/HCI/Education … Researchers
Data-Driven Decision Making
(not just gut feeling)
Mining App Stores: Educational/Learning Analytics
6
TouchDevelop Team@MSR
TouchDevelop Users
SE/HCI/Education … Researchers
Data-Driven Decision Making
(not just gut feeling)
Our Previous Study on TouchDevelop Scripts and Users
• Characteristic and longitudinal study – 17,322 scripts and 4,275 users (July 2011-Feb 2013)
• Scripts – Structural features
– Code reuse
– Change of features over time
• Users – Expertise
– Publishing and learning activities over time
7
Sihan Li, Tao Xie, and Nikolai Tillmann. A Comprehensive Field Study of End-User Programming on Mobile Devices. VL/HCC 2013. http://web.engr.illinois.edu/~taoxie/publications/vlhcc13-study.pdf
Previous Characteristic Study
What do TouchDevelop scripts look like?
What is the code-reuse ratio of TouchDevelop scripts?
What kinds of TouchDevelop users are there?
8
How are TouchDevelop scripts changing over time?
What is the users’ progress of developing TouchDevelop scripts?
Structural Features: Findings
9
EMC: # external method call sites
What do TouchDevelop scripts look like?
Structural Features: Findings
73% less than 100 LOC, avg. 133 LOC
Small in size
9
EMC: # external method call sites
What do TouchDevelop scripts look like?
Structural Features: Findings
73% less than 100 LOC, avg. 133 LOC
106 EMC out of 114 MC per script Every two lines contain an EMC
Small in size
Heavily use of external code
9
EMC: # external method call sites
What do TouchDevelop scripts look like?
Structural Features: Findings
73% less than 100 LOC, avg. 133 LOC
106 EMC out of 114 MC per script Every two lines contain an EMC
Implications: Heavily rely on external methods to achieve functionalities. Enriching the built-in APIs and library scripts.
Small in size
Heavily use of external code
9
EMC: # external method call sites
What do TouchDevelop scripts look like?
Types of Code Reuse
• Inherit code from a parent script
• Use the existing library scripts
• Use TouchDevelop built-in APIs
• Copy and paste
10
Types of Code Reuse
• Inherit code from a parent script
• Use the existing library scripts
• Use TouchDevelop built-in APIs
• Copy and paste
10
Code Reuse: Findings
11
What is the code-reuse ratio of TouchDevelop scripts?
Code Reuse: Findings
58% scripts have a parent High code-reuse ratio
11
What is the code-reuse ratio of TouchDevelop scripts?
Code Reuse: Findings
58% scripts have a parent
74% parent-child pairs are written by the same user
High code-reuse ratio
Reuse their own code
11
What is the code-reuse ratio of TouchDevelop scripts?
Code Reuse: Findings
58% scripts have a parent
74% parent-child pairs are written by the same user
Quite a lot less than 10 LOC
High code-reuse ratio
Reuse their own code
Small modifications
11
What is the code-reuse ratio of TouchDevelop scripts?
Code Reuse: Findings
58% scripts have a parent
74% parent-child pairs are written by the same user
Quite a lot less than 10 LOC
Implications: Users tend to reuse code. Lack of code reuse across different users. Improve tool support for code reuse.
High code-reuse ratio
Reuse their own code
Small modifications
11
What is the code-reuse ratio of TouchDevelop scripts?
Expertise of Users: Metrics
• Activity of script-publishing – # scripts published
• Quality of scripts and comments – # positive reviews received
• Grasp of the TouchDevelop language – # TouchDevelop language features used (e.g.,
programming concepts, built-in APIs)
12
Clusters of Users
74.2% 21.5% 4.2%
13
Clusters of Users
74.2% 21.5% 4.2%
13
Expertise of Users: Findings
14
What kinds of TouchDevelop users are there?
Expertise of Users: Findings
74% novices, 22% ordinary users, 4% experts Big differences between users’ expertise
Most users are novices
14
What kinds of TouchDevelop users are there?
Expertise of Users: Findings
74% novices, 22% ordinary users, 4% experts Big differences between users’ expertise
Implications: Give novices high priority when designing tools and services. Provide customized services (e.g., UI) to different kinds of users.
Most users are novices
14
What kinds of TouchDevelop users are there?
Changes of Scripts: Findings
15
How are TouchDevelop scripts changing over time?
Changes of Scripts: Findings
# scripts , # large scripts , # library , code-reuse ratio .
Important increasing trends
15
How are TouchDevelop scripts changing over time?
Changes of Scripts: Findings
# scripts , # large scripts , # library , code-reuse ratio .
Implications: Enriching TouchDevelop language and Improving the management of libraries to better accommodate these trends.
Important increasing trends
15
How are TouchDevelop scripts changing over time?
Evolution of Users: Findings
16
What is users’ progress of developing TouchDevelop scripts?
Evolution of Users: Findings
22.1% users active initially, less active later 9.6% users not active initially, more active later 68.3% users publish one or two scripts, and then leave
Three patterns of publishing scripts
16
What is users’ progress of developing TouchDevelop scripts?
Evolution of Users: Findings
22.1% users active initially, less active later 9.6% users not active initially, more active later 68.3% users publish one or two scripts, and then leave
Implications: Useful to know why users leave or become less active and why they become more active. Design better strategies to retain users.
Three patterns of publishing scripts
16
What is users’ progress of developing TouchDevelop scripts?
Evolution of Users: Findings
17
Evolution of Users: Findings
16.3% users learn most initially, and only a few later 12.4% users learn some initially, and quite a lot at certain point 71.3% users learn a few initially, and stop learning
Three patterns of learning language features
17
Evolution of Users: Findings
16.3% users learn most initially, and only a few later 12.4% users learn some initially, and quite a lot at certain point 71.3% users learn a few initially, and stop learning
Implications: Users have different learning behaviors. Provide an adaptive tutoring system based on the history of users’ language-feature usage.
Three patterns of learning language features
17
Overview of This Study
• Motivation – As mobile gaming grows, it is important to analyze
game applications and how it compares to other applications.
• Characteristic study – 31,699 scripts, including 539 game scripts
• Goal – To discover how TouchDevelop games compare to
non-games – To discover what makes some games more popular
18
Characteristic study on Games
• RQ1: What do TouchDevelop game scripts look like when compared to non-game scripts?
• RQ2: How do games compare to non-games in terms of popularity?
• RQ3: What features do more popular games have that less popular games lack?
19
Research Question 1
• RQ1: What do TouchDevelop game scripts look like when compared to non-game scripts?
20
Structural Comparison
• Finding: Game scripts are longer than non-game scripts.
• Implication: Expanding IDE support could enhance the creation of large scripts.
21
Mean #LOC
Median #LOC
Game 369 272
Non-Game 210 50
Structural Comparison cont.
• Finding: Game scripts typically have more actions and events than non-game scripts
• Implication: IDE expansion could be highly beneficial.
22
Mean # action
Mean # event
Game 11.3 3.2
Non-Game 6.1 1.6
Structural Comparison cont.
• Finding: Game scripts invoke more methods than non-game scripts.
• Implication: Games include more features than non-games. Emphasis on games in marketing could be beneficial.
23
Mean # MC
Mean MC density
Game 347 0.94
Non-Game 165 0.79
Research Question 2
• RQ2: How do games compare to non-games in terms of popularity?
24
Comparison of Popularity
• Finding: Game scripts are more popular than non-game scripts
• Implication: TouchDevelop should market games as important types of scripts.
25
Mean # run
Mean # install
Mean # likes
Game 216 35 1.80
Non-Game 23 3 0.32
Research Question 3
• RQ3: What features do more popular games have that less popular games lack?
26
Game Popularity Comparison
• Metric: Top 25% of frequently run games as popular
• Finding: Popular games are longer than unpopular games
• Implication: Expansion of the IDE could encourage larger scripts
27
Games Mean # LOC
Median # LOC
Popular 462 384
Unpopular 363 255
Game Popularity Comparison cont.
• Finding: Popular and unpopular game scripts have similar numbers of actions and events
• Implication: Large numbers of actions and events may not contribute to popularity
28
Games Mean # action
Mean # event
Popular 14.1 3.6
Unpopular 11.1 3.2
Comparison of Actions and Events
0
2
4
6
8
10
12
14
16
Mean Actions Mean Events
Non-Games Games Popular Games Unpopular Games
29
# action and events
Comparison of Mean and Median Length
30
0
50
100
150
200
250
300
350
400
450
500
Mean Length Median Length
Non-Games Games Popular Games Unpopular Games
# LOC
Conclusion
• Previous study [VL/HCC 13] reports a number of valuable findings
• This study focuses on game vs. non-games popular vs. unpopular games – Games are typically longer and more feature-rich than
non-games – Popular games are typically longer but not necessarily
more feature rich than unpopular games
31
Thank you!
ACKNOWLEDGMENTS. This work is supported in part by NSF grants CCF-0845272, CCF- 0915400, CNS-0958235, CNS-1160603, CCF-1349666, CNS-1318419, a Microsoft Research award, and NSF of China No. 61228203. We thank Nikolai Tillmann for his help and feedback in conducting the study described in this paper.
32
Conclusion
• Previous study [VL/HCC 13] reports a number of valuable findings
• This study focuses on game vs. non-games popular vs. unpopular games – Games are typically longer and more feature-rich than
non-games – Popular games are typically longer but not necessarily
more feature rich than unpopular games
33