Mobile operating systems - Application Benchmarking
-
Upload
nicolas-demetriou -
Category
Technology
-
view
363 -
download
0
description
Transcript of Mobile operating systems - Application Benchmarking
![Page 1: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/1.jpg)
Mobile Operating Systems: Android
Nicos Demetriou
Operating Systems for New Architectures Fall Semester 2012
![Page 2: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/2.jpg)
Outline
• Application Development
• Experiments
– Emulator, VM, Real device
– Metrics
– Applications
– Results
• Conclusions
2
![Page 3: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/3.jpg)
Android OS - Application development
• Tools needed: – Android SDK – Java JDK, Eclipse – Android Development Tools plugin
• Application components: – Activities: Interactive Tasks – Services: Non-interactive Tasks – Content providers: Database Server – Broadcast receivers: Capture event responses – Intents: Component activation mechanism
• Device Emulator inside Android SDK
3
![Page 4: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/4.jpg)
Android OS - Application development
4
![Page 5: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/5.jpg)
Android OS - Application development
• Graphical User Interface
– Layouts and Views (Widgets)
– Only the main thread can directly call View methods
– Call-back mechanism for other threads to command the execution of a View method in the main thread
• Manifest File (AndroidManifest.xml)
– Describes the application and components used
5
![Page 6: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/6.jpg)
Android OS - Application development <manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.android.notepad" >
<application android:icon="@drawable/app_notes“ android:label="@string/app_name" >
<provider android:name="NotePadProvider"
android:authorities="com.example.notepad.provider.NotePad" />
<activity android:name="NoteEditor"
android:theme="@android:style/Theme.Light"
android:configChanges="keyboardHidden|orientation">
<intent-filter android:label="@string/resolve_edit">
<action android:name="android.intent.action.VIEW" />
<action android:name="android.intent.action.EDIT" />
<action android:name="com.android.notes.action.EDIT_NOTE" />
<category android:name="android.intent.category.DEFAULT" />
<data android:mimeType="vnd.android.cursor.item/vnd.google.note" />
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.INSERT" />
<category android:name="android.intent.category.DEFAULT" />
<data android:mimeType=“vnd.android.cursor.dir/vnd.google.note" />
</intent-filter>
</activity>
</application>
<uses-sdk android:minSdkVersion="3" android:targetSdkVersion="4"/>
</manifest>
6
![Page 7: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/7.jpg)
Android OS - Application development
• Bin/
– classes/ holds the compiled Java classes
– classes.dex holds the executable created from the compiled Java classes
– nameapp.ap_ holds your application’s resources, packaged as a ZIP file
– nameapp.apk is the actual Android application (where nameapp is the name of your application)
7
![Page 8: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/8.jpg)
Android OS - Application development
• src: It contains all source code file (.java file) of android application
• res/
– drawable: it contains images or pictures
– layout: contains XML files that contain the UI layout used in Project or view window of an application
– values: Arrays, colors, dimensions, strings, and styles are declared in XML files
– raw: it contains non-complied file (e.g. Audio file .mp3, video file .mpg)
8
![Page 9: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/9.jpg)
Android OS - Application development
• assets: External content reference
• libs: Holds any third-party Java JARs your application requires
• gen:
– The R.java file is an index (id) into all the resources defined in the file
– This class files are used in source code file or it gives for reference/location user interface object that is used in the source code
– Automatically generated files
9
![Page 10: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/10.jpg)
Android OS - Application development
10
![Page 11: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/11.jpg)
Android OS - Application development
• Deploying an Android application to apk file
11
![Page 12: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/12.jpg)
Focus on • Testing applications on:
– SDK Emulator
– Real device
– Virtual Machine running Android
• Share experience
• Compare metrics: – CPU usage
– Memory
• Using low level commands (adb)
12
![Page 13: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/13.jpg)
Experimenting – SDK Emulator
• Supported in all operating systems
• Android Emulator
• Supports various versions of Android
• Functionality of real device (buttons, keyboard)
• Android Virtual Device manager – Customize device
• Use shell commands through terminal
• Used along with Eclipse to debug applications
• Device used: Nexus One with Android 2.3/4.0
13
![Page 14: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/14.jpg)
Experimenting – SDK Emulator
• Android Virtual Device manager
14
![Page 15: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/15.jpg)
Experimenting – SDK Emulator
• How to execute it
1. Run as an Android Application in Eclipse
2. AVD Manager
3. Through terminal: ..\Android\android-sdk\platform-tools>adb devices
..\Android\android-sdk\tools>emulator -avd AndroidEmulator
• Basic Commands
a) Load .apk files in android-sdk\platform-tools b) ..\Android\android-sdk\platform-tools>adb install [apk]
c) adb shell
d) adb pull <file>
e) adb push <file>
15
![Page 16: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/16.jpg)
Experimenting – SDK Emulator
16
![Page 17: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/17.jpg)
Experimenting – Virtual Machine
• VMWare Player with Android 2.2
• Android OS Image free for various hardware – http://www.android-x86.org/download
• Live CD or installation on disk
• Minor changes to add network support
• Little complicate to add USB flash as SDCard
• Install applications from SDCard or internet
• Customizable hardware
17
![Page 18: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/18.jpg)
Experimenting – Virtual Machine
18
![Page 19: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/19.jpg)
Experimenting – Real Mobile Device
• Model: Huawei Sonic U8650
• Android v2.3.3 (Gingerbread)
• Install applications from Google Play or SDCard
• More stable
19
![Page 20: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/20.jpg)
Experimenting – Real Mobile Device
20
![Page 21: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/21.jpg)
Experiments – Specifications
HUAWEI Emulator VM CPU 600MHz (ARMv6) 204MHz 1,60GHz
Memory 177MB 504MB 882MB Disk 160MB 197MB 2GB
Display Resolution 320x480 480x720 800x600 Android OS v2.3 v2.3/4.0 v2.2
21
![Page 22: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/22.jpg)
Experiments - Metrics
• CPU
• Memory
• Filesystem / IO
• Graphics
• Network utilization
• Evaluation
– Benchmark Applications
22
![Page 23: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/23.jpg)
Experiments - Metrics
• Benchmark Applications
23
Application CPU Memory I/O Graphics Network
Quadrant Standard Edition x x x x
Linpack for Android x
RealPi BenchMark x
BenchMark x x x x
0xBench x x x x x
MemBench x
An3DBench x
SpeedTest x
![Page 24: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/24.jpg)
Experiments - Quadrant Standard Edition
• CPU, I/O, 3D graphics Benchmark
• Compares your device with other brand devices
24
![Page 25: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/25.jpg)
Experiments - Quadrant Standard Edition
25
0
2000
4000
6000
8000
10000
12000
CPU Memory IO Graphics
Quadrant Standard Edition
VM
Emulator
Emulator 4.0
Real Device
![Page 26: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/26.jpg)
Experiments – Linpack for Android
26
• CPU benchmark
• Measure of system’s floating point computing power
– N x N linear equations Ax = b
• Reflection of the Android Dalvik
• Results in MFLOPS
![Page 27: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/27.jpg)
Experiments – Linpack for Android
27
0
10
20
30
40
50
60
70
80
MFLOPS Time (s) Norm Res
Single Thread
VM
Emulator
Android 4.0
Real Device
0
20
40
60
80
100
120
140
160
180
MFLOPS Time (s) Norm Res
MultiThread
VM
Emulator
Android 4.0
Real Device
![Page 28: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/28.jpg)
Experiments – RealPi Benchmark
• CPU and memory performance
• Calculates the value of Pi to many decimal places
• AGM + FFT formula (Arithmetic Geometric Mean
28
![Page 29: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/29.jpg)
Experiments – RealPi Benchmark
29
0,00
500,00
1000,00
1500,00
2000,00
2500,00
3000,00
3500,00
Time for 1000000 digits (s)
RealPi Benchmark
VM
Emulator
Android 4.0
Real Device
![Page 30: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/30.jpg)
Experiments - Benchmark
• 2D Graphic performance
• CPU performance
• Memory performance
• File system performance
30
![Page 31: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/31.jpg)
Experiments - Benchmark
31
0,00
500,00
1000,00
1500,00
2000,00
2500,00
3000,00
3500,00
Totalgraphics
score
Total CPUscore
Totalmemory
score
Total filesystemscore
Benchmark
VM
Emulator
Android 4.0
Real Device
![Page 32: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/32.jpg)
Experiments – 0xBench
• Comprehensive benchmark suite
• C library and system call – LinPack
– SciMark2
• OpenGL
• 2D Canvas
• Garbage collection in Dalvik
• JavaScript engine
• Open source
32
![Page 33: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/33.jpg)
Experiments – 0xBench
33
0
10
20
30
40
50
60
VM
Emulator
Android 4.0
Real Device
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
GarbageCollection
VM
Emulator
Android 4.0
Real Device
![Page 34: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/34.jpg)
Experiments - MemBench
• Simple Memory Benchmark
• Integer array copy and sum
• Runs 4 times
34
![Page 35: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/35.jpg)
Experiments - MemBench
35
0
10
20
30
40
50
Copy MB/sec Add MB/sec
MemBench
VM
Emulator
Emulator v4.0
Real Device
![Page 36: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/36.jpg)
Experiments - An3DBench
• Based on jPCT 3d engine
• Runs 7 tests from fill rate to complex scenes
36
![Page 37: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/37.jpg)
Experiments - An3DBench
37
0
10
20
30
40
50
60
70
High ObjectCount (fps)
MultipleLihgt (fps)
HighPolygon
count (fps)
Keyframeanimation
(fps)
Game level(fps)
An3DBench
VM
Emulator
Emulator v4.0
Real Device
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Total
An3DBench
VM
Emulator
Emulator v4.0
Real Device
![Page 38: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/38.jpg)
Experiments - SpeedTest
• Discover your Download, Upload, and Ping
• Real-time graphs show connection consistency
38
![Page 39: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/39.jpg)
Experiments - SpeedTest
39
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Ping (ms) Download (kbps) Upload (kbps)
SpeedTest
VM
Emulator
Emulator v4.0
Real Device
![Page 40: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/40.jpg)
Other benchmark applications for Android
40
Application CPU Memory I/O Graphics Network
AnTuTu Benchmark x x x x x
SmartBench 2012 x x x x
NBench x x
CPUBenchmark x
NenaMark x
BrowserMark x
![Page 41: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/41.jpg)
Conclusions
• Android Application Development A bit tricky
• CPU is the major factor
• Newer versions of Android perform better
• Real devices are more stable
• Some applications are not supported by VM and Emulator
• But…good for testing No harm to OS
41
![Page 42: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/42.jpg)
Other options – Blue Stacks
42
![Page 43: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/43.jpg)
References
• Developing Applications – http://developer.android.com/tools/building/index.html
• Virtual Machine / Emulator – http://forum.xda-developers.com/showthread.php?t=1474956 – http://developer.android.com/tools/devices/emulator.html – http://tutorialfor-android.blogspot.com.es/2012/07/how-to-download-apk-from-google-play-
to.html – http://www.vladan.fr/how-to-install-android-in-vmware-workstation/ – http://quinxy.com/2011/03/12/running-android-2-2-in-vmware-in-less-than-5-minutes/
• Experimenting – http://www.phonearena.com/news/10-useful-Android-benchmark-apps_id26876 – http://www.techhive.com/article/255977/how_to_benchmark_your_android_device.html – http://www.howtogeek.com/111594/how-to-benchmark-your-android-device-5-free-apps/ – http://code.google.com/p/0xbench/
• Book & paper – Beginning Android Application Development – Wei-Meng Lee – Evaluating Android OS for Embedded Real-time Systems - Cl´audio Maia, Lu´ıs Nogueira, Lu´ıs
Miguel Pinho
43
![Page 44: Mobile operating systems - Application Benchmarking](https://reader034.fdocuments.in/reader034/viewer/2022050817/5538e452550346e0428b4871/html5/thumbnails/44.jpg)
Mobile Operating Systems: Android
Nicos Demetriou
Operating Systems for New Architectures Fall Semester 2012