8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
1/35
Designing An Android
Sensor Subsystem
Pitfalls and Considerations
Jen Costillo
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
2/35
Simple Choices
Userexperience
Batteryperformance
7/15/2012 2Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
3/35
Established
or Innovative Product?
Established
Will I be making another
new product in 6
months?
Is the reference design
considered good enough
for the application?
Innovation-Driven
Do I have new sensors
types?
Are features more
important than release
date?
Are money and
resources noproblem?
7/15/2012 3Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
4/35
Forsaking Reference Designs
7/15/2012 4Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
5/35
Going On Your Own
If you make your own,
Youre on your own
Integration pains
Test time Gesture testing
becomes a challenge
Calibration blues
Larger mechanicalfootprint
But
power
Control code size
Control mechanicalfootprint
In-house expertise
7/15/2012 Costillo- OSCON 2012 5
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
6/35
Android Universe
7/15/2012 Costillo- OSCON 2012 6
Sensor Hub/Coprocessor Sensors
Interface Kernel
Driver Sensor Driver
Sensor HAL
Sensor Service Sensor Manager
Sensor JNI
SensorManager
Android Application
Hardware
Linux Kernel
Libraries
Frameworks
Application
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
7/35
HARDWAREHardware
7/15/2012 7Costillo- OSCON 2012
Linux Kernel
Libraries
Frameworks
Application
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
8/35
Hardware Architecture
7/15/2012 8Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
9/35
Sampling Rates:
The 3 RatesUnder-sampling
Inaccurate, sluggishresponse
Slight power savings
Sampling
Rate
Over-sampling
Accurate, smoothresponse
Power-hungry
7/15/2012 9Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
10/35
Wake up events and power
considerationsApplication
Processor only
Internal
Coprocessor
External
Processor
D
Reference supported
Most power hungry
Reference supported
Most work done foryou
More processor selection
More outcome controlMost customized
Footprint impact
7/15/2012 10Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
11/35
Hardware Summary
PowerConsumption= sensorsn+any dedicated
processor
Latency =Max(sensorsn)
+ dedicatedprocessing
time
SensorSolution
Use tie-breaker criteria
7/15/2012 Costillo- OSCON 2012 11
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
12/35
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
13/35
Sensor
Kernel Driver
7/15/2012 13Costillo- OSCON 2012
CoprocessorMicrocontroller
Application Processor
PeripheralInterface
SharedMemory
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
14/35
LIBRARIES ANDSERVICES
Libraries
7/15/2012 14Costillo- OSCON 2012
Hardware
Frameworks
Application
Linux Kernel
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
15/35
Sensor HAL and Services
HALdevice///libsensors
Serviceframeworks/base/services/sensorservice
Managerframeworks/base/libs/gui
7/15/2012 15Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
16/35
Sensor Fusion
7/15/2012 Costillo- OSCON 2012 16
Sensors
Linux Kernel
Libraries
Sensor Hub
http://en.wikipedia.org/wiki/Sensor_fusion
https://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.html
http://en.wikipedia.org/wiki/Sensor_fusionhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttps://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.htmlhttp://en.wikipedia.org/wiki/Sensor_fusionhttp://en.wikipedia.org/wiki/Sensor_fusion8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
17/35
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
18/35
Gesture Detection Comparison
Make Buy
Completesolution
Already Tested&tuned
MinimalSchedule Impact
Compact code
In-houseexpertise
Off-load AppPro
Application
Powerful processor
Sensor hub
Off-load to cheaperpower
Wake up EventHandling
7/15/2012 Costillo- OSCON 2012 18
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
19/35
Calibration
7/15/2012 19Costillo- OSCON 2012
Use of Calibration Gesture in the
Compass App By Catch.com
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
20/35
FRAMEWORKS
7/15/2012 20Costillo- OSCON 2012
Hardware
Application
Linux Kernel
Libraries
Frameworks
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
21/35
Virtual Sensors
Leverages 1+ physical or other virtualsensors
Multiple Options
Google (version 3)
Reference Vendor
Sensor Vendors
7/15/2012 Costillo- OSCON 2012 21
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
22/35
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
23/35
Android Virtual Sensors
Gyro Accel Orientation
Compass Accel Rotation
7/15/2012 Costillo- OSCON 2012 23
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
24/35
Virtual Sensors Challenges
Garbage In- Garbage Out
Latency
Non-Synchronizedsamples
Implementation
Dependencies Multi-vendor problems,
verify Vendor ID
http://www.invensense.com/midc/presentations/James%20Lim.pdf7/15/2012 24Costillo- OSCON 2012
Sensor1
Sensor
2Virtual
Output
Sensor
1
Sensor
2
Virtual
1
Virtual
2
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
25/35
APPLICATIONS
7/15/2012 25Costillo- OSCON 2012
Hardware
Linux Kernel
Libraries
Application
Frameworks
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
26/35
Using Sensors
7/15/2012 Costillo- OSCON 2012 26
mSensorManager =
getSystemService(Context.SENSOR_SERVICE);
mSensorManager.registerListener(
mSensorListener,
mSensorManager.getDefaultSensor(
Sensor.TYPE_ACCELEROMETER),
SensorManager.SENSOR_DELAY_NORMAL);
void onSensorChanged(SensorEvent event) {
// get sensor data
float x =
event.values[SensorManager.DATA_X];
}
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
27/35
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
28/35
Types of Sensor Problems
Bias
Drift
Settling Time Jitter/Noise
Environmental
Interference
7/15/2012 Costillo- OSCON 2012 28
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
29/35
Bias
Problem: Data is offby a constant value.
Sources:
static calibrationfailure
Solutions:
Calculate linear
offset at start ofapplication
Recalibrate locally
7/15/2012 Costillo- OSCON 2012 29
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
30/35
Drift
Problem: Shift of
data without cause
Sources:
Magneticinterference
Poor HW calibration
Solutions:
Increase smoothing
techniques
7/15/2012 Costillo- OSCON 2012 30
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
31/35
Settling Time Problem: Extended
time before
finalized steady
data.
Sources: Latency
Sensitivity
Solutions:
Limit additional
processing
7/15/2012 Costillo- OSCON 2012 31
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
32/35
Noise
Problem: Data jumpsaround constantly
Sources:
Sensor
Calibration
Poor filtering
Solutions:
High pass filter
Linear averaging
FFT
7/15/2012 Costillo- OSCON 2012 32
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
33/35
Environmental Interference
Problem: Inconsistentresults
Sources:
Magnetometer
EMI
Solutions:
Reference Device
Calibration gesture
7/15/2012 Costillo- OSCON 2012 33
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
34/35
Best Practices in
Application Development
Select the right sensor for the job.
Use the Correct Data Rate.
UI or GAMING are the most common.
Use Sensor In Context
Customize for your hardware and system
capabilities
Magnetometer-based sensors are the mosttouchy.
Keep the Gesture UI simple.
7/15/2012 34Costillo- OSCON 2012
8/12/2019 Topics in Designing an Android Sensor Subsystem_ Pitfalls and Considerations Presentation
35/35
Additional resources
http://processors.wiki.ti.com/index.php/Android_Sensor_PortingGuidehttp://www.kandroid.org/online-pdk/guide/sensors.html
http://invensense.com/midc/
http://developer.android.com/reference/android/hardware/SensorEvent.html
7/15/2012 35Costillo- OSCON 2012
http://processors.wiki.ti.com/index.php/Android_Sensor_PortingGuidehttp://www.kandroid.org/online-pdk/guide/sensors.htmlhttp://invensense.com/midc/http://developer.android.com/reference/android/hardware/SensorEvent.htmlhttp://developer.android.com/reference/android/hardware/SensorEvent.htmlhttp://invensense.com/midc/http://www.kandroid.org/online-pdk/guide/sensors.htmlhttp://www.kandroid.org/online-pdk/guide/sensors.htmlhttp://www.kandroid.org/online-pdk/guide/sensors.htmlhttp://processors.wiki.ti.com/index.php/Android_Sensor_PortingGuideTop Related