Handling displacement effects in on-body sensor-based activity recognition
-
Upload
oresti-banos -
Category
Science
-
view
130 -
download
0
description
Transcript of Handling displacement effects in on-body sensor-based activity recognition
Handling displacement effects in on-body sensor-based activity recognition
IWAAL 2013, San José (Costa Rica)
Oresti Baños, Miguel Damas, Héctor Pomares, and Ignacio Rojas Department of Computer Architecture and Computer Technology, CITIC-UGR,
University of Granada, SPAIN
DG-Research Grant #228398
Context
• On-body activity recognition is becoming true…
– Portable/Wearable
– Unobtrusive
– Fashionable
2
Context
• On-body activity recognition is becoming true… Really? – Reliability
• Different performance depending on who uses the system (age, height, gender,…) and due to people changes during the lifelong use (conditions, ageing,…) Reliable? Perdurable?
– Usability/Applicability
• Application-specific systems require to put on several diverse systems to provide different functionalities Portable? Unobtrusive? Fashionable? Tractable?
– Robustness
• Sensor anomalies (decalibration, loose of attachment, displacement,…) Robust?
3
Problem statement
Collect a training dataset
Train and test the model
The AR system is “ready”
4
[-0
.14
, 3
.41
, 4
,21
, … ,
6.1
1]
[-0
.84
, 3
.21
, 4
.21
, … ,
6.1
1]
[-0
.81
, 5
.71
, 4
.21
, … ,
6.2
2]
[-0
.14
, 3
.92
, 4
.23
, … ,
7.8
2]
u p s1,s2,…,sk fℝ(s1,s2,…,sk) c
Problem statement
INVARIANT SENSOR SETUP (IDEALLY) GOOD RECOGNITION
5
Problem statement
SENSOR SETUP CHANGES RECOGNITION PERFORMANCE MAY DROP
6
[-0
.14
, 3
.41
, 4
,21
, … ,
6.1
1]
[-0
.84
, 3
.21
, 4
.21
, … ,
6.1
1]
[-0
.81
, 5
.71
, 4
.21
, … ,
6.2
2]
[-0
.14
, 3
.92
, 4
.23
, … ,
7.8
2]
u p s1,s2,…,sk fℝ(s1,s2,…,sk) c
Concept of sensor displacement
• Categories of sensor displacement
– Static: position changes can remain static across the execution of many activity instances, e.g. when sensors are attached with a displacement each day
– Dynamic: effect of loose fitting of the sensors, e.g. when attached to cloths
• Sensor displacement new sensor position signal space change
• Sensor displacement effect depends on
– Original/end position and body part
– Activity/gestures/movements performed
– Sensor modality (ACC, GYR, MAG)
7
Sensor displacement = rotation + translation (angular displacement) (linear displacement)
Sensor displacement effects
Changes in the signal space forward propagates on the activity recognition process (e.g., variations in the feature space)
RCIDEAL LCIDEAL= LCSELF
8
RCSELF
Dealing with sensor displacement: Feature Fusion
9
[-0
.14
, 3
.41
, 4
,21
, … ,
6.1
1]
[-0
.84
, 3
.21
, 4
.21
, … ,
6.1
1]
[-0
.81
, 5
.71
, 4
.21
, … ,
6.2
2]
[-0
.14
, 3
.92
, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k)
c u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k)
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk)
fℝ(s11,s12,…,s1k, s21,s22,…,s2k,…,
sM1,sM2,…,sMk)
Dealing with sensor displacement: Decision Fusion
10
[-0
.14
, 3
.41
, 4
,21
, … ,
6.1
1]
[-0
.84
, 3
.21
, 4
.21
, … ,
6.1
1]
[-0
.81
, 5
.71
, 4
.21
, … ,
6.2
2]
[-0
.14
, 3
.92
, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k) c1
c=φ(c1,c2,…,cM)
u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k) c2
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk) cM
Multi-Sensor Hierarchical Classifier
11
SM
S2
S1 α11
∑ C12
C1N
C11
∑
C21
C22
C2N
∑
CM1
CM2
CMN
∑
Decisio
n
Class level Source level Fusion
β11
α12 β12
α1N β1N
α21 β21
α22 β22
α2N β2N
αM1 βM1
αM2 βM2
αMN βMN
γ11,…,1N δ11,…,1N
γ21,…,2N δ21,…,2N
γM1,…,MN δM1,…,MN
[-0
.14
, 3
.41
, 4
,21
, … ,
6.1
1]
[-0
.84
, 3
.21
, 4
.21
, … ,
6.1
1]
[-0
.81
, 5
.71
, 4
.21
, … ,
6.2
2]
[-0
.14
, 3
.92
, 4
.23
, … ,
7.8
2]
S1
S2
SM
u1 p1 s11,s12,…,s1k fℝ(s11,s12,…,s1k)
u2 p2 s21,s22,…,s2k fℝ(s21,s22,…,s2k)
uM pM sM1,sM2,…,sMk fℝ(sM1,sM2,…,sMk)
O. Banos, M. Damas, H. Pomares, F. Rojas, B. Delgado-Marquez, and O. Valenzuela. Human activity recognition based on a sensor weighting hierarchical classifier. Soft Computing, 17:333-343, 2013.
Study of sensor displacement effects
• Analyze
– Variability introduced by sensors self-positioning with respect to an ideal setup
– Effects of large sensor displacements (extreme de-positioning)
– Robustness of sensor fusion to displacement
• Scenarios
– Ideal-placement
– Self-placement
– Induced-displacement
Ideal Self Induced
12
O. Banos, M. Damas, H. Pomares, I. Rojas, M. Attila Toth, and O. Amft. A benchmark dataset to evaluate sensor displacement in activity recognition. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pages 1026-1035, New York, NY, USA, 2012.
Dataset: activity set
• Activities intended for: – Body-general motion: Translation | Jumps | Fitness
– Body-part-specific motion: Trunk | Upper-extremities | Lower-extremities
13
Dataset: Study setup
• Cardio-fitness room
• 9 IMUs (XSENS) ACC, GYR, MAG
• Laptop data storage and labeling
• Camera offline data validation
http://crnt.sourceforge.net/CRN_Toolbox/Home.html 14
Dataset: Experimental protocol
• Scenario description
• Protocol
Round Sensor Deployment #subjects #anomalous sensors
1st Self-placement 17 3/9
2nd Ideal-placement 17 0/9
- Mutual-displacement 3 {4,5,6 or 7}/9
15
Preparation phase (sensor positioning & wiring, Xsens-Laptop
bluetooth connection, camera set up)
Exercises execution (20 times/1 min. each)
Battery replacement, data downloading
Data postprocessing (relabeling, visual
inspection, evaluation)
Round
Experimental setup
• Data considerations
– Data domain: ACC, GYR, MAG and combinations (ACC-GYR, ACC-MAG, GYR-MAG, ACC-GYR-MAG)
– ALL sensors
• Activity recognition methods
– No preprocessing
– Segmentation: 6 seconds sliding window
– Features: MEAN, STD, MAX, MIN, MCR
– Reasoner: C4.5 decision tree
16
• Sensor displacement scenarios
– Ideal (no displacement)
– Self (3 out of all sensors)
– Induced (7 out of all sensors)
• Evaluation
– Ideal: 5-fold cross validation, 100 times
– Self/Mutual: tested on a system trained on ideal-placement data
Fusion systems performance
17
Feature Fusion Decision Fusion (MSHC)
Fusion systems performance
18
40% 30% 15% 20% 40% 15% 35% 20% 15% 15% 15% 10% 5% 10%
Feature Fusion Decision Fusion (MSHC)
Fusion systems performance
19
60% 45% 25% 50%
55% 35% 45% 35% 30% 55% 35% 30% 10% 30%
Feature Fusion Decision Fusion (MSHC)
Conclusions and final remarks
• Sensor anomalies (here displacement) may seriously damage the performance of activity recognition systems, especially single sensor based systems
• Sensor fusion is proposed to deal with these anomalies
• Feature fusion approaches (the most widely used) has been demonstrated to be very sensitive to sensor displacement
• Decision fusion cope much better with the effects of sensor displacement, even when a majority of the sensors are highly de-positioned
• From the analyzed sensor magnitudes, GYR outstands as the most robust modality to displacement with a performance drop of less than 5% for the self-placement scenario and 10% for the extreme displacement 20
Thank you for your attention. Questions?
Oresti Baños Legrán
Dep. Computer Architecture & Computer Technology Faculty of Computer & Electrical Engineering (ETSIIT)
University of Granada, Granada (SPAIN) Email: [email protected]
Phone: +34 958 241 778 Fax: +34 958 248 993
Work supported in part by the HPC-Europa2 project funded by the European Commission - DG Research in the Seventh Framework Programme under grant agreement no. 228398 and the FPU Spanish grant AP2009-2244. 21