120 TEMPLATES IN TOTAL - IEEE Bilkentieee.bilkent.edu.tr/grc2017/docs/sample_pdf.pdf · Exp e rim e...
1
Experimental Results TEMPLATES Aras Yurtman and Billur Barshan Department of Electrical and Electronics Engineering, Bilkent University [email protected], [email protected] ● Physical therapy often requires repeating certain exercise movements. ● Patients first perform the required exercises under supervision in a hospital or rehabilitation center. PARTIAL AND SUBJECTIVE FEEDBACK ● Most patients continue their exercises at home. NO FEEDBACK ● The intensity of a physical therapy session is estimated by the number of correct executions. OBJECTIVE : to detect and evaluate all exercise executions in a physical therapy session by using wearable motion sensors based on template recordings Introduction Algorithm ● 5 wearable motion sensors, each containing a tri-axial accelerometer, gyroscope, and magnetometer ● 8 exercise types performed by 5 subjects ● Each exercise is assumed to have 3 execution types: one correct and two erroneous (fast and low-amplitude execution) ● one template for each execution type of each exercise of each subject 120 TEMPLATES IN TOTAL ● To simulate a physical therapy session, for each exercise, each subject performs the exercise 10 times in the correct way, then 10 times with type-1 error, and finally 10 times with type-2 error. Between these 3 blocks, the subject is idle. 1,200 TEST EXECUTIONS IN TOTAL Dataset ● Standard DTW ● measures the similarity between two signals that are different in time or speed ● matches two signals by transforming their time axes nonlinearly to maximize the similarity ● Multi-Template Multi-Match DTW (MTMM-DTW) has been developed based on DTW to ● detect multiple occurrences of multiple template signals in a long test signal ● both detect and classify the occurrences Features of MTMM-DTW: ● The number of templates, occurrences, their positions, and lengths of the template and test signals may be arbitrary. ● The signals may be multi-D. ● A threshold factor can be selected to prevent relatively short matches compared to the matching template. ● The amount of overlap between the matched subsequences can be adjusted. ● Any modification to the DTW algorithm may be used in MTMM-DTW. EXECUTION TYPES: ● correct ● type-1 error (executed too fast) ● type-2 error (executed in low amplitude) 5 subjects 8 exercises 3 execution types 10 repetitions 1 of 3 executions selected ✕ ✕ ✕ = = 120 templates 1,200 executions EXPERIMENTS SIMULATING A PHYSICAL THERAPY SESSION For each exercise, each subject has simulated a therapy session by executing the exercise ● 10 times in the correct way, ● waiting idly, ● 10 times with type-1 error, ● waiting idly, and then ● 10 times with type-2 error. 0 50 100 150 200 -20 0 20 subject 3, exercise 1, unit 2 time (s) acceleration (m/s 2 ) x y z 0 50 100 150 200 -5 0 5 time (s) angular rate (rad/s) x y z 0 50 100 150 200 -1 0 1 time (s) magnetic field (relative) x y z 0 10 20 30 40 50 -20 0 20 subject 3, exercise 1 (templates), unit 2 time (s) acceleration (m/s 2 ) x y z 0 10 20 30 40 50 -5 0 5 time (s) angular rate (rad/s) x y z 0 10 20 30 40 50 -1 0 1 time (s) magnetic field (relative) x y z 1 2 3 4 5 LEG EXERCISES 6 7 8 ARM EXERCISES TEST XSENS MTx SENSOR UNIT Number of total executions 1,200 Number of executions detected 1,125 Accuracy of exercise classification 93.5% Accuracy of exercise and execution type classification 88.7% Misdetection rate (MDs / positives) 8.6% False alarm rate (FAs / negatives) 4.9% Sensitivity 91.4% Specificity 95.1% 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 1 of subject 5 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 2 of subject 5 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 3 of subject 5 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 4 of subject 5 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 5 of subject 5 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 6 of subject 5 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 7 of subject 5 1000 2000 3000 4000 5000 0 5 10 15 samples distance Experiment 8 of subject 5 ● We apply the proposed MTMM-DTW algorithm to each test signal with the 24 template signals of the same subject for 8 exercise types × 3 execution types. ● Each detected exercise must be at least half the length of the matching template. ● Detections with a normalized DTW distance larger than 10 are omitted. Conclusion ● The proposed system can be used in tele-rehabilitation to provide feedback to the patient exercising remotely and assessing the effectiveness of the exercising session. ● In previous systems, each execution is recorded separately or cropped manually. ● Our system ● automatically detects the individual executions and idle time periods, ● classifies each execution as one of the exercise types, ● evaluates its correctness, and ● identifies the error type if any. References [1] A. Yurtman and B. Barshan, “Automated evaluation of physical therapy exercises using multi-template dynamic time warping on wearable sensor signals,” Comp. Meth. and Prog. Biomed., 117(2):189–207, Nov. 2014. [2] P. Tormene, T. Giorgino, S. Quaglini, and M. Stefanelli, “Matching incomplete time series with dynamic time warping: an algorithm and an application to post-stroke rehabilitation,” Artif. Intell. Med., 45(1):11–34, Jan. 2009.
Transcript of 120 TEMPLATES IN TOTAL - IEEE Bilkentieee.bilkent.edu.tr/grc2017/docs/sample_pdf.pdf · Exp e rim e...
E x p e r i m e n t a l R e s u l t s
TE
MP
LA
TE
S
Aras Yurtman and Billur BarshanDepartment of Electrical and Electronics Engineering, Bilkent University
[email protected], [email protected]
● Physical therapy often requires repeating certain exercise movements.
● Patients first perform the required exercises under supervision in a hospital or rehabilitation center. PARTIAL AND SUBJECTIVE FEEDBACK
● Most patients continue their exercises at home. NO FEEDBACK
● The intensity of a physical therapy session is estimated by the number of correct executions.
OBJECTIVE : to detect and evaluate all exercise executions in a physical therapy session by using wearable motion sensors based on template recordings
I n t r o d u c t i o n
A l g o r i t h m
● 5 wearable motion sensors, each containing a tri-axial accelerometer, gyroscope, and magnetometer
● 8 exercise types performed by 5 subjects
● Each exercise is assumed to have 3 execution types: one correct and two erroneous (fast and low-amplitude execution)
● one template for each execution type of each exercise of each subject 120 TEMPLATES IN TOTAL
● To simulate a physical therapy session, for each exercise, each subject performs the exercise 10 times in the correct way, then 10 times with type-1 error, and finally 10 times with type-2 error. Between these 3 blocks, the subject is idle. 1,200 TEST EXECUTIONS IN TOTAL
D a t a s e t
● Standard DTW
● measures the similarity between two signals that are different in time or speed
● matches two signals by transforming their time axes nonlinearly to maximize the similarity
● Multi-Template Multi-Match DTW (MTMM-DTW) has been developed based on DTW to
● detect multiple occurrences of multiple template signalsin a long test signal
● both detect and classify the occurrences
Features of MTMM-DTW:
● The number of templates, occurrences, their positions, and lengths of the template and test signals may be arbitrary.
● The signals may be multi-D.
● A threshold factor can be selected to prevent relatively short matches compared to the matching template.
● The amount of overlap between the matched subsequences can be adjusted.
● Any modification to the DTW algorithm may be used in MTMM-DTW.
EXECUTION TYPES:
● correct
● type-1 error (executed too fast)
● type-2 error (executed in low amplitude)
5subjects
8exercises
3execution
types10
repetitions
1 of 3executions selected✕ ✕ ✕
=
=
120templates
1,200executions
EXPERIMENTS SIMULATING A PHYSICAL THERAPY SESSION
For each exercise, each subject has simulated a therapy session by executing the exercise
● 10 times in the correct way,
● waiting idly,
● 10 times with type-1 error,
● waiting idly, and then
● 10 times with type-2 error.
0 50 100 150 200-20
0
20subject 3, exercise 1, unit 2
time (s)
accele
ration (
m/s
2)
x
y
z
0 50 100 150 200-5
0
5
time (s)
angula
r ra
te (
rad/s
)
x
y
z
0 50 100 150 200-1
0
1
time (s)magnetic f
ield
(re
lative)
x
y
z
0 10 20 30 40 50-20
0
20subject 3, exercise 1 (templates), unit 2
time (s)
accele
ration (
m/s
2)
x
y
z
0 10 20 30 40 50-5
0
5
time (s)
angula
r ra
te (
rad/s
)
x
y
z
0 10 20 30 40 50-1
0
1
time (s)magnetic f
ield
(re
lative)
x
y
z
1 2
3
4
5
LE
G E
XE
RC
ISE
S
6 7
8
AR
M E
XE
RC
ISE
S
TEST
XSENS MTx
SENSOR UNIT
Number of total executions 1,200
Number of executions detected 1,125
Accuracy of exercise classification 93.5%
Accuracy of exercise and execution type classification 88.7%
Misdetection rate (MDs / positives) 8.6%
False alarm rate (FAs / negatives) 4.9%
Sensitivity 91.4%
Specificity 95.1%
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 1 of subject 5
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 2 of subject 5
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 3 of subject 5
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 4 of subject 5
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 5 of subject 5
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 6 of subject 5
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 7 of subject 5
1000 2000 3000 4000 50000
5
10
15
samples
dist
ance
Experiment 8 of subject 5
● We apply the proposed MTMM-DTW algorithm to each test signal with the 24 template signals of the same subjectfor 8 exercise types × 3 execution types.
● Each detected exercise must be at least half the length of the matching template.
● Detections with a normalized DTW distance larger than 10 are omitted.
C o n c l u s i o n
● The proposed system can be used in tele-rehabilitation to provide feedback to the patient exercising remotely and assessing the effectiveness of the exercising session.
● In previous systems, each execution is recorded separately or cropped manually.
● Our system
● automatically detects the individual executions and idle time periods,
● classifies each execution as one of the exercise types,
● evaluates its correctness, and
● identifies the error type if any.
R e f e r e n c e s
[1] A. Yurtman and B. Barshan, “Automated evaluation of physical therapy exercises using multi-template dynamic time warping on wearable sensor signals,” Comp. Meth. and Prog. Biomed., 117(2):189–207, Nov. 2014.
[2] P. Tormene, T. Giorgino, S. Quaglini, and M. Stefanelli, “Matching incomplete time series with dynamic time warping: an algorithm and an application to post-stroke rehabilitation,” Artif. Intell. Med., 45(1):11–34, Jan. 2009.
