IoT for Next-Generation Racket Sports Training · 2019-06-21 · practice. Index Terms—IoT...

2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal

> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <

1

Abstract—We propose an IoT framework for next-generation

racket sports training. To validate its performance, a wireless

wearable sensing device (WSD) based on MEMS

(microelectromechanical systems) motion sensors was used to

recognize different badminton strokes and classify skill levels

from different badminton players. The system includes a

customized sensor node for data collection, a mobile app and a

cloud-based data processing unit. The WSD developed is

low-cost, easy-to-use and computationally efficient compared to

video-based methods for analyzing badminton strokes. It offers

the advantage of dynamic monitoring of multiple players in

indoor and outdoor environments. In this paper we present the

hardware design, mobile software implementation, and data

processing algorithms of the system. Twelve right-handed male

subjects wore the WSD on their wrists while each performed 30

trials of different strokes in a real badminton court. The results

show that our system is capable of recognizing three different

actions, i.e., smashes, clears and drops, with an accuracy rate of

97%. The skill assessment function can differentiate between

professional, sub-elite, and amateur players from their stroke

performance. This IoT framework aims to change the way of

racket sports training from experience-driven (subjective) to

data-driven (objective), and which can be easily extended to

analyze the motions and skill levels of players in other racket

sports (e.g., tennis, table tennis and squash) for training and/or

practice.

Index Terms—IoT Wearable Devices, Micro Inertial

Measurement Unit, Sports Analysis, Badminton Strokes, Motion

Assessment

I. INTRODUCTION

acket sports, such as badminton, tennis, table tennis and

squash, are amongst the most popular recreational sports

activities. In addition to physical fitness, racket sports

players need aerobic stamina, agility, strength, speed, and

precision simultaneously. Meanwhile, racket sports are highly

1Department of Mechanical and Biomedical Engineering, City University of

Hong Kong (CityU), Hong Kong SAR, China.

2Department of Electronic Engineering, CityU, Hong Kong SAR, China.

3Department of Computer Science, CityU, Hong Kong SAR, China.

4Shenzhen Academy of Robotics (SZAR), Shenzhen, China. #Contact Authors: Dr. Rosa H. M. Chan ([email protected]) is Associate Professor in the Department of Electronic Engineering at City

University of Hong Kong. Dr. Wen J. Li ([email protected]) is Chair

Professor of Biomedical Engineering at CityU; he is also an affiliated professor at the SZAR.

technical requiring sophisticated motor coordination for agile

steps and racket movements. Generally, a badminton player

should be able to strike a shuttlecock with different motions

(with corresponding forces) called ‘strokes’. This requires

techniques which are constructed and delivered through precise

basic skills, powerful strength, and coordination [1].

Badminton can be characterized by three types of racket

movements: short drops, long clears and hard smashes [2].

Timely execution of these strokes contributes to success in

matches [3]. In recent years, numerous researchers have

investigated the movement of racket sports players [4], since

observable changes from athletes are vital in the coaching

process. The majority of conventional studies focus on using

video-based methods to track athletes’ actions and performance.

However, such video-based methods are not utilized in

day-to-day training due to limitations such as large

computation load [5], costly equipment, and environmental

constraints [6]. Also, adoption of multiple physical markers

tends to incur a marker crossover phenomenon and extra cost.

Thus, conventional means cannot provide timely feedback to

players. Instead, to capture the motion in daily practice outside

the lab, researchers have shifted their efforts to recognize

activities using inertial sensors worn on the body. The

advancement in MEMS and Bluetooth Low Energy (BLE)

technologies makes feasible long-term motion data collection

and transmission in a small package with relatively long battery

life.

Recently, a number of research studies in sports activities

recognition have utilized MEMS motion sensors [7]. For

example, a support system for golf players using wearable

sensors was designed to collect the athletes’ kinematics data [8].

An inertial information database was constructed for

professional horseback riders with 16 motion sensors. Data on

several motion features: elbow angle, knee angle, backbone

angle, hip position, and knee-elbow distance were extracted [9].

Another example is a volleyball application investigating the

frequency of jumps in volleyball games using a tri-axial

accelerometer [10]. On the other hand, there are already similar

research work analyzing strokes and smashing motions in

various racket-related sports using video-based and MEMS

technologies [11]. For example, in a study of baseball players,

researchers collected data from a body sensor network (made of

MEMS-based acceleration sensors) to generate motion

transcripts, and used them to measure coordination among limb

segments and joints [12]. However, each sensor node’s surface

area is about 3. 2cm x 6.6 cm, which makes it impractical to be

worn by baseball players in a real game. The possibility of

using MEMS-based gyroscopes for skill assessments of tennis

IoT for Next-Generation Racket Sports

Training

Yufan Wang1, Student Member, IEEE, Meng Chen1, Xinyu Wang3, Rosa H. M. Chan2*, Senior Member,

IEEE and Wen J. Li1,4*, Fellow, IEEE

R




2

players was investigated as well [13]. However, they merely

collected motion data from three sets of two-axis gyroscopes

placed in different locations of four different participants, and

no data segmentation or motion differentiation were reported in

that work. In our previous work [14], we used an inertial

measurement unit to assess the skill level of volleyball spikers.

The system was demonstrated to be stable and capable of

assessing spike-motion differences between professional,

sub-elite and amateur volleyball players. However, the

collected data could only be saved on a local Secure Digital (SD)

card and processed offline. For the device presented in this

work, we have further improved the stability of the sensing

device and implemented the real-time transmission of motion

data via Bluetooth technology. To the best of our knowledge,

the system we describe here is one of the first that is capable of

racket sports action recognition and skill assessment using a

single sensor unit, communicating through Bluetooth Low

Energy technology, and computing in the Cloud.

With proliferation of consumers’ passion for a healthy and fit

lifestyle in recent years, the wearable electronic market has

grown exponentially. Many enthusiastic racket sports players

are looking for a more professional and efficient way to track

their motion data and assess their performance with data

collected during competitions. Amateurs likewise may want to

learn the correct motions for a particular action in a sport in

order to improve their skill level.

Here, using the example of badminton, we propose a novel

and “smart” racket action recognition and skill assessment

system using a low-power MEMS inertial measurement unit

(IMU) with BLE and Cloud Technology. This application

demonstrates the possibilities in using IoT framework for

next-generation racket sports training. The system is capable of

classifying different actions and differentiating skill levels

between professional athletes and badminton amateurs. At the

same time, it provides feedback for the quality of a player’s

smashes and clears. First, we developed a wireless wearable

sensing device (smaller than any existing commercial products

to the best of our knowledge) with an overall size of

18mm×17mm×2mm, to collect inertial data. Second, we

designed a mobile app that can visualize experimental results

and upload data from the experiments to the cloud server.

Third, we used machine-learning algorithms to classify

different badminton actions at 97% prediction accuracy.

Moreover, we have shown that this system can discriminate

skill levels between professional badminton athletes and

badminton amateurs in term of different actions, reaching a

high prediction accuracy of 83.3% for smashes and 90% for

clears. Foreseeably, this framework can be extended to

recognize actions and analyze skill levels of players in other

racket sports, such as tennis, table tennis and squash.

II. SYSTEM SETUP

Here we detail the complete technical solution developed to

build a smart badminton action recognition system based on

low-power Bluetooth communication and Cloud technology. It

contains hardware and software parts as well as sensor

placement locations. The general system setup principles are

shown in Fig. 1. The system consists of a sensor node, a

high-speed camera, a mobile device, a cloud server and a

laptop. We used this system to collect motion data from

badminton players and to recognize different badminton

actions. The general work principle for the system is that data

collected by an inertial measurement unit was sent to a mobile

device through Bluetooth Low Energy (BLE). Once the mobile

phone received the data, it sent the motion raw data to the

remote server via Cloud technology. After collecting all data,

users could then view badminton players’ data with a client

Wi-Fi.

A. Hardware System

Fig. 2 shows the wireless IMU design. It comprises a

microprocessor with Bluetooth wireless communication

module, a MEMS motion sensing chip (with 3-axes

accelerometer and 3-axes gyroscope), an on/off switcher and a

coin cell battery (on the back of the circuit board). DA14583

(Dialog Semiconductor, Reading, United Kingdom) fully

integrates radio transceiver and baseband processor for

Bluetooth Low Energy, which saves space for communication

and processing.

Fig. 1. System setup and sensor placement on human subjects.

Fig. 2. (a) Circuit board and dimension of the Wearable Sensing Device (WSD)

which (b) can be embedded in wrist band.




3

TABLE 1 COMPARISON BETWEEN DIFFERENT BLUETOOTH CHIPS [15]

Company Product

Power

Consumption

TX (mA)

Power

Consumption

RX (mA)

Chip Size

Dialog Semiconductor

DA14583 4.9 4.9 5 mm 5 mm

Nordic

Semiconductor nRF51822 8.0 9.7 6 mm 6 mm

Texas

Instruments CC2640 6.1 5.9 5 mm 5 mm

Qualcomm CSR1010 16.0 18.0 5 mm 5 mm

We considered two major parameters while selecting chips

for our BLE module: power consumption and chip size. Table

1 summarizes the specification of this Bluetooth chip and

comparison among its congeneric products. This unit enables a

small and power efficient Bluetooth smart system to

communicate with the motion sensor and send motion data to

the peripheral mobile device. It was programmed using Dialog

Semiconductor Software Development Kits (SDK). Apart from

inertial sensors mentioned above, there are several commercial

inertial sensor systems available, such as MTi0series from

Xsens North America, Inc. and the wireless accelerometer from

Noraxon. However, the ranges of these sensors were not

suitable for our application. According to the results from

Wang et al. [16], a 16g sensor range is enough to analyze all the

badminton actions. Therefore, we mainly considered full-scale

range and chip size for IMU chip selection. Table 2

summarizes the specification of the inertial sensor used in this

study and a comparison among its congeneric products. As

shown in Table 2, we utilized BMI160 from BOSCH because

of its suitable sensor range (16 g) and small size (2.5 mm × 3.0

mm × 0.83 mm). BMI160 is a highly integrated, low power

IMU that provides orientation and acceleration readings in

x-y-z dimensions. Using this sensor module, we kept the entire

WSD size within an 18 mm × 17 mm package that weighs only

2.2 g.

TABLE 2 COMPARISON BETWEEN DIFFERENT IMU CHIPS

Company Product

Gyroscope-

Full-scale Range (°/sec)

/ Sensitivity

(LSB/°/sec)

Accelerometer-

Full-scale Range (g) /

Sensitivity

(LSB/g)

Chip Size

Bosch Sensortec

[17]

BMI160 2000/16.4 16/2048 2.5 mm 3 mm

InvenSense

[18] MPU9250 2000/16.4 16/2048 3 mm 3 mm

ST [19]

LSM9DS1 2000/14.3 16/1366 3.5 mm 5 mm

In this study, a high-speed camera was used for two

purposes. It provides validation for the inertial information

received from sensors, and for auto-segmentation in the data

processing. We considered eight major parameters while

selecting the camera for our system: frame rate, image

resolution, exposure time (shutter speed), sensitivity, bit depth,

colour or monochrome, and camera interface.

Since the users’ playing approach varied, we chose the

BASLER acA2000-165um camera, which is capable of

freezing fast moving objects in an indoor sports center

environment as well as providing high definition. Table 3

shows the specifications of the acA2000-165um camera.

TABLE 3 SPECIFICATIONS OF THE ACA2000-165 µM CAMERA [20]

Product acA2000-165 µm

Resolution 2,048 px×1,088 px

Frame rate 165 fps Mono/Color ±4800 µT

Interface USB 3.0

Exposure control Programmable via camera API Pixel depth 10,12 bits

B. Interface Software

To receive and visualize the IMU data collected from the

BLE peripheral node, we wrote a software application on the

mobile phone. This mobile application is based on the

Evothings framework, a development tool to create the mobile

apps for Internet of Things (IoT). It is an open-source software

developed with Java Script programming language. The

software we developed can be divided into three modules: BLE

Connection, Sensor Data Display and Sensor Data Cloud. The

BLE connection module is based on Evothings and Cordova

BLE Plugin that implements BLE support for Android, IOS,

and Windows 8.1. Fig. 3 shows the inertial sensor data from a

badminton player when he is smashing.

Our system adopts a new cloud-based method to save the

data received from BLE into the remote server. Moreover, this

method also supports building a cloud-based badminton actions

database for the use of other researchers. In this module, the

Cordova HTTP plugin is used to recognize the cloud saving

function. Once the collecting process ends, any user can look

up the sensor data by visiting a designated website.

Fig. 3. Android mobile app software for collecting and displaying 3-axes sensor

data from the WSD.

http://vr95.com:8006/all.aspx




4

III. METHODS

Our experiment was conducted at HU FA KUANG Sport

Centre in City University of Hong Kong. We recruited twelve

right-handed male badminton players, including four amateurs,

four sub-elites and four elite badminton players. Their

demographics are shown in Table 4.

The experimental procedures were reviewed and approved by

the Ethics Committee of the City University of Hong Kong, and

all participants provided written informed consent before

participation. Elite players represented their region and had

played in international competitions more than 10 times.

Sub-elite players played in local competitions but had no

experience playing in international competitions. Amateurs are

badminton beginners who have never played in competitions.

Table 4 shows specific physical information about all the

subjects. As badminton is a wrist-based sport, each

right-handed subject wore the designed sensor on their right

wrist when performing badminton basic training. Such

configuration is comfortable and unobtrusive.

After a 20-minute warmup supervised by a professional

coach, each subject performed 20 straight smashes, short drops,

and long clears, respectively. As shown in Fig. 4, the coach

served the shuttlecock to position “1” (for drops), position “2”

(for smashes), and position “3” (for clears). The subject

performed the actions at different positions. Every subject had

to hit the shuttlecock to the destination inside the right half

court; otherwise, we did not count it as a successful action.

Fig. 5 shows the six-axis synchronized raw data from

players at different levels. Fig. 6 displays the raw data captured

by the WSD. The first two rows show the angular velocity and

acceleration from a clear action, while the second and third two

rows show the inertial information from 6-axes for the drop and

smash actions.

Fig. 4. The positions of badminton players to perform drops (1), smashes (2)

and clears (3).

Fig. 5. Example sensor data recorded at wrist during smash. Angular velocities

of (a) an amateur, (c) a sub-elite and (e) an elite and accelerations of the (b) amateur, (d) sub-elite and (f) elite are crucial to discriminate the skill levels.

Fig. 6. Raw sensor data plot from three different strokes.




5

Fig. 7. Data processing flow to recognize badminton activities and assess skill levels from a single wrist-worn sensor.

TABLE 4 SUBJECT DESCRIPTIONS

Subject Age Height Body Mass

Elite A 22 184 cm 80 kg

Elite B 25 179 cm 74 kg

Elite C 22 175 cm 68 kg

Elite D 21 182 cm 75 kg

Sub-elite A 29 175 cm 74 kg

Sub-elite B 26 180 cm 76 kg

Sub-elite C 25 174 cm 70 kg

Sub-elite D 22 176 cm 69 kg

Amateur A 25 183 cm 70 kg

Amateur B 26 179 cm 71 kg

Amateur C 28 174 cm 74 kg

Amateur D 23 170 cm 64 kg

IV. DATA PROCESSING

A. Badminton Actions Recognition System

After data collection, a typical machine learning data

processing method was implemented in our badminton actions

recognition and skill level assessment system shown in Fig. 7.

This framework includes preprocessing, segmentation, feature

extraction, dimensionality reduction and classification. Each

stage of this framework can be implemented using a variety of

methods. To demonstrate performance in this section, we

utilized the Support Vector Machine (SVM) classifier.

In data preprocessing, data points associated with the subject

failing to hit the shuttlecock inside the target area were

removed. We first loaded the raw data Ṧ(t)ij from each subject.

Then we applied a 3-point filter moving average to reduce the

effect of noise and obtain a clearer S(t)ij signal. The statistical

and morphology features were extracted and each dataset Ẋi =

(f1 …fm) was merged into a large matrix Ẋ. Segmentation was

processed automatically by finding the peak of the signal. This

window-based method can realize real-time data processing

[21].

In this study, we extracted 15 statistical features and 3

morphological features as inputs for badminton actions

recognition and classification, as detailed in Table 5. These

features included 1) mean and variance from the six axes and

root mean square (RMS) from three acceleration axes; 2) the

maximum acceleration in x-axis, 3-axis acceleration data and

3-axis angular velocity data. We compiled a badminton actions

(smashes, clears and drops) database from the inertial sensor.

a) Principle Component Analysis (PCA)

Principle components were identified to alleviate the

computing load and bandwidth requirements during

communication with the cloud server. We used PCA to

preprocess the data before classification because PCA shows

better performance compared to nonlinear dimensionality

reduction [22].

Eighteen features extracted from the raw badminton actions’

data can be expressed as vectors, where f=[f1,f2,…,f18]. These

new features are linear combinations of the original features

and can be expressed as fn=[fn1,fn2,…fnm], where m represents

the dimension to be reduced:

1 1 2 2= + +m i i mi mf a f a f a f (1)

where aij are eigenvalues of the covariance matrix. As we have

only one node, (1) can be simplified to

1 1 2 2 m= + +m mf a f a f a f . (2)

b) Support Vector Machine

Support vector machine (SVM) is a supervised learning

algorithm used for solving a binary classification problem. As

shown by the Mercer’s condition [23], SVM exhibits some

distinct advantages such as good generalization ability, and

robustness by free choice of model parameters in processing

high dimensional and linear inseparable problems over other

supervised learning algorithms, such as Naïve Bayes and

Linear discriminant analysis. This algorithm is also suitable to

our application because there are some intersections for clears

and smashes action (patterns are similar). That comprises a

linear inseparable problem.




6

TABLE 5 STATISTICAL AND MORPHOLOGICAL FEATURES

No. Symbol Description

1 Aax Mean value of acceleration from x axis

2 Aay Mean value of acceleration from y axis

3 Aaz Mean value of acceleration from y axis

4 Dax Variance of acceleration from x axis

5 Day Variance of acceleration from y axis

6 Daz Variance of acceleration from z axis

7 Agx Mean value of angular velocity from x axis

8 Agy Mean value of angular velocity from y axis

9 Agz Mean value of angular velocity from z axis

10 Dgx Variance of angular velocity from x axis

11 Dgy Variance of angular velocity from y axis

12 Dgz Variance of angular velocity from z axis

13 Max The maximum acceleration from the x-axis

14 Ma The magnitude of the 3-axis acceleration

15 Mg The magnitude of the 3-axis angular velocity

16 RMSax Root mean square of acceleration from x axis

17 RMSay Root mean square of acceleration from y axis

18 RMSaz Root mean square of acceleration from z axis

In this study, as we have three labels including smash, clear,

and drop, we choose a one-versus-one strategy [24] in which

three SVMs are constructed using corresponding data from the

other two classes and then a voting scheme is applied. This is a

binary classification problem solved by SVM:

Given a training data set 1

, , 1, 1n

m

i ii

T y R y

X X

where, X is a m-dimensional matrix; yi is a binary label, which

belongs to either 1 or -1; n is the total number of samples; and i

is the current sample number. The main idea of using SVM is

to map the training data set into a higher-dimensional feature

space and then to classify the training data set with hyperplanes.

The problem that finds the maximum margin hyperplane

(MMH) can be converted to an optimization problem that can

be described as follows:

,

2

argmin1

2b

. .s t ( ) 1, 1,...,iy b i n i

x

(3)

where is a normal vector of a hyperplane and b is an offset of

a hyperplane from the origin along the normal vector.

According to the Lagrangian multipliers under the

Karush-Kuhn-Tucker (KKT) conditions, (3) can be

reformulated as follows

2270

1

( , , )

1

21

nT

i ii

L b

y b

ix (4)

where α represents the Lagrangian multipliers vector. The

derivative of (4) with respect to results in

270

1

n

i i

i

y

ix (5)

The derivative of (4) with respect to b results in

270

1

0n

i i

i

y

(6)

Then, after (5) and (6) are substituted into (4), we obtain a

simplified Lagrangian dual problem.

270 270

1 1

1arg max

2

n n

i i j i ji i

y y

i jx x

. .s t 0, 1,...,i i n

270

1

0n

i ii

y

(7)

Since there are some overlap data from clears and smash

actions, which means that our case is not linear separable, we

add a slack variable ξi and an error penalty constant C to find a

tradeoff between a large margin and an error penalty.

Following the aforementioned procedure, we obtained the

simplified Lagrangian dual problem in the case of non-linear

separable problems as:

270 270

1 1

1arg max

2

n n

i i j i j

i i

y y

i jx , x

. .s t 0, 1,...,iC i n

270

1

0n

i i

i

y

(8)

By using Sequential Minimal Optimization (SMO) algorithm

[25], we can obtain the Lagrange multipliers αi. According to

(4), we can calculate the final and find an optimization

hyperplane. The decision function for classification is:

90

1

sgnn

T

i i

i

d y b

X i jx , x (9)

where yi refers to the class label of a support vector; i and b0

refer to two constants; and X refers to the testing set of

badminton actions samples whose labels are yi. To investigate

the influence of parameters on classification performance, we

randomly chose some parameters for testing tabulated in Table

6.

360 datasets were collected from 12 subjects, each of whom

performed 30 trials for three different actions. We used nine

subjects’ datasets (270 datasets) for the training, and the rest of

the datasets (90 datasets) from a different three subjects for

testing classifier performances. During the training process, we

used 10-fold cross validation to avoid the overfitting problem

and find the best parameters of the SVM classifier.




7

TABLE 6 RANDOMLY CHOSEN PARAMETERS OF SVM

Penalty Parameter (C) Gamma Kernel

1 0.0001 Linear

100 0.0005 Polynomial

1000 0.001 RBF

5000 0.005 Sigmoid

10000 0.01

50000 0.1

We compared C values ranging from 1 to 50000, Gamma

values ranging from 0.0001 to 0.1, and several different types

of kernels. We achieved the best classifier when C=1, and when

using the linear kernel function. Table 7 shows the average

classification results when using SVM following PCA

(SVM+PCA).

TABLE 7 SVM+PCA CLASSIFICATION RESULTS OF RECOGNIZING DIFFERENT

STROKES

Actions Precision Recall F1-score

Clears 91% 1.00 0.95

Drops 100% 1.00 1.00

Smash 100% 0.90 0.95

AVERAGE 97% 0.97 0.97

As shown in Table 7, the recognition accuracies for three

different actions (clears, drops, and smash) are 91%, 100% and

100%, respectively. This result demonstrated clear distinction

between different actions. On average, the precision of

classifying different actions can reach 97%, which means our

system is highly effective.

B. Skill Assessment System

Similar to the above analysis, we changed the label from

different actions to different skill levels. We labeled three

different levels (Elite, Sub-elite and Amateur) as shown in the

Table 8. We used nine subjects’ datasets (90 datasets) from

each group for the training, and the rest (30 datasets) from

another three different subjects from each group for testing

classifier performances. 10-fold cross validation was used

again to avoid the overfitting problem and find the best

parameters of the SVM classifier. This was repeated 64 times

to ensure that all possible combinations of testing sets with

three subjects of different skill levels were covered. Table 8,

Table 9 and Table 10 show the skill assessment results in terms

of different actions.

TABLE 8 SVM+PCA CLASSIFICATION RESULTS OF SKILL ASSESSMENT IN

SMASH STOKES

Skill Level Precision Recall F1-score

Elite 100% 0.90 0.87

Sub-elite 70% 0.77 0.74

Amateurs 80% 0.78 0.89

AVERAGE 83.3% 0.82 0.83

As shown in the Table 8, the recognition accuracy of elite,

sub-elite, and amateur players through smash strokes are 100%,

70% and 80% respectively. For skill assessment of clear

strokes as shown in Table 9, the average classification precision

is 90.3%, which demonstrates clear distinction in performance

between amateurs to elites.


CLEAR STROKES


Elite 100% 1.00 1.00

Sub-elite 82% 0.90 0.86

Amateurs 89% 0.80 0.84

AVERAGE 90.3% 0.90 0.90


DROP STROKES


Elite 100% 0.45 0.62

Sub-elite 0% 0.00 0.00

Amateurs 0% 0.00 0.00

AVERAGE 33% 0.15 0.21

On the contrary, the classification accuracy of skill level from

wrist motion during drop stokes is very low, particularly for

sub-elite players and amateurs, as shown in Table 10.

C. Comparison of Different Classifiers

For stroke recognition, we compared k-Nearest-Neighbor

(kNN) non-parametric classifier and Naïve Bayes (NB)

classifier, as shown in Table 11, to determine whether

SVM+PCA is the best classifier for our data. We tested

different k values (from 1 to 11) to find the best estimator for

our data, achieving the best model results when k=5. The

results from testing two other algorithms demonstrated that the

computational efficient of PCA+SVM is also sufficiently

accurate.

TABLE 11 BADMINTON ACTIONS CLASSIFICATION ACCURACIES OF DIFFERENT

ALGORITHMS

Classification

Algorithm

Parameters Accuracy

SVM+PCA C = 1,

Gamma =0.0001 97%

SVM C = 1,

Gamma =0.0001 94%

Nearest

Neighbor

K = 5 94%

Naïve Bayes N.A. 90%




8

TABLE 12 SMASH LEVEL CLASSIFICATION ACCURACIES OF DIFFERENT

ALGORITHMS

Classification

Algorithm

Parameters Accuracy

SVM+PCA C = 1,

Gamma =0.001 83.3%

SVM C = 1,

Gamma =0.001 74%

Nearest

Neighbor

K = 5 78%


Table 12 and Table 13 compare skill assessment system

performance in a similar way. The results for smash and clear

are listed only, because the average accuracy of assessing skill

levels in drops is very low. Again, PCA+SVM shows

advantages in dealing with a linear inseparable problem over

k-Nearest-Neighbor and Naïve Bayes classifier.

TABLE 13 CLEAR LEVEL CLASSIFICATION ACCURACIES OF DIFFERENT

ALGORITHMS

Classification

Algorithm

Parameters Accuracy

SVM+PCA C = 1,

Gamma =0.001 90%

SVM C = 1,

Gamma =0.001 86%

Nearest

Neighbor

K = 7 82%


V. DISCUSSION

Quantifying sport activities is of great interest since it allows

trainers and coaches to assess an athlete’s performance. This

paper presents a smart badminton actions recognition system

comprising of Bluetooth Low Energy technology, MEMS

inertial measurement unit, Cloud technology, and machine

learning algorithms. The complete platform for badminton

actions motion data analysis includes three parts: a wearable

sensor, a mobile app, and cloud server. The data collected by

an inertial measurement unit (IMU) is sent to a mobile phone

through Bluetooth Low Energy (BLE). Once the mobile phone

receives the data, it sends the motion data to the remote server

by Cloud technology. After data collection, users can analyze

badminton players’ data on a server in real time or afterwards.

The results shown in Table 7 indicate that our smart system

can classify at least three different badminton strokes clearly

with an average accuracy of 97%. The system can

automatically provide data statistics of badminton players,

which can help coaches and athletes to learn about real

condition changes during a match or a training session.

Classification results of the action recognition system strongly

support the assumption that wrist motion is crucial in

badminton playing. And, we have shown that different strokes

require distinct wrist motion in execution.

As for the skill assessment system, test samples from elite

players are recognized precisely, showing that the elite players

have a distinctive motion compared to the other two groups of

players in all strokes. It is thus feasible to identify level of

performance from wrist motion data of smash and clear. Based

on the results thus far, we could possibly identify elite players

just by observing their clear strokes. However, drop strokes

are relatively flexible. Moreover, amateurs and sub-elites play

similarly while elite players’ drop strokes are consistently

different from those of amateur and sub-elites. Using this

system, we can compile a database of badminton action

movements from players at different levels, which can then be

used by sports scientists and professional coaches for further

study and research.

Motion analysis is an important factor in building

self-awareness of athletes in playing sports. Using MEMS

sensor to capture motion data can help badminton players or

other racket sports players improve their skills, which will play

a significant role for next-generation racket sports training.

With the advances in MEMS sensors and wireless

communication technology, as well as cloud computing, it is

possible to use wearable sensing devices to automatically

recognize different actions that can provide statistics during

matches, which will allow athletes themselves or their coaches

to assess their performance in real time. In this study, we

investigated some major features in characterizing each data

segment. The performance levels represented by the skill data

were thus estimated by SVM, kNN and NB classifiers.

Comparisons of these classifiers show SVM achieves high

accuracy in stroke recognition (97%) and in assessing levels of

players in executing clear stokes (90.3%). Therefore, we

envision that the IoT framework presented in this paper will

play an important role in sports analysis where wrist actions are

important.

ACKNOWLEDGMENT

The authors thank CityU Men’s Badminton Team and South

China Athlete Association (SCAA) for their participation in

this study. The authors would also like to thank the Hong Kong

Innovation Technology Commission (Project no. UIM/326),

the Hong Kong Research Grants Council (Project no.

CityU/11213817), and the Shenzhen Overseas High Level

Talent (Peacock Plan) Program (grant number:

KQTD20140630154026047) for partially funding this research

project.

REFERENCES

[1] A. Lees, “Science and the major racket sports: A

review,” J. Sports Sci., vol. 21, no. 9, pp. 707–732,

2003.

[2] J. Wang and J. Moffit, “Teaching Badminton Based on

Student Skill Levels.,” Strateg., vol. 22, no. 6, pp.

14–18, 2009.

[3] M. Kwan, M. S. Andersen, C. L. Cheng, W. T. Tang,

and J. Rasmussen, “Investigation of high-speed

badminton racket kinematics by motion capture,”

Sport. Eng., vol. 13, no. 2, pp. 57–63, 2011.

[4] G. Zhu, Q. Huang, C. Xu, L. Xing, W. Gao, and H.

Yao, “Human behavior analysis for highlight ranking

in broadcast racket sports video,” IEEE Trans.

Multimed., vol. 9, no. 6, pp. 1167–1181, 2007.

[5] M. Nagasawa, Y. Hatori, M. Kakuta, T. Hayashi, and




9

Y. Sekine, “Smash motion analysis for badminton from

image,” in Proceedings of the IEEE Image Electronics

and Visual Computing Workshop, 2012, pp. 1–8.

[6] Z. Chen, R. Li, C. Ma, X. Li, X. Wang, and K. Zeng,

“3D vision based fast badminton localization with

prediction and error elimination for badminton robot,”

in Proceedings of the World Congress on Intelligent

Control and Automation (WCICA), 2016, pp.

3050–3055.

[7] I. Bisio, A. Delfino, F. Lavagetto, and A. Sciarrone,

“Enabling IoT for In-Home Rehabilitation:

Accelerometer Signals Classification Methods for

Activity and Movement Recognition,” IEEE Internet

Things J., vol. 4, no. 1, pp. 135–146, 2017.

[8] T. Mitsui, S. Tang, and S. Obana, “Support system for

improving golf swing by using wearable sensors,” in

2015 8th International Conference on Mobile

Computing and Ubiquitous Networking, ICMU 2015,

2015, pp. 100–101.

[9] J. Lee, “A Three-Dimensional Motion Anlaysis of

Horse Rider in Wireless Sensor Network

Environments,” Int. J. Adv. Comput. Sci. Appl., vol. 5,

no. 11, pp. 50–55, 2014.

[10] J. M. Jarning, K.-M. Mok, B. H. Hansen, and R. Bahr,

“Application of a tri-axial accelerometer to estimate

jump frequency in volleyball,” Sport. Biomech., vol.

3141, no. May, pp. 1–11, 2015.

[11] S. Akpinar, E. Devrilmez, and S. Kirazci,

“Coincidence-Anticipation Timing Requirements are

Different in Racket Sports,” Percept. Mot. Skills, vol.

115, no. 2, pp. 581–593, 2012.

[12] H. Ghasemzadeh and R. Jafari, “Body sensor networks

for baseball swing training: Coordination analysis of

human movements using motion transcripts,” in 2010

8th IEEE International Conference on Pervasive

Computing and Communications Workshops,

PERCOM Workshops 2010, 2010, pp. 792–795.

[13] A. Ahmadi, D. Rowlands, and D. A. James, “Towards a

wearable device for skill assessment and skill

acquisition of a tennis player during the first serve,”

Sport. Eng., vol. 2, no. 3–4, pp. 129–136, 2010.

[14] Y. Wang, Y. Zhao, R. H. M. Chan, and W. J. Li,

“Volleyball Skill Assessment using a Single Wearable

Micro Inertial Measurement Unit,” IEEE

Access,accepted.

[15] D. Semiconductor, “Bluetooth Low Energy 4.3,” 2016.

[Online]. Available:

https://www.dialog-semiconductor.com/sites/default/fi

les/da14583_ds_3v0.pdf.

[16] Z. Wang, C. Zhao, and M. Guo, “Badminton Stroke

Recognition and Skill Assessment Based on Body

Sensor Networks,” IEEE Trans. Human-Machine Syst.,

vol. 46, no. 5, pp. 1–14, 2016.

[17] Bosch Sensortec, “BMI160 Datasheet,” Prod. Specif.,

pp. 1–4, 2015.

[18] S. Jose, “MPU9250 Datasheet,” Prod. Specif., vol. 1,

no. 408, pp. 1–4, 2016.

[19] ST-Microelectronics, “LSM9DS1 Datasheet,” no.

August, pp. 1–74, 2013.

[20] Basler, “Basler ace User Manual,” 2011. [Online].

Available:

https://graftek.biz/system/files/2576/original/Basler_A

ce_USB_3.0_Manual.pdf?1479057814.

[21] G. Okeyo, L. Chen, H. Wang, and R. Sterritt,

“Dynamic sensor data segmentation for real-time

knowledge-driven activity recognition,” Pervasive

Mob. Comput., vol. 10, pp. 155–172, 2014.

[22] Y. Bengio and P. Vincent, “Out-of-Sample Extensions

for LLE , Isomap , MDS , Eigenmaps , and Spectral

Clustering,” NIPS Proc., pp. 1–10, 2003.

[23] W. M. Campbell, D. E. Sturim, and D. A. Reynolds,

“Support vector machines using GMM supervectors for

speaker verification,” IEEE Signal Process. Lett., vol.

13, no. 5, pp. 308–311, 2006.

[24] E. Approaches, A. Rocha, S. K. Goldenstein, and S.

Member, “Multiclass from Binary : Expanding

One-Versus-All,” IEEE Trans. Neural Networks

Learn. Syst., vol. 25, no. 2, pp. 289–302, 2014.

[25] S. S. Keerthi and S. K. Shevade, “SMO algorithm for

least-squares SVM formulations.,” Neural Comput.,

vol. 15, no. 2, pp. 487–507, 2003.




10

Yufan Wang received his BS degree in

communications engineering from Beijing

Jiaotong University (BJTU) in 2014,

where he was also a volleyball player in

the varsity volleyball team. His BJTU

team won the Chinese University

Volleyball Association (CUVA) Super

Cup Championship in Nanjing in 2010.

Since 2014, he has been pursuing his Ph. D. research work at

the City University of Hong Kong. He has represented CityU

to win the Volleyball Champion of the University Federation of

Hong Kong (USFHK) in 15/16; he has also represented Hong

Kong Men’s Volleyball Team to win the Volleyball Champion

of the Four-Regions (Mainland China, Taiwan, Hong Kong,

and Macau) in 15/16.

His current research interests are in the area of wearable

cyber physical devices, inertial measurement unit, artificial

intelligence, and sports motion analysis.

Meng Chen received his B. S. degree in

software engineering from Shenzhen

Univeristy, Shenzhen, China, in 2009. He

worked as a research assistant in City

University of Hong Kong’s Shenzhen

Reasearch Institute from 2013 to 2015, in

City University of Hong Kong from 2015

to 2016, and in Shenzhen Academy of

Robotics from 2016 to 2017. He has been pursuing his Ph. D.

degree at the City Univerisity of Hong Kong since 2017. His

research interests include wireless sensor networks, network

transmission protocols, and artificial intelligence.

Xinyu Wang is currently a Ph. D. student

at the Department of Computer Science,

City University of Hong Kong. He

received his B. E. degree from East China

Jiaotong University in 2011. He served as

a software engineer in Tencent during

2011 to 2013, developing information

security systems. He was a research

assistant at the City University of Hong Kong from 2013 to

2016, and worked on research topics related to the security of

applications and systems. His research interests include cloud

computing, network security, and big data.

Rosa H. M. Chan (M’01-SM’17) is

currently an Associate Professor in the

Department of Electronic Engineering at

City University of Hong Kong. She

received the B. Eng. (1st Hon.) degree in

Automation and Computer-Aided

Engineering from The Chinese University

of Hong Kong in 2003. She was later

awarded the Croucher Scholarship and Sir Edward Youde

Memorial Fellowship for Overseas Studies in 2004. She

received her Ph. D. degree in Biomedical Engineering in 2011

from the University of Southern California (USC), where she

also received her M. S. degrees in Biomedical Engineering,

Electrical Engineering, and Aerospace Engineering. Her

research interests include mathematical modeling of neural

systems, development of neural prostheses, and brain-machine

interface applications.

Wen J. Li (F’11) received his B. S. and

M. S. degrees in aerospace engineering

from the University of Southern

California (USC), in 1987 and 1989,

respectively, and his Ph. D. degree in

aerospace engineering from the University

of California, Los Angeles (UCLA), in

1997.

He is currently a chair professor in the Department of

Mechanical and Biomedical Engineering, City University of

Hong Kong. From September 1997 to October 2011, he was

with the Department of Mechanical and Automation

Engineering, The Chinese University of Hong Kong. His

industrial experience includes the Aerospace Corporation (El

Segundo, CA), NASA Jet Propulsion Laboratory (Pasadena,

CA), and Silicon Microstructures, Inc. (Fremont, CA). His

current research interests include intelligent cyber physical

sensors, super-resolution microscopy and nanoscale sensing

and manipulation. Dr. Li is an IEEE Fellow and served as the

President of the IEEE Nanotechnology Council in 2016 and

2017.

IoT for Next-Generation Racket Sports Training · 2019-06-21 · practice. Index Terms—IoT...

Documents

Transcript of IoT for Next-Generation Racket Sports Training · 2019-06-21 · practice. Index Terms—IoT...