Research ArticleHuAc Human Activity Recognition Using Crowdsourced WiFiSignals and Skeleton Data
Linlin Guo Lei Wang Jialin Liu Wei Zhou and Bingxian Lu
Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province School of SoftwareDalian University of Technology Dalian China
Correspondence should be addressed to Lei Wang leiwangieeeorg
Received 18 September 2017 Accepted 27 November 2017 Published 11 January 2018
Academic Editor Kuan Zhang
Copyright copy 2018 Linlin Guo et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
The joint of WiFi-based and vision-based human activity recognition has attracted increasing attention in the human-computerinteraction smart home and security monitoring fields We propose HuAc the combination of WiFi-based and Kinect-basedactivity recognition system to sense human activity in an indoor environmentwith occlusion weak light and different perspectivesWe first construct a WiFi-based activity recognition dataset named WiAR to provide a benchmark for WiFi-based activityrecognition Then we design a mechanism of subcarrier selection according to the sensitivity of subcarriers to human activitiesMoreover we optimize the spatial relationship of adjacent skeleton joints and draw out a corresponding relationship between CSIand skeleton-based activity recognition Finally we explore the fusion information of CSI and crowdsourced skeleton joints toachieve the robustness of human activity recognition We implemented HuAc using commercial WiFi devices and evaluated it inthree kinds of scenarios Our results show that HuAc achieves an average accuracy of greater than 93 using WiAR dataset
1 Introduction
Human activity recognition is an important research problemin the social life pervasive computing and security monitor-ing fields [1ndash3] Daily activities [4] were seen as an importantmeans of communicating in our daily life and we cancommunicate through body language like hands and headrather than speaking Therefore human activity recognitionsystems have been proposed in terms of application demandtechnical support and auxiliary devices
Previous works related to activity recognition are roughlydivided into three categories including wearable-basedvision-based and WiFi-based Wearable-based sensing be-havior has been popular and widely used in elder healthcaresmart sensing sports application and tracking [1 5 6]Researchers leverage the collecting information via sensorsto recognize human behavior and analyze human health con-dition However it has several limitations such as increasingthe burden of users the inconvenience of routine life andsensors with limited power Vision-based activity recognitionhas been popular and achieves high accuracy The lightshadowing privacy protection and angle factors increase the
difficulty of activity recognition and constrain the applicationfields Microsoft released Kinect technology and Kinect canprovide skeleton information using built-in sensors [7 8]Although Kinect-based activity recognition solves the light-environment problem and can track the skeleton joints of anactivity with high accuracy it cannot recognize the imperfectactivity due to the crowded room the presence of obstacles andout of the monitoring range
With the coverage ofWiFi signals and the improvement ofwireless infrastructures in public places WiFi-based activityrecognition systems [4 9ndash11] leverage the change pattern ofWiFi signals reflected by a human body to recognize theactivity WiFi-based activity recognition systems [12ndash14] notonly ease the burden of wearable-based users but also cansense the presence of obstacles in comparison with Kinect-based works For example WiVi [14] can sense the userrsquosbehavior through the wall and RF-Capture [11] tracks the3D positions of a human body when the person is occludedcompletely and captures the human figure without wearabledevices
We are interested in BodyScan system [15] and it isestimated on the idea of the combination of the wearable
HindawiWireless Communications and Mobile ComputingVolume 2018 Article ID 6163475 15 pageshttpsdoiorg10115520186163475
2 Wireless Communications and Mobile Computing
sensors andWiFi signals Moreover it overcomes key limita-tions of existing wearable devices by providing a contactlessand privacy-preserving approach to capture a rich varietyof human activities Based on this work we explore thecombination of CSI and skeleton data to sense human behav-ior According to the works mentioned above we explorethree issues of activity recognition in this paper First weconstruct a WiFi-based activity recognition dataset namedWiAR to provide a benchmark for previous works Secondwe design the mechanism of subcarrier selection to improvethe robustness of activity recognition in the WiAR datasetThird we combine WiFi signals with crowdsourced skeletondata to improve the accuracy and robustness of activityrecognition breaking the limitations of Kinect technologyThe contributions of our work are summarized as follows
(i) We propose a HuAc system to recognize humanactivity and also construct a WiFi-based activityrecognition dataset named WiAR as a benchmark toevaluate the performance of existing activity recogni-tion systems We use the kNN Random Forest andDecision Tree algorithms to verify the effectiveness ofthe WiAR dataset
(ii) We detect the start and end of the activity usingthe moving variance of CSI Moreover we leverage119870-means algorithm to cluster effective subcarriersaccording to subcarrierrsquos sensitivity and improve therobustness of activity recognition
(iii) We develop a selection method of skeleton jointsbased on KARDrsquos work named SSJ and it considersthe spatial relationship and the angle of adjacentjoints as auxiliary information of human activityrecognition to improve the accuracy of tracking
(iv) We implement the fusion framework of CSI andskeleton data to sense the activity and solve thelimitations of CSI-based and skeleton-based activityrecognition respectively Experimental results showthat HuAc achieves the accuracy of greater than 93
The rest of this paper is organized as follows We intro-duce the related work in Section 2 Section 3 introducespreliminaries of WiFi-based activity recognition and wedescribe the overview of HuAc in Section 4 Section 5describes Kinect module and WiFi module is shown inSection 6 Section 7 describes the process of human activityrecognition Section 8 evaluates the performance of HuAcsystem and we give a case study about a motion-sensinggame using WiFi signals in Section 9 Section 10 lists severaldiscussions and we give the conclusion of this paper inSection 11
2 Related Work
In this section related works on human activity recognitioncan be divided into two categories Kinect-basedWiFi-based
21 Kinect-Based Activity Recognition Vision-based activityrecognition has been proposed and developed in the com-puter vision field With the release of Kinect researchers
explore the human activity recognition using depth informa-tion and skeleton joints data provided by Kinect [7 8 16]Biswas and Basu [8] leverage the histogram of depth infor-mation to recognize eight gestures Moreover the differencesbetween continuous frames can obtain the motion profile todescribe various gestures Other works [7 16] leverage depthinformation in combination with color image to improve theaccuracy of gestures recognition The limitations of Kinect-based activity recognition contain the restriction of sensingfield skeleton joints overlapping and position-dependencefactors HuAc system explores the spatial relationship ofskeleton joints to describe the trajectory of an activity andcombines with CSI to improve the robustness of humanactivity recognition in a dynamic environment
22 WiFi-Based Activity Recognition Early works [17ndash19]explore the attenuation characteristics of WiFi signals tolocate the position of someone and count the number ofpeople in the indoor environment Researchers study thesignal pattern reflected by a human body to sense humanbehavior [11 20ndash22] These works describe human behav-ior recognition using coarse-grained RSSI information Forexample WiGest [18] studies the relationship between RSSIfluctuation and gestures to controlmedia player actions with-out training Therefore we explore the relationship betweenRSSI fluctuation and humanmovement to detect the presenceof an activity
With the requirement of the practical application andthe limitations of RSSI an increasing number of researchersbegin to explore fine-grained channel state information (CSI)to sense human behavior Compared with RSSI CSI cancapture the tiny behavior [2 9 23ndash28] in terms of locationspeed and directionWiFall system [2] detects a fall behaviorby learning the specific CSI pattern E-eyes [9] recognizeswalking activity and in-place activity by adopting movingvariance of CSI and fingerprint technique Walking activitycauses significant pattern changes of the CSI amplitudeover time since it involves significant body movementsand location changes In-place activity (watching TV) onlyinvolves relative smaller body movements and will not causesignificant amplitude changes with repetitive patterns Therelationship between an activity and the place where anactivity occurs motivates the novel idea on human activityrecognition CARM [10] shows the correlation between CSIvalue and human activity by constructingCSI-speed andCSI-activity model WiDance [28] explores the Doppler shiftsreflected by human behavior to predict the motion direc-tion for the Exergames We design the combination systemof Kinect-based and WiFi-based methods to recognize anactivity in different environments such as gaming systemsupermarket and elder health applications
3 Preliminaries
31 RSSI and CSI Received Signal Strength Indicator (RSSI)[29] in the level of packet represents signal-to-interference-plus-noise ratio (SINR) over the channel bandwidth asfollows
RSSI = 10 lg (1198812) (1)
Wireless Communications and Mobile Computing 3
Number Name(1) Head(2) Neck(3) Right shoulder(4) Right elbow(5) Right hand(6) Le shoulder(7) Le elbow(8) Le hand(9) Torso(10) Right hip(11) Le hip(12) Right knee(13) Right foot(14) Le knee(15) Le foot
62
1
7
8
3
4
5
11 10
9
14 12
15 13
Figure 1 Skeleton joints
where 119881 is signal voltage RSSI is the received signal strengthin decibels (dB) and mapped into the distance accordingto Log-distance path loss model to roughly locate users ordevices
Channel State Information (CSI) depicts multipath prop-agation at the granularity of OFDM subcarrier in the fre-quency domain It contains amplitude and phase measure-ments as follows
ℎ = |ℎ| 119890119895 sin 120579 (2)
where |ℎ| and 120579 are the amplitude and phase respectivelyThe variable ℎ shows CSI value of each subcarrier We studythe characteristics of each subcarrier to sense activity in thefollowing work
32 Kinect Technology Kinect (RGB-D camera) refers to theadvanced RGBdepth sensing hardware and the software-based technology that interprets the GRBdepth informationThe hardware contains a normal RGB camera a depthsensor (infrared projector and infrared camera) and a four-microphone array which is able to provide depth signalsRGB images and audio signals simultaneously Kinect-basedactivity recognition algorithm frequently fails due to occlu-sions overlapping joints (limbs close to the body) or clutter(other objects in the scene) [7] A skeleton reported by Kinectcontains 15 joints in Figure 1 We explore the correspondingrelationship between skeleton joints and CSI to analyzethe characteristics of an activity Moreover we explore thefusion information to improve the accuracy of human activityrecognition The details of Kinect-based activity recognitionare listed in Section 5
33 WiAR Constructing WiFi-Based Activity Dataset Atpresent there is noWiFi-based public activity dataset as wellas vision-based public activity dataset Due to the sensitivityof WiFi signals it is hard for peer researchers to reproduceand evaluate previous works Therefore we construct theWiAR dataset which collectsWiFi signals reflected by sixteenactivities in three indoor environments such as empty roommeeting room and office listed in Table 1 Each activity isperformed 50 times by 10 volunteers which consist of five
Kinect applications
Public Family ResearchCrowdsourcing
Skeleton joints data
Collecting WiFi signals
Input of HuAc system+
Figure 2 The framework of crowdsourced dataset
females and five males and the height of human body rangesfrom 150 cm to 185 cm
The environmental complexity according to the roomlayout divides into three levels including empty environmentnormal environment and complex environment First emptyenvironment describes no people and furniture around itWeobtain the high-quality WiFi signals from the empty roomdue to less noise and treat it as a baseline of WiAR datasetThen the normal environment contains furniture and work-ing people Compared with the empty environment themultipath effect reflected by the furniture enriches collectingWiFi signals Finally a complex environment with furnitureand moving people increases the difficulty of human activityrecognition The performance of WiAR dataset is given inSection 8
34 Crowdsourced WiFi Signals and Skeleton Joints Crowd-sourced-based applications [30ndash37] have been increasinglydeveloped by collecting data and reducing the cost in theInternet field For the macrolevel network the work [30]proposed a crowdsensing-oriented mobile cyber-physicalsystem to provide the practical usage of the vita For themicrolevel wireless network related works [38ndash41] leveragecrowdsensing WiFi signals to detect the userrsquos location
In our work we attempt to collect WiFi signals andcrowdsourced skeleton joints to reduce the training burdenfor collecting activity dataset We obtain the activity labelby leveraging the help from Kinectrsquos user The framework ofcrowdsourced WiFi signals and skeleton joints are shown inFigure 2
4 Overview of HuAc
41 Observations The following observations come from thecombination of our results and previous works [20 42ndash44]
The Impact of Indoor Environment on WiFi Signals Has aDifference with Time RSSI and CSI keep stability in thestatic indoor environment and RSSI fluctuation ranges from0 dB to 5 dB (empty environment 0ndash3 dB home environ-ment 0ndash7 dB office 0ndash5 dB dynamic environment 5ndash10 dB)Although RSSI sharply changes with environmental changeit cannot describe the fine-grained change of indoor environ-ment due to themultipath effectHowever CSI is able to sensethe change of fine-grained environment and detects whathappened in an indoor environment Specifically RSSI onlycan find the environmental change and cannot sense how
4 Wireless Communications and Mobile Computing
Table 1 WiFi-based activity recognition dataset (WiAR)
Granularity Activities Environments Devices
Activity Forward kick side kick bend walk phone sit down squat drinkwater
Empty room meetingroom office
Router laptop with5300 card
Gestures Horizontal arm wave two-hand wave high throw toss paper drawtick draw x hand clap high arm wave
Empty room meetingroom office
Router laptop with5300 card
Subcarrier selection-K-means-
Distribution of CSIFeatures extraction
Outlier detection-Threshold-
Smooth data -Weighted Moving Average-
Preprocessing
PreprocessingJoints overlap
detection
3D Joints normalization
CSI features set
Selection of skeleton joints
Skeleton features set
Posture analysis
Activity recognition
SSJ Selection of skeleton joints
HuAc framework
SVM
WiFi signals
Skeleton joints
Predict_label
Predict_label
Accuracy
Train_label
Train_label
50 100 1500 200 250 300
30
25
20
15
Figure 3 The framework of HuAc system
the environment changes CSI can find what causes environ-mental change and also can recognize how the environmentchanges such as tracking sensing environment and activityrecognition
It Is Hard to Distinguish Similar Activities Existing works [215 45] explore the similar activity recognition For exampleWiFall [2] extracts seven features to describe fall behaviorbecause similar activity causes the similar patterns of CSIand it is difficult to distinguish them only using anomalydetectionThe following RT-Fall system adopts the CSI phasedifference to segment fall and fall-like activities because thephase difference of CSI is a more sensitive signature than CSIamplitude for activity recognition The phase of CSI dependson the variation of LOS (Line-of-Sight) lengthTherefore thebreakthrough point of the similar activity recognition restson the physical difference between similar activities
The Same Activity Operated by Different People Has VariousSignal Patterns According to our observations the amplitudeof CSI reflected by the same activity changes continuously inthe different time and environments Therefore we cannotrecognize activity with high accuracy according to the ampli-tude of CSI The changing pattern of signals reflected by anactivity can describe the characteristic of activity as verifiedby Smokey [25] Therefore we explore the changing patternof signals to recognize an activity
The Impact of Activity with Different Directions on ActivityRecognition In order to explore the impact of direction onactivity recognition we design a simple and clear experiment
on the playground because the playground does not haverich multipath effect and other wireless devices We explorethe impact of four directions including east west north andsouth on the change pattern of signals and the differencebetween face and back to the AP is biggest Moreover CSIdata we collect in the playground contains less noise than thatin an indoor environment
42 Framework of HuAc The HuAc framework consists oftheKinect-basedmodule andWiFi-basedmodule in Figure 3We describe details of each module respectively
Kinect module consists of the preprocessing and postureanalysis We detect the overlap of skeleton joints using thestatisticalmethod and complete the normalization of skeletonjoints In order to obtain effective features of skeleton jointswe analyze postures of an activity according to the sequenceof skeleton joints Moreover we design a selection methodof skeleton joints named SSJ according to the result ofposture analysis Finally we extract features of skeleton jointsaccording to effective skeleton joints and also consider thespatial relationship of adjacent joints as auxiliary informationto sense human activity
WiFi module consists of the preprocessing and featuresextraction In the preprocessing stage we detect and removethe outlier data of an activity sequence according to thevariance of RSSI reflected by an activity After removingoutlier data we leverage the weighted moving average tosmooth the activity data For features extraction we firstanalyze the amplitude distribution of CSI reflected by anactivity to evaluate the sensitivity of the subcarrier on an
Wireless Communications and Mobile Computing 5
(a) (b)
Figure 4 Skeleton structure [7] (a) A skeleton structure contains 15skeleton joints (b) The white circle represents skeleton joints with-out direction such as shoulder and hip The gray circle representsthe neck and the torso which has a weak effect on the upper-bodyactivity and the lower-body activity except the squatThe black circlerepresents normal skeleton joints
activity Then we use 119870-means algorithm to cluster effec-tive subcarriers Finally we extract important features fromeffective subcarriers to improve the stability of human activityrecognition
We use the combination information of CSI features setand the skeleton features set as an input of SVM to recognizehuman activity Compared with the result of predict label wegive a feedback to the previous process of HuAc frameworkby using a train label respectively
5 Kinect Module
We mainly describe the details of Kinect module on thehuman activity recognition Kinect module contains thepreprocessing and posture analysis
51 Preprocessing The collected skeleton data contain emptyvalues due to the overlap of skeleton joints or the occlusionin the motion-sensing game Therefore we need to detectthe overlapping joints and replace the invalid values byrecovering the true value of the overlapping joints Weleverage the relationship between the coordinates of adjacentjoints to detect the overlapping joints Certainly we discardthe sample of an activity when the percent of invalid jointsexceeds the threshold
After recovering the invalid data we normalize thecoordinates of skeleton joints due to the differences of peoplersquosheight and the distance between the user and the sensorThe work [7] extracts 11 joints (except right shoulder leftshoulder right hip and left hip) from 15 joints in Figure 4and we explore 30 subcarriers with the similar patternreflected by a human body Therefore we select 15 jointsto match the 15 subcarriers Let 119869119894 be one of the 15 jointsdetected by the Kinect and the coordinates vector 119891 is givenby
119891 = 1198951 1198952 119895119894 11989514 11989515 (3)
300250 350200 400 450X (cm)
0
100
200
300
400
Y(c
m)
Figure 5 High arm wave tracking using skeleton data The activityhas two active joints (right hand right elbow) and the directionchanges with every clockwise movement However adjacent jointshave the slight change in a certain range
where 119895119894 is the vector containing the 3D normalized coordi-nates of the 119894th joint 119869119894 detected by Kinect Thus
119895119894 = 119869119894119904 + 119879119894 1 le 119894 le 15 (4)
where 119904 is the scale factor which normalizes the skeletonaccording to the distance ℎ between the neck and the torsojoints of a reference skeleton and
119904 = 10038171003817100381710038171198699 minus 11986921003817100381710038171003817ℎ (5)
The translation matrix 119879 needs to set the origin of thecoordinate system to the torso After preprocessing phase weobtain high-quality skeleton data
52 Postures Analysis An activity consists of subactivitysequence over time According to the skeleton structure ahuman body is divided into two parts including upper bodyand lower body Upper body contains five joints (right elbowleft elbow right hand left hand and head) and two baselinejoints (neck torso) as in Figure 4 Lower body contains fourjoints (right foot left foot right knee and left knee) Wereproduce the tracking of skeleton joints using QT tool andplot the trajectory chart of each activity We observe that theadjacent joints keep the similar track in Figure 5 and somejoints have slight movement influenced by human activityFor example when the right elbow and right hand move inthe clockwise direction to complete the horizontal arm wavewe observe that right hip and left hip have slight movement
According to the change of joints sequence we cansegment an activity into several subactivities in terms ofdirection and pause factor Horizontal arm wave behaviorconsists of four postures (subactivities) as in Figure 6 Eachsubactivity roughly contains 14 frames and 119865 119894 representsthe 119894th frame (packet) of the activity reported by Kinect Wecan evaluate the rough activity according to the sequence of
6 Wireless Communications and Mobile Computing
1 2 3 4
F_1 F_17 F_21 F_26 F_34 F_40 F_48 F_57 F_66
Figure 6 Postures of horizontal arm wave
subactivity Except for related joints of each subactivity torsoand hip joints have a weak swing We neglect the impactof weak swing on the activity recognition We pay moreattention to the selection of skeleton joints in the followingsection
53 SSJ Selecting Skeleton Joints We design a selectionmethod of skeleton joints named SSJ to describe a fine-grained subactivity After postures analysis we know therelationship between a subactivity and key skeleton jointsWe expend the coordinated system of human skeleton tominiature coordinated system of subactivity skeleton by theabove-mentioned relationship The miniature coordinatedsystem needs to determine a fixed skeleton joint and differentsubactivities have different fixed skeleton joints For examplewe observe that shoulder joint is a fixed joint from the processof high arm wave behavior Therefore we determine thestarting point coordinate of theminiature coordinated systemcorresponding to the subactivity
6 WiFi Module
We introduce the design details of WiFi module on thehuman activity recognition WiFi module consists of thepreprocessing and features extraction
61 Preprocessing The collected data with noises increasesthe difficulty of activity recognition due to the tiny differencesbetween noises and WiFi signals reflected by a fine-grainedactivity Outlier data also weaken the quality of collectingdata Therefore we detect outlier using the variance-basedmethod and remove high-frequency signals using the low-pass filter Moreover we reduce the sawtooth wave of thefiltered signal by using the weighted moving average
611 Outlier Detection and Removing High Frequency Out-lier has an important impact on the quality of collectingdata because outlier increases or decreases the fluctuationstrength of WiFi signals We analyze the RSSI distributionof an activity to evaluate the possible experience-thresholdThen we combine the variance of RSSI and the experience-threshold to detect outlier After removing outlier data theactivity corresponds to the low-frequency change of CSIaccording to the waveform of CSI reflected by an activityTherefore we adopt the low-pass filter to remove the high-frequency data in Figure 7
612 Weighted Moving Average For filtered signal signaldata still contain sawtooth wave Because CSI is sensitive toindoor layout or human movement and the received CSIfluctuation caused by the environment is hard to distinguishfrom the fluctuation caused by a fine-grained activity There-fore we smooth the CSI data using the weighted movingaverage as proposed in WiFall [2] We randomly select15 subcarriers from 30 subcarriers which correspond to15 skeleton joints of Kinect technology Each CSI streamcontains 15 subcarriers as CSI1CSI2 CSI15 CSI1199051 is thefirst subcarrier of CSI at time 119905 CSI11 CSI1199051 indicatesthe CSI sequence of first subcarrier in the time period 119905 Thelatest CSI has weight 119898 the second latest 119898 minus 1 and so onThe expression of CSI series is shown as follows
CSI1199051 = 1119898 + (119898 minus 1) + sdot sdot sdot + 1 times (119898 times CSI1199051
+ (119898 minus 1) times CSI119905minus11 + sdot sdot sdot + 1 times CSI119905minus119898minus11) (6)
where CSI1199051 is the averaged new CSI The value of119898 decidesin what degree the current value is related to historicalrecords In our study we select119898 according to the experienceand trial method We first set 119898 as 5 which means thelength of 5 packets A weighted moving average algorithmand median filter have the similar effect on the originalsignals recorded by the receiver in Figure 7They can removethe galling of signals and alleviate the sharp change ofsignals With the119898 increasing the weighted moving averagealgorithm becomes more smooth than the low-pass filter andthe median filter Finally we set119898 to 10 because each activityproduces a sharp change in 10 packet periods62 Feature Extraction Plenty of related works summarizethe importance of features extraction for human activityrecognition in a dynamic indoor environment We segmentactivity after smoothing CSI and extract features of eachactivity according to activity characteristics Kinect-basedfeatures extraction quotes the work [3]
621 Activity Segmentation Activity segmentation mainlydetects the start and end of an activity and removes thenonactivity packets from a sample which corresponds tothe whole activity We propose two methods to detect thestart and end of an activity and improve the robustness ofsegmentation algorithm First we remove the first secondand the last-second data sequence of an activity to reduce
Wireless Communications and Mobile Computing 7
Original signal Median filter
Low-pass filter Weighted moving average
15
20
25A
mpl
itude
of C
SI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
200100Packets
18
20
22
Am
plitu
de o
f CSI
100 2000Packets
Figure 7 Methods of signal filtering
the error of true activity sequence in our experimentalenvironment But this method is invalid in the practicalenvironment due to the unknown time which each activitystartsTherefore we leveragemoving variance ofCSI to detectthe start and end of each activity Moving variance of CSIdescribes the difference of the local packets reflected by theactivity Packet sequences on the corresponding activity aredefined as119883 = 1199091 1199092 119909119899119883 represents data sequence (asample) of an activity and 119909119894 represents the 119894th packet in thedata sequenceWe often use the standard deviation instead ofthe variance of CSI as follows
120590119894 = radicsum1198981 (119909119894+119895minus1 minus 119909)2119898 (119894 = 1 2 119899 minus 119898) (7)
where 119898 represents step-size and 119909 is the mean value ofsamples
We construct a window per 10 packets from the packetsequence of each sample and compute the variance of thewindow Then we construct the moving variance histogramand compare with other strength windows Finally we candetect the sharp points of each activity and roughly recognize
Subplot 1 two-hand wave
Subplot 2 hand clap
Sharp change Activityperiods20
22
24
Am
plitu
de o
f CSI
21
23
25
Am
plitu
de o
f CSI
50 100 150 200 2500Packets
50 100 150 2000Packets
Figure 8 Segmentation point of similar activity
the start and end of each activity from the data sequenceThestart and end of the activity period are shown in Figure 8Thered circle describes a sharp change of CSI at the start point ofcollecting data but it is not the true start of an activityThe red
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
2 Wireless Communications and Mobile Computing
sensors andWiFi signals Moreover it overcomes key limita-tions of existing wearable devices by providing a contactlessand privacy-preserving approach to capture a rich varietyof human activities Based on this work we explore thecombination of CSI and skeleton data to sense human behav-ior According to the works mentioned above we explorethree issues of activity recognition in this paper First weconstruct a WiFi-based activity recognition dataset namedWiAR to provide a benchmark for previous works Secondwe design the mechanism of subcarrier selection to improvethe robustness of activity recognition in the WiAR datasetThird we combine WiFi signals with crowdsourced skeletondata to improve the accuracy and robustness of activityrecognition breaking the limitations of Kinect technologyThe contributions of our work are summarized as follows
(i) We propose a HuAc system to recognize humanactivity and also construct a WiFi-based activityrecognition dataset named WiAR as a benchmark toevaluate the performance of existing activity recogni-tion systems We use the kNN Random Forest andDecision Tree algorithms to verify the effectiveness ofthe WiAR dataset
(ii) We detect the start and end of the activity usingthe moving variance of CSI Moreover we leverage119870-means algorithm to cluster effective subcarriersaccording to subcarrierrsquos sensitivity and improve therobustness of activity recognition
(iii) We develop a selection method of skeleton jointsbased on KARDrsquos work named SSJ and it considersthe spatial relationship and the angle of adjacentjoints as auxiliary information of human activityrecognition to improve the accuracy of tracking
(iv) We implement the fusion framework of CSI andskeleton data to sense the activity and solve thelimitations of CSI-based and skeleton-based activityrecognition respectively Experimental results showthat HuAc achieves the accuracy of greater than 93
The rest of this paper is organized as follows We intro-duce the related work in Section 2 Section 3 introducespreliminaries of WiFi-based activity recognition and wedescribe the overview of HuAc in Section 4 Section 5describes Kinect module and WiFi module is shown inSection 6 Section 7 describes the process of human activityrecognition Section 8 evaluates the performance of HuAcsystem and we give a case study about a motion-sensinggame using WiFi signals in Section 9 Section 10 lists severaldiscussions and we give the conclusion of this paper inSection 11
2 Related Work
In this section related works on human activity recognitioncan be divided into two categories Kinect-basedWiFi-based
21 Kinect-Based Activity Recognition Vision-based activityrecognition has been proposed and developed in the com-puter vision field With the release of Kinect researchers
explore the human activity recognition using depth informa-tion and skeleton joints data provided by Kinect [7 8 16]Biswas and Basu [8] leverage the histogram of depth infor-mation to recognize eight gestures Moreover the differencesbetween continuous frames can obtain the motion profile todescribe various gestures Other works [7 16] leverage depthinformation in combination with color image to improve theaccuracy of gestures recognition The limitations of Kinect-based activity recognition contain the restriction of sensingfield skeleton joints overlapping and position-dependencefactors HuAc system explores the spatial relationship ofskeleton joints to describe the trajectory of an activity andcombines with CSI to improve the robustness of humanactivity recognition in a dynamic environment
22 WiFi-Based Activity Recognition Early works [17ndash19]explore the attenuation characteristics of WiFi signals tolocate the position of someone and count the number ofpeople in the indoor environment Researchers study thesignal pattern reflected by a human body to sense humanbehavior [11 20ndash22] These works describe human behav-ior recognition using coarse-grained RSSI information Forexample WiGest [18] studies the relationship between RSSIfluctuation and gestures to controlmedia player actions with-out training Therefore we explore the relationship betweenRSSI fluctuation and humanmovement to detect the presenceof an activity
With the requirement of the practical application andthe limitations of RSSI an increasing number of researchersbegin to explore fine-grained channel state information (CSI)to sense human behavior Compared with RSSI CSI cancapture the tiny behavior [2 9 23ndash28] in terms of locationspeed and directionWiFall system [2] detects a fall behaviorby learning the specific CSI pattern E-eyes [9] recognizeswalking activity and in-place activity by adopting movingvariance of CSI and fingerprint technique Walking activitycauses significant pattern changes of the CSI amplitudeover time since it involves significant body movementsand location changes In-place activity (watching TV) onlyinvolves relative smaller body movements and will not causesignificant amplitude changes with repetitive patterns Therelationship between an activity and the place where anactivity occurs motivates the novel idea on human activityrecognition CARM [10] shows the correlation between CSIvalue and human activity by constructingCSI-speed andCSI-activity model WiDance [28] explores the Doppler shiftsreflected by human behavior to predict the motion direc-tion for the Exergames We design the combination systemof Kinect-based and WiFi-based methods to recognize anactivity in different environments such as gaming systemsupermarket and elder health applications
3 Preliminaries
31 RSSI and CSI Received Signal Strength Indicator (RSSI)[29] in the level of packet represents signal-to-interference-plus-noise ratio (SINR) over the channel bandwidth asfollows
RSSI = 10 lg (1198812) (1)
Wireless Communications and Mobile Computing 3
Number Name(1) Head(2) Neck(3) Right shoulder(4) Right elbow(5) Right hand(6) Le shoulder(7) Le elbow(8) Le hand(9) Torso(10) Right hip(11) Le hip(12) Right knee(13) Right foot(14) Le knee(15) Le foot
62
1
7
8
3
4
5
11 10
9
14 12
15 13
Figure 1 Skeleton joints
where 119881 is signal voltage RSSI is the received signal strengthin decibels (dB) and mapped into the distance accordingto Log-distance path loss model to roughly locate users ordevices
Channel State Information (CSI) depicts multipath prop-agation at the granularity of OFDM subcarrier in the fre-quency domain It contains amplitude and phase measure-ments as follows
ℎ = |ℎ| 119890119895 sin 120579 (2)
where |ℎ| and 120579 are the amplitude and phase respectivelyThe variable ℎ shows CSI value of each subcarrier We studythe characteristics of each subcarrier to sense activity in thefollowing work
32 Kinect Technology Kinect (RGB-D camera) refers to theadvanced RGBdepth sensing hardware and the software-based technology that interprets the GRBdepth informationThe hardware contains a normal RGB camera a depthsensor (infrared projector and infrared camera) and a four-microphone array which is able to provide depth signalsRGB images and audio signals simultaneously Kinect-basedactivity recognition algorithm frequently fails due to occlu-sions overlapping joints (limbs close to the body) or clutter(other objects in the scene) [7] A skeleton reported by Kinectcontains 15 joints in Figure 1 We explore the correspondingrelationship between skeleton joints and CSI to analyzethe characteristics of an activity Moreover we explore thefusion information to improve the accuracy of human activityrecognition The details of Kinect-based activity recognitionare listed in Section 5
33 WiAR Constructing WiFi-Based Activity Dataset Atpresent there is noWiFi-based public activity dataset as wellas vision-based public activity dataset Due to the sensitivityof WiFi signals it is hard for peer researchers to reproduceand evaluate previous works Therefore we construct theWiAR dataset which collectsWiFi signals reflected by sixteenactivities in three indoor environments such as empty roommeeting room and office listed in Table 1 Each activity isperformed 50 times by 10 volunteers which consist of five
Kinect applications
Public Family ResearchCrowdsourcing
Skeleton joints data
Collecting WiFi signals
Input of HuAc system+
Figure 2 The framework of crowdsourced dataset
females and five males and the height of human body rangesfrom 150 cm to 185 cm
The environmental complexity according to the roomlayout divides into three levels including empty environmentnormal environment and complex environment First emptyenvironment describes no people and furniture around itWeobtain the high-quality WiFi signals from the empty roomdue to less noise and treat it as a baseline of WiAR datasetThen the normal environment contains furniture and work-ing people Compared with the empty environment themultipath effect reflected by the furniture enriches collectingWiFi signals Finally a complex environment with furnitureand moving people increases the difficulty of human activityrecognition The performance of WiAR dataset is given inSection 8
34 Crowdsourced WiFi Signals and Skeleton Joints Crowd-sourced-based applications [30ndash37] have been increasinglydeveloped by collecting data and reducing the cost in theInternet field For the macrolevel network the work [30]proposed a crowdsensing-oriented mobile cyber-physicalsystem to provide the practical usage of the vita For themicrolevel wireless network related works [38ndash41] leveragecrowdsensing WiFi signals to detect the userrsquos location
In our work we attempt to collect WiFi signals andcrowdsourced skeleton joints to reduce the training burdenfor collecting activity dataset We obtain the activity labelby leveraging the help from Kinectrsquos user The framework ofcrowdsourced WiFi signals and skeleton joints are shown inFigure 2
4 Overview of HuAc
41 Observations The following observations come from thecombination of our results and previous works [20 42ndash44]
The Impact of Indoor Environment on WiFi Signals Has aDifference with Time RSSI and CSI keep stability in thestatic indoor environment and RSSI fluctuation ranges from0 dB to 5 dB (empty environment 0ndash3 dB home environ-ment 0ndash7 dB office 0ndash5 dB dynamic environment 5ndash10 dB)Although RSSI sharply changes with environmental changeit cannot describe the fine-grained change of indoor environ-ment due to themultipath effectHowever CSI is able to sensethe change of fine-grained environment and detects whathappened in an indoor environment Specifically RSSI onlycan find the environmental change and cannot sense how
4 Wireless Communications and Mobile Computing
Table 1 WiFi-based activity recognition dataset (WiAR)
Granularity Activities Environments Devices
Activity Forward kick side kick bend walk phone sit down squat drinkwater
Empty room meetingroom office
Router laptop with5300 card
Gestures Horizontal arm wave two-hand wave high throw toss paper drawtick draw x hand clap high arm wave
Empty room meetingroom office
Router laptop with5300 card
Subcarrier selection-K-means-
Distribution of CSIFeatures extraction
Outlier detection-Threshold-
Smooth data -Weighted Moving Average-
Preprocessing
PreprocessingJoints overlap
detection
3D Joints normalization
CSI features set
Selection of skeleton joints
Skeleton features set
Posture analysis
Activity recognition
SSJ Selection of skeleton joints
HuAc framework
SVM
WiFi signals
Skeleton joints
Predict_label
Predict_label
Accuracy
Train_label
Train_label
50 100 1500 200 250 300
30
25
20
15
Figure 3 The framework of HuAc system
the environment changes CSI can find what causes environ-mental change and also can recognize how the environmentchanges such as tracking sensing environment and activityrecognition
It Is Hard to Distinguish Similar Activities Existing works [215 45] explore the similar activity recognition For exampleWiFall [2] extracts seven features to describe fall behaviorbecause similar activity causes the similar patterns of CSIand it is difficult to distinguish them only using anomalydetectionThe following RT-Fall system adopts the CSI phasedifference to segment fall and fall-like activities because thephase difference of CSI is a more sensitive signature than CSIamplitude for activity recognition The phase of CSI dependson the variation of LOS (Line-of-Sight) lengthTherefore thebreakthrough point of the similar activity recognition restson the physical difference between similar activities
The Same Activity Operated by Different People Has VariousSignal Patterns According to our observations the amplitudeof CSI reflected by the same activity changes continuously inthe different time and environments Therefore we cannotrecognize activity with high accuracy according to the ampli-tude of CSI The changing pattern of signals reflected by anactivity can describe the characteristic of activity as verifiedby Smokey [25] Therefore we explore the changing patternof signals to recognize an activity
The Impact of Activity with Different Directions on ActivityRecognition In order to explore the impact of direction onactivity recognition we design a simple and clear experiment
on the playground because the playground does not haverich multipath effect and other wireless devices We explorethe impact of four directions including east west north andsouth on the change pattern of signals and the differencebetween face and back to the AP is biggest Moreover CSIdata we collect in the playground contains less noise than thatin an indoor environment
42 Framework of HuAc The HuAc framework consists oftheKinect-basedmodule andWiFi-basedmodule in Figure 3We describe details of each module respectively
Kinect module consists of the preprocessing and postureanalysis We detect the overlap of skeleton joints using thestatisticalmethod and complete the normalization of skeletonjoints In order to obtain effective features of skeleton jointswe analyze postures of an activity according to the sequenceof skeleton joints Moreover we design a selection methodof skeleton joints named SSJ according to the result ofposture analysis Finally we extract features of skeleton jointsaccording to effective skeleton joints and also consider thespatial relationship of adjacent joints as auxiliary informationto sense human activity
WiFi module consists of the preprocessing and featuresextraction In the preprocessing stage we detect and removethe outlier data of an activity sequence according to thevariance of RSSI reflected by an activity After removingoutlier data we leverage the weighted moving average tosmooth the activity data For features extraction we firstanalyze the amplitude distribution of CSI reflected by anactivity to evaluate the sensitivity of the subcarrier on an
Wireless Communications and Mobile Computing 5
(a) (b)
Figure 4 Skeleton structure [7] (a) A skeleton structure contains 15skeleton joints (b) The white circle represents skeleton joints with-out direction such as shoulder and hip The gray circle representsthe neck and the torso which has a weak effect on the upper-bodyactivity and the lower-body activity except the squatThe black circlerepresents normal skeleton joints
activity Then we use 119870-means algorithm to cluster effec-tive subcarriers Finally we extract important features fromeffective subcarriers to improve the stability of human activityrecognition
We use the combination information of CSI features setand the skeleton features set as an input of SVM to recognizehuman activity Compared with the result of predict label wegive a feedback to the previous process of HuAc frameworkby using a train label respectively
5 Kinect Module
We mainly describe the details of Kinect module on thehuman activity recognition Kinect module contains thepreprocessing and posture analysis
51 Preprocessing The collected skeleton data contain emptyvalues due to the overlap of skeleton joints or the occlusionin the motion-sensing game Therefore we need to detectthe overlapping joints and replace the invalid values byrecovering the true value of the overlapping joints Weleverage the relationship between the coordinates of adjacentjoints to detect the overlapping joints Certainly we discardthe sample of an activity when the percent of invalid jointsexceeds the threshold
After recovering the invalid data we normalize thecoordinates of skeleton joints due to the differences of peoplersquosheight and the distance between the user and the sensorThe work [7] extracts 11 joints (except right shoulder leftshoulder right hip and left hip) from 15 joints in Figure 4and we explore 30 subcarriers with the similar patternreflected by a human body Therefore we select 15 jointsto match the 15 subcarriers Let 119869119894 be one of the 15 jointsdetected by the Kinect and the coordinates vector 119891 is givenby
119891 = 1198951 1198952 119895119894 11989514 11989515 (3)
300250 350200 400 450X (cm)
0
100
200
300
400
Y(c
m)
Figure 5 High arm wave tracking using skeleton data The activityhas two active joints (right hand right elbow) and the directionchanges with every clockwise movement However adjacent jointshave the slight change in a certain range
where 119895119894 is the vector containing the 3D normalized coordi-nates of the 119894th joint 119869119894 detected by Kinect Thus
119895119894 = 119869119894119904 + 119879119894 1 le 119894 le 15 (4)
where 119904 is the scale factor which normalizes the skeletonaccording to the distance ℎ between the neck and the torsojoints of a reference skeleton and
119904 = 10038171003817100381710038171198699 minus 11986921003817100381710038171003817ℎ (5)
The translation matrix 119879 needs to set the origin of thecoordinate system to the torso After preprocessing phase weobtain high-quality skeleton data
52 Postures Analysis An activity consists of subactivitysequence over time According to the skeleton structure ahuman body is divided into two parts including upper bodyand lower body Upper body contains five joints (right elbowleft elbow right hand left hand and head) and two baselinejoints (neck torso) as in Figure 4 Lower body contains fourjoints (right foot left foot right knee and left knee) Wereproduce the tracking of skeleton joints using QT tool andplot the trajectory chart of each activity We observe that theadjacent joints keep the similar track in Figure 5 and somejoints have slight movement influenced by human activityFor example when the right elbow and right hand move inthe clockwise direction to complete the horizontal arm wavewe observe that right hip and left hip have slight movement
According to the change of joints sequence we cansegment an activity into several subactivities in terms ofdirection and pause factor Horizontal arm wave behaviorconsists of four postures (subactivities) as in Figure 6 Eachsubactivity roughly contains 14 frames and 119865 119894 representsthe 119894th frame (packet) of the activity reported by Kinect Wecan evaluate the rough activity according to the sequence of
6 Wireless Communications and Mobile Computing
1 2 3 4
F_1 F_17 F_21 F_26 F_34 F_40 F_48 F_57 F_66
Figure 6 Postures of horizontal arm wave
subactivity Except for related joints of each subactivity torsoand hip joints have a weak swing We neglect the impactof weak swing on the activity recognition We pay moreattention to the selection of skeleton joints in the followingsection
53 SSJ Selecting Skeleton Joints We design a selectionmethod of skeleton joints named SSJ to describe a fine-grained subactivity After postures analysis we know therelationship between a subactivity and key skeleton jointsWe expend the coordinated system of human skeleton tominiature coordinated system of subactivity skeleton by theabove-mentioned relationship The miniature coordinatedsystem needs to determine a fixed skeleton joint and differentsubactivities have different fixed skeleton joints For examplewe observe that shoulder joint is a fixed joint from the processof high arm wave behavior Therefore we determine thestarting point coordinate of theminiature coordinated systemcorresponding to the subactivity
6 WiFi Module
We introduce the design details of WiFi module on thehuman activity recognition WiFi module consists of thepreprocessing and features extraction
61 Preprocessing The collected data with noises increasesthe difficulty of activity recognition due to the tiny differencesbetween noises and WiFi signals reflected by a fine-grainedactivity Outlier data also weaken the quality of collectingdata Therefore we detect outlier using the variance-basedmethod and remove high-frequency signals using the low-pass filter Moreover we reduce the sawtooth wave of thefiltered signal by using the weighted moving average
611 Outlier Detection and Removing High Frequency Out-lier has an important impact on the quality of collectingdata because outlier increases or decreases the fluctuationstrength of WiFi signals We analyze the RSSI distributionof an activity to evaluate the possible experience-thresholdThen we combine the variance of RSSI and the experience-threshold to detect outlier After removing outlier data theactivity corresponds to the low-frequency change of CSIaccording to the waveform of CSI reflected by an activityTherefore we adopt the low-pass filter to remove the high-frequency data in Figure 7
612 Weighted Moving Average For filtered signal signaldata still contain sawtooth wave Because CSI is sensitive toindoor layout or human movement and the received CSIfluctuation caused by the environment is hard to distinguishfrom the fluctuation caused by a fine-grained activity There-fore we smooth the CSI data using the weighted movingaverage as proposed in WiFall [2] We randomly select15 subcarriers from 30 subcarriers which correspond to15 skeleton joints of Kinect technology Each CSI streamcontains 15 subcarriers as CSI1CSI2 CSI15 CSI1199051 is thefirst subcarrier of CSI at time 119905 CSI11 CSI1199051 indicatesthe CSI sequence of first subcarrier in the time period 119905 Thelatest CSI has weight 119898 the second latest 119898 minus 1 and so onThe expression of CSI series is shown as follows
CSI1199051 = 1119898 + (119898 minus 1) + sdot sdot sdot + 1 times (119898 times CSI1199051
+ (119898 minus 1) times CSI119905minus11 + sdot sdot sdot + 1 times CSI119905minus119898minus11) (6)
where CSI1199051 is the averaged new CSI The value of119898 decidesin what degree the current value is related to historicalrecords In our study we select119898 according to the experienceand trial method We first set 119898 as 5 which means thelength of 5 packets A weighted moving average algorithmand median filter have the similar effect on the originalsignals recorded by the receiver in Figure 7They can removethe galling of signals and alleviate the sharp change ofsignals With the119898 increasing the weighted moving averagealgorithm becomes more smooth than the low-pass filter andthe median filter Finally we set119898 to 10 because each activityproduces a sharp change in 10 packet periods62 Feature Extraction Plenty of related works summarizethe importance of features extraction for human activityrecognition in a dynamic indoor environment We segmentactivity after smoothing CSI and extract features of eachactivity according to activity characteristics Kinect-basedfeatures extraction quotes the work [3]
621 Activity Segmentation Activity segmentation mainlydetects the start and end of an activity and removes thenonactivity packets from a sample which corresponds tothe whole activity We propose two methods to detect thestart and end of an activity and improve the robustness ofsegmentation algorithm First we remove the first secondand the last-second data sequence of an activity to reduce
Wireless Communications and Mobile Computing 7
Original signal Median filter
Low-pass filter Weighted moving average
15
20
25A
mpl
itude
of C
SI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
200100Packets
18
20
22
Am
plitu
de o
f CSI
100 2000Packets
Figure 7 Methods of signal filtering
the error of true activity sequence in our experimentalenvironment But this method is invalid in the practicalenvironment due to the unknown time which each activitystartsTherefore we leveragemoving variance ofCSI to detectthe start and end of each activity Moving variance of CSIdescribes the difference of the local packets reflected by theactivity Packet sequences on the corresponding activity aredefined as119883 = 1199091 1199092 119909119899119883 represents data sequence (asample) of an activity and 119909119894 represents the 119894th packet in thedata sequenceWe often use the standard deviation instead ofthe variance of CSI as follows
120590119894 = radicsum1198981 (119909119894+119895minus1 minus 119909)2119898 (119894 = 1 2 119899 minus 119898) (7)
where 119898 represents step-size and 119909 is the mean value ofsamples
We construct a window per 10 packets from the packetsequence of each sample and compute the variance of thewindow Then we construct the moving variance histogramand compare with other strength windows Finally we candetect the sharp points of each activity and roughly recognize
Subplot 1 two-hand wave
Subplot 2 hand clap
Sharp change Activityperiods20
22
24
Am
plitu
de o
f CSI
21
23
25
Am
plitu
de o
f CSI
50 100 150 200 2500Packets
50 100 150 2000Packets
Figure 8 Segmentation point of similar activity
the start and end of each activity from the data sequenceThestart and end of the activity period are shown in Figure 8Thered circle describes a sharp change of CSI at the start point ofcollecting data but it is not the true start of an activityThe red
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Wireless Communications and Mobile Computing 3
Number Name(1) Head(2) Neck(3) Right shoulder(4) Right elbow(5) Right hand(6) Le shoulder(7) Le elbow(8) Le hand(9) Torso(10) Right hip(11) Le hip(12) Right knee(13) Right foot(14) Le knee(15) Le foot
62
1
7
8
3
4
5
11 10
9
14 12
15 13
Figure 1 Skeleton joints
where 119881 is signal voltage RSSI is the received signal strengthin decibels (dB) and mapped into the distance accordingto Log-distance path loss model to roughly locate users ordevices
Channel State Information (CSI) depicts multipath prop-agation at the granularity of OFDM subcarrier in the fre-quency domain It contains amplitude and phase measure-ments as follows
ℎ = |ℎ| 119890119895 sin 120579 (2)
where |ℎ| and 120579 are the amplitude and phase respectivelyThe variable ℎ shows CSI value of each subcarrier We studythe characteristics of each subcarrier to sense activity in thefollowing work
32 Kinect Technology Kinect (RGB-D camera) refers to theadvanced RGBdepth sensing hardware and the software-based technology that interprets the GRBdepth informationThe hardware contains a normal RGB camera a depthsensor (infrared projector and infrared camera) and a four-microphone array which is able to provide depth signalsRGB images and audio signals simultaneously Kinect-basedactivity recognition algorithm frequently fails due to occlu-sions overlapping joints (limbs close to the body) or clutter(other objects in the scene) [7] A skeleton reported by Kinectcontains 15 joints in Figure 1 We explore the correspondingrelationship between skeleton joints and CSI to analyzethe characteristics of an activity Moreover we explore thefusion information to improve the accuracy of human activityrecognition The details of Kinect-based activity recognitionare listed in Section 5
33 WiAR Constructing WiFi-Based Activity Dataset Atpresent there is noWiFi-based public activity dataset as wellas vision-based public activity dataset Due to the sensitivityof WiFi signals it is hard for peer researchers to reproduceand evaluate previous works Therefore we construct theWiAR dataset which collectsWiFi signals reflected by sixteenactivities in three indoor environments such as empty roommeeting room and office listed in Table 1 Each activity isperformed 50 times by 10 volunteers which consist of five
Kinect applications
Public Family ResearchCrowdsourcing
Skeleton joints data
Collecting WiFi signals
Input of HuAc system+
Figure 2 The framework of crowdsourced dataset
females and five males and the height of human body rangesfrom 150 cm to 185 cm
The environmental complexity according to the roomlayout divides into three levels including empty environmentnormal environment and complex environment First emptyenvironment describes no people and furniture around itWeobtain the high-quality WiFi signals from the empty roomdue to less noise and treat it as a baseline of WiAR datasetThen the normal environment contains furniture and work-ing people Compared with the empty environment themultipath effect reflected by the furniture enriches collectingWiFi signals Finally a complex environment with furnitureand moving people increases the difficulty of human activityrecognition The performance of WiAR dataset is given inSection 8
34 Crowdsourced WiFi Signals and Skeleton Joints Crowd-sourced-based applications [30ndash37] have been increasinglydeveloped by collecting data and reducing the cost in theInternet field For the macrolevel network the work [30]proposed a crowdsensing-oriented mobile cyber-physicalsystem to provide the practical usage of the vita For themicrolevel wireless network related works [38ndash41] leveragecrowdsensing WiFi signals to detect the userrsquos location
In our work we attempt to collect WiFi signals andcrowdsourced skeleton joints to reduce the training burdenfor collecting activity dataset We obtain the activity labelby leveraging the help from Kinectrsquos user The framework ofcrowdsourced WiFi signals and skeleton joints are shown inFigure 2
4 Overview of HuAc
41 Observations The following observations come from thecombination of our results and previous works [20 42ndash44]
The Impact of Indoor Environment on WiFi Signals Has aDifference with Time RSSI and CSI keep stability in thestatic indoor environment and RSSI fluctuation ranges from0 dB to 5 dB (empty environment 0ndash3 dB home environ-ment 0ndash7 dB office 0ndash5 dB dynamic environment 5ndash10 dB)Although RSSI sharply changes with environmental changeit cannot describe the fine-grained change of indoor environ-ment due to themultipath effectHowever CSI is able to sensethe change of fine-grained environment and detects whathappened in an indoor environment Specifically RSSI onlycan find the environmental change and cannot sense how
4 Wireless Communications and Mobile Computing
Table 1 WiFi-based activity recognition dataset (WiAR)
Granularity Activities Environments Devices
Activity Forward kick side kick bend walk phone sit down squat drinkwater
Empty room meetingroom office
Router laptop with5300 card
Gestures Horizontal arm wave two-hand wave high throw toss paper drawtick draw x hand clap high arm wave
Empty room meetingroom office
Router laptop with5300 card
Subcarrier selection-K-means-
Distribution of CSIFeatures extraction
Outlier detection-Threshold-
Smooth data -Weighted Moving Average-
Preprocessing
PreprocessingJoints overlap
detection
3D Joints normalization
CSI features set
Selection of skeleton joints
Skeleton features set
Posture analysis
Activity recognition
SSJ Selection of skeleton joints
HuAc framework
SVM
WiFi signals
Skeleton joints
Predict_label
Predict_label
Accuracy
Train_label
Train_label
50 100 1500 200 250 300
30
25
20
15
Figure 3 The framework of HuAc system
the environment changes CSI can find what causes environ-mental change and also can recognize how the environmentchanges such as tracking sensing environment and activityrecognition
It Is Hard to Distinguish Similar Activities Existing works [215 45] explore the similar activity recognition For exampleWiFall [2] extracts seven features to describe fall behaviorbecause similar activity causes the similar patterns of CSIand it is difficult to distinguish them only using anomalydetectionThe following RT-Fall system adopts the CSI phasedifference to segment fall and fall-like activities because thephase difference of CSI is a more sensitive signature than CSIamplitude for activity recognition The phase of CSI dependson the variation of LOS (Line-of-Sight) lengthTherefore thebreakthrough point of the similar activity recognition restson the physical difference between similar activities
The Same Activity Operated by Different People Has VariousSignal Patterns According to our observations the amplitudeof CSI reflected by the same activity changes continuously inthe different time and environments Therefore we cannotrecognize activity with high accuracy according to the ampli-tude of CSI The changing pattern of signals reflected by anactivity can describe the characteristic of activity as verifiedby Smokey [25] Therefore we explore the changing patternof signals to recognize an activity
The Impact of Activity with Different Directions on ActivityRecognition In order to explore the impact of direction onactivity recognition we design a simple and clear experiment
on the playground because the playground does not haverich multipath effect and other wireless devices We explorethe impact of four directions including east west north andsouth on the change pattern of signals and the differencebetween face and back to the AP is biggest Moreover CSIdata we collect in the playground contains less noise than thatin an indoor environment
42 Framework of HuAc The HuAc framework consists oftheKinect-basedmodule andWiFi-basedmodule in Figure 3We describe details of each module respectively
Kinect module consists of the preprocessing and postureanalysis We detect the overlap of skeleton joints using thestatisticalmethod and complete the normalization of skeletonjoints In order to obtain effective features of skeleton jointswe analyze postures of an activity according to the sequenceof skeleton joints Moreover we design a selection methodof skeleton joints named SSJ according to the result ofposture analysis Finally we extract features of skeleton jointsaccording to effective skeleton joints and also consider thespatial relationship of adjacent joints as auxiliary informationto sense human activity
WiFi module consists of the preprocessing and featuresextraction In the preprocessing stage we detect and removethe outlier data of an activity sequence according to thevariance of RSSI reflected by an activity After removingoutlier data we leverage the weighted moving average tosmooth the activity data For features extraction we firstanalyze the amplitude distribution of CSI reflected by anactivity to evaluate the sensitivity of the subcarrier on an
Wireless Communications and Mobile Computing 5
(a) (b)
Figure 4 Skeleton structure [7] (a) A skeleton structure contains 15skeleton joints (b) The white circle represents skeleton joints with-out direction such as shoulder and hip The gray circle representsthe neck and the torso which has a weak effect on the upper-bodyactivity and the lower-body activity except the squatThe black circlerepresents normal skeleton joints
activity Then we use 119870-means algorithm to cluster effec-tive subcarriers Finally we extract important features fromeffective subcarriers to improve the stability of human activityrecognition
We use the combination information of CSI features setand the skeleton features set as an input of SVM to recognizehuman activity Compared with the result of predict label wegive a feedback to the previous process of HuAc frameworkby using a train label respectively
5 Kinect Module
We mainly describe the details of Kinect module on thehuman activity recognition Kinect module contains thepreprocessing and posture analysis
51 Preprocessing The collected skeleton data contain emptyvalues due to the overlap of skeleton joints or the occlusionin the motion-sensing game Therefore we need to detectthe overlapping joints and replace the invalid values byrecovering the true value of the overlapping joints Weleverage the relationship between the coordinates of adjacentjoints to detect the overlapping joints Certainly we discardthe sample of an activity when the percent of invalid jointsexceeds the threshold
After recovering the invalid data we normalize thecoordinates of skeleton joints due to the differences of peoplersquosheight and the distance between the user and the sensorThe work [7] extracts 11 joints (except right shoulder leftshoulder right hip and left hip) from 15 joints in Figure 4and we explore 30 subcarriers with the similar patternreflected by a human body Therefore we select 15 jointsto match the 15 subcarriers Let 119869119894 be one of the 15 jointsdetected by the Kinect and the coordinates vector 119891 is givenby
119891 = 1198951 1198952 119895119894 11989514 11989515 (3)
300250 350200 400 450X (cm)
0
100
200
300
400
Y(c
m)
Figure 5 High arm wave tracking using skeleton data The activityhas two active joints (right hand right elbow) and the directionchanges with every clockwise movement However adjacent jointshave the slight change in a certain range
where 119895119894 is the vector containing the 3D normalized coordi-nates of the 119894th joint 119869119894 detected by Kinect Thus
119895119894 = 119869119894119904 + 119879119894 1 le 119894 le 15 (4)
where 119904 is the scale factor which normalizes the skeletonaccording to the distance ℎ between the neck and the torsojoints of a reference skeleton and
119904 = 10038171003817100381710038171198699 minus 11986921003817100381710038171003817ℎ (5)
The translation matrix 119879 needs to set the origin of thecoordinate system to the torso After preprocessing phase weobtain high-quality skeleton data
52 Postures Analysis An activity consists of subactivitysequence over time According to the skeleton structure ahuman body is divided into two parts including upper bodyand lower body Upper body contains five joints (right elbowleft elbow right hand left hand and head) and two baselinejoints (neck torso) as in Figure 4 Lower body contains fourjoints (right foot left foot right knee and left knee) Wereproduce the tracking of skeleton joints using QT tool andplot the trajectory chart of each activity We observe that theadjacent joints keep the similar track in Figure 5 and somejoints have slight movement influenced by human activityFor example when the right elbow and right hand move inthe clockwise direction to complete the horizontal arm wavewe observe that right hip and left hip have slight movement
According to the change of joints sequence we cansegment an activity into several subactivities in terms ofdirection and pause factor Horizontal arm wave behaviorconsists of four postures (subactivities) as in Figure 6 Eachsubactivity roughly contains 14 frames and 119865 119894 representsthe 119894th frame (packet) of the activity reported by Kinect Wecan evaluate the rough activity according to the sequence of
6 Wireless Communications and Mobile Computing
1 2 3 4
F_1 F_17 F_21 F_26 F_34 F_40 F_48 F_57 F_66
Figure 6 Postures of horizontal arm wave
subactivity Except for related joints of each subactivity torsoand hip joints have a weak swing We neglect the impactof weak swing on the activity recognition We pay moreattention to the selection of skeleton joints in the followingsection
53 SSJ Selecting Skeleton Joints We design a selectionmethod of skeleton joints named SSJ to describe a fine-grained subactivity After postures analysis we know therelationship between a subactivity and key skeleton jointsWe expend the coordinated system of human skeleton tominiature coordinated system of subactivity skeleton by theabove-mentioned relationship The miniature coordinatedsystem needs to determine a fixed skeleton joint and differentsubactivities have different fixed skeleton joints For examplewe observe that shoulder joint is a fixed joint from the processof high arm wave behavior Therefore we determine thestarting point coordinate of theminiature coordinated systemcorresponding to the subactivity
6 WiFi Module
We introduce the design details of WiFi module on thehuman activity recognition WiFi module consists of thepreprocessing and features extraction
61 Preprocessing The collected data with noises increasesthe difficulty of activity recognition due to the tiny differencesbetween noises and WiFi signals reflected by a fine-grainedactivity Outlier data also weaken the quality of collectingdata Therefore we detect outlier using the variance-basedmethod and remove high-frequency signals using the low-pass filter Moreover we reduce the sawtooth wave of thefiltered signal by using the weighted moving average
611 Outlier Detection and Removing High Frequency Out-lier has an important impact on the quality of collectingdata because outlier increases or decreases the fluctuationstrength of WiFi signals We analyze the RSSI distributionof an activity to evaluate the possible experience-thresholdThen we combine the variance of RSSI and the experience-threshold to detect outlier After removing outlier data theactivity corresponds to the low-frequency change of CSIaccording to the waveform of CSI reflected by an activityTherefore we adopt the low-pass filter to remove the high-frequency data in Figure 7
612 Weighted Moving Average For filtered signal signaldata still contain sawtooth wave Because CSI is sensitive toindoor layout or human movement and the received CSIfluctuation caused by the environment is hard to distinguishfrom the fluctuation caused by a fine-grained activity There-fore we smooth the CSI data using the weighted movingaverage as proposed in WiFall [2] We randomly select15 subcarriers from 30 subcarriers which correspond to15 skeleton joints of Kinect technology Each CSI streamcontains 15 subcarriers as CSI1CSI2 CSI15 CSI1199051 is thefirst subcarrier of CSI at time 119905 CSI11 CSI1199051 indicatesthe CSI sequence of first subcarrier in the time period 119905 Thelatest CSI has weight 119898 the second latest 119898 minus 1 and so onThe expression of CSI series is shown as follows
CSI1199051 = 1119898 + (119898 minus 1) + sdot sdot sdot + 1 times (119898 times CSI1199051
+ (119898 minus 1) times CSI119905minus11 + sdot sdot sdot + 1 times CSI119905minus119898minus11) (6)
where CSI1199051 is the averaged new CSI The value of119898 decidesin what degree the current value is related to historicalrecords In our study we select119898 according to the experienceand trial method We first set 119898 as 5 which means thelength of 5 packets A weighted moving average algorithmand median filter have the similar effect on the originalsignals recorded by the receiver in Figure 7They can removethe galling of signals and alleviate the sharp change ofsignals With the119898 increasing the weighted moving averagealgorithm becomes more smooth than the low-pass filter andthe median filter Finally we set119898 to 10 because each activityproduces a sharp change in 10 packet periods62 Feature Extraction Plenty of related works summarizethe importance of features extraction for human activityrecognition in a dynamic indoor environment We segmentactivity after smoothing CSI and extract features of eachactivity according to activity characteristics Kinect-basedfeatures extraction quotes the work [3]
621 Activity Segmentation Activity segmentation mainlydetects the start and end of an activity and removes thenonactivity packets from a sample which corresponds tothe whole activity We propose two methods to detect thestart and end of an activity and improve the robustness ofsegmentation algorithm First we remove the first secondand the last-second data sequence of an activity to reduce
Wireless Communications and Mobile Computing 7
Original signal Median filter
Low-pass filter Weighted moving average
15
20
25A
mpl
itude
of C
SI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
200100Packets
18
20
22
Am
plitu
de o
f CSI
100 2000Packets
Figure 7 Methods of signal filtering
the error of true activity sequence in our experimentalenvironment But this method is invalid in the practicalenvironment due to the unknown time which each activitystartsTherefore we leveragemoving variance ofCSI to detectthe start and end of each activity Moving variance of CSIdescribes the difference of the local packets reflected by theactivity Packet sequences on the corresponding activity aredefined as119883 = 1199091 1199092 119909119899119883 represents data sequence (asample) of an activity and 119909119894 represents the 119894th packet in thedata sequenceWe often use the standard deviation instead ofthe variance of CSI as follows
120590119894 = radicsum1198981 (119909119894+119895minus1 minus 119909)2119898 (119894 = 1 2 119899 minus 119898) (7)
where 119898 represents step-size and 119909 is the mean value ofsamples
We construct a window per 10 packets from the packetsequence of each sample and compute the variance of thewindow Then we construct the moving variance histogramand compare with other strength windows Finally we candetect the sharp points of each activity and roughly recognize
Subplot 1 two-hand wave
Subplot 2 hand clap
Sharp change Activityperiods20
22
24
Am
plitu
de o
f CSI
21
23
25
Am
plitu
de o
f CSI
50 100 150 200 2500Packets
50 100 150 2000Packets
Figure 8 Segmentation point of similar activity
the start and end of each activity from the data sequenceThestart and end of the activity period are shown in Figure 8Thered circle describes a sharp change of CSI at the start point ofcollecting data but it is not the true start of an activityThe red
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
4 Wireless Communications and Mobile Computing
Table 1 WiFi-based activity recognition dataset (WiAR)
Granularity Activities Environments Devices
Activity Forward kick side kick bend walk phone sit down squat drinkwater
Empty room meetingroom office
Router laptop with5300 card
Gestures Horizontal arm wave two-hand wave high throw toss paper drawtick draw x hand clap high arm wave
Empty room meetingroom office
Router laptop with5300 card
Subcarrier selection-K-means-
Distribution of CSIFeatures extraction
Outlier detection-Threshold-
Smooth data -Weighted Moving Average-
Preprocessing
PreprocessingJoints overlap
detection
3D Joints normalization
CSI features set
Selection of skeleton joints
Skeleton features set
Posture analysis
Activity recognition
SSJ Selection of skeleton joints
HuAc framework
SVM
WiFi signals
Skeleton joints
Predict_label
Predict_label
Accuracy
Train_label
Train_label
50 100 1500 200 250 300
30
25
20
15
Figure 3 The framework of HuAc system
the environment changes CSI can find what causes environ-mental change and also can recognize how the environmentchanges such as tracking sensing environment and activityrecognition
It Is Hard to Distinguish Similar Activities Existing works [215 45] explore the similar activity recognition For exampleWiFall [2] extracts seven features to describe fall behaviorbecause similar activity causes the similar patterns of CSIand it is difficult to distinguish them only using anomalydetectionThe following RT-Fall system adopts the CSI phasedifference to segment fall and fall-like activities because thephase difference of CSI is a more sensitive signature than CSIamplitude for activity recognition The phase of CSI dependson the variation of LOS (Line-of-Sight) lengthTherefore thebreakthrough point of the similar activity recognition restson the physical difference between similar activities
The Same Activity Operated by Different People Has VariousSignal Patterns According to our observations the amplitudeof CSI reflected by the same activity changes continuously inthe different time and environments Therefore we cannotrecognize activity with high accuracy according to the ampli-tude of CSI The changing pattern of signals reflected by anactivity can describe the characteristic of activity as verifiedby Smokey [25] Therefore we explore the changing patternof signals to recognize an activity
The Impact of Activity with Different Directions on ActivityRecognition In order to explore the impact of direction onactivity recognition we design a simple and clear experiment
on the playground because the playground does not haverich multipath effect and other wireless devices We explorethe impact of four directions including east west north andsouth on the change pattern of signals and the differencebetween face and back to the AP is biggest Moreover CSIdata we collect in the playground contains less noise than thatin an indoor environment
42 Framework of HuAc The HuAc framework consists oftheKinect-basedmodule andWiFi-basedmodule in Figure 3We describe details of each module respectively
Kinect module consists of the preprocessing and postureanalysis We detect the overlap of skeleton joints using thestatisticalmethod and complete the normalization of skeletonjoints In order to obtain effective features of skeleton jointswe analyze postures of an activity according to the sequenceof skeleton joints Moreover we design a selection methodof skeleton joints named SSJ according to the result ofposture analysis Finally we extract features of skeleton jointsaccording to effective skeleton joints and also consider thespatial relationship of adjacent joints as auxiliary informationto sense human activity
WiFi module consists of the preprocessing and featuresextraction In the preprocessing stage we detect and removethe outlier data of an activity sequence according to thevariance of RSSI reflected by an activity After removingoutlier data we leverage the weighted moving average tosmooth the activity data For features extraction we firstanalyze the amplitude distribution of CSI reflected by anactivity to evaluate the sensitivity of the subcarrier on an
Wireless Communications and Mobile Computing 5
(a) (b)
Figure 4 Skeleton structure [7] (a) A skeleton structure contains 15skeleton joints (b) The white circle represents skeleton joints with-out direction such as shoulder and hip The gray circle representsthe neck and the torso which has a weak effect on the upper-bodyactivity and the lower-body activity except the squatThe black circlerepresents normal skeleton joints
activity Then we use 119870-means algorithm to cluster effec-tive subcarriers Finally we extract important features fromeffective subcarriers to improve the stability of human activityrecognition
We use the combination information of CSI features setand the skeleton features set as an input of SVM to recognizehuman activity Compared with the result of predict label wegive a feedback to the previous process of HuAc frameworkby using a train label respectively
5 Kinect Module
We mainly describe the details of Kinect module on thehuman activity recognition Kinect module contains thepreprocessing and posture analysis
51 Preprocessing The collected skeleton data contain emptyvalues due to the overlap of skeleton joints or the occlusionin the motion-sensing game Therefore we need to detectthe overlapping joints and replace the invalid values byrecovering the true value of the overlapping joints Weleverage the relationship between the coordinates of adjacentjoints to detect the overlapping joints Certainly we discardthe sample of an activity when the percent of invalid jointsexceeds the threshold
After recovering the invalid data we normalize thecoordinates of skeleton joints due to the differences of peoplersquosheight and the distance between the user and the sensorThe work [7] extracts 11 joints (except right shoulder leftshoulder right hip and left hip) from 15 joints in Figure 4and we explore 30 subcarriers with the similar patternreflected by a human body Therefore we select 15 jointsto match the 15 subcarriers Let 119869119894 be one of the 15 jointsdetected by the Kinect and the coordinates vector 119891 is givenby
119891 = 1198951 1198952 119895119894 11989514 11989515 (3)
300250 350200 400 450X (cm)
0
100
200
300
400
Y(c
m)
Figure 5 High arm wave tracking using skeleton data The activityhas two active joints (right hand right elbow) and the directionchanges with every clockwise movement However adjacent jointshave the slight change in a certain range
where 119895119894 is the vector containing the 3D normalized coordi-nates of the 119894th joint 119869119894 detected by Kinect Thus
119895119894 = 119869119894119904 + 119879119894 1 le 119894 le 15 (4)
where 119904 is the scale factor which normalizes the skeletonaccording to the distance ℎ between the neck and the torsojoints of a reference skeleton and
119904 = 10038171003817100381710038171198699 minus 11986921003817100381710038171003817ℎ (5)
The translation matrix 119879 needs to set the origin of thecoordinate system to the torso After preprocessing phase weobtain high-quality skeleton data
52 Postures Analysis An activity consists of subactivitysequence over time According to the skeleton structure ahuman body is divided into two parts including upper bodyand lower body Upper body contains five joints (right elbowleft elbow right hand left hand and head) and two baselinejoints (neck torso) as in Figure 4 Lower body contains fourjoints (right foot left foot right knee and left knee) Wereproduce the tracking of skeleton joints using QT tool andplot the trajectory chart of each activity We observe that theadjacent joints keep the similar track in Figure 5 and somejoints have slight movement influenced by human activityFor example when the right elbow and right hand move inthe clockwise direction to complete the horizontal arm wavewe observe that right hip and left hip have slight movement
According to the change of joints sequence we cansegment an activity into several subactivities in terms ofdirection and pause factor Horizontal arm wave behaviorconsists of four postures (subactivities) as in Figure 6 Eachsubactivity roughly contains 14 frames and 119865 119894 representsthe 119894th frame (packet) of the activity reported by Kinect Wecan evaluate the rough activity according to the sequence of
6 Wireless Communications and Mobile Computing
1 2 3 4
F_1 F_17 F_21 F_26 F_34 F_40 F_48 F_57 F_66
Figure 6 Postures of horizontal arm wave
subactivity Except for related joints of each subactivity torsoand hip joints have a weak swing We neglect the impactof weak swing on the activity recognition We pay moreattention to the selection of skeleton joints in the followingsection
53 SSJ Selecting Skeleton Joints We design a selectionmethod of skeleton joints named SSJ to describe a fine-grained subactivity After postures analysis we know therelationship between a subactivity and key skeleton jointsWe expend the coordinated system of human skeleton tominiature coordinated system of subactivity skeleton by theabove-mentioned relationship The miniature coordinatedsystem needs to determine a fixed skeleton joint and differentsubactivities have different fixed skeleton joints For examplewe observe that shoulder joint is a fixed joint from the processof high arm wave behavior Therefore we determine thestarting point coordinate of theminiature coordinated systemcorresponding to the subactivity
6 WiFi Module
We introduce the design details of WiFi module on thehuman activity recognition WiFi module consists of thepreprocessing and features extraction
61 Preprocessing The collected data with noises increasesthe difficulty of activity recognition due to the tiny differencesbetween noises and WiFi signals reflected by a fine-grainedactivity Outlier data also weaken the quality of collectingdata Therefore we detect outlier using the variance-basedmethod and remove high-frequency signals using the low-pass filter Moreover we reduce the sawtooth wave of thefiltered signal by using the weighted moving average
611 Outlier Detection and Removing High Frequency Out-lier has an important impact on the quality of collectingdata because outlier increases or decreases the fluctuationstrength of WiFi signals We analyze the RSSI distributionof an activity to evaluate the possible experience-thresholdThen we combine the variance of RSSI and the experience-threshold to detect outlier After removing outlier data theactivity corresponds to the low-frequency change of CSIaccording to the waveform of CSI reflected by an activityTherefore we adopt the low-pass filter to remove the high-frequency data in Figure 7
612 Weighted Moving Average For filtered signal signaldata still contain sawtooth wave Because CSI is sensitive toindoor layout or human movement and the received CSIfluctuation caused by the environment is hard to distinguishfrom the fluctuation caused by a fine-grained activity There-fore we smooth the CSI data using the weighted movingaverage as proposed in WiFall [2] We randomly select15 subcarriers from 30 subcarriers which correspond to15 skeleton joints of Kinect technology Each CSI streamcontains 15 subcarriers as CSI1CSI2 CSI15 CSI1199051 is thefirst subcarrier of CSI at time 119905 CSI11 CSI1199051 indicatesthe CSI sequence of first subcarrier in the time period 119905 Thelatest CSI has weight 119898 the second latest 119898 minus 1 and so onThe expression of CSI series is shown as follows
CSI1199051 = 1119898 + (119898 minus 1) + sdot sdot sdot + 1 times (119898 times CSI1199051
+ (119898 minus 1) times CSI119905minus11 + sdot sdot sdot + 1 times CSI119905minus119898minus11) (6)
where CSI1199051 is the averaged new CSI The value of119898 decidesin what degree the current value is related to historicalrecords In our study we select119898 according to the experienceand trial method We first set 119898 as 5 which means thelength of 5 packets A weighted moving average algorithmand median filter have the similar effect on the originalsignals recorded by the receiver in Figure 7They can removethe galling of signals and alleviate the sharp change ofsignals With the119898 increasing the weighted moving averagealgorithm becomes more smooth than the low-pass filter andthe median filter Finally we set119898 to 10 because each activityproduces a sharp change in 10 packet periods62 Feature Extraction Plenty of related works summarizethe importance of features extraction for human activityrecognition in a dynamic indoor environment We segmentactivity after smoothing CSI and extract features of eachactivity according to activity characteristics Kinect-basedfeatures extraction quotes the work [3]
621 Activity Segmentation Activity segmentation mainlydetects the start and end of an activity and removes thenonactivity packets from a sample which corresponds tothe whole activity We propose two methods to detect thestart and end of an activity and improve the robustness ofsegmentation algorithm First we remove the first secondand the last-second data sequence of an activity to reduce
Wireless Communications and Mobile Computing 7
Original signal Median filter
Low-pass filter Weighted moving average
15
20
25A
mpl
itude
of C
SI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
200100Packets
18
20
22
Am
plitu
de o
f CSI
100 2000Packets
Figure 7 Methods of signal filtering
the error of true activity sequence in our experimentalenvironment But this method is invalid in the practicalenvironment due to the unknown time which each activitystartsTherefore we leveragemoving variance ofCSI to detectthe start and end of each activity Moving variance of CSIdescribes the difference of the local packets reflected by theactivity Packet sequences on the corresponding activity aredefined as119883 = 1199091 1199092 119909119899119883 represents data sequence (asample) of an activity and 119909119894 represents the 119894th packet in thedata sequenceWe often use the standard deviation instead ofthe variance of CSI as follows
120590119894 = radicsum1198981 (119909119894+119895minus1 minus 119909)2119898 (119894 = 1 2 119899 minus 119898) (7)
where 119898 represents step-size and 119909 is the mean value ofsamples
We construct a window per 10 packets from the packetsequence of each sample and compute the variance of thewindow Then we construct the moving variance histogramand compare with other strength windows Finally we candetect the sharp points of each activity and roughly recognize
Subplot 1 two-hand wave
Subplot 2 hand clap
Sharp change Activityperiods20
22
24
Am
plitu
de o
f CSI
21
23
25
Am
plitu
de o
f CSI
50 100 150 200 2500Packets
50 100 150 2000Packets
Figure 8 Segmentation point of similar activity
the start and end of each activity from the data sequenceThestart and end of the activity period are shown in Figure 8Thered circle describes a sharp change of CSI at the start point ofcollecting data but it is not the true start of an activityThe red
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Wireless Communications and Mobile Computing 5
(a) (b)
Figure 4 Skeleton structure [7] (a) A skeleton structure contains 15skeleton joints (b) The white circle represents skeleton joints with-out direction such as shoulder and hip The gray circle representsthe neck and the torso which has a weak effect on the upper-bodyactivity and the lower-body activity except the squatThe black circlerepresents normal skeleton joints
activity Then we use 119870-means algorithm to cluster effec-tive subcarriers Finally we extract important features fromeffective subcarriers to improve the stability of human activityrecognition
We use the combination information of CSI features setand the skeleton features set as an input of SVM to recognizehuman activity Compared with the result of predict label wegive a feedback to the previous process of HuAc frameworkby using a train label respectively
5 Kinect Module
We mainly describe the details of Kinect module on thehuman activity recognition Kinect module contains thepreprocessing and posture analysis
51 Preprocessing The collected skeleton data contain emptyvalues due to the overlap of skeleton joints or the occlusionin the motion-sensing game Therefore we need to detectthe overlapping joints and replace the invalid values byrecovering the true value of the overlapping joints Weleverage the relationship between the coordinates of adjacentjoints to detect the overlapping joints Certainly we discardthe sample of an activity when the percent of invalid jointsexceeds the threshold
After recovering the invalid data we normalize thecoordinates of skeleton joints due to the differences of peoplersquosheight and the distance between the user and the sensorThe work [7] extracts 11 joints (except right shoulder leftshoulder right hip and left hip) from 15 joints in Figure 4and we explore 30 subcarriers with the similar patternreflected by a human body Therefore we select 15 jointsto match the 15 subcarriers Let 119869119894 be one of the 15 jointsdetected by the Kinect and the coordinates vector 119891 is givenby
119891 = 1198951 1198952 119895119894 11989514 11989515 (3)
300250 350200 400 450X (cm)
0
100
200
300
400
Y(c
m)
Figure 5 High arm wave tracking using skeleton data The activityhas two active joints (right hand right elbow) and the directionchanges with every clockwise movement However adjacent jointshave the slight change in a certain range
where 119895119894 is the vector containing the 3D normalized coordi-nates of the 119894th joint 119869119894 detected by Kinect Thus
119895119894 = 119869119894119904 + 119879119894 1 le 119894 le 15 (4)
where 119904 is the scale factor which normalizes the skeletonaccording to the distance ℎ between the neck and the torsojoints of a reference skeleton and
119904 = 10038171003817100381710038171198699 minus 11986921003817100381710038171003817ℎ (5)
The translation matrix 119879 needs to set the origin of thecoordinate system to the torso After preprocessing phase weobtain high-quality skeleton data
52 Postures Analysis An activity consists of subactivitysequence over time According to the skeleton structure ahuman body is divided into two parts including upper bodyand lower body Upper body contains five joints (right elbowleft elbow right hand left hand and head) and two baselinejoints (neck torso) as in Figure 4 Lower body contains fourjoints (right foot left foot right knee and left knee) Wereproduce the tracking of skeleton joints using QT tool andplot the trajectory chart of each activity We observe that theadjacent joints keep the similar track in Figure 5 and somejoints have slight movement influenced by human activityFor example when the right elbow and right hand move inthe clockwise direction to complete the horizontal arm wavewe observe that right hip and left hip have slight movement
According to the change of joints sequence we cansegment an activity into several subactivities in terms ofdirection and pause factor Horizontal arm wave behaviorconsists of four postures (subactivities) as in Figure 6 Eachsubactivity roughly contains 14 frames and 119865 119894 representsthe 119894th frame (packet) of the activity reported by Kinect Wecan evaluate the rough activity according to the sequence of
6 Wireless Communications and Mobile Computing
1 2 3 4
F_1 F_17 F_21 F_26 F_34 F_40 F_48 F_57 F_66
Figure 6 Postures of horizontal arm wave
subactivity Except for related joints of each subactivity torsoand hip joints have a weak swing We neglect the impactof weak swing on the activity recognition We pay moreattention to the selection of skeleton joints in the followingsection
53 SSJ Selecting Skeleton Joints We design a selectionmethod of skeleton joints named SSJ to describe a fine-grained subactivity After postures analysis we know therelationship between a subactivity and key skeleton jointsWe expend the coordinated system of human skeleton tominiature coordinated system of subactivity skeleton by theabove-mentioned relationship The miniature coordinatedsystem needs to determine a fixed skeleton joint and differentsubactivities have different fixed skeleton joints For examplewe observe that shoulder joint is a fixed joint from the processof high arm wave behavior Therefore we determine thestarting point coordinate of theminiature coordinated systemcorresponding to the subactivity
6 WiFi Module
We introduce the design details of WiFi module on thehuman activity recognition WiFi module consists of thepreprocessing and features extraction
61 Preprocessing The collected data with noises increasesthe difficulty of activity recognition due to the tiny differencesbetween noises and WiFi signals reflected by a fine-grainedactivity Outlier data also weaken the quality of collectingdata Therefore we detect outlier using the variance-basedmethod and remove high-frequency signals using the low-pass filter Moreover we reduce the sawtooth wave of thefiltered signal by using the weighted moving average
611 Outlier Detection and Removing High Frequency Out-lier has an important impact on the quality of collectingdata because outlier increases or decreases the fluctuationstrength of WiFi signals We analyze the RSSI distributionof an activity to evaluate the possible experience-thresholdThen we combine the variance of RSSI and the experience-threshold to detect outlier After removing outlier data theactivity corresponds to the low-frequency change of CSIaccording to the waveform of CSI reflected by an activityTherefore we adopt the low-pass filter to remove the high-frequency data in Figure 7
612 Weighted Moving Average For filtered signal signaldata still contain sawtooth wave Because CSI is sensitive toindoor layout or human movement and the received CSIfluctuation caused by the environment is hard to distinguishfrom the fluctuation caused by a fine-grained activity There-fore we smooth the CSI data using the weighted movingaverage as proposed in WiFall [2] We randomly select15 subcarriers from 30 subcarriers which correspond to15 skeleton joints of Kinect technology Each CSI streamcontains 15 subcarriers as CSI1CSI2 CSI15 CSI1199051 is thefirst subcarrier of CSI at time 119905 CSI11 CSI1199051 indicatesthe CSI sequence of first subcarrier in the time period 119905 Thelatest CSI has weight 119898 the second latest 119898 minus 1 and so onThe expression of CSI series is shown as follows
CSI1199051 = 1119898 + (119898 minus 1) + sdot sdot sdot + 1 times (119898 times CSI1199051
+ (119898 minus 1) times CSI119905minus11 + sdot sdot sdot + 1 times CSI119905minus119898minus11) (6)
where CSI1199051 is the averaged new CSI The value of119898 decidesin what degree the current value is related to historicalrecords In our study we select119898 according to the experienceand trial method We first set 119898 as 5 which means thelength of 5 packets A weighted moving average algorithmand median filter have the similar effect on the originalsignals recorded by the receiver in Figure 7They can removethe galling of signals and alleviate the sharp change ofsignals With the119898 increasing the weighted moving averagealgorithm becomes more smooth than the low-pass filter andthe median filter Finally we set119898 to 10 because each activityproduces a sharp change in 10 packet periods62 Feature Extraction Plenty of related works summarizethe importance of features extraction for human activityrecognition in a dynamic indoor environment We segmentactivity after smoothing CSI and extract features of eachactivity according to activity characteristics Kinect-basedfeatures extraction quotes the work [3]
621 Activity Segmentation Activity segmentation mainlydetects the start and end of an activity and removes thenonactivity packets from a sample which corresponds tothe whole activity We propose two methods to detect thestart and end of an activity and improve the robustness ofsegmentation algorithm First we remove the first secondand the last-second data sequence of an activity to reduce
Wireless Communications and Mobile Computing 7
Original signal Median filter
Low-pass filter Weighted moving average
15
20
25A
mpl
itude
of C
SI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
200100Packets
18
20
22
Am
plitu
de o
f CSI
100 2000Packets
Figure 7 Methods of signal filtering
the error of true activity sequence in our experimentalenvironment But this method is invalid in the practicalenvironment due to the unknown time which each activitystartsTherefore we leveragemoving variance ofCSI to detectthe start and end of each activity Moving variance of CSIdescribes the difference of the local packets reflected by theactivity Packet sequences on the corresponding activity aredefined as119883 = 1199091 1199092 119909119899119883 represents data sequence (asample) of an activity and 119909119894 represents the 119894th packet in thedata sequenceWe often use the standard deviation instead ofthe variance of CSI as follows
120590119894 = radicsum1198981 (119909119894+119895minus1 minus 119909)2119898 (119894 = 1 2 119899 minus 119898) (7)
where 119898 represents step-size and 119909 is the mean value ofsamples
We construct a window per 10 packets from the packetsequence of each sample and compute the variance of thewindow Then we construct the moving variance histogramand compare with other strength windows Finally we candetect the sharp points of each activity and roughly recognize
Subplot 1 two-hand wave
Subplot 2 hand clap
Sharp change Activityperiods20
22
24
Am
plitu
de o
f CSI
21
23
25
Am
plitu
de o
f CSI
50 100 150 200 2500Packets
50 100 150 2000Packets
Figure 8 Segmentation point of similar activity
the start and end of each activity from the data sequenceThestart and end of the activity period are shown in Figure 8Thered circle describes a sharp change of CSI at the start point ofcollecting data but it is not the true start of an activityThe red
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
6 Wireless Communications and Mobile Computing
1 2 3 4
F_1 F_17 F_21 F_26 F_34 F_40 F_48 F_57 F_66
Figure 6 Postures of horizontal arm wave
subactivity Except for related joints of each subactivity torsoand hip joints have a weak swing We neglect the impactof weak swing on the activity recognition We pay moreattention to the selection of skeleton joints in the followingsection
53 SSJ Selecting Skeleton Joints We design a selectionmethod of skeleton joints named SSJ to describe a fine-grained subactivity After postures analysis we know therelationship between a subactivity and key skeleton jointsWe expend the coordinated system of human skeleton tominiature coordinated system of subactivity skeleton by theabove-mentioned relationship The miniature coordinatedsystem needs to determine a fixed skeleton joint and differentsubactivities have different fixed skeleton joints For examplewe observe that shoulder joint is a fixed joint from the processof high arm wave behavior Therefore we determine thestarting point coordinate of theminiature coordinated systemcorresponding to the subactivity
6 WiFi Module
We introduce the design details of WiFi module on thehuman activity recognition WiFi module consists of thepreprocessing and features extraction
61 Preprocessing The collected data with noises increasesthe difficulty of activity recognition due to the tiny differencesbetween noises and WiFi signals reflected by a fine-grainedactivity Outlier data also weaken the quality of collectingdata Therefore we detect outlier using the variance-basedmethod and remove high-frequency signals using the low-pass filter Moreover we reduce the sawtooth wave of thefiltered signal by using the weighted moving average
611 Outlier Detection and Removing High Frequency Out-lier has an important impact on the quality of collectingdata because outlier increases or decreases the fluctuationstrength of WiFi signals We analyze the RSSI distributionof an activity to evaluate the possible experience-thresholdThen we combine the variance of RSSI and the experience-threshold to detect outlier After removing outlier data theactivity corresponds to the low-frequency change of CSIaccording to the waveform of CSI reflected by an activityTherefore we adopt the low-pass filter to remove the high-frequency data in Figure 7
612 Weighted Moving Average For filtered signal signaldata still contain sawtooth wave Because CSI is sensitive toindoor layout or human movement and the received CSIfluctuation caused by the environment is hard to distinguishfrom the fluctuation caused by a fine-grained activity There-fore we smooth the CSI data using the weighted movingaverage as proposed in WiFall [2] We randomly select15 subcarriers from 30 subcarriers which correspond to15 skeleton joints of Kinect technology Each CSI streamcontains 15 subcarriers as CSI1CSI2 CSI15 CSI1199051 is thefirst subcarrier of CSI at time 119905 CSI11 CSI1199051 indicatesthe CSI sequence of first subcarrier in the time period 119905 Thelatest CSI has weight 119898 the second latest 119898 minus 1 and so onThe expression of CSI series is shown as follows
CSI1199051 = 1119898 + (119898 minus 1) + sdot sdot sdot + 1 times (119898 times CSI1199051
+ (119898 minus 1) times CSI119905minus11 + sdot sdot sdot + 1 times CSI119905minus119898minus11) (6)
where CSI1199051 is the averaged new CSI The value of119898 decidesin what degree the current value is related to historicalrecords In our study we select119898 according to the experienceand trial method We first set 119898 as 5 which means thelength of 5 packets A weighted moving average algorithmand median filter have the similar effect on the originalsignals recorded by the receiver in Figure 7They can removethe galling of signals and alleviate the sharp change ofsignals With the119898 increasing the weighted moving averagealgorithm becomes more smooth than the low-pass filter andthe median filter Finally we set119898 to 10 because each activityproduces a sharp change in 10 packet periods62 Feature Extraction Plenty of related works summarizethe importance of features extraction for human activityrecognition in a dynamic indoor environment We segmentactivity after smoothing CSI and extract features of eachactivity according to activity characteristics Kinect-basedfeatures extraction quotes the work [3]
621 Activity Segmentation Activity segmentation mainlydetects the start and end of an activity and removes thenonactivity packets from a sample which corresponds tothe whole activity We propose two methods to detect thestart and end of an activity and improve the robustness ofsegmentation algorithm First we remove the first secondand the last-second data sequence of an activity to reduce
Wireless Communications and Mobile Computing 7
Original signal Median filter
Low-pass filter Weighted moving average
15
20
25A
mpl
itude
of C
SI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
200100Packets
18
20
22
Am
plitu
de o
f CSI
100 2000Packets
Figure 7 Methods of signal filtering
the error of true activity sequence in our experimentalenvironment But this method is invalid in the practicalenvironment due to the unknown time which each activitystartsTherefore we leveragemoving variance ofCSI to detectthe start and end of each activity Moving variance of CSIdescribes the difference of the local packets reflected by theactivity Packet sequences on the corresponding activity aredefined as119883 = 1199091 1199092 119909119899119883 represents data sequence (asample) of an activity and 119909119894 represents the 119894th packet in thedata sequenceWe often use the standard deviation instead ofthe variance of CSI as follows
120590119894 = radicsum1198981 (119909119894+119895minus1 minus 119909)2119898 (119894 = 1 2 119899 minus 119898) (7)
where 119898 represents step-size and 119909 is the mean value ofsamples
We construct a window per 10 packets from the packetsequence of each sample and compute the variance of thewindow Then we construct the moving variance histogramand compare with other strength windows Finally we candetect the sharp points of each activity and roughly recognize
Subplot 1 two-hand wave
Subplot 2 hand clap
Sharp change Activityperiods20
22
24
Am
plitu
de o
f CSI
21
23
25
Am
plitu
de o
f CSI
50 100 150 200 2500Packets
50 100 150 2000Packets
Figure 8 Segmentation point of similar activity
the start and end of each activity from the data sequenceThestart and end of the activity period are shown in Figure 8Thered circle describes a sharp change of CSI at the start point ofcollecting data but it is not the true start of an activityThe red
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Wireless Communications and Mobile Computing 7
Original signal Median filter
Low-pass filter Weighted moving average
15
20
25A
mpl
itude
of C
SI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
100 2000Packets
15
20
25
Am
plitu
de o
f CSI
200100Packets
18
20
22
Am
plitu
de o
f CSI
100 2000Packets
Figure 7 Methods of signal filtering
the error of true activity sequence in our experimentalenvironment But this method is invalid in the practicalenvironment due to the unknown time which each activitystartsTherefore we leveragemoving variance ofCSI to detectthe start and end of each activity Moving variance of CSIdescribes the difference of the local packets reflected by theactivity Packet sequences on the corresponding activity aredefined as119883 = 1199091 1199092 119909119899119883 represents data sequence (asample) of an activity and 119909119894 represents the 119894th packet in thedata sequenceWe often use the standard deviation instead ofthe variance of CSI as follows
120590119894 = radicsum1198981 (119909119894+119895minus1 minus 119909)2119898 (119894 = 1 2 119899 minus 119898) (7)
where 119898 represents step-size and 119909 is the mean value ofsamples
We construct a window per 10 packets from the packetsequence of each sample and compute the variance of thewindow Then we construct the moving variance histogramand compare with other strength windows Finally we candetect the sharp points of each activity and roughly recognize
Subplot 1 two-hand wave
Subplot 2 hand clap
Sharp change Activityperiods20
22
24
Am
plitu
de o
f CSI
21
23
25
Am
plitu
de o
f CSI
50 100 150 200 2500Packets
50 100 150 2000Packets
Figure 8 Segmentation point of similar activity
the start and end of each activity from the data sequenceThestart and end of the activity period are shown in Figure 8Thered circle describes a sharp change of CSI at the start point ofcollecting data but it is not the true start of an activityThe red
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
8 Wireless Communications and Mobile Computing
15
16
17
18
19
20
21
22
23
Am
plitu
de o
f CSI
50 100 150 200 250 3000Packet index
First subcarrierSecond subcarrier
Tenth subcarrierTwentieth subcarrier
Figure 9 The fluctuation of different subcarriers reflected by thehorizontal arm wave behavior
rectangle represents the duration of activity Moreover theblack dotted line roughly represents the true start and end ofthe activity According to our experimental results detectingthe start and end of the activity still causes a small error dueto the sensitivity of signals
622 Subcarrier Selection and Feature Detection Accordingto our observation subcarriers have the similar tendency forthe same activity in Figure 9 but they have different sensitiv-ity Therefore we select the obvious subcarriers reflected byan activity using119870-means to achieve the robustness of humanactivity recognition Thirty subcarriers are divided into 3clusters using 119870-means algorithm in Figure 10 Accordingto the output of 119870-means algorithm on subcarriers CSIfeatures we extract include variance the envelope of CSIsignal entropy the velocity of signal change median absolutedeviation the period of motion and normalized standarddeviation Finally we construct the features set of CSI
7 HuAc Activity Recognition
We explore the relationship between CSI-based and skeleton-based methods on human activity recognition in Figure 11The CSI-based method leverages the signal pattern to rec-ognize an activity The skeleton-based method uses thecoordinate change of skeleton joints to recognize the sameactivity From the opinion of experiment results an activitywith back to the AP has more complex CSI pattern and hasthe smaller amplitude than that with face to AP
We mainly introduce several classification algorithmsused by the human activity recognition field includingkNN Random Forest Decision Tree and SVM In thefollowing sections we verify that the performance of SVMoutperforms others We select SVM classification algorithmto recognize sixteen activities in the WiAR dataset CSIfeatures set and skeleton features set as the inputs of SVMtrain the optimal model to achieve the stable accuracyof activity recognition The outputs of SVM contain the
1
0
051
005
z
yx
0
05
1
15
Figure 10 Clustering subcarriers
119886119888119888119906119903119886119888119910 119901119903119890119889119894119888119905 119897119886119887119890119897 and 119901119903119900119887 119890119904119905119894119898119886119905119890119904 We evaluatethe performance of classification algorithm according to theaccuracy and achieve the accuracy of activity recognitionusing the119901119903119890119889119894119888119905 119897119886119887119890119897 According to thematch level between119905119903119886119894119899 119897119886119887119890119897 and119901119903119890119889119894119888119905 119897119886119887119890119897 we obtain the false positive rateand the false negative rate We analyze the result and give afeedback on the previous step According to the feedback wepay more attention to the activity with low accuracy
8 Implementation and Evaluation
81 Implementation
811 Experimental Setup We use a commercial TP-Linkwireless router as the transmitter operating in IEEE 80211nAPmode at 24GHzAThinkpad 400 laptop runningUbuntu1004 is used as a receiver which is equipped with off-the-shelf Intel 5300 card and a modified firmware During theprocess of receivingWiFi signals the receiver pings 30 pktssfrom the router and records the RSSI and CSI from eachpacket Three experimental environments including emptyroom meeting room and office are shown in Figure 12
812 Experimental Data We deal with data from threecases ForWiFi-based activity data we collect activity data indifferent indoor environment For skeleton data we directlyleverage the KARD dataset [3] to get the skeleton data Forenvironmental data we mainly collect data from the emptyroom meeting room and office with the human Our goalis to explore the impact of the environmental factor on theWiFi signals and analyze the differences between an activityand environmental change on WiFi signals according to theabove-mentioned three kinds of data
We collectWiFi signals to construct a new dataset namedWiAR which contains 16 activities with 50 times performedby ten volunteers The details of WiAR have been introducedin Section 3 The KARD contains RGB video (avi) depthvideo (avi) and 15 skeleton points (txt) Each volunteerperforms 18 activities 3 times each with ages ranging from20ndash30 years and height from 150ndash180 cm In this paper weonly select 16 activities as target activity listed in Table 1
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Wireless Communications and Mobile Computing 9
(a) (b) (c)
Face to APBack to AP
50 100 150 200 2500Packet index
12
14
16
18
20
22
24
26
28
CSI
(d)
Figure 11 Skeleton joints sequence and CSI change of squat behavior (a)ndash(c) represent the skeleton sequence of squat behavior (d) is theCSI change reflected by squat behavior in terms of face to AP and back to AP
AP Receiver
(a) Empty room
AP Receiver
Meeting desk
1m 3m
(b) Meeting room
AP
ReceiverDesk
(c) Office
Figure 12 Experimental scenarios
We design three experimental schemes to analyze theaccuracy of activity recognition First we collect RSSI andCSI to recognize an activity as the reference point Second weleverage the skeleton data of KARD to recognize an activityby using our method and previous method [3] in the similarindoor environment Third we propose a fusion scheme
which CSI combines with skeleton data to recognize anactivity Moreover we design another experimental schemein which volunteer performs an activity with repeating 10times The goal of the experimental scheme is to investigatethe periodic regularity of CSI change influenced by the sameactivity
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
10 Wireless Communications and Mobile Computing
Table 2 Performance comparison by four classification algorithms
Method 10 subcarriers 30 subcarriersA B C A B C
kNN 0875 0916 0947 0916 0895 0947Random Forest 0885 0906 0958 0906 0895 0948Decision Tree 08542 0822 0916 0865 0834 0917SVM 09625 09688 0975 094375 090625 09375
82 Evaluation of WiAR Dataset We analyze activity data ofall volunteers to evaluate the performance of WiAR datasetusing kNN with voting Random Forest and Decision Treealgorithms
We study the impact of subcarriers and antennae on theperformance of activity recognition by using four classifica-tion algorithms shown in Table 2 It shows that the accuracyusing SVM outperforms other classification algorithms and10 subcarriers obtained by subcarrier selection mechanismincrease 426 when compared with activity recognitionusing 30 subcarriers Three antennae such as A B and Cincrease the diversity of CSI data and keep more than 80of activity recognition accuracy The four algorithms verifythe effectiveness of WiAR dataset
83 Evaluation of Activity Recognition
831 Performance of Activity Recognition Using RSSI Thesection evaluates the performance of RSSI on the humanactivity recognition The difficulty we encounter in theprocess of activity recognition using RSSI is how to dealwith the multipath effect caused by indoor environment andreflection effect caused by human behavior We select anindoor environment as a reference environment which keepsstatic and only contains a volunteer and an operator Weleverage RSSI variance as an input of SVM to obtain the 89of average recognition accuracy in the static environmentWhen other people move and are close to the control area ofWiFi signals the accuracy of activity recognition decreasesto 77 with the high stability Several activities face the lowaccuracy such as two-hand wave forward kick side kickand high throw The average false positive rate is 89 andincreases to 153 in a dynamic environment Thereforehuman activity recognition using RSSI needs the help of CSI-based method to improve the accuracy and the robustness ofhuman activity recognition
832 Performance of Activity Recognition Using CSI Thissection elaborates the impact of interference factors onhuman activity recognition using CSI in the following fouraspects human diversity similar activities different indoorenvironments and the size of a training set Moreoverwe keep the fixed position of volunteers and the distancebetween receiver device and transmitter device in the wholeexperiment
The Impact of Human Diversity on the Accuracy Humandiversity not only increases the diversity information of CSIbut also raises the difficulty of activity recognition because
different people have different motion styles such as speedheight and strength We achieve 9342 of average recogni-tion accuracy for all volunteers in Figure 13(a) We select twovolunteers including volunteer A and volunteer B to verifythe impact of human diversity on the accuracy VolunteerA which often regularly exercises obtains 971 of averagerecognition accuracy Volunteer B which rarely exercisesin the routine lives achieves 923 of average recognitionaccuracy Therefore the exercise experience increases thedifferences between activities due to standard activity andimproves the recognition accuracy
The Impact of Similar Activity on theAccuracyWe explore twogroup similar activities including high arm wave horizontalarmwave high throw and toss paper in Figure 13(b)The firstgroup activity achieves 925of average recognition accuracyand 946 for the second groupThe false positive for similaractivity is higher than independent activity For exampleforward kick and side kick also belong to the similar activityand the difference between them is the moving directionIn order to obtain the better accuracy we will consider theimpact of moving direction on the signal change in the futurework
The Impact of Indoor Environment on the Accuracy As shownin Figure 12 there are three experimental environmentsincluding empty room meeting room and office in termsof the complexity The accuracy about three environments isshown in Figure 13(c)The accuracy of themeeting roomwith947 outperforms the other two environments and thenaccuracy was 93 for empty room and 87 for office due tomultipath effectThemeeting room generates 26of averageerror and 98 of average error in the office due to pathsexcessively reflected by the body We will deeply explore themultipath effect using the amplitude and phase of CSI in thefuture work
The Impact of Training Size on the Accuracy We design threeproof schemes to analyze the accuracy of human activityrecognition by using different training sizes in Figure 13(d)We first introduce three activity sets and three training setsActivity set 1 consists of horizontal arm wave high armwave high throw and toss paper Activity set 2 containstwo-hand wave and handclap activity Activity set 3 consistsof phone draw tick draw x and drink water Moreoverthese activity sets come from the same people With thetraining size increasing the accuracy of activity recognitionis improved by about 10 for the activity set 1 Activity set1 has a low accuracy because activity set 1 contains more
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Wireless Communications and Mobile Computing 11
Volunteer AVolunteer BFusion of volunteers
0
02
04
06
08
1Ac
cura
cy o
f act
ivity
(
)
5 10 150Activity types
(a)
Volunteer B Volunteer CVolunteer ASimilar activities
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
Horizontal arm waveHigh arm wave
High throwToss paper
(b)
Average accuracy of activityAverage error of activity
Meeting room OfficeEmpty roomExperimental environments
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(c)
30 training samples50 training samples70 training samples
Set 2 Set 3Set 1Activity sets
0
02
04
06
08
1
Accu
racy
of a
ctiv
ity (
)
(d)
Figure 13 Performance analysis of activities using CSI (a) Sixteen activities include horizontal arm wave high arm wave two-hand wavehigh throw draw x draw tick toss paper forward kick side kick bend handclap walk phone drink water sit down and squat (b) Fouractivities contain horizontal arm wave high armwave high throw and toss paper (c)The impact of experimental environments on accuracy(d) The impact of training samples on accuracy of three activity sets
similar activities Although activity set 3 also contains similaractivities the accuracy is better than activity set 1 due to thestrength of activity
833 Performance between Kinect-Based and WiFi-BasedActivity Recognition It is hard for the waveform of RSSI withnoise to keep the stability when controlling area changesduring collecting data Therefore we use waveform shape ofRSSI to recognize an activity that is not a better choice forthe current level of technology Waveform pattern of CSI candescribe an activity with credibility and fine-grainedwayThemapping relationship between CSI-based and Kinect-basedactivity recognition for various activities is represented byusing several parameters shown in Table 3 The environmen-tal factor is evaluated by using the number of multipathsand the complexity of the indoor environment In order toextend the application field of activity sensing we constructthe mapping relationship between CSI-based and Kinect-based activity recognition The mapping relationship canavoid information loss For example once one of the two
datasets is lost activity recognition system still works by usinganother dataset information
We evaluate the performance of human activity recog-nition from KARD dataset [3] The highest recognition rateis 100 (side kick handclap) while the worst is 80 (highthrow) We propose a selection method of skeleton jointsnamed SSJ to improve the accuracy of activity recognitionand reduce the computing cost SSJ achieves 9315 of theaverage recognition accuracy Existing three activities suchas high arm wave draw kick and sit down achieve thelow accuracy of 80 75 and 70 respectively Table 4shows the performance of fourmethods includingCSI-basedKARD-based (skeleton joints) SSJ-based and HuAc Tablerow of the bold font shows that skeleton-based methodoutperforms CSI-based method on the accuracy of activityrecognition Table row of the italic font shows that severalactivities are sensitive to CSI HuAc improves the accuracyof activity recognition and increases the stability of activityrecognition in a dynamic indoor environment We focus
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
12 Wireless Communications and Mobile Computing
Table 3 Mapping relation between WiFi and Kinect
WiFi KinectTechniques CSI Skeleton jointsGranularity Subcarriers (15) Joints (15)
Parameters Similarity coefficient median absolute deviation varianceenvironment factor
Distance between joints angle between adjacent jointsvariance sequence of key joints
Table 4 Accuracy of activity for CSI-based and Kinect-based
Activities WiFi KARD [3] SSJ HuAcHorizontal arm wave 90 92 100 100High arm wave 100 96 80 95Two-hand wave 931 96 100 100High throw 90 80 100 100Draw x 100 96 100 93Draw tick 100 90 75 93Toss paper 100 90 100 100Forward kick 87 96 100 100Side kick 100 100 90 100Bend 957 96 100 100Hand clap 92 100 100 100Walk 100 100 100 100Phone 100 96 100 100Drink water 100 86 100 100Sit down 90 100 70 91Squat 967 100 90 90
attention on the stability of activity recognition algorithm orsystem in the future work
9 Case Study Motion-Sensing GameUsing WiFi Signals
We introduce the application based on our work in themotion-sensing game At present Kinect provides the anglewith limitations in which the horizontal viewing angle is575∘ and 435∘ for vertical viewing angle and distance withlimitation ranges from 05m to 45m Moreover Kinect losesthe sensing ability when barrier occurs and occludes gameuser in the control area An interesting point of our workis that we pay more attention to the activity itself and wedo not care about the user location However Kinect needsto adjust the location of a user before activity recognition toachieve well sensingTherefore we will propose a frameworkinstead of Kinect in the future when the accuracy of humanactivity recognition usingWiFi can satisfy the requirement inan indoor environment
We list a motion-sensing game using WiFi signals inFigure 14 One or two people are located in the middle of thetransmission and receiving terminal and prolong the distancebetween the TV and userThe area below the blue dashed linerepresents the control area and our work can sense humanbehavior within 10m and achieve a better performance
in the range of black circle The user operates the sameactivity as well as the TV set and receiving terminal collectscorresponding data By the phase of signals processing weachieve an activity with the probability and match it withthe game of TV set Once the matching result satisfies thethreshold value activity recognition matches success in themotion-sensing game using WiFi signals
10 Discussion and Future Work
101 Extending to Shadow Recognition In our research weconsider the relationship between the WiFi signals andskeleton data on the human activity recognition Moreoverwe describe the interesting topic of the shadow activityrecognition Shadow is an important issue to vision-basedactivity recognition or monitoring however WiFi-basedactivity recognition can sense human behavior through wallor shadow First we explore the characteristics of CSI toenhance the sensing ability by using the high-precisiondevice Second WiFi signals can help vision-based activityrecognition to improve the ability of sensing environment Inthis study we also need to consider the material attenuationAccording to our observations there is a little differencebetween the impact of wall reflection and body reflection ontheWiFi signals WiVi [14] leverages the nulling technique toexplore the through-wall sensing behavior by using CSI and
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Wireless Communications and Mobile Computing 13
TV set
Transmission terminal of signals
Receiving terminal of signals
Figure 14 Motion-sensing game using WiFi signals
analyzing the offset of signals from reflection and attenuationof the wallWe recommend researchers to read this paper andtheir following work [11]
102 Extending to Multiple People Activity Recognition Mul-tiple people activity recognition needs multiple APs to obtainmore signals information reflected by a human body Atpresent existing works can locate target location [46] anddetect the number [19] of multiple people using CSI inthe indoor environment Kinect-based activity recognitionsystem recognizes two skeletons (six skeletons for Kinect 20)and locates skeletons of six people Therefore the combina-tion of WiFi signals and Kinect facilitates the developmentof multiple people activity recognition In the future ourteam wants to deeply research the character of WiFi signalsand propose a novel framework to facilitate the practicalapplication of human activity recognition in the social lives
103 Data Fusion Skeleton data detect the position of eachjoint for each activity and track the trajectory of humanbehavior CSI can sense a fine-grained activity withoutattaching device in the complex indoor environment Thebalance point between CSI and skeleton joints and the selec-tion method of effective features are important factors forimproving the quality of fusion information Moreover timesynchronization of fusion information is also an importantchallenge in the human activity recognition field
11 Conclusion
In ourworkwe construct aWiFi-based public activity datasetnamedWiAR and designHuAc a novel framework of humanactivity recognition using CSI and crowdsourced skeleton
joints to improve the robustness and accuracy of activityrecognition First we leverage the moving variance of CSIto detect the rough start and end of an activity and adoptthe distribution of CSI to describe the detail of each activityMoreover we also select several effective subcarriers byusing 119870-means algorithm to improve the stability of activityrecognition Then we design SSJ method on the basis ofKARD to recognize similar activities by leveraging spatialrelationship and the angle of adjacent joints Finally wesolve the limitations of CSI-based and skeleton-based activityrecognition using fusion information Our results show thatHuAc achieves 93 of average recognition accuracy in theWiAR dataset
Conflicts of Interest
The authors declare that they have no conflicts of interest
Acknowledgments
The work is supported by National Natural Science Foun-dation of China with no 61733002 and the Fundamen-tal Research Funds for the Central University with noDUT17LAB16 and no DUT2017TB02 This work is alsosupported by Tianjin Key Laboratory of Advanced Network-ing (TANK) School of Computer Science and TechnologyTianjin University Tianjin 300350 China
References
[1] L Chen J Hoey C D Nugent D J Cook and Z Yu ldquoSensor-based activity recognitionrdquo IEEE Transactions on Systems Manand Cybernetics Part C Applications and Reviews vol 42 no 6pp 790ndash808 2012
[2] C Han K Wu Y Wang and L M Ni ldquoWiFall device-free falldetection by wireless networksrdquo in Proceedings of the 33rd IEEEConference on Computer Communications (IEEE INFOCOMrsquo14) pp 271ndash279 Toronto Canada May 2014
[3] S Gaglio G Lo Re and M Morana ldquoHuman activity recog-nition process using 3-D posture datardquo IEEE Transactions onHuman-Machine Systems vol 45 no 5 pp 586ndash597 2015
[4] H Abdelnasser K A Harras and M Youssef ldquoWiGest demoa ubiquitous WiFi-based gesture recognition systemrdquo in Pro-ceedings of the IEEE Conference on Computer CommunicationsWorkshops (INFOCOM WKSHPS rsquo15) pp 17-18 IEEE HongKong May 2015
[5] A Bulling U Blanke and B Schiele ldquoA tutorial on humanactivity recognition using body-worn inertial sensorsrdquo ACMComputing Surveys vol 46 no 3 article 33 2014
[6] A Avci S Bosch M Marin-Perianu R Marin-Perianu and PHavinga ldquoActivity recognition using inertial sensing for health-care wellbeing and sports applicationsA surveyrdquo inProceedingsof the ARCS 2010
[7] J Han L Shao D Xu and J Shotton ldquoEnhanced computervision with microsoft kinect sensor A reviewrdquo IEEE Transac-tions on cybernetics vol 43 no 5 pp 1318ndash1334 2013
[8] K Biswas and S Basu ldquoGesture recognition using MicrosoftKinectrdquo in Proceedings of the 5th International Conference onAutomation Robotics andApplications (ICARA rsquo11) pp 100ndash103December 2011
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
14 Wireless Communications and Mobile Computing
[9] Y Wang J Liu Y Chen M Gruteser J Yang and H Liu ldquoE-eyes Device-free location-oriented activity identification usingfine-grained WiFi signaturesrdquo in Proceedings of the 20th ACMAnnual International Conference on Mobile Computing andNetworking MobiCom 2014 pp 617ndash628 USA September 2014
[10] W Wang A X Liu M Shahzad K Ling and S Lu ldquoUnder-standing and modeling of WiFi signal based human activityrecognitionrdquo in Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking MobiCom2015 pp 65ndash76 Paris France September 2015
[11] F Adib C-Y Hsu H Mao D Katabi and F Durand ldquoCap-turing the human figure through a wallrdquo ACM Transactions onGraphics vol 34 no 6 article no 219 2015
[12] J Wilson and N Patwari ldquoSee-through walls motion trackingusing variance-based radio tomography networksrdquo IEEE Trans-actions on Mobile Computing vol 10 no 5 pp 612ndash621 2011
[13] F Adib Z Kabelac and D Katabi ldquoMulti-person localizationvia RF body reflectionsrdquo in Proceedings of the 12th USENIXSymposium on Networked Systems Design and ImplementationNSDI 2015 pp 279ndash292 usa May 2015
[14] F Adib and D Katabi ldquoSee through walls with WiFirdquo in Pro-ceedings of the Conference on Applications Technologies Archi-tectures and Protocols for Computer Communication (ACMSIGCOMM rsquo13) pp 75ndash86 August 2013
[15] B Fang N D Lane M Zhang A Boran and F KawsarldquoBodyScan Enabling radio-based sensing on wearable devicesfor contactless activity and vital signmonitoringrdquo inProceedingsof the 14th Annual International Conference on Mobile SystemsApplications and Services MobiSys 2016 pp 97ndash110 SingaporeJune 2016
[16] Z Cheng L Qin Y Ye Q Huang and Q Tian ldquoHuman dailyaction analysis with multi-view and color-depth datardquo LectureNotes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics)Preface vol 7584 no 2 pp 52ndash61 2012
[17] P Bahl and V N Padmanabhan ldquoRADAR an in-building RF-based user location and tracking systemrdquo in Proceedings of the19thAnnual Joint Conference of the IEEEComputer andCommu-nications Societies (IEEE INFOCOM rsquo00) vol 2 pp 775ndash784 TelAviv Israel March 2000
[18] J Gjengset J Xiong G McPhillips and K Jamieson ldquoPhaserEnabling phased array signal processing on commodity WiFiaccess pointsrdquo in Proceedings of the 20th ACM Annual Inter-national Conference on Mobile Computing and NetworkingMobiCom 2014 pp 153ndash163 USA September 2014
[19] C Xu B Firner R S Moore et al ldquoScpl indoor device-free multi-subject counting and localization using radio signalstrengthrdquo in Proceedings of the 12th International Conference onInformation Processing in Sensor Networks (IPSN rsquo13) pp 79ndash90Philadelphia Pa USA April 2013
[20] K Kleisouris B Firner R Howard Y Zhang and R P MartinldquoDetecting intra-room mobility with signal strength descrip-torsrdquo in Proceedings of the 11th ACM International Symposiumon Mobile Ad Hoc Networking and Computing MobiHoc 2010pp 71ndash80 USA September 2010
[21] Z Yang C Wu and Y Liu ldquoLocating in fingerprint spacewireless indoor localization with little human interventionrdquoin Proceedings of the 18th Annual International Conference onMobile Computing and Networking (Mobicom rsquo12) pp 269ndash280August 2012
[22] L Sun S Sen and D Koutsonikolas ldquoBringing mobility-awareness to WLANs using PHY layer informationrdquo in Pro-ceedings of the 10th ACM International Conference on EmergingNetworking Experiments and Technologies CoNEXT 2014 pp53ndash65 Australia December 2014
[23] Y Zeng P H Pathak and P Mohapatra ldquoAnalyzing shopperrsquosbehavior throughWiFi signalsrdquo in Proceedings of the 2ndWork-shop on Physical Analytics WPA 2015 pp 13ndash18 Italy
[24] K Ali A X Liu W Wang and M Shahzad ldquoKeystroke recog-nition using WiFi signalsrdquo in Proceedings of the 21st AnnualInternational Conference on Mobile Computing and NetworkingMobiCom 2015 pp 90ndash102 France September 2015
[25] X Zheng JWang L Shangguan Z Zhou and Y Liu ldquoSmokeyUbiquitous smoking detection with commercial WiFi infras-tructuresrdquo in Proceedings of the 35th Annual IEEE InternationalConference on Computer Communications IEEE INFOCOM2016 USA April 2016
[26] L Sun S Sen D Koutsonikolas and K-H Kim ldquoWiDrawEnabling hands-free drawing in the air on commodity WiFidevicesrdquo in Proceedings of the 21st Annual International Confer-ence on Mobile Computing and Networking MobiCom 2015 pp77ndash89 France September 2015
[27] G Wang Y Zou Z Zhou K Wu and L M Ni ldquoWe canhear you with Wi-Firdquo in Proceedings of the ACM InternationalConference on Mobile Computing and Networking (MobiComrsquo14) pp 593ndash604 Maui Hawaii USA September 2014
[28] K Qian C Wu Z Zhou Y Zheng Z Yang and Y Liu ldquoInfer-ring Motion Direction using Commodity Wi-Fi for InteractiveExergamesrdquo in Proceedings of the the 2017 CHI Conference pp1961ndash1972 Denver Colorado USA May 2017
[29] Z Yang Z Zhou and Y Liu ldquoFrom RSSI to CSI indoor local-ization via channel responserdquo ACM Computing Surveys vol46 no 2 article 25 2013
[30] X Hu T H S Chu H C B Chan and V C M Leung ldquoVitaa crowdsensing-oriented mobile cyber-physical systemrdquo IEEETransactions on Emerging Topics in Computing vol 1 no 1 pp148ndash165 2013
[31] V Radu and M K Marina ldquoHiMLoc indoor smartphonelocalization via activity aware Pedestrian Dead Reckoning withselective crowdsourced WiFi fingerprintingrdquo in Proceedings ofthe International Conference on Indoor Positioning and IndoorNavigation (IPIN rsquo13) pp 1ndash10 Montbeliard France October2013
[32] X Hu X Li E C-H Ngai V C M Leung and P KruchtenldquoMultidimensional context-aware social network architecturefor mobile crowdsensingrdquo IEEE Communications Magazinevol 52 no 6 pp 78ndash87 2014
[33] X Hu and V C M Leung ldquoTorwards context-aware mobilecrowdsensing in vehicular social networksrdquo of IEEE ISCCGC2015
[34] B Lu Z Zeng L Wang B Peck D Qiao and M SegalldquoConfining Wi-Fi coverage A crowdsourced method usingphysical layer informationrdquo in Proceedings of the 13th AnnualIEEE International Conference on Sensing Communication andNetworking SECON 2016 UK June 2016
[35] Z Ning X Hu Z Chen et al ldquoA Cooperative Quality-awareService Access System for Social Internet of Vehiclesrdquo IEEEInternet of Things Journal pp 1-1
[36] Z Ning F Xia N Ullah X Kong and X Hu ldquoVehicular SocialNetworks Enabling Smart Mobilityrdquo IEEE CommunicationsMagazine vol 55 no 5 pp 16ndash55 2017
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Wireless Communications and Mobile Computing 15
[37] Z Ning XWang X Kong andWHou ldquoA Social-aware GroupFormation Framework for Information Diffusion in Narrow-band Internet of Thingsrdquo IEEE Internet of Things Journal pp1-1
[38] A Rai K K Chintalapudi V N Padmanabhan and R SenldquoZee Zero-effort crowdsourcing for indoor localizationrdquo inProceedings of the 18th annual international conference onMobilecomputing and networking (Mobicom rsquo12) pp 293ndash304 August2012
[39] Y Sungwon P Dessai M Verma and M Gerla ldquoFreeLoccalibration-free crowdsourced indoor localizationrdquo in Proceed-ings of the 32nd IEEE Conference on Computer Communications(INFOCOM rsquo13) pp 2481ndash2489 Turin Italy April 2013
[40] Z-P Jiang W Xi X Li et al ldquoCommunicating is crowd-sourcing Wi-Fi indoor localization with CSI-based speedestimationrdquo Journal of Computer Science andTechnology vol 29no 4 pp 589ndash604 2014
[41] Y Chen L Shu A M Ortiz N Crespi and L Lv ldquoLocatingin crowdsourcing-based dataspaceWireless indoor localizationwithout special devicesrdquoMobile Networks and Applications vol19 no 4 pp 534ndash542 2014
[42] L Cheng and JWang ldquoHow can I guardmy AP Non-intrusiveuser identification for mobile devices using WiFi signalsrdquo inProceedings of the 17th ACM International Symposium onMobileAd Hoc Networking and Computing MobiHoc 2016 pp 91ndash100Germany July 2016
[43] Z Zhou Z Yang C Wu W Sun and Y Liu ldquoLiFi Line-Of-Sight identification with WiFirdquo in Proceedings of the 33rd IEEEConference on Computer Communications (INFOCOM rsquo14) pp2688ndash2696 May 2014
[44] YWang J Yang Y Chen H Liu M Gruteser and R PMartinldquoTracking human queues using single-point signalmonitoringrdquoin Proceedings of the 12th Annual International Conference onMobile Systems Applications and Services MobiSys 2014 pp42ndash54 USA June 2014
[45] H Wang D Zhang Y Wang J Ma Y Wang and S Li ldquoRT-Fall A Real-Time and Contactless Fall Detection System withCommodity WiFi Devicesrdquo IEEE Transactions on Mobile Com-puting vol 16 no 2 pp 511ndash526 2017
[46] X Guo D Zhang K Wu and L M Ni ldquoMODLoc Localizingmultiple objects in dynamic indoor environmentrdquo IEEE Trans-actions on Parallel and Distributed Systems vol 25 no 11 pp2969ndash2980 2014
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
International Journal of
AerospaceEngineeringHindawiwwwhindawicom Volume 2018
RoboticsJournal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Active and Passive Electronic Components
VLSI Design
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Shock and Vibration
Hindawiwwwhindawicom Volume 2018
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawiwwwhindawicom
Volume 2018
Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom
The Scientific World Journal
Volume 2018
Control Scienceand Engineering
Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom
Journal ofEngineeringVolume 2018
SensorsJournal of
Hindawiwwwhindawicom Volume 2018
International Journal of
RotatingMachinery
Hindawiwwwhindawicom Volume 2018
Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawiwwwhindawicom Volume 2018
Hindawiwwwhindawicom Volume 2018
Navigation and Observation
International Journal of
Hindawi
wwwhindawicom Volume 2018
Advances in
Multimedia
Submit your manuscripts atwwwhindawicom
Top Related